Face Detection by Jason Schneider

Every analysis of the picture-taking habits of Americans by subject category has come to the same inescapable conclusion—people are by far the most popular photographic subjects, and they all have faces. Shooting a frame-filling close-up of a cute kid or your aunt Martha doesn’t usually pose much of a technical problem for the auto-focus (AF) and auto-exposure (AE) systems built into today’s digital point & shoot cameras—they’ll focus on her face and generally give you an excellent exposure. But when you place that person’s face off center near the edge of the frame, compose a double portrait of mom and dad, or photograph a group of people arrayed around the dinner table, things don’t always work out as you planned because the camera doesn’t see the world as you do. Conventional AF systems, even those with multiple AF zones, will sometimes miss the faces and focus on something else that’s in between them, farther away, or closer. The result: The people in your picture will be out of focus, blurry, and over- or underexposed—not a good thing. Sure, you can anticipate and override your cameras potential mistakes by locking focus on your intended subject--just hold in the shutter button partway, recompose and shoot. But in the excitement of the moment, it’s easy to forget this simple procedure. And even though it only takes a second to autofocus the intelligent way by adding a bit of manual control to the process, in that brief instant your little angel’s perfect expression can well have vanished.

Here’s how face detection looks when using a Nikon Coolpix.

Fortunately for everyone who takes people pictures, many leading camera manufacturers have addressed this sticky problem head-on (no pun intended) and have come up with ingenious solutions known as Face Recognition, Face Detection or Face Priority Mode. Basically, face identification is a form of pattern recognition—when you turn on the camera’s face-detection system, the auto-focus and auto-exposure systems analyze the objects appearing within the borders of the frame, and compare them with algorithms stored in the camera’s electronic memory. When face patterns are detected (up to 10 in some systems, fewer in others), the camera focuses sharply on your intended subjects, and also adjusts the exposure so the subjects are properly exposed. Some of the latest iterations of face detection even claim to be able to identify facial patterns in profile with a high degree of accuracy (check the specs to see if a camera you’re considering offers this capability).

Not all face detection systems work exactly the same way when it comes to identifying and prioritizing multiple faces within a frame, but all provide some kind of readout on the LCD and/or in the viewfinder—usually a frame around the prioritized face—to let you know what the camera is focusing on so you can override its decision. Some systems can be turned off when you’re photographing subjects other than people, such as landscapes or animals. Others are on all the time, the manufacturers claiming that their systems can tell with a high degree of certainty when there are faces in the picture. No face detection systems can perform miracles of course, such as simultaneously achieving a sharp, perfectly exposed image of uncle Joe who is 27 feet away and cousin Steve whose face is 3 feet from the camera. However, based on my personal hands-on experience with Fuji, Canon and Nikon cameras incorporating face recognition systems, all do a remarkably good job well over 90% of the time, and they have the great advantage of doing so automatically without the need for thought or user intervention.

With Face Detection turned on Fuji Finepix F40fd identifies main face (green frame) and subsidiary faces (white frames) and calculates optimum focus and exposure. System prioritizes faces closest to center of frame.

The only thing you must bear in mind with face detection is that the operational procedure varies from camera to camera. If your camera has user-selectable face recognition system, you’ve got to turn it on when you’re taking people pictures and turn it off when you’re shooting, say, scenic vistas or macro close-ups of flowers. This is usually a simple procedure—push the button to turn the system on and a face detection icon appears on the LCD. What happens if you don’t turn off face detection when shooting, say, landscapes? Based on my field tests, it seldom causes any focusing or exposure errors. The theory is that something like a face-shaped object in the foreground could mess things up, but this is highly unlikely. However face detection systems will focus on faces appearing in photographs and paintings of faces as well as statues, etc. They will not focus on dog, cat, and cow faces—apparently they are sophisticated enough to differentiate human from animal faces, and, as mentioned some manufacturers claim their systems can even pick out faces that are in profile or three-quarter view as well as those facing the camera frontally.

While it’s not possible to provide a detailed comparison of the specific performance characteristics of each manufacturer’s face recognition system or the slightly different iterations among the various camera models in this article, here’s a hands-on example of how it works using a typical 2007 point and shoot digital camera—the 8.3 megapixel Fuji FinePix F40fd (yes, the last two italicized letters stand for face detection). To enable face detection, turn the camera on, press the face detection button on the back of the camera once (it has an icon of a person showing the head in autofocus brackets), push lightly on the shutter release and an identical icon now appears in the upper left-hand corner of the 2-1/2-inch LCD confirming that the system is activated. If you now point the camera at a person or a group of people, a green frame will appear over the “primary subject” and a white frame may appear over “subsidiary subjects.” If you now take the picture, the primary subject will be sharply focused and well exposed and the subsidiary subject or subjects will be as sharply focused and as well exposed as possible given their varying distances from the camera, their relative brightness, and whether they’re within the range of the built-in flash.

With face detection turned on camera focuses properly on off-center face (top image), but with system turned off (bottom) sharpest focus is on background details in window onß left and face is less sharp.

As previously mentioned, the system works very well in most cases, and you are unlikely to improve the result by manual intervention, but it can’t perform miracles, such as extending depth of field or flash range beyond the laws of physics. If you’re shooting a single subject, such as an off-center portrait, the green frame will appear over the subject’s face (it varies in size to match the facial outline) and the camera will focus on the face, not on distant objects in the center of the frame. The exposure will also favor the face, giving accurate, natural-looking skin tones, though the background may be under- or overexposed depending on its brightness, distance from the camera, etc. The Fuji F40fd also performs well when shooting a double portrait, and will almost always display a green frame over one of the subjects rather than focusing on, say, the distant tree in the middle of the frame. And occasionally, perhaps 2 to 3 percent of the time (especially with faces that are shadowed, in profile, wearing strange hats or scarves etc.) the face detection system will either fail to identify a face or put a frame around something other than a face. The remedy in such cases is simple—turn off face detection by pressing the face detection button (you’ll be prompted by a big “face detection off” icon in the center of the LCD) press the shutter button lightly to turn the system off, lock focus on the subject you select, recompose, and take the picture.

With face detection turned on (image on top) camera identifies and focuses on woman closest to camera, and man in the distance is less sharp. With face detection turn off, central AF zone of camera focuses on man in the distance and woman in foreground is slightly less sharp.

Face recognition is just the latest example of how advanced electronic technology can help people take better, more satisfying pictures automatically without having to think about how their cameras work. It’s the hot new digital point-and-shoot feature of 2007-2008—in 2006 it was built-in anti-shake, a feature that is now present on nearly 90% of point-and-shoots and many DSLRs. Based on the response of the world’s leading camera manufacturers it’s clear that they consider face detection quite important—Canon, for example now offers nearly two dozen different models with this feature. And cameras that incorporate face recognition technology are now available from Canon, Fuji, Nikon, Panasonic, Pentax, Samsung and Sony. Some outstanding point-and-shoot examples: The Canon PowerShot SD1000 Digital Elph, the Fuji FinePix F40fd, the Pentax Optio A30, the Nikon Coolpix 5000, and the Samsung NV11. Just as this article was being completed Panasonic announced the Lumix L10, a full-featured FourThirds-system DSLR that qualifies as the first DSLR with a sophisticated built-in face detection system! Like anti-shake, face detection is a feature that delivers real-world advantages and we expect to it built into an increasing number of ultra-zoom cameras with electronic viewfinders and in more consumer digital SLRs. Let’s face it—it’s an idea whose time has come.