Apples iOS facial recognition could lead to Kinect-like interaction

Apple has included facial recognition technology in iOS 5, 9to5Mac discovered earlier this week. Its not something Apple is advertising about with the software update yet, but as it develops, it could become one of the most significant additions ever introduced to Apples mobile operating system.

The recognition tech was presumably acquired through Apples 2010 purchase of Polar Rose, a company that specialized in face detection algorithms. 9t05Mac found iOS 5 APIs that use the tech, which means they should provide an easy way for developers to implement facial recognition for various purposes in their own apps, quickly and easily.

The unearthed APIs are described as highly sophisticated, and can determine where a users mouth, and left and right eyes are located, as well as process images taken by the iPhone for face detection. Aside from providing Apple an easy way to introduce Faces (which recognizes specific people in iPhoto) to both its own Photos app and any third-party apps that access that library, it should also open the door for much more advanced facial recognition applications.

You could create apps that track a users eye movement and dynamically change content accordingly, for instance. App developers might even be able to use data gathered from facial recognition APIs to identify so-called hotspots, providing insight about where a user is looking most within an app and arranging content accordingly. In time, an iPhone app might even be able to assess the emotional state of the user, based on whether theyre frowning or smiling, and address the user in a manner appropriate to the! ir mood. It might also be able to tell how engaged users are with mobile ads and content, which might be useful for iAd customers, among others.

Apple could also use the tech to implement something many have been asking for on iOS device in a unique way: user account switching. Currently, iOS devices dont have user accounts the way a Mac does. On the iPad especially, which is a shared device for many, it makes sense to offer multiple accounts that offer different levels of access to different sets of content. Facial recognition could intelligently and automatically switch iOS user profiles, setting restrictions if a child picks up a device, and changing the app load-out and home screenarrangementfor different family members.

But before all that, which is admittedly something still likely quite a way off in terms of the development of iOS, we should see implementations that improve Apples existing video products. So features like FaceTime, for instance, could get the ability to judge which person deserves focus during a group video chat (which is also likely in the works). A similar system has been described as one of the possible motivations behind Googles recent acquisition of facial recognition company PittPatt, for use with Google+ Hangouts.

Whatever else it leads to, facial recognition should provide developers with the opportunity to create some impressive new apps, so long as they keep in mind what Facebook learned the hard way: People dont like it when you implement recognition without asking nicely first.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.