Mark Patrick, Mouser Electronics

September 2017 was a significant month in smartphone technology – the month that Apple announced its revolutionary iPhone X. The headline-grabbing features included a large OLED screen and the ability to charge wirelessly, all with a sub-$1000 price tag. However, it was a security feature that stole the show. The iPhone X used its ability to recognise faces (known as ‘Face ID’) to unlock the device – and to authorise payments. Facial recognition has been around for a while but this was the first rollout in a mainstream consumer product.

How facial recognition started
Now linked with another ‘hot’ technology trend (artificial intelligence – AI), facial recognition has been of interest to researchers since the 1960s. More than half a century ago, US intelligence agencies funded a trio of computer scientists (Woody Bledsoe, Helen Chan Wolf and Charles Bisson) to pioneer automated facial recognition techniques.

In this early stage, the process relied on human operators to extract coordinates of key facial features such as the locations of the corner of eyes, tip of the nose and edge of the mouth. The system then computed the distances between pairs of points to give sizes of key facial features, before storing the data in a database.

One of the greatest challenges the team faced was the huge number of factors that could impact pattern matching. Subject distance (from the camera), rotation of the head, lighting conditions, expression, facial hair growth and aging could all impact results. The team concluded that this could be addressed by creating a program to normalize the distances, as if the face was in a straight-on orientation. First, the program had to determine any lean, tilt or rotation of the head. Once these angles were known, then their effect on feature sizes could be determined and normalized values for a straight-on view were deduced. The algorithm included a ‘standard head’ model that was generated from data derived from analysis of seven test subjects. While, until then, facial recognition had seen limited success and accuracy, the team’s approach heralded a significant step forward in facial recognition accuracy.

Through a combination of increased processing speed / power and higher resolution imaging systems, the technology behind facial recognition has moved on significantly. In fact, modern systems are now able to match, or even surpass, the recognition rates of human operators. This has resulted in several well-reported uses, all of which have contributed to facial recognition becoming considered a mainstream technology.

One obvious application is within the law enforcement / security sector. In fact, face recognition was recently used during the popular Notting Hill Carnival as well as at the Remembrance Sunday ceremony, both in London, UK. However, alongside these trials, there are a couple of other usages that are really delivering facial recognition as a mainstream technology.

Real world usage of facial recognition
Electronic passport (‘ePassport’) gates have become a familiar sight to international travellers in a bid to reduce immigration delays due to increased passenger volumes in already crowded airports. Passengers that have modern biometric (‘chipped’) passports can use the gates that take a photo of the traveller and then use facial recognition algorithms to make a detailed comparison with the photo in their passport (which is stored electronically on the ‘chip’ embedded in the passport.

As the conditions are very controlled (subject position, lighting etc) and the rules for passport photos prevent distractions such as hats or sunglasses the technology generally works well and failure rates are quite low, allowing passengers to flow through the immigration area quickly. In fact, the technology is now migrating to the boarding gate where some airlines are replacing boarding card barcodes with facial recognition to allow passengers to board the aircraft.

In the US (mainly because privacy laws are tougher and consumer / regulator concerns are higher in Europe), Facebook has been using facial recognition technology to ‘enhance’ the user experience. Their image database is unique in that it contains billions of images, which users have neatly tagged with the names of the people contained in the image. AI technology backed by machine learning and a deep learning neural network powers Facebook’s DeepFace technology that calculates a unique number for each individual based on their facial features.

When a new photo is uploaded containing a face that is already on Facebook, DeepFace can identify the individual(s) and prompt the user to tag the image, as well as informing the subject that an image containing them has just been added to the site.

While these applications are playing a significant role in bringing facial recognition to the mainstream, many observers believe that Apple’s Face ID will be the technology that truly pushes it into everyday use. Apple’s ability to combine slick technology with highly effective marketing has a track record of creating ‘must-have’ features that define sectors. A recent report by Strategy Analytics has already said that Face ID is proving very popular with the all-important early adopters.

In order for facial recognition to be successful, it must be robust. Early facial recognition has been ‘fooled’ by holding a photo in front of the camera. However, Face ID is far more sophisticated using multiple sensors, AI and imaging software to create a 3D face map that cannot be replicated by a 2D photo. 30,000 dots from an infrared projector combine to create the 3D face map while the infrared flood illuminator means that even distractions such as sunglasses or a dark environment (including night time) do not impede the facial recognition. A face image is stored so securely on the phone that even Apple cannot access it and this is compared to the Face ID image to authenticate the user. As the technology is ‘always on’ facial recognition is almost instantaneous, making the solution far more convenient than typing a password.

Facial recognition is significantly more secure than techniques such as Touch ID fingerprint biometrics. In fact, Apple states that the chance of fingerprint ID being bypassed is 1 in 50,000 while the likelihood that Face ID can be bypassed is 1 in 1,000,000 – a factor of 20 better. The user needs to make a deliberate attempt to unlock the phone – they must be looking directly at the phone and eyes must be open – so it cannot be bypassed by holding the phone in front of someone who is asleep.

Over a billion images were analysed in developing Face ID with Apple using this data to train their neural network. Yet, even with this huge amount of data, Face ID continues to learn. Each time a face is captured as part of an access request, machine learning algorithms and a ‘neural engine’ capture more information. As a result, the face map is continuously updated, allowing the system to cope with the effects of aging or other facial changes, temporary or permanent.

What comes next?
Facial recognition has become big business and following all of the recent activity, many businesses are building the technology into their applications and products. Allied Market Research are predicting that the market for facial recognition will be worth around $9.6 billion by 2022, a CAGR of 21.3% over the next four years.

The rapid growth is due, in part, to the availability of plug and play hardware modules and associated software that allows the incorporation of facial recognition technology by designers with relatively little experience in the area. One example of a modular solution is the B5T HVC Face Detection Sensor Module from Omron Electronics. The 60mm x 40mm package includes a fully integrated human vision component (HVC) module with a camera that is capable of wide angle and long-range operation. Also in the B5T HVC are a microprocessor and USB and UART interfaces which allow the module to be controlled as well as facilitating the transfer of image data to an external system. Integral to the system is Omron’s OKAO technology which ensures rapid and accurate facial recognition based upon unique image sensing algorithms

In just 1.1 seconds, the B5T can capture an image, detect and recognise a face that is around 4 feet (1.3m) away, with a high level of confidence based on the accuracy of the system. The system can even make an assessment of the subject’s likely mood based on a small database of 5 stored expressions. Estimating blinks or gazes takes just one second.

The module has an input resolution of 640 x 480 pixels and can detect at angles of 49° to the horizontal and 37° to the vertical. Human bodies can be detected up to 2.8 meters away and features such as hands can be detected at distances of up to 1.5 meters.

The module incorporates a total of ten sensing functions to recognise non-verbal intentions. Alongside the expression estimation already mentioned these include detecting age and gender as well as detecting faces, hands and bodies. As the module takes care of the image processing locally, it is a relatively easy task to add external intelligence and incorporate the device into Internet of Things (IoT) applications.

Download Techmezine Android app on google play store: Download Now