Making a facial recognition system for Southeast Asia

Pritha Vijay, lead product manager

. July 21, 2023 . Regional

Grab uses facial recognition as an additional layer of security to ensure that users and driver-partners are who they say they are.

This was first introduced in Malaysia in 2019 as part of broader efforts to boost overall safety standards for everyone using our platform. Facial recognition verification has since been rolled out across the region.

When a new user signs up, we register their face to improve safeguards against crimes and to help with incident investigations. Passengers will only need to go through such a verification once.

For driver-partners, we prompt a verification step every time they log on to ensure that booking requests are only taken on by registered, licensed drivers who have passed background checks.

We also track the time taken for driver-partners to submit their selfies. Taking too long could be a sign of individuals sharing or renting accounts from verified accounts. Such driver-partners may be asked for a follow-up selfie verification at a randomised time.

The app also identifies key facial features and compares them against a database of users who have been barred from our platform for offences such as theft and harassment, among others. If there’s a match, the user will be blocked and prevented from proceeding.

Making facial recognition available to more devices

Facial recognition tech is becoming more and more common as a biometric security layer. To make this work on phones, some devices have extra hardware features such as an infrared camera that can sense a more accurate 3D map of the face. Other phones require a well-lit picture at least, and need better front-facing cameras to help capture that.

But what do you do to make facial recognition accessible to a bigger range of smartphones, including lower end models which might not have these features?

The app begins by performing a liveness check that requires users to nod their heads.

Additionally, Grab’s facial recognition backend was expanded on a set of spoof versions of images in its database. This allows it to better pick out false face data, such as people showing photographs to their phones or wearing masks of other faces.

We relied on data annotators to manually label, tag, and classify data to synthesise image spoofing cues to train our models. Common spoofing cues include distortion and interferences that appear when cameras capture images of screen replays, or printed masks.

A database for Southeast Asian faces

Facial recognition systems are only as good as the data they are trained on.

According to a report published by US-based National Institute of Standards and Technology in 2019, most facial recognition algorithms exhibit bias. The 189 major facial recognition algorithm systems studied falsely identified Black and Asian faces 10 to 100 times more often than they did white people.

This is where Grab’s efforts to collect large and comprehensive face datasets come into play. Our database covers a wide range of demographic groups in Southeast Asia, and has allowed us to build a far less biased facial recognition model that serves the region better.

And as mask-wearing dominated everyday life in the past few years, we also wanted to develop a facial verification model that can verify identities even when people have their masks on. We began to add masked selfies into the training data, allowing the model to process both masked and unmasked selfies.

Engineering

Making a facial recognition system for Southeast Asia

Making facial recognition available to more devices

A database for Southeast Asian faces

Komsan Chiyadis

Komsan Chiyadis