Trying to implement facial recognition in your applications can be a daunting task. With the use of TMS Web Core and the vast amount of available JavaScript libraries, this can be made much easier. There is no need for complex functions or algorithms. Just include the library in your HTML and you can access all the functionality. To showcase this, we've made a demo with the following library: face-api. This library contains a lot of different functionality that you can use. You can detect faces and match them with previously created models. But you can also use it to detect the age, sex and expressions of the person. You can even draw landmarks on the detected face. You can find the demo here (requires a webcam installed).

WebGL vs WebAssembly

The library that we used, gives us the option to use WebGL or WebAssembly. whilst WebGL is really fast it does have a initial warm-up time. the first detection of a face will take 5000-6000ms and every subsequent detection will be around 100ms. Performance is also heavily dependent on the installed GPU, as having a dedicated GPU really boosts the performance. Meanwhile, WebAssembly uses the CPU. We recently did some experimentation with this technology. You can read all about it in this blogpost. WebAssembly doesn't have the initial warm-up time, but every detection takes about 700-900ms. As we only do one detection when logging in, we've opted to use WebAssembly . 

In the following chart we only used one model to start face recognition (the demo uses 7 models to get all necessary data). this chart also doesn't take the warm-up time in account. But it should give you a fair comparison between different systems and set-ups. 

picture taken from Tensorflow.js which is used in the back-end of the face-api library.

Integrating the Face-Api using external classes

People who use TMS WEB Core will already know that you can use asm blocks to write JavaScript in Pascal methods. But you can also use the external keyword to wrap classes and functions from libraries. These external classes and methods can then be used directly in your Pascal code, negating the need to use asm blocks. This makes it also very easy to integrate these libraries in different applications as you'll only need to include the pas file in your project. 

Here you can find an example of a procedure:

procedure LoadAgeGenderNetModel(AURL: string); async; external name 'faceapi.nets.ageGenderNet.load';

TFaceMatcher = class external name 'faceapi.FaceMatcher' (TJSObject)
  public
    constructor new; overload;
    constructor new(AInput: TJSObject); overload;
    labeledDescriptors: TJSArray;
    function FindBestMatch(Descriptor: TJSObject): TFaceMatch; external name 'findBestMatch';
    function ToJSON: JSValue; external name 'toJSON';
    class function FromJSON(AJSON: TJSObject): TFaceMatcher; external name 'fromJSON';
  end

To use this method you also need to add the following code right under the interface

{$mode objfpc}
{$modeswitch externalclass}

Demo

For our demo we have used facial recognition to use as authentication. When we can't match the face, the application will ask to register yourself. This data will be saved in an indexedDb. The model to test against will be updated and stored in local storage. When a face is recognized, we will fetch the data of that person and display it in another form. except for opening forms and writing/reading from the databases, the actual detecting and matching only requires 5 lines of code when using the libfaceapi unit. 

You can find the source code here.