Blog

All Blog Posts  |  Next Post  |  Previous Post

TMS WEB Core v1.2 tips & tricks part 5: Accessing microphone and camera

Bookmarks: 

Tuesday, June 25, 2019

Over the years, increasingly new web APIs have given web developers access to areas of the operating system that were originally only accessible from native applications. This is also the case with the microphone and camera that is attached to a desktop computer or integrated in laptops, tablets and smartphones.

As the main vision and mission of TMS WEB Core is to bring OO RAD component based development for web client applications, it is expected that TMS WEB Core also brings easy to use components for using your device microphone or cameras (yes that is plural as it is possible to select between front and back cameras on a mobile device for example) from a TMS WEB Core application.

Security / privacy

An important first remark to make is that before a web client application can access a microphone or camera, user authorization is required. So, clearly, no web client application can secretly in the back listen to your conversations or watch what you are doing! Together with this authorization is also the requirement that the web application comes from a HTTPS URL so there is a reliable identification of the author or company hosting the application possible. So, when your TMS WEB Core application runs from a HTTPS domain and you want to access a microphone or camera, the user will see a prompt to allow this access:



Microphone

To use the microphone, we have introduced the non-visual component TWebMediaCapture. This component can be used to capture audio via the microphone when TWebMediaCapture.Capture is set to mctAudio. The standard recording mode is mrmManual. This means that recording is started by calling WebMediaCapture.Start and the recording is stopped by calling WebMediaCapture.Stop. After the recording is stopped, the data that was recorded is asynchronously returned via the event OnStopCapture that has the event signature:

TMediaCaptureCloseEvent = procedure(Sender: TObject; ABinary: TJSUint8Array; ABase: string) of object;
This event returns the audio via a JavaScript array or as a base64 encoded string. An important remark is that different browsers might have a different default audio file format. In the Chrome browser this is the webm format for example, in Firefox it is ogg. With this component, we have created a TMS WEB Core dictaphone app. You can discover this application here: https://download.tmssoftware.com/tmsweb/demos/tmsweb_Dictaphone/

The code to start and stop the recording is simply done from a click on an image:


procedure TForm1.WebImageControl1Click(Sender: TObject);
begin
  if not FRecording then
  begin
    WebMediaCapture1.Start;
    FRecording := True;
    WebImageControl1.URL := GetStopImage;
  end
  else
  begin
    WebMediaCapture1.Stop;
    FRecording := False;
    WebImageControl1.URL := GetRecordImage;
  end;
end;

The code to capture the recorded data is here:

procedure TForm1.WebMediaCapture1StopCapture(Sender: TObject;
  ABinary: TJSUint8Array; ABase: string);
begin
  if WebIndexedDbClientDataset1.RecordCount > 0 then
    WebIndexedDbClientDataset1.RecNo := 1;

  WebIndexedDbClientDataset1.Insert;
  WebIndexedDbClientDataset1.FieldByName('descr').AsString := WebEdit1.Text;
  WebIndexedDbClientDataset1.FieldByName('time').AsDateTime := Now;
  WebIndexedDbClientDataset1.FieldByName('base').AsString := ABase;
  WebIndexedDbClientDataset1.Post;
  WebEdit1.Text := 'SoundClip ' + IntToStr(WebIndexedDbClientDataset1.RecordCount + 1);
end;

This code shows another interesting concept, that is to store the recorded audio in an IndexedDB dataset. That is a dataset hosted in your browser which is private and accessible at a later time, so you can listen again to captured audio snippets when you open the web application again at a later time.

One more interesting option is to let the microphone record automatically when a certain audio level is reached. To use the TWebMediaCapture this way, set WebMediaCapture.RecordingMode to mrmAudio. With the properties WebMediaCaptature.Sensitivity, WebMediaRecorder.FFTSize, WebMediaRecorder.SmoothingTimeConstant it can be controlled at what noise level and what duration of the noise level the recording will start and stop.

Camera

Thanks to the new TWebCamera component, taking pictures from a TMS WEB Core web client application is equally easy. Drop the component on the form and start the camera by calling WebCamera.Start. To take a picture, call one of the three properties depending on the format you wish for the captured image data:

property WebCamera.SnapShotAsBase64: string;
property WebCamera.SnapShotAsImageData: TJSImageData;
property WebCamera.SnapShotAsUint8Array: TJSUint8Array;
A very simple use is to call WebCamera.SnapShotAsBase64 and assign the result to a HTML image element, i.e. what is wrapped in TWebImageControl via:

WebImageControl1.URL := WebCamera1.SnapShotAsBase64;
The image format returned van be selected between JPEG or PNG with WebCamera.BaseFormat.

As mentioned, TWebCamera can handle multiple cameras. To select a camera, the property WebCamera.CameraType can be used that can select between rear, front or default camera. To detect the cameras available, the TWebCamera.CameraDevices collection can be used. You can use:

for i := 0 to WebCamera.CameraDevices.Count - 1 do
begin
  WebListBox1.Items.Add(WebCamera.CameraDevices[i].Name);
end;

For camera & microphone use, we have published 3 demos you can discover at: https://www.tmssoftware.com/site/tmswebcoreintro.asp#demos There is even a camera demo that integrates with a QR and barcode scanning library so you can detect and decode pictures taken from a QR code or barcode. Of course, the TMS WEB Core distribution contains the full source code of these demos

Discover these and more exciting features in TMS WEB Core that let you create web client applications with unprecedented capabilities.

You can download the trial version, go ahead with the standalone version you purchased or with TMS WEB Core and additional tools that are all included in TMS ALL-ACCESS. Our team looks forward to all your comments, feedback, feature-requests to help steering the development of TMS WEB Core to the next stages!

Bruno Fierens


Bookmarks: 

This blog post has not received any comments yet.



Add a new comment

You will receive a confirmation mail with a link to validate your comment, please use a valid email address.
All fields are required.



All Blog Posts  |  Next Post  |  Previous Post