Blog
All Blog Posts | Next Post | Previous Post
Visiting the TMS lab day 2: Adding artificial intelligence to TMS WEB Core apps
Tuesday, October 2, 2018
There is no doubt about it that artificial intelligence is a hot topic. Wondering how this cool technology could be used by Delphi developers creating web applications, we researched how we could leverage existing technology. In the area of artificial intelligence, the Tensorflow.js library is one of the most popular and powerful libraries for use in web clients. So, we embarked on investigating how we could enable using TensorFlow.js easily from our beloved Pascal language in a TMS WEB Core web client application.
First sample: sequential prediction
This first small sample is based on creating first a model with sample data ourselves and then let the machine predict a value based on the model. The model is simply a series of points (X,Y values). After we feed the point data into the model, it is up to the model to predict the next point on a line.
In Pascal, this translates to using a class TTMSTFPredictSequentialNext that internally uses the Tensorflow model tf.keras.models.Sequential. This is made available as a TComponent with following public interface:
TTMSTFPredictSequentialNext = class(TComponent)
public
procedure createModelWithSample(xs, ys: TJSArray);
function predictYforX(aValue: JSValue): JSValue;
end;
To feed the model with sample data, a Javascript array of X values and Y values is created, initialized with the dots clicked on the user-interface in a TWebPaintBox. Drawing the dot on the TWebPaintBox is just like a Delphi developer would do this for years with the TCanvas in a TPaintbox:
procedure TForm2.DrawDot(X, Y: Integer); begin WebPaintBox1.Canvas.Brush.Style := bsSolid; WebPaintBox1.Canvas.Brush.Color := clRed; WebPaintBox1.Canvas.Pen.Style := psSolid; WebPaintBox1.Canvas.Pen.Width := 2; WebPaintBox1.Canvas.Pen.color := clRed; WebPaintBox1.Canvas.Ellipse(X-4, Y-4, X+4, Y+4); end;
Initialize the model
var xs,ys: TJSArray; begin tfModel := TTMSTFPredictSequentialNext.create; xs := TJSArray.New; ys := TJSArray.New; FillData(xs, ys); NormalizeData(xs, ys); tfModel.createModelWithSample(xs, ys); end;
Note that here we of course need to first normalize the X value, i.e. convert it to the range 0..1 where 0 is the minimum X value and 1 is the maximum X value. The predicted Y value is normalized as well, so here we need to denormalize it, i.e. convert the value between 0..1 back to Y pixel coordinates based on the minimum and maximum Y value.
var predictY: single; X,Y: integer; begin X := LastPointX + 50; predictY := tfModel.predictYforX(normalize(X))); y := denormalize(predictY); end;
This is a sample prediction:
or you can discover how this works by directly using the online TMS WEB Core demo.
Second sample: digit identification
In the second sample, we show how existing model data can be loaded in the TensorFlow model and can be used to identify a drawn number. The model data is taken from a Tensorflow example using a MNIST database. It is loaded into the model with:
tfModel := TTMSTFDigitIdentifier.Create;
tfModel.loadModelFromUrl('ModelJs/model.json');procedure TForm2.WebPaintBox1MouseDown(Sender: TObject; Button: TMouseButton;
Shift: TShiftState; X, Y: Integer);
begin
doPaint := true;
remX := X;
remY := Y;
end;
procedure TForm2.WebPaintBox1MouseMove(Sender: TObject; Shift: TShiftState; X,
Y: Integer);
begin
if doPaint then
begin
WebPaintBox1.Canvas.Pen.Width := 30;
WebPaintBox1.Canvas.Pen.Color := colorStroke;
WebPaintBox1.Canvas.MoveTo(X, Y);
WebPaintBox1.Canvas.LineTo(X + 30, Y);
remX := X;
remY := Y;
end;
end;
procedure TForm2.WebPaintBox1MouseUp(Sender: TObject; Button: TMouseButton;
Shift: TShiftState; X, Y: Integer);
begin
doPaint := false;
end;The next step is to get the image data from the TWebPaintBox, normalize its size, i.e. reduce it to a 28x28 image and call the method model.IdentifyDigit() to let the identification happen. This identifyDigit() method expects the image data as type JSImageData, defined in Pascal as TJSImageData. When it is recognized, IdentifyDigit returns a number from 0 to 9, when not, it returns -1. The returned value is set as caption of a panel.
The code used is:
var
ImgData: TJSImageData;
v: JSValue;
begin
ImgData := normalizeddigit.Canvas.Context.getImageData(0, 0, 28, 28);
v := tfModel.identifyDigit(x);
if v = -1 then
pnlDigit.Caption := '?'
else
pnlDigit.Caption := Format('%d', [v]);
end;
Summary
The area of artificial intelligence is big. Different problems require different models and different approaches. We just presented two small samples here, but it is clear that much more can be done, not only with TensorFlow but also with other libraries. In our lab, we did the experimental work to use this in a convenient way from our beloved Pascal language for these two samples, but clearly, there is still quite some work ahead to expose all capabilities as convenient to use Pascal classes. We're curious and very interested to hear what AI problems you might want to solve and thus, with what priority and on what we should focus first.
Want to learn more about Tensorflow?
You can visit this tutorial if you wish to learn more about the exciting possibilities of ML, AI and Tensorflow: https://www.guru99.com/tensor-tensorflow.html
Lab visit feedback & win!
Our team loves to hear what you think about what is brewing in the lab, how you plan to use the upcoming features, what priority our team should give to it and if you have possibly interesting and/or creative ideas to make this even more powerful for Delphi developers. To reward your interaction & feedback, we'll pick 3 blog comments on October 15 that we liked the most and first prize is a free TMS WEB Core license, the 2nd and 3rd prize is a 50% discount coupon on TMS WEB Core. Let yourself hear to increase your chances!
Meanwhile, you can go ahead and explore the new & exciting territories of web client development that become available for Delphi developers with TMS WEB Core! You can download the trial version that is generally available, go ahead with the standalone version you purchased or with TMS WEB Core and additional tools that are all included in TMS ALL-ACCESS. Note also that in October, you can still take advantage of purchasing TMS WEB Core at launch price from 295EUR for a single developer license. From Nov 1, regular pricing will be active on TMS WEB Core (395EUR for a single developer license).
Bruno Fierens
This blog post has received 4 comments.
2. Saturday, October 6, 2018 at 9:16:09 AM
The digit identification is based on sample model data from the TensorFlow examples that could indeed be more accurate. I assume that with more training sample data, it would be more accurate. We''ll continue to research this integration and possibilities and at the same time also experiment with other AI libraries.
Bruno Fierens
3. Thursday, December 2, 2021 at 12:41:56 PM
Hello Bruno.This is an excellent example of TMS WEB core power. Is the full source code available for this example as TMS WEB core project?. It would be nice as I am taking a course in TensorFlow with JavaScript and would like to find out how to integrate tensorFlow.js libraries into TM WEB core project.
Regrads,
Ertan
Ertan Baykal
Regards,
Ertan
Ertan Baykal
Baykal Ertan
4. Thursday, December 2, 2021 at 4:34:11 PM
The source code of these projects can be downloaded here:http://www.tmssoftware.net/public/tensorflow.zip
Bruno Fierens
All Blog Posts | Next Post | Previous Post
I''ve long integrated neural networks into Delphi for prototype development but was starting to hit problems as the NN sw package I use is 20 years old(Neuroshell 2 :-) ) and only supported on Windows 32 bit. I can now look to further develop our prototype testing with TMSwebcore + TensorFlow to customers globally in a simple manner.
Thanks again!
knowles adrian