keyTMS ALL-ACCESS
local_offerOne money and time saving bundle
mouseCross Platform UI
bar_chartReporting
languageWeb
swap_horizCommunication
storageData
mouseWindows UI
  •  
  •  
  •  
  1. Home
  2. Blog

Blog

All Blog Posts  |  Next Post  |  Previous Post

Visiting the TMS lab day 5: Using external classes in pas2js

Monday, October 8, 2018



For todays visit in the lab, we dive deep into the workings of the pas2js compiler. This technical article is therefore provided by Michael Van Canneyt, one of the two masterminds behind the pas2js compiler. Michael Van Canneyt has been involved in the Pascal language development community for over 20 years, wrote numerous articles, books, open-source Pascal projects, commercial Pascal based desktop and server applications and much more. In other words, a very highly respected Pascal language professional we have the honour to work with!

Using external classes in pas2js

Using Classes in Object Pascal goes without saying. JavaScript also has objects and classes - sort of. Meanwhile, the amount of JavaScript APIs and libraries dwarfs probably the amount of available libraries in any other language. So, the natural question is: how do we use all these native JavaScript objects and classes in Object Pascal as efficiently as possible?

One way would be to actually create classes in Object Pascal which mimic the classes in JavaScript, and use assembler blocks to access the fields and methods of the actual underlying object. This approach has the advantage that it allows you to reshape the native JavaScript object into something that is more pascal-friendly. Or, if you don’t like the API of a JavaScript object, this approach also allows you to create a better API. The disadvantage of this approach is that it requires a lot of work, code and is error prone.

Another approach is to let the compiler understand JavaScript classes: provide the compiler with an object pascal description of a JavaScript object, but without an implementation in pascal: an external class definition.

In essence, this is nothing more than an extension of the ‘external’ declaration that programmers have used for ages to access functions in a library. Nobody thinks twice when looking at the following external function declaration:

function CreateFile(lpFileName: LPCWSTR; dwDesiredAccess, 
wShareMode: DWORD; lpSecurityAttributes: PSecurityAttributes; dwCreationDisposition,
dwFlagsAndAttributes: DWORD; hTemplateFile: THandle): THandle; stdcall; external kernelbase name 'CreateFileW';

Or, if you want, an extension of an ‘Interface’ for a COM or other object which you import using a type library:

IInterface = interface   ['{00000000-0000-0000-C000-000000000046}']   
function QueryInterface(const IID: TGUID; out Obj): HResult; stdcall;
function _AddRef: Integer; stdcall;
function _Release: Integer; stdcall;
end;

This is also an external definition: the compiler knows what methods to expect, and can generate code to call each of the methods. In Object pascal, you can add property definitions to an interface: this is actually a convenience feature for the pascal programmer, since COM by itself does not know properties. When reading or writing the property, the compiler just knows it must insert calls to the getter and setter methods of the interface.

So, extending this idea, it is only a natural extension to be able to say

TJSHTMLCollection = class external name 'HTMLCollection'  
private 
  FLength : NativeInt; external name 'length';  
public    
  function item(aIndex : Integer) : TJSNode;
  function namedItem(aName : string) : TJSNode;
  property Items[aIndex : Integer] : TJSNode read item; default;    
  property NamedItems[aName : String] : TJSNode read namedItem;
  property length : NativeInt Read FLength;  
end;
To let the compiler know that there exists a class called HTMLCollection in JavaScript, and that whenever we create an instance of TJSHTMLCollection, the compiler should actually generate the necessary code to create a HTMLCollection class.

A big advantage of using such an external class definition is that it is very efficient at runtime: the transpiled code emitted by the compiler will contain direct method calls for all methods of external classes, there will be no extra generated intermediate code or conversion code.
This means that writing

var
  a: TJSNode;  
begin
  a:=Coll[0];  
end;


Where Coll is of type TJSHTMLCollection presented above, will map directly on 
{
    var a = null;
    a=Coll.item(0);
}


Which is about as efficient as it can get for transpiled code.

An external class definition is also easy to understand: the above does not really differ a lot from an interface definition. Any Object Pascal programmer will at a glance understand what API this class offers. The only new concept to grasp is the header:

TJSHTMLCollection = class external name 'HTMLCollection'


The external name ‘HTMLCollection’ is almost verbatim what one can find in a function definition when importing a function from an external library, only now applied to complete classes. One can also view it as an equivalent of the GUID identifier in an interface definition.

The example shows that array properties for external classes are supported, just as they are in interface declarations. It also shows how to declare a read-only field: JavaScript does not know properties as we know them in Object Pascal, but only knows fields. For JavaScript objects that are backed by some native API, these fields can be read-only. The ‘length’ field in the above JavaScript HTMLCollection object is an example. If we were to declare it as 

TJSHTMLCollection = class external name 'HTMLCollection'
   length : NativeInt;
end;


The compiler would know there is a field length, but it would be read-write. Obviously, this is not good. Since we can declare read-only or write-only properties, their syntax is used to solve this problem:

TJSHTMLCollection = class external name 'HTMLCollection'
private
  FLength : NativeInt; external name 'length';
public
  property length : NativeInt read FLength;
end;


From this definition it is clear for the programmer that there is a read-only property length. At the same time compiler knows that when it needs to read the property, it can use the name ‘length’. Why this syntax and not introduce something else, for example:

TJSHTMLCollection = class external name 'HTMLCollection'
    length : NativeInt; ReadOnly;
end;


Besides the fact that we don’t want to pollute the language with too many keywords, the external syntax was needed anyway, for two reasons:
First, JavaScript allows field names to start with the $ character, so a syntax to be able to give a valid pascal name was needed. The ‘external name’ was again reused:

TSomeJavaScriptClass = class external name ‘SomeJSClass' 
  MetaField : NativeInt; external name ‘$metaField';
end;

This tells the compiler that there is a ‘MetaField’ field in pascal, but that the JavaScript name is $metaField. Secondly, some JavaScript identifiers are valid pascal keywords.

This can be solved using the & escape character:

TSomeJavascriptClass = class external name ‘SomeJSClass'
    &end : NativeInt;
end;

Or using the external name syntax:

TSomeJavascriptClass = class external name ‘SomeJSClass'
    _end : NativeInt; external name ‘end’
end;

The same 2 reasons for having the external syntax apply of course for methods as well.

There is a special use for the ‘external name’ construct, namely to allow using object properties. The following is valid javascript:

MyObj[SomeName]=123;
A JavaScript is also a named array of arbitrary JavaScript values. This is translated in the TJSObject class declaration as follows:

TJSObject = class external name 'Object'
private
  function GetProperties(Name: String): JSValue; external name '[]';
  procedure SetProperties(Name: String; const AValue: JSValue); external name '[]';
public
  property Properties[Name: String]: JSValue read GetProperties write SetProperties; default;
end;

The external name ‘[]’ incantation tells the compiler that this function or procedure is in fact an array property access. As a result the above statement syntax can be kept:
MyObj[SomeName] := 123;


The same construct can be found in the JavaScript Array object declaration of TJSArray, and several other object definitions.

The above highlights the possibilities of the external class syntax. This syntax is actually nothing new or invented specially for pas2js: the Free Pascal Java bytecode compiler uses the same syntax to import Java classes, and uses a similar syntax to import Objective C class definitions on Mac OS, thus giving the pascal programmer access to all the possibilities offered by the platform.

In order to access JavaScript APIs, it is necessary to create external object definitions. The pas2js RTL delivers such definitions: units JS and web consist almost entirely from external definitions. The same is true for library imports such as libjquery, webaudio and webbluetooth.

How to create such units ? Three possible approaches exist:

-Manually:
This is how the JS, Web and Libjquery units were made, based on official documentation, the object declarations were coded manually. Needless to say, this is a cumbersome and error-prone approach.

-If a WebIDL specification is available, the webidl2pas tool can be used
This is discussed below.

-Using the classtopas function
The classtopas function is a function in the class2pas unit. It comes in several forms:
function ClassToPas(Const aName : string; Obj: TJSObject; aAncestor : string = ''; recurse : Boolean = False): string;
function ClassToPas(Const aJSName,aPascalName : string; Obj: TJSObject; aAncestor : string = ''; recurse : Boolean = False): string;
procedure ClassToPas(Const aJSName,aPascalName,aAncestor : string; Obj: TJSObject; aDecl : TStrings; recurse : Boolean = False);

The meaning of these arguments is intuitively clear from their names.
How to use this function ?

Get a reference to an instance of the object you want to create a declaration for, and call the function or procedure.

The demos of the pas2js compiler show how to do this.

program democlasstopas;

uses Web,Classes, JS, class2pas, browserconsole;

procedure ShowRTLProps(aClassName,aJSClassName : String; O : TJSObject);  Var    S : TStrings;    I : Integer;    
begin
    S:=TStringList.Create;
    try
      ClassToPas(aClassName,aJSClassName,'',O,S);
      for I:=0 to S.Count-1 do
         Writeln(S[i]);
    finally
      S.Free;
    end;
end;

var
   o : TJSObject;    
begin
    // get the new JavaScript object:
asm
      $mod.o = new Ext.form.Panel;
end;
MaxConsoleLines:=5000;
ShowRTLProps('Ext.form.Panel','TExtJSForm',o);  end.


Combine this with the following HTML page:

  

<!DOCTYPE html>  
<html>   
<head>   
<meta charset="utf-8"/>   
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/extjs/6.0.0/ext-all.js"></script>   
<script type="application/javascript" src="democlasstopas.js"></script>    
</head>    
<body>   
<script type="application/javascript">    rtl.run();   
</script>   
</body>  
</html>

Will result in the following output when the page is opened in the browser:


The declaration is far from perfect: it still needs a lot of manual tweaking. The argument types of functions are unknown (if they are declared at all) the return types too. Objects do not have an actual type other than the TJSObject.

But it is a start, and it should be easy to create a program that allows you to enter a class name in an edit box, and which outputs a class declaration in a memo.

It should be clear that the ClassToPas function is limited. However, if you are lucky, a WebIDL file exists for the API you are trying to access. WebIDL is a standard of sorts to describe Javascript APIS: https://www.w3.org/TR/WebIDL-1/ Version 2 is in preparation on https://heycam.github.io/webidl/

If such an IDL exists you can use the webidl2pas tool. This is a commandline tool (it could easily be converted to a GUI tool) which takes as input a WebIDL file, and outputs a pascal file with external class definitions.

Basically, this means that the command webidl2pas -i webaudio.idl -o webaudio.pas

Will create a unit webaudio.pas from the input file webaudio.idl. The output can be tweaked somewhat, the command webidl2pas -h or webidl2pas --help will give an overview of all available options.

Using these tools, there is no excuse for not exploiting the many Javascript libraries out there in your next TMS Web Core project!

Thanks Michael Van Canneyt for this excellent article with many very interesting insights!

Bruno Fierens




This blog post has not received any comments yet.



Add a new comment

You will receive a confirmation mail with a link to validate your comment, please use a valid email address.
All fields are required.



All Blog Posts  |  Next Post  |  Previous Post