A key benefit of working on JavaScript is the immense collection of JavaScript libraries, many of them free, that we can use in our TMS WEB Core projects, often just by adding a link via a CDN. This is the main purpose of this blog series - exploring how we might use such libraries to extend the reach of our projects (and our coding skills) far beyond, and far more quickly, than what we might be able to accomplish otherwise. 

Another related benefit, however, is that these projects typically run inside a modern browser environment. The browser itself might be hidden, in the case of projects developed with the Miletus or Electron frameworks, or when an application is run full-screen or similarly in some form of kiosk mode, but the environment is there all the same. Often, we have an adversarial view of the browser environment (seemingly arbitrary and often inconvenient sandboxing rules, inexplicable variations in terms of browser rendering, unexpected performance bottlenecks, etc.),

However, there is an impressive array of underlying APIs that we can take advantage of. As modern browsers have evolved, many new APIs have been added, such as the Web Audio API we covered here. Other APIs have not evolved much at all, but they may provide core functionality that we can use in our projects. Today we're going to have a look at one of those - the History API.

Motivation.

The back button. When browsing around normally, we're likely to be accustomed to using the back button in our browser of choice to, well, go back to where we were, relative to where we are. Historically, this worked pretty well, with each "page" being its own thing. Moving backward (and occasionally forward) through our browsing history was (and often still is) not an uncommon thing to do.

The situation changed with the advent of "Single Page Apps" or SPAs, such as those that we might create with TMS WEB Core. In these apps (which might now be the norm rather than the exception these days), the user might move around quite a lot, visiting sections of the app, viewing content in pop-up windows, scrolling tables, and so on. All while the main "page" that they're visiting (the overarching URL for the website) remains the same. From the perspective of the browser, this is just one page. When the user hits the "back" button, then, they go back to the previous website (if available), exiting the app entirely. This can be an inconvenience for some users or for some applications, at best, or a jarring and fundamentally problematic experience for others.  

What is the problem exactly? Well, moving away from the app and then returning may present several challenges, depending on how the application has been designed.

• If it is a secure app, the user may be prompted to log in again. And with other mechanisms, like multi-factor authentication, logging in again can sometimes be a headache.
• In some applications, the user may be able to navigate down several layers into an application. This "dive" may be lost when returning to the app, impacting precious user productivity.
• If the user (like me) has a mouse with extra buttons for things like "back" and "forward", it may be very intuitive to use these buttons to close a popup window or otherwise navigate around an app. This is certainly commonplace in many modern web applications. Having a web app that doesn't work this way is perhaps the new non-normal.
• If the user (also like me) has their browser configured to close the tab when the back button is clicked and when no history is available, then the user also has to find their way back to the app first, and then they have to potentially deal with the login and navigation aspects of the app to get back to where they were. Not ideal.

There may be other problems as well, but these are sufficient to justify doing something about it. The History API provides the tools we need to address the biggest issues.

History Queue.

As the user goes about their browsing, a queue is built up inside the browser, where each successive new page visited is added to the queue. This might happen whether the page was reached via a link from a previous page (like clicking the link on a search results page), by entering a new URL in the browser's address bar, performing a search from the browser's search bar, or by clicking a bookmark of some kind.

This queue is maintained separately for each browser tab. Clicking and holding the back or forward buttons in the browser will show a list of other pages in the queue, in either direction. If the user is currently viewing a new page, the forward button will be disabled. If they've moved back to the first page, the back button will be disabled. If a new tab is opened, this queue will be empty and both the back and forward buttons will be disabled. Nothing new here, this all works as one would expect, and this has been the case since browsers were first invented.

Using the History API, we can replicate to some degree the functionality of the back and forward buttons in our own button components. We might do this in a Miletus or Electron app where these buttons aren't accessible, or if our app is run in a full-screen mode, where we might like this functionality to still be available.

procedure TForm1.WebButtonBackClick(Sender: TObject);
begin
  window.history.back;
end;

procedure TForm1.WebButtonForwardClick(Sender: TObject);
begin
  window.history.forward;
end;

Manipulating History.

What we can do, though, is a little bit of queue manipulation. We can determine the size of the queue using window.history.length. If this returns a value of 1, then we know that we're at the beginning of the queue and that there is nothing to move back or forward to, so we can disable both of our buttons. And, in all likelihood, both of the browser buttons will already be similarly disabled. If this returns a value greater than 1, then we know that there may be pages in the queue, either before our page or after, and that either button may be enabled or disabled. The History API does not tell us where our current page (our app) is in the queue, however, which is disappointing.  

One way we could address this is to check the window.history.length value, and if it is not 1, then re-launch our application in a new tab or window. We usually can't close our current tab as we didn't open it ourselves (remember those browser sandboxing rules?), so a message would have to be displayed by our app indicating that it should be closed. The new window will only appear if popups for our page have not been disabled by the user. Not really ideal, but depending on the application, this might be an option.

Another option is to pretend that we're at the end of the queue, even if we're not, and keep track of the position from that point. While we can't really remove entries from the queue, we can add new entries. And adding a new entry generally gets rid of any future entries. Just like when visiting websites, if you go back a few steps and then forward in a different direction, the previously queued "forward" entries are cleared and replaced by pages in the new direction taken. We can also replace our current entry. But why would we want to do that? Well, in addition to URLs, the History API also allows us to store state information.

The State of History.

What kind of state? Well, whatever we want, actually. There may be a limit to the size of the state data stored in some browsers, so best to store just what is needed. But even then, the limits are quite generous - we're not likely to hit any browser limits just by storing navigation information. The History API gives us three main tools to work with.

1. Replace the state value and URL associated with our current page in the queue.
   
2. Push a new state value and a new URL to the queue.
3. Pop the last state value and URL from the queue.

The state value is something we define and use ourselves, so we can do whatever we like here. As we're ultimately using JavaScript, and as we should be really fond of JSON by now (or at least keeping an open mind on becoming fond of JSON!), we'll use JSON to store information that allows us to configure elements in our app. In our example app, we'll just be using buttons, but the state we're storing could contain anything that might help us show or hide elements, forms, menus, or entire sections of our project.

The URL is a little special here in that what we're actually using, with the History API, is the mechanism that browsers use to handle in-page links - the hash feature. If we pass new values for the URL, appending values after the URL, prefixed with "#", the History API will update the URL that we see in the browser by appending this value to the current URL. 

In a traditional web page, we would use this inside an anchor element's href attribute (a link, essentially) in the page to refer to the id of an HTML element elsewhere in the same page that we wanted to navigate to, and the browser would scroll so that the element was at the top of the browser view. This is how the "Table of Contents" links are created in these blog posts, for example. 

In our current situation, the hash value in the URL will initially serve as a visual aid, indicating what the state of our page is, but this can be put to another good use if we want to implement a routing function. More on that in a little bit.

History Example App.

To help illustrate and test what we're covering here, let's create a new TMS WEB Core app using the usual Bootstrap Application template. We'll need some buttons that we can change. These will serve as a proxy for different parts of our application - different forms, for example. We'll add a couple of TWebMemos that can be scrolled, so we can see how to deal with that. We'll need back and forward buttons to replicate what is in the browser. Maybe a label to show what our position is in the History API queue - or rather our interpretation of our position. And a couple of extra buttons to show how we might create a new application window, or close the existing one. Here's what it looks like.

History Example Application.

The idea is that if we click on one of the four main buttons (A, B, C, or D), they'll change color. Then, using the Back and Forward buttons, we'll see the buttons change state to coincide with whatever they were previously. Here's our button click handler.

procedure TForm1.WebButton1Click(Sender: TObject);
begin
  (sender as TWebButton).ElementHandle.classList.toggle('btn-primary');
  (sender as TWebButton).ElementHandle.classList.toggle('btn-secondary');
  Position := Position + 1;
  window.history.pushState(CaptureState,'',URL);
  UpdateNav;
end;

All we're doing here is swapping the Bootstrap class from "btn-primary" to "btn-secondary". As they are all set to "btn-primary" to begin with, the first toggle removes "btn-primary" and the second adds "btn-secondary". The next time the button is clicked, the reverse happens. Each time we click a button, we'll add a new History API queue entry, so we increment our "Position" in the queue (a Form variable), push our new state value and URL (another Form variable), and update our navigation buttons. The navigation update looks like this.

procedure TForm1.UpdateNav;
begin
  webLabel1.Caption := IntToStr(Position)+' / '+IntToStr(window.history.length);

  if Position <= 2
  then WebButtonBack.Enabled := False
  else WebButtonBack.Enabled := True;

  if Position = window.history.length
  then WebButtonForward.Enabled := False
  else WebButtonForward.Enabled := True;
end;

The state we're storing contains the state of the four buttons (their class properties), the scroll position of the two TWebMemos, the current "Position" value, and the current "URL" value.

function TForm1.CaptureState: JSValue;
begin
  if WebButton1.ElementHandle.className.contains('btn-primary')
  then URL := '#Yeah'
  else URL := '#Nope';

  if WebButton2.ElementHandle.className.contains('btn-primary')
  then URL := URL+'/Yeah'
  else URL := URL+'/Nope';

  if WebButton3.ElementHandle.className.contains('btn-primary')
  then URL := URL+'/Yeah'
  else URL := URL+'/Nope';

  if WebButton4.ElementHandle.className.contains('btn-primary')
  then URL := URL+'/Yeah'
  else URL := URL+'/Nope';

  Scroll1 := WebMemo1.ElementHandle.scrollTop;
  Scroll2 := WebMemo2.ElementHandle.scrollTop;

  asm
    Result = {
      "Position":this.Position,
      "url":this.URL,
      "buttons":[
        WebButton1.className,
        WebButton2.className,
        WebButton3.className,
        WebButton4.className,
      ],
      "Scroll1":this.Scroll1,
      "Scroll2":this.Scroll2
    };
  end;
end;

Here, our URL value is just a reference to the four buttons, as either "Yeah" or "Nope" depending on whether its class is "btn-primary" or "btn-secondary". These are combined into a string starting with "#" and using "/" as a delimiter. We can think of this as the overall state of our page, and we'll see shortly how we can use this to actually set the state of our page as well. 

Note also that our components have been assigned ElementID properties that correspond to their Delphi component names, which allows us to use them interchangeably in both Delphi and JavaScript. JavaScript automatically creates variables corresponding to all of the HTML element id values, a handy feature for our purposes, but one that is not at all popular in certain JavaScript developer circles. We aren't going to spend any time worrying about those folks!

We've seen above how we can add an entry to the history queue, using the pushState() method. This takes as parameters the state, the page title, and the URL. The page title is apparently not used at all by any modern browser, so leaving it empty has no effect. But how do we know when the back or forward button has been clicked?  Well, one way is to add a JavaScript event listener to the page.

procedure TForm1.WebFormCreate(Sender: TObject);
begin

  // What to do when we hit back/forward button
  asm
    window.addEventListener('popstate', function(popstateEvent)  {
      pas.Unit1.Form1.RevertState(popstateEvent.state);
    });
  end;

end;

When it fires, it will call a Delphi method, RevertState(), and pass it the state value that was stored with the page at that spot in the queue. This method will then "configure" the app to correspond to the state that we were in previously. In our example, this involves setting the state of the four buttons (adjusting their class properties), as well as the scroll position of the two TWebMemo components.

procedure TForm1.RevertState(StateData: JSValue);
begin
  asm
    if (StateData !== null) {
      this.Position = StateData.Position;
      this.URL = StateData.URL;
      WebButton1.className = StateData.buttons[0];
      WebButton2.className = StateData.buttons[1];
      WebButton3.className = StateData.buttons[2];
      WebButton4.className = StateData.buttons[3];
      this.Scroll1 = StateData.Scroll1;
      this.Scroll2 = StateData.Scroll2;
    }
  end;
  WebMemo1.ElementHandle.scrollTop := Scroll1;
  WebMemo2.ElementHandle.scrollTop := Scroll2;

  // Disable Back button
  if Position = 1 then
  begin
    Position := Position + 1;
    window.history.pushState(CaptureState,'',URL);
  end;

  UpdateNav;
end;

Once we've recreated our state, we then check and see what "Position" we're at, based on the value that was in the state we've just been handed. If we're already back at Position 1, then we know we've gone too far. So we immediately push our current state back onto the history queue, effectively making it impossible to go back to Position 1 - and thus disabling going back so far as to exit our application or to close our browser page unexpectedly.

procedure TForm1.WebFormCreate(Sender: TObject);
begin

  // Set our current state as the state we want to go back to
  Position := 1;
  window.history.replaceState(CaptureState,'',URL);
  Position := 2;
  window.history.pushState(CaptureState,'',URL);

...

end;

And, to reiterate, the "state" of our application in this example is just the classes assigned to our four buttons and the scroll positions of the two TWebMemo components. In an actual project, this may very well be considerably more complex. For example, instead of a button, there may be a certain form loaded into an element on the page. Or maybe a chart is displayed with a specific set of parameters. Regardless, the idea is to store whatever information we need to configure our application in the same way, should the user return to this place in the history queue.  How finely tuned these "steps" in the queue are will be something that you, as the developer, have to determine.  We'll cover a more complex example later in this post.

Routing.

Routing in this context refers to using the URL of a page to indicate where to go within the app. In some respects, it is similar to passing values using URL query parameters. Even better though, we can change the URL without reloading our application, passing in multiple new values at the same time, just by adjusting what comes after the "#" in the URL. 

In other web applications, this might manifest itself as a series of extra path elements in the URL. For example, www.example.com/sales/customer/fred - where we'd be expecting to land in the sales module, in the customer section, looking at Fred's customer profile. In our case, we'll start this path with a #, but anything after is fair game for helping us route the user to a particular part of our application.

In our example application, we've constructed the URL to contain a list of button states. We can then read this URL and set those states, just as we would if we were hitting the back or forward button to go back to a state we had visited previously. 

To help out with this, TMS WEB Core has made available a WebFormHashChange event that is available in our Form. Using this, we can then parse the URL (the part after the "#") and then set our application state accordingly. Note carefully, though, that we don't necessarily generate this URL - this might have been saved as a bookmark or manually typed in. As a result, we have to do a little more error checking, handling scenarios where the information is incomplete or incorrect, for example.

procedure TForm1.WebFormHashChange(Sender: TObject; oldURL, newURL: string);
begin
  asm
    var route = newURL.split('#')[1].split('/');

    console.log('Routes: '+route.length);
    for (var i = 0; i < route.length; i++) {
      console.log('Route '+i+': '+route[i]);
    }

    if (route.length > 3) {
      WebButton4.classList.remove('btn-primary');
      WebButton4.classList.remove('btn-secondary');
      if (route[3].toUpperCase().indexOf('NOPE') > -1) {
        WebButton4.classList.add('btn-secondary');
      } else {
        WebButton4.classList.add('btn-primary');
      }
    }

    if (route.length > 2) {
      WebButton3.classList.remove('btn-primary');
      WebButton3.classList.remove('btn-secondary');
      if (route[2].toUpperCase().indexOf('NOPE') > -1) {
        WebButton3.classList.add('btn-secondary');
      } else {
        WebButton3.classList.add('btn-primary');
      }
    }

    if (route.length > 1) {
      WebButton2.classList.remove('btn-primary');
      WebButton2.classList.remove('btn-secondary');
      if (route[1].toUpperCase().indexOf('NOPE') > -1) {
        WebButton2.classList.add('btn-secondary');
      } else {
        WebButton2.classList.add('btn-primary');
      }
    }

    if (route.length > 0) {
      WebButton1.classList.remove('btn-primary');
      WebButton1.classList.remove('btn-secondary');
      if (route[0].toUpperCase().indexOf('NOPE') > -1) {
        WebButton1.classList.add('btn-secondary');
      } else {
        WebButton1.classList.add('btn-primary');
      }
    }

  end;
end;

procedure TForm1.WebFormCreate(Sender: TObject);
begin
...
  // Set inital state
  if (pos('#',window.location.href) > 0)
  then WebFormHashChange(Sender,'', window.location.href);
...
end;

program Project1;
 uses
  Vcl.Forms,
  System.SysUtils,
  WEBLib.Forms,
  WEBLib.WebTools,
  Unit1 in 'Unit1.pas' {Form1: TWebForm} {*.html},
  Unit2 in 'Unit2.pas' {Form2: TWebForm} {*.html};

{$R *.res}

begin
  Application.Initialize;
  Application.AutoFormRoute := True;
  Application.MainFormOnTaskbar := True;

  if (Uppercase(GetQueryParam('form')) = 'FORM2')
  then Application.CreateForm(TForm2, Form2)
  else Application.CreateForm(TForm1, Form1);

  Application.Run;
end.

To load up Form2, we can then use a URL like this.

localhost:8000/Project1/Project1.html?form=Form2

Note that if you wanted to use a URL without a hash value or a query parameter, this would most likely be handled with considerable help from the actual web server (like Apache), where it would use URL rewriting rules to take such a URL and insert the appropriate query parameter syntax or hash formatting before passing it to our application.  Without doing that, the web server would interpret such a URL (without a ? or # symbol and only / symbols) as referencing a different file that it needs to serve before our application would even get a chance to see it.

With the routing sorted out, our History Example application is now complete.

History Example Application.

Practical Example.

That worked out pretty well, but let's try to do something with a more complex and practical project - the TMS WEB Core Template project that recently made its debut in recent a blog post. Check out the first post, about TMS XData, here. And the second, about the AdminLTE template and TMS WEB Core, here. There is even a bonus post which is perhaps the most relevant to this discussion, here, as it covers how different forms and subforms are managed.   There is quite a lot of material there to cover, so let's recap a few of the relevant highlights for our purposes here.

• The client is a TMS WEB Core project that uses the AdminLTE template, providing CSS and JavaScript to help with interacting with a variety of elements including menus, info-boxes, and other containers.
• MainForm is used as the main application conductor, containing common functions and form variables that help control how the application works overall.
  
• MainForm has a TWebHTMLDiv component, divHost, which is used to hold one top-level form. Might be easiest to think of these forms as dashboards.
• Dashboards are Delphi Forms designed to contain menus, headers, and footers for the page, along with related functionality common to that particular subset of the application.  For example, we might have separate dashboards for "administration", "human resources", "sales", and so on, with menus and other links specific to each.
• The main part of a dashboard contains another TWebHTMLDiv, divSubForm, which is used to hold a second-level form - the individual "page" or "subform".
  
• Pages are also Delphi Forms, but contain specific UI elements and code related to a particular function, usually but not always related to the parent dashboard.
• There are already a couple of these pages in place, like "user profile" and "user actions" that are not specific to a dashboard, but can be loaded as if they were.
• Typically each dashboard would have a default page of its own (each Form will have a default SubForm).
  

Only one top-level dashboard is visible at a time, and it has the ability to host only one second-level page at a time.

MainForm
--> divHost (eg: LoginForm)
--> divHost (eg: AdministratorForm)
----> divSubForm (eg: AdministratorSubForm)
----> divSubForm (eg: UserProfileSubForm)
----> divSubForm (eg: UserActionsSubForm)

To start with, we can add a similar block of code, and Form variables (Position, StartPosition, and URL), to MainForm to initialize the history queue, adding to what is already in WebFormCreate. Here we're adding a couple of history items so that the back button will not leave this page. And we start Position off at whatever window.history.length is, and then use StartPosition to keep track of where we're at.

   // Set our current state as the state we want to go back to
  Position := window.history.length;
  StartPosition := window.history.length;
  URL := window.location.href;
  window.history.pushState(CaptureState, '', URL);
  Position := window.history.length;
  window.history.pushState(CaptureState, '', URL);

  // What to do when we hit back/forward button
  asm
    window.addEventListener('popstate', function(popstateEvent)  {
      pas.UnitMain.MainForm.RevertState(popstateEvent.state);
    });
  end;

We'll then need to populate four methods - CaptureState, RevertState, WebFormHashChange, and UpdateNav - all methods in MainForm.  

Let's start with the easiest one - UpdateNav. Here, we just want to enable or disable whatever back and forward buttons we may have, if any, anywhere on our page. We can specify which buttons might be used for this by adding a class, either nav-history-back or nav-history-forward. The buttons, when clicked (if not disabled) can call the same methods we used before to trigger the browser back or forward action, which we'll also add to MainForm.

procedure TMainForm.UpdateNav;
begin
  asm
    var backbtns = document.getElementsByClassName('nav-history-back');
    if (this.Position <= (this.StartPosition+1)) {
      for (var i = 0; i < backbtns.length; i++) {
        backbtns[i].setAttribute('disabled','');
      }
    } else {
      for (var i = 0; i < backbtns.length; i++) {
        backbtns[i].removeAttribute('disabled');
      }
    }

    var forwardbtns = document.getElementsByClassName('nav-history-forward');
    if (this.Position == window.history.length) {
      for (var i = 0; i < forwardbtns.length; i++) {
        forwardbtns[i].setAttribute('disabled','');
      }
    } else {
      for (var i = 0; i < forwardbtns.length; i++) {
        forwardbtns[i].removeAttribute('disabled');
      }
    }
  end;
end;

function TMainForm.CaptureState: JSValue;
begin
   // Return state of some kind
   asm
     Result = {
       "Position": this.Position,
       "URL": this.URL,
       "Form": this.CurrentFormName,
       "SubForm": this.CurrentSubFormName
     }
   end;
end;

procedure TMainForm.RevertState(StateData: JSValue);
var
 PriorForm: String;
 PriorSubForm: String;
begin
  asm
    if (StateData !== null) {
      this.Position = StateData.Position;
      this.URL = StateData.URL;
      PriorForm = StateData.Form;
      PriorSubForm = StateData.SubForm;
    }
  end;

  // Disable Back button
  if Position <= StartPosition then
  begin
    Position := Position + 1;
    window.history.pushState(CaptureState, '', URL);
    UpdateNav;
  end
  else
  begin

    if (PriorForm <> CurrentFormName) and (PriorForm <> '')
    then LoadForm(PriorForm);

    if (PriorSubForm <> CurrentSubFormName) and (PriorSubForm <> '')
    then LoadSubForm(PriorSubForm, False);
  end;
end;

 procedure TMainForm.WebFormHashChange(Sender: TObject; oldURL, newURL: string);
var
  NewForm: String;
  NewSubForm: String;
begin
  asm
    if (newURL.split('#').length == 2) {
      if (newURL.split('#')[1].split('/').length == 2) {
        NewForm = newURL.split('#')[1].split('/')[0];
        NewSubForm = newURL.split('#')[1].split('/')[1];
      }
    }
  end;

  if (NewForm <> CurrentFormName) and (NewForm <> '')
  then LoadForm(NewForm);

  if (NewSubForm <> CurrentSubFormName) and (NewSubForm <> '')
  then LoadSubForm(NewSubForm, True);

end;

    // Save to History
    if NewInstance then
    begin
      MainForm.Position := MainForm.Position + 1;
      window.history.pushState(MainForm.CaptureState, '', MainForm.URL);
    end;

Then, after the new SubForm is created, we update the current URL to reflect the page we're currently looking at.

  procedure AfterSubCreate(AForm: TObject);
  begin
    LogAction('Load SubForm: '+SubForm+' Loaded ('+IntToStr(MillisecondsBetween(Now, ElapsedTime)-500)+'ms)', False);
    URL := '#'+CurrentFormName+'/'+CurrentSubFormName;
    window.history.replaceState(MainForm.CaptureState, '', URL);
    UpdateNav;
  end;

Mission accomplished!

History Example Project Download
Template Demo Repository on GitHub

Follow Andrew on 𝕏 at @WebCoreAndMore or join our 𝕏 Web Core and More Community.