In this post, we're going to have a quick look at the Web Audio API, the current standard for everything audio-related in modern web browsers. We'll create a music player-type app that can be used to playback standard audio files like MP3 or WAV. And while the Web Audio API itself doesn't require a JavaScript library (it is included in modern browsers already), a few additional JavaScript libraries will be playing a supporting role. And just for fun, we'll use our newly minted D3 Charting talents to create a basic visualizer as well.

Motivation

Credit is due to TMS WEB Core customer David Schwartz for suggesting this as a topic, as well as his book recommendation - Working with the Web Audio API by Josh Reiss, a pretty solid reference work that covers more than enough of the Audio Web API for what is needed here, written by an expert in the field.

Getting Started.

To create a player app, naturally, we'll need some kind of user interface. Being a creature of habit, my applications all tend to look the same. But for those who are not a fan of my particular style choices, the general idea here is that the example projects are available for you to endlessly customize as you like. Keeping that in mind, we'll start with the familiar boilerplate added to the Project.html file that has become commonplace of late. Bootstrap, Font Awesome, D3, and Tabulator, to start with. Each of these could of course be replaced with something else. A different font. A different icon library. A different table library. Nothing but options.

    <!-- Bootstrap 5.2.1-->
    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.2.1/dist/js/bootstrap.bundle.min.js" integrity="sha384-u1OknCvxWvY5kfmNBILK2hRnQC3Pr17a+RTT6rIHI7NnikvbZlHgTPOOmMi466C8" crossorigin="anonymous"></script>
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.2.1/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-iYQeCzEYFbKjA/T2uDLTpkwGzCiq6soy8tYaI1GyVh/UjpbCx/TYkiZhlZB6+fzT" crossorigin="anonymous">

    <!-- FontAwesome v6 Free -->
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@fortawesome/fontawesome-free@6/css/all.min.css">

    <!-- Google Fonts: Cairo -->
    <link rel="preconnect" href="https://fonts.googleapis.com">
    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
    <link href="https://fonts.googleapis.com/css2?family=Cairo&display=swap" rel="stylesheet">

    <!-- Tabulator -->
    <script src="https://cdn.jsdelivr.net/npm/tabulator-tables@5/dist/js/tabulator.min.js"></script>
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/tabulator-tables@5/dist/css/tabulator.min.css">

    <!-- D3 -->
    <script src="https://cdn.jsdelivr.net/npm/d3@7/dist/d3.min.js"></script>

    <!-- Our own custom stylesheet -->
    <link rel="stylesheet" href="player.css">

And as it would be fun to use this player on mobile devices while offline, we'll use the PWA template to create our example rather than the Bootstrap template, though we'll still be using Bootstrap throughout. The custom stylesheet is more about having a place to put all the Tabulator overrides than anything else, but because it's there, we'll be adding a few other little touches as well.

For the main UI, naturally, we'll want player controls, a place for the visualizer, a place for cover art, and a list of songs that we're playing. Cover art is typically square. We'll want the player to fill the vertical space available, so the list will grow or shrink based on that space. There are a lot of different ways to make an app responsive. Here, we're going to put everything inside a set of <div> elements and then have those <div> elements explicitly sized to fit whatever space we have as best we can. the WebFormShow and WebFormResize methods will then both point to this code to set the size and position of the <div> elements.

procedure TForm1.WebFormShow(Sender: TObject);
var
  PageWidth: Integer;
  PageHeight: Integer;
  MarginSpacing: Integer;
begin
  // This is the constraint we're working within
  PageWidth := Form1.Width;
  PageHeight := Form1.Height;

  // Set Width of main UI
  if (PageWidth > 500) then
  begin
    divMain.Width := 500;
    divMain.Left := Trunc((PageWidth - 500) / 2);
  end
  else
  begin
    divMain.Width := PageWidth - 10;
    divMain.Left := 5;
  end;

  // Set Height of main UI
  divMain.Top := 5;
  divMain.Height := PageHeight - 10;

  // Set height of top section
  divTop.Height := divMain.width - 144;

  // Add spacing to icons at top-left to deal with variable height
  MarginSpacing := max(Trunc((divTop.Height - 205) / 4),0);
  btnLoadPlaylist.Margins.Top := MarginSpacing;
  btnSavePlaylist.Margins.Top := MarginSpacing;
  btnSettings.Margins.Top := MarginSpacing;
  btnShare.Margins.Top := MarginSpacing;

end;

For most standard vertical displays, this works out pretty well. For horizontal or square displays, well, not so much.  The <div> elements would have to be rearranged left-right instead of top-bottom. Similarly, rearranging could also be helpful when it comes to supporting left-handed and right-handed users. For example, swapping the icons at the bottom, so the play button is on the left. Or rearrange the icons so that the play button is in the middle. No end to the options here. But here's what we're going with for this example.

For the Volume and Scrubber sliders, the TTMSFNCTrackBar component is used, which is part of the FNC UI Pack.  This is a bit of a departure from using just stock TMS Web Core components. But the TWebTrackBar component didn't allow for enough customization and attempts to use CSS to override those proved to be more of a headache than anything else. One of those instances where the different browsers are really not on the same page. 

The FNC component, on the other hand, allows for customizing pretty much everything. So while I'm generally not a fan of <canvas> elements, in this case, it works pretty well. I even get all the rounded corners that I'm so very fond of! But this does introduce a bit of a restriction in that you must have the TMS FNC UI Pack (or TMS FNC Component Pack) installed in your development environment in order to successfully build the example project. To the best of my knowledge, this is also available for Visual Studio Code and Lazarus.

The music icon at the top left was set up with a bit of color cycling and a drop-shadow behind it, just for a little extra flair. Here's our CSS so far. Color #212529 is equivalent to Bootstrap's "dark" color.

html, body {
  background-color: #000;
}

#labelElapsed,
#labelRemaining {
  font-size: 14px;
  font-family: "Cairo";
  color: #fff;
}

#buttonPlayer {
  filter: drop-shadow( 0px 0px 1px #212529)
          drop-shadow( 0px 0px 1px #212529)
          drop-shadow( 0px 0px 10px white )
          drop-shadow( 0px 0px 10px white );
  animation: color-change 30s infinite;
}
@keyframes color-change {
  0%   { color: #f00; }
  5%   { color: #f00; }
  10%  { color: #f00; }
  15%  { color: #f40; }
  20%  { color: #f80; }
  25%  { color: #fb0; }
  30%  { color: #ff0; }
  35%  { color: #ff4; }
  40%  { color: #ff8; }
  45%  { color: #ffb; }
  50%  { color: #fff; }
  55%  { color: #fff; }
  60%  { color: #fff; }
  65%  { color: #fbf; }
  70%  { color: #f8f; }
  75%  { color: #f4f; }
  80%  { color: #f0f; }
  85%  { color: #f0b; }
  90%  { color: #f08; }
  95%  { color: #f04; }
 100%  { color: #f00; }
}

We'll go about filling in the rest of the interface as we go, but the idea here is that it is pretty easy to set up a mockup like this and have it be the actual template for the app, properly resizable for different device dimensions, along with whatever styling is needed, before writing any other code for the app.

Web Audio API Helper Unit.

There are certain things in there that are not Web Audio API-related, but we'll overlook that for the sake of simplicity and just refer to it as the Web Audio API Helper unit, regardless of what else gets tossed in there. As it has no visual components, we'll use a TWebDataModule for this. With a little luck, the main Unit1.pas will not have any "asm... ...end" blocks in it. This should also make it easier to create an entirely new UI using the same underlying functionality.

The Playlist.

To start with, we'll put the code to initialize the Tabulator-based playlist in there, as well as the non-Delphi parts of the functions needed to manage this list. The idea with the playlist is to hold a list of MP3 or WAV files or anything else that web browsers might support in terms of audio formats. There are undoubtedly many, many more. If extra metadata is available (cover art, song title, artist title, and so on) then that's what we'd prefer to show in the playlist. If we don't have that, then the filename will have to suffice.

There are four buttons to wire up here.  The Load/Save playlist buttons at the top, which load and save entire playlists. And the Add/Delete buttons at the bottom, for adding or removing individual playlist items.

We'll need to add our new unit, WebAudioAPIHelper, to the uses clause in our main Unit1.pas. And then we can initialize the playlist by calling a function within it, passing it the ElementID of the component that we're using to host the Tabulator table. A pretty basic approach, nothing too fancy. In our Unit1.pas we then have this function.

implementation

uses WebAudioAPIHelper;

{$R *.dfm}

procedure TForm1.WebFormCreate(Sender: TObject);
begin
  WAAH.InitializePlaylist(divTrackListHolder.ElementID);
end;

And then in the WebAudioAPIHelper.pas unit, we have the Tabulator function that creates the table.  This will be built up considerably from here, but just to get us started, it looks like this.

procedure TWAAH.InitializePlaylist(ElementID: String);
begin
  // This creates the Tabulator instance for the playlist
  asm
    this.tabPlaylist = new Tabulator("#"+ElementID, {
      index: "ID",
      movableRows: true,
      selectable: 1,
      columns: [
        { title: "ID", field: "ID", width: 50 },
        { title: "Cover Art", field: "COVER" },
        { title: "Track Name", field: "TRACK" },
        { title: "Artist Name", field: "ARTIST" }
      ]
    });
  end;
end;

And this produces our initial playlist, which looks like this. Lots of room for improvement, but we'll get to that in a moment.

Tabulator-driven Playlist.

To get files into our player, we can use the TWebOpenDialog component. For demo purposes, there are a handful of tracks from https://www.free-stock-music.com included in the project. Just some MP3 files with the requisite cover art and other metadata to test with. The intent here is not to promote the website, the artists, or anything of that nature, just some data to play with. If you happen to like these tracks, though, check out their website!

Flow by Helkimer | https://soundcloud.com/helkimer
Music promoted by https://www.free-stock-music.com
Creative Commons Attribution 3.0 Unported License
https://creativecommons.org/licenses/by/3.0/deed.en_US

Overnight by BatchBug | https://soundcloud.com/batchbug
Music promoted by https://www.free-stock-music.com
Creative Commons Attribution 3.0 Unported License
https://creativecommons.org/licenses/by/3.0/deed.en_US
 
Energy by Ametryo | https://soundcloud.com/ametryo
Music promoted by https://www.free-stock-music.com
Attribution-NoDerivs 3.0 Unported (CC BY-ND 3.0)
https://creativecommons.org/licenses/by-nd/3.0/

For TWebOpenDialog, we'll set the "Accept" property to include .MP3 and .WAV files. TWebOpenDialog component has callback functions that return data in various formats, which currently include Text, Base64, Stream, and Array Buffer. This is binary data, so we'll want to use one of the non-text variations to get this content into our app.  Not really any reason to convert the data to Base64 (yet!), and as Stream is more of a Delphi thing, let's go with the Array Buffer. We'll also want to be able to load more than one file at a time. Here's what we've got hiding behind the "Plus" button. Note that this is an asynchronous operation.

procedure TForm1.btnLoadTrackClick(Sender: TObject);
var
  i: Integer;
begin
  // Open file dialog
  await(string, WebOpenDialogTracks.Perform);

  // If files were selected, iterate through them
  i := 0;
  while (i < WebOpenDialogTracks.Files.Count) do
  begin
    WebOpenDialogTracks.Files.Items[i].GetFileAsArrayBuffer;
    i := i + 1;
  end;
end;

procedure TForm1.WebOpenDialogTracksGetFileAsArrayBuffer(Sender: TObject; AFileIndex: Integer; ABuffer: TJSArrayBufferRecord);
begin
  WAAH.LoadTrack( WebOpenDialogTracks.Files.Items[AfileIndex].Name, ABuffer);
end;

On the JavaScript end, we'll need to add a new record to the playlist and include any information that we can get from the file that is being passed. For WAV files, this isn't much. For MP3 files, there may very well be a lot of data, or no data at all. Extracting that data is not something that is included in the Web Audio API, however. So we've got another JavaScript library to help us out with that - jsmediatags. As usual, we'll need to add this to our Project.html file. This JavaScript library can read and extract metadata from a variety of file types, not just .MP3 files. But we're primarily interested in .MP3 files and in particular the cover image.

<script src="https://cdn.jsdelivr.net/npm/jsmediatags@3.9.7/dist/jsmediatags.min.js"></script>

Adding to the trouble, using a "try" clause in JavaScript is one step too far for the Delphi IDE to try and make sense of, so the asm block in this instance is wrapped in {$IFNDEF WIN32}...{$ENDIF} so that the IDE tries harder to ignore what is inside. It will still work OK without this extra wrapper, but certain IDE functions (particularly class completion) start to get tripped up.

procedure TWAAH.LoadTrack(TrackName: String; TrackData: TJSArrayBufferRecord);
var
  TrackID: Integer;
begin
  // We've got a track to load.  So let's load it.
  {$IFNDEF WIN32}
  asm
    // An ID field is used as a unqiue index to the playlist
    var TrackID = this.tabPlaylist.getCalcResults().top.ID + 1;
    if (isNaN(TrackID)) {
      TrackID = 1;
    }

    // These are the fields we're looking to populate
    var Track = TrackName;
    var Artist = '';
    var Album = '';
    var Cover = 'images/missingartwork.png';

    // Convert TJSArrayBufferRecord to Blob
    var TrackBlobData = new Blob([TrackData.jsarraybuffer]);

    // Extract meatadata, if available
    var tag = '';
    async function readMetadataAsync (file) {
      return new Promise((resolve, reject) => {
        jsmediatags.read(file, {
                  onSuccess: resolve,
                  onError: reject
        })
          })
    }
    try {
      var tag = await readMetadataAsync(TrackBlobData);

      if (tag.tags.title !== undefined) {
        Track = tag.tags.title;
      }

      if (tag.tags.artist !== undefined) {
        Artist = tag.tags.artist;
      }

      if (tag.tags.album !== undefined) {
        Album = tag.tags.album;
      }
      if (tag.tags.picture !== undefined) {
        var image = tag.tags.picture;
        var base64String = "";
        for (var i = 0; i < image.data.length; i++) {
          base64String += String.fromCharCode(image.data[i]);
        }
        Cover = "data:"+tag.tags.picture.format+";base64,"+window.btoa(base64String);
      }
    }
    catch (e) {
      console.log('Data not available for [ '+TrackName+' ]');
    }

    // This is needed to be able to store TrackData as text, so it can be
    // included in the playlists that are saved/loaded
    // https://stackoverflow.com/questions/9267899/arraybuffer-to-base64-encoded-string
    function arrayBufferToBase64(ab){
      var dView = new Uint8Array(ab);
      var arr = Array.prototype.slice.call(dView);
      var arr1 = arr.map(function(item){
        return String.fromCharCode(item);
      });
      return window.btoa(arr1.join(''));
    }

    // Add new track to playlist
    this.tabPlaylist.addRow({
      ID: TrackID,
      TRACKNAME: TrackName,
      TRACKDATA: arrayBufferToBase64(TrackData.jsarraybuffer),
      TRACK: Track,
      ARTIST: Artist,
      ALBUM: Album,
      COVER: Cover
    })
    // If update is true, then make this
    .then(function(row) {
      if (pas.Unit1.Form1.PlayerState == 'Paused') {
        row.select();
        row.getTable().scrollToRow(row);
      }
    });

  end;
  {$ENDIF}
end;

procedure TWAAH.InitializePlaylist(ElementID: String);
begin
  // This creates the Tabulator instance for the playlist
  asm
    this.tabPlaylist = new Tabulator("#"+ElementID, {
      index: "ID",
      layout: "fitColumns",
      movableRows: true,
      movableColumns: false,
      selectable: true,
      rowHeight: 50,
      headerVisible: false,
      columns: [
        { title: "ID", field: "ID", width: 50, topCalc: "max", visible: false },
        { title: "Filename", field: "TRACKNAME", visible: false },
        { title: "Filedata", field: "TRACKDATA", visible: false },
        { title: "Cover Art", field: "COVER", width: 50, cssClass: "NoPadding", resizable: false,
            formatter: "image",
            formatterParams: {width: 50, height:50  }
        },
        { title: "Track Name", field: "TRACK", resizable: false,
            formatter: function(cell, formatterParams, onRendered) {
              var album = cell.getRow().getCell('ALBUM').getValue();
              if ((album !== undefined) && (album !== '')){
                album = ' ['+album+']';
              }
              return '<strong>'+cell.getValue()+'</strong><br />'+cell.getRow().getCell('ARTIST').getValue()+album;
            }
        },
        { title: "Artist Name", field: "ARTIST", visible: false },
        { title: "Album Name", field: "ALBUM", visible: false },
      ]
    });

    this.tabPlaylist.on("rowSelected", function(row) {
      if (pas.Unit1.Form1.PlayerState == 'Paused') {
        divCover.innerHTML = '<image src='+row.getCell("COVER").getValue()+' width=100% height=100%>'
      }
    });
  end;
end;

Implementing the trash function (removing entries from the playlist) is then just a matter of removing the currently selected row(s) from the table.  

procedure TWAAH.RemoveTrack;
begin
  // Remove selected tracks from the playlist
  asm
    var rows = this.tabPlaylist.getSelectedRows();
    for (var i = 0; i < rows.length; i++) {
      this.tabPlaylist.deleteRow(rows[i]);
    }
  end;
end;

Once upon a time, this would have been a fairly traumatic thing. Today, though, this has the benefit of having the playlist work offline, without having to load up any other data - it is all in the playlist. Perhaps this is an area for improvement, or perhaps not. If one were to load up a collection of .WAV files or other lossless data, this might be more of an issue, certainly.

Note that everything being passed around should all be 'local' in the sense that nothing is being sent to or retrieved from a server at this point.  Loading 250 MB from local storage is likely a non-issue. Loading 250 MB from a remote server is likely a show-stopper. In any event, this approach makes loading and saving playlists rather simple. For saving, we'll also be using another JavaScript library to make our lives a little easier - StreamSaver.js.

<script src="https://cdn.jsdelivr.net/npm/streamsaver@2.0.6/StreamSaver.min.js"></script>

procedure TForm1.btnSavePlaylistClick(Sender: TObject);
begin
  WAAH.SavePlaylist;
end;

procedure TForm1.btnLoadPlaylistClick(Sender: TObject);
begin
  // Open file dialog
  await(string, WebOpenDialogPlaylist.Perform);

  // If playlist was selected, then deal with it
  if WebOpenDialogPlaylist.Files.Count = 1
  then WebOpenDialogPlaylist.Files.Items[0].GetFileAsText;
end;

procedure TForm1.WebOpenDialogPlaylistGetFileAsText(Sender: TObject;
  AFileIndex: Integer; AText: string);
begin
  WAAH.LoadPlaylist(AText);
end;

procedure TWAAH.SavePlaylist;
begin
  // Just save the entire contents of the Tabulator table
  asm
    var playlist = JSON.stringify(this.tabPlaylist.getData());
    var blob = new Blob([playlist], {type: "text/plain;charset=utf-8"});
    var streamSaver = window.streamSaver;
    const fileStream = streamSaver.createWriteStream('New Playlist.playlist', {
      size: blob.size // Makes the procentage visiable in the download
    })
    const readableStream = blob.stream()
    window.writer = fileStream.getWriter()
    const reader = readableStream.getReader()
    const pump = () => reader.read()
      .then(res => res.done
            ? writer.close()
            : writer.write(res.value).then(pump))
    pump()
  end;
end;

procedure TWAAH.LoadPlaylist(data: String);
begin
  // Just set the contents of the Tabulator table
  asm
    this.tabPlaylist.setData(JSON.parse(data)).
    then(function() {
      var table = pas.WebAudioAPIHelper.WAAH.tabPlaylist;
      if (table.getDataCount() > 0) {
        table.selectRow(table.getRowFromPosition(1));
      }
    });
  end;
end;

Web Audio API Intro.

Alright. After all that, let's set about making some noise. Or rather, playing some music. The Web Audio API (documentation) can be thought of as a collection of a relatively small number of building blocks, referred to as AudioNodes. There are "source" AudioNodes that are used to directly generate or otherwise present sound information for subsequent use. These include the following.

• AudioScheduledSourceNode
• OscillatorNode
• AudioBufferSourceNode
• MediaElementAudioSourceNode
• MediaStreamAudioSourceNode
• MediaStreamTrackAudioSourceNode

For our player application, we're primarily just interested in the AudioBuferSourceNode. We'll have a look at the other sources in the next blog post.

At the other end, there are "destination" AudioNodes - where the sound will be played or otherwise output. There are only a few of those to contend with.

• AudioDestinationNode
• MediaStreamAudioDestinationNode
• AnalyzerNode

We'll be using AudioDestinationNode to actually output audio. We'll also have a look at AnalyzerNode as this can be used to produce the data we need for our visualizer.

Between the source and destination AudioNodes, there are a number of "intermediate" nodes. These are used for virtually everything else, including things related to handling different audio channels, altering the audio data in many different ways, and handling the overall flow of data. 

When these are all combined, we end up with an AudioGraph that is contained within an AudioContext (typically one AudioContext per application). Put another way, we'll need an AudioContext that consists of a source AudioNode, a destination AudioNode, and whatever we want to put between the source and destination, like a node to address the volume of the output, normally referred to as "gain" in an audio setting.

The other gotcha here is that the Web Audio API doesn't really know much about audio file formats. It assumes its data is in a particular format (PCM, if you're interested), and provides a decodeAudioData() function to get data into that format.

The first and most obvious question might well be about what formats such a function might support. It turns out that this is not such an easy question to answer, as it relies on the audio support of the browser to function. And not all browsers support the same audio formats. Fun, hey? Well, for our purposes, they all support .MP3 and .WAV, so we're not going to worry about it much. A number of other formats (OGG, AAC, FLAC etc.) may very well be supported, but it would be a good idea to test different formats on different browsers if this is something that is important to you.

To start with, let's try just playing one of our .MP3 files, whichever one happens to be selected when we click the play button. First, we'll initialize our AudioContext and set up our first AudioNode. Then we'll load the .MP3 player data into the appropriate node and see what happens. This data is currently stored in the table as a Base64 encoded string and an ArrayBuffer.

The idea is that the Base64 string will be stored in the playlist file, whereas the ArrayBuffer is likely to incur less overhead when processing - to get our audio processed as soon as possible. Storing the decoded audio is likely problematic as it is rather large. If a playlist file is loaded, it will initially have only the Base64 version, so the ArrayBuffer version will be created. Kind of a nuisance, but the alternative (what most players do I imagine) is to just store a reference to a file. Doesn't work so well here as a browser can't arbitrarily access any old file at any old time it likes. It is also a bit of an issue when it comes to memory management, something browsers (and web app developers) routinely ignore, as we'll be doing here.

procedure TWAAH.PlayAudio(Start: Double);
begin
  asm
    // might be called from other JS events, so let's fully qualify everything
    var WAAH = pas.WebAudioAPIHelper.WAAH;

    // If playlist is loaded, ArrayBuffers will need to be regenerated
    function base64ToArrayBuffer(base64) {
      var binary_string = window.atob(base64);
      var len = binary_string.length;
      var bytes = new Uint8Array( len );
      for (var i = 0; i < len; i++) {
        bytes[i] = binary_string.charCodeAt(i);
      }
      return bytes.buffer;
    }

    // If it is already playing, then do everything we can to stop it and
    // clear out any data or memory.  Unclear if this is sufficient.
    if (this.AudioSrc !== undefined) {
      WAAH.AudioSrc.stop();
      WAAH.AudioSrc.disconnect();
      WAAH.AudioSrc.buffer = null;
      WAAH.AudioSrc = this.AudioCtx.createBufferSource();
    }
    else {
      WAAH.AudioSrc = this.AudioCtx.createBufferSource();
    }

    // If a row is selected, start playing the first one.
    var rows = WAAH.tabPlaylist.getSelectedRows();
    if (rows.length !== 0) {

      // Get the ArrayBuffer from the table if it is there.
      // If it isn't there, convert the BAse64 data into an ArrayBuffer
      var AudioData = new Uint8Array(rows[0].getCell('TRACKBUF').getValue()).buffer;
      if (AudioData == undefined) {
        AudioData = base64ToArrayBuffer(rows[0].getCell('TRACKDATA').getValue());
        rows[0].getCell('TRACKDATABUF').setValue(new Uint8Array(AudioData));
      }

      // Convert ArrayBuffer into audio data the Web Audio API understands
      WAAH.AudioCtx.decodeAudioData(
        AudioData,
        (buffer) => {

          WAAH.AudioSrc.buffer = buffer;
          WAAH.AudioSrc.connect(WAAH.AudioCtx.destination);

          // Kind of resuming a pause here but also good for having the
          // scrubber be able to set the play position.  Not ideal.
          WAAH.AudioSrc.start(Start*buffer.duration);

          // We don't have a "position" with AudioBufferSource Node, so
          // we need to keep track of timestamps ourselves.  Boo.
          WAAH.AudioStart = WAAH.AudioCtx.currentTime - (Start*buffer.duration);;
          pas.Unit1.Form1.PlayNow = Start*buffer.duration;
          pas.Unit1.Form1.PlayerDuration = buffer.duration;
        },

        // Guess it got some data it didn't understand :(
        (e) => {
          console.log("Error decoding audio data: "+e.error);
        }
      );
    }
  end;
end;

This is perhaps a surprising aspect of this part of the API. Seems the general gist of things is to create AudioBufferSourceNode objects whenever you want to play something other than what is currently playing.  You can't call "start()" more than once on an AudioBufferSourceNode, and you can't position where the playing occurs.

There are other AudioNodes that have these features, but as we're angling for some other things that are only available in an AudioBufferSourceNode, we'll just take our lumps here and continue on with this approach. As I'm rather new to the Web Audio API, I don't know how best to free up AudioBufferSourceNode objects. The documentation simply states that they'll be garbage collected when there are no longer any references to them.  Just like any other JavaScript library. Something to keep in mind.

Scrubbing.

In order to give feedback in the UI, there are some form variables in the main unit that are used to keep track of where the playing is at. Getting a current play position from an AudioBufferSourceNode is conspicuously absent as well, so we have to resort to using timers and that sort of thing. The AudioContext object has a "currentTime" value (a double) that represents the number of seconds since the AudioContext object was first initialized. This is a higher precision timer than what might be available in other JavaScript timers, so it is used as the base for other calculations. To use this in Delphi, we call a JavaScript function that in turn sets the form variable.

procedure TWAAH.GetCurrentPosition;
begin
  asm
    if (this.AudioSrc !== undefined) {
      pas.Unit1.Form1.PlayerNow = this.AudioCtx.currentTime - this.AudioStart;
    }
    else {
      pas.Unit1.Form1.PlayerNow = -1;
      pas.Unit1.Form1.PlayerDuration = -1;
    }
  end;
end;

procedure TForm1.tmrNowPlayingTimer(Sender: TObject);
begin
  WAAH.GetCurrentPosition;
  if (PlayerNow < PlayerDuration) then
  begin
    labelElapsed.Caption := FormatDateTime('hh:mm:ss', PlayerNow / 86400);
    labelRemaining.Caption := FormatDateTime('hh:mm:ss', Max((PlayerDuration - PlayerNow) / 86400,0));
    trackbarScrubber.Value := Max(Min(Trunc(10000.0*(PlayerNow / PlayerDuration)),10000),0);
    tmrNowPlaying.Interval := 50;
  end
  else if (PlayerNow = -1) then
  begin
    tmrNowPlaying.enabled := False;
    tmrNowPlaying.Interval := 1000;
  end
  else
  begin
    WAAH.LoadNextTrack;
  end;
end;

procedure TForm1.btnPlayClick(Sender: TObject);
begin
  if PlayerState = 'Paused' then
  begin
    WAAH.PlayAudio(trackbarScrubber.Value / 10000);
    PlayerState := 'Playing';
    tmrNowPlaying.Enabled := True;
    btnPlay.Caption := '<i class="fa-solid fa-pause fa-3x"></i>';
  end
  else
  begin
    WAAH.PauseAudio;
    PlayerState := 'Paused';
    tmrNowPlaying.Enabled := False;
    tmrNowPlaying.Interval := 1000;
    btnPlay.Caption := '<i class="fa-solid fa-play fa-3x"></i>';
  end;
end;

procedure TForm1.trackbarScrubberClick(Sender: TObject);
begin
  if PlayerState <> 'Paused' then
  begin
    WAAH.PlayAudio(trackbarScrubber.Value / 10000);
    tmrNowPlaying.Enabled := True;
  end
end;

Previous and Next Tracks.

For the previous track, we just check if the time played is less than 2s. If it is, we go to the previous track. If it isn't, we just reload the current track  In both the previous and next track buttons, we'll loop around the playlist.

procedure TForm1.btnRewindClick(Sender: TObject);
begin
  tmrNowPlaying.Enabled := False;
  if PlayerNow < 2.0
  then WAAH.PlayAudio(0)
  else WAAH.LoadPreviousTrack;
  tmrNowPlaying.Interval := 1000;
  tmrNowPlaying.Enabled := True;
end;
procedure TForm1.btnForwardClick(Sender: TObject);
begin
  tmrNowPlaying.Enabled := False;
  WAAH.LoadNextTrack;
  tmrNowPlaying.Interval := 1000;
  tmrNowPlaying.Enabled := True;
end;

The JavaScript functions then deal with the Tabulator calculations.

procedure TWAAH.LoadPreviousTrack;
begin
  asm
    var rows = this.tabPlaylist.getSelectedRows();
    if (rows.length !== 0) {
      var rowposition = rows[0].getPosition();
      if (rowposition == 1) {
        rowposition = this.tabPlaylist.getDataCount()
      }
      else {
        rowposition = rowposition - 1;
      }
      this.tabPlaylist.deselectRow();
      this.tabPlaylist.selectRow(this.tabPlaylist.getRowFromPosition(rowposition));
      pas.WebAudioAPIHelper.WAAH.PlayAudio(0);
    }
  end;
end;

procedure TWAAH.LoadNextTrack;
begin
  asm
    var rows = this.tabPlaylist.getSelectedRows();
    if (rows.length !== 0) {
      var rowposition = rows[0].getPosition();
      if (rowposition < this.tabPlaylist.getDataCount()) {
        rowposition = rowposition + 1;
      }
      else {
        rowposition = 1;
      }
      this.tabPlaylist.deselectRow();
      this.tabPlaylist.selectRow(this.tabPlaylist.getRowFromPosition(rowposition));
      pas.WebAudioAPIHelper.WAAH.PlayAudio(0);
    }
  end;
end;

Shuffling.

The playlist is a collection of rows in a (Tabulator) table. The table is deliberately not sorted so that the user can rearrange the rows to their liking. To shuffle the playlist, all we're doing is moving each row to be in front of another randomly chosen row in the table. It might be a curiosity that the same row could be moved (and probably is moved) more than once, but the randomization effect is the same.

procedure TWAAH.ShufflePlaylist;
begin
  asm
    var rowcount = this.tabPlaylist.getDataCount();
    for (var i = 1; i <= rowcount; i++) {
      var row = this.tabPlaylist.getRowFromPosition(i);
      row.move(this.tabPlaylist.getRowFromPosition(1+Math.floor(Math.random() * (rowcount - 1)),true));
    }
  end;
end;

Volume.

In order to implement a volume control, we'll need to add another AudioNode to our AudioContext AudioGraph.  The idea is that we want to go from an AudioBufferSourceNode (which contains the music data) to a "gain node" and then to the destination (speakers, for example). At the moment, the source is directly connected to the destination. This new node will be persistent, so we can create it when we create the initial AudioContext.

procedure TWAAH.InitializeAudio;
begin
  asm
    this.AudioCtx = new (window.AudioContext || window.webkitAudioContext)();
    this.AudioGain = new GainNode(this.AudioCtx);
  end;
end;

WAAH.AudioSrc.connect(WAAH.AudioCtx.destination);

WAAH.AudioSrc.connect(WAAH.AudioGain).connect(WAAH.AudioCtx.destination);

procedure TWAAH.SetVolume(volume: Integer);
begin
  asm
    this.AudioGain.gain.value = volume;
  end;
end;

procedure TForm1.trackbarVolumeChanged(Sender: TObject);
begin
  WAAH.SetVolume(trackbarVolume.value / 100.0);
end;

GainNodes could also be added individually to each track. For example, perhaps there is a song in the playlist that is unusually quiet or unusually loud. It wouldn't be difficult to add a separate "volume adjustment" to the individual tracks. This would then be saved with the playlist and could be added as an extra AudioNode in the PlayAudio function. 

Similarly, having a clipping function for each track is also not much of a stretch - starting the track a bit later or ending a bit earlier to clip any quiet time out of the track, for example. We'll get more into this kind of thing in the next blog post, but this is one of the powerful ideas behind the Web Audio API. Being able to connect many different AudioNodes together, and adjust the properties of each along the way, to achieve whatever result you're after.

Visualizer.

The last topic we're going to cover today is the visualizer. While the AudioBufferSourceNode gives us very little in terms of what is going on, there's another AudioNode that we can use to get quite a lot of information - the AnalyserNode. This is capable of generating on-the-fly a set of data that describes the current waveform being played, using a bunch of math that I've long forgotten - Fourier transforms and that sort of thing. 

We can then take that data and hand it over to D3 to display it. And this all happens fast enough that we can create a pretty reasonable visualizer from it. The initial setup is just the same as we've done with the volume mechanism.  A new AudioNode needs to be inserted into the AudioGraph. This one, too, is persistent, so we can do the same thing we've just done. Here's the new InitializeAudio procedure.

procedure TWAAH.InitializeAudio;
begin
  asm
    this.AudioCtx = new (window.AudioContext || window.webkitAudioContext)();
    this.AudioGain = new GainNode(this.AudioCtx);
    this.AudioAnalyser = new AnalyserNode(this.AudioCtx)
  end;
end;

WAAH.AudioSrc.connect(WAAH.AudioGain).connect(WAAH.AudioAnalyser).connect(WAAH.AudioCtx.destination);

    AnalyserNode.fftSize = 256;
    const dataArray = new Uint8Array(AnalyserNode.frequencyBinCount);
    AnalyserNode.getByteFrequencyData(dataArray)

procedure TWAAH.DrawVisualizer;
var
  VisWidth: Integer;
  VisHeight: Integer;
begin
  VisWidth := Form1.divVisualizer.width;
  VisHeight := Form1.divVisualizer.Height;
  asm
    const h = VisHeight;
    const w = VisWidth;

    var colors
    let svg;

    svg = d3.select('#divVisualizer').append('svg')
        .attr('width', w)
        .attr('height', h)
        .attr('id', 'visualizer-svg');

    this.AudioAnalyser.fftSize = 256;
    const dataArray = new Uint8Array(this.AudioAnalyser.frequencyBinCount);

    const colorScale = d3.scaleSequential(d3.interpolateSinebow)
      .domain([1, 255])

    const y = d3.scaleLinear()
      .domain([0, 255])
      .range([h, 0])

    svg.selectAll('rect')
      .data(dataArray)
      .enter().append('rect')
      .attr('width', ((w / dataArray.length) * 0.8))
      .attr('x', function (d, i) { return (((w / dataArray.length) * i) + ((w / dataArray.length) * 0.1)) })

    function renderFrame () {
      requestAnimationFrame(renderFrame)
      pas.WebAudioAPIHelper.WAAH.AudioAnalyser.getByteFrequencyData(dataArray)

      svg.selectAll('rect')
        .data(dataArray)
        .attr('height', function (d) { return (h - y(d)) })
        .attr('y', function (d) { return y(d) })
        .attr('fill', function (d) { return d === 0 ? 'black' : colorScale(d) })
    }
    renderFrame()
  end;
end;

This gets us our fancy visualizer.

The End Result!

Next Time.

That should about cover it for today (I hope!). Next time, we'll create another Web Audio API application, but one modeled more on something like GarageBand, looking at how to combine a larger and more varied collection of AudioNodes to create more interesting audio effects, applied to either existing tracks or for generating new sounds entirely.

Example download

Related Links
Web Audio API: Part 1 of 2
Web Audio API: Part 2 of 2

Follow Andrew on 𝕏 at @WebCoreAndMore or join our 𝕏 Web Core and More Community.