Blog

All Blog Posts  |  Next Post  |  Previous Post

Resumable uploads for large files to DropBox, Google Drive and OneDrive in Delphi

Tuesday, October 11, 2022

Introducing the long awaited resumable uploads feature in TMS FNC Cloud Pack!

Uploading files with REST by using the default upload HTTP request can be problematic when you need to upload large files:

  • The maximum file size that can be uploaded with a single request is limited
  • If a problem occurs during the upload, the whole file needs to uploaded again
  • Internet connection issues and/or stability can cause the upload to fail

All this can be a thing of the past with the new resumable uploads feature in TMS FNC Cloud Pack.


How Does It Work?

Instead of executing a single request that transfers the entire file at once, the resumable upload uses the chunked uploads technique. This means the file to be uploaded is split in smaller parts and each part is uploaded separately. If the upload is interrupted it can be continued later from the last chunk that was successfully uploaded. This technique keeps the amount of data that needs to be re-uploaded in case of a failed upload to a minimum and makes it possible to upload large files.

Supported Services

Resumable uploads are supported for the following cloud storage services:

TMS Software Delphi  Components TMS Software Delphi  Components TMS Software Delphi  Components TMS Software Delphi  Components

Box.netDropBoxGoogle Drive and Microsoft OneDrive



Getting Started

How to start a resumable upload

  • Add a new TTMSFNCCloudFile object to the CloudFiles collection and set the full path and filename of the file that is going to be uploaded in the FilePath property
  • Call the UploadResumableFile method with the TTMSFNCCloudFile object as parameter

var
  FCloudFile: TTMSFNCCloudFile;
begin
  FCloudFile := TMSFNCCloudStorageServices1.CloudFiles.Add;
  FCloudFile.FilePath := 'c:\uploads\example.zip';
  TMSFNCCloudStorageServices1.UploadResumableFile(FCloudFile);
end;


How to monitor upload progress

Assign the OnUploadResumableFile, OnUploadResumableFileFailed, OnUploadResumableFileFinished events

  • The OnUploadResumableFile event is triggered each time a chunk has been uploaded successfully. The ACloudFile.Position parameter value can be used to determine how much data has already been uploaded.

  • The OnUploadResumableFileFailed event is triggered when an error occurs during upload. Depending on the error, the update will be need to be resumed or restarted. The error message is contained in the ARequestResult.ResultString parameter value.

  • The OnUploadResumableFinished event is triggered when the upload is complete. A reference to the TTMSFNCCloudFile object that was used for the upload is available in the ACloudFile parameter.

In the sample below, upload progress is displayed in a TProgressBar and if an error occurs it is displayed in a message box.

procedure TForm1.Button1Click(Sender: TObject);
var
  FCloudFile: TTMSFNCCloudFile;
begin
  TMSFNCCloudStorageServices1.CloudFiles.Clear;
  FCloudFile := TMSFNCCloudStorageServices1.CloudFiles.Add;
  FCloudFile.FilePath := 'example.zip';
  TMSFNCCloudStorageServices1.UploadResumableFile(FCloudFile);
  ProgressBar1.Min := 0;
  ProgressBar1.Max := FCloudFile.GetFileSize;
  ProgressBar1.Value := 0;
end;

procedure TForm1.TMSFNCCloudStorageServices1UploadResumableFile(
  Sender: TObject; ACloudFile: TTMSFNCCloudFile; const ARequestResult: TTMSFNCCloudBaseRequestResult);
begin
  Label1.Text := 'file uploading ... ' + ACloudFile.FilePath;
  ProgressBar1.Value := ACloudFile.Position;
end;

procedure TForm1.TMSFNCCloudStorageServices1UploadResumableFileFailed(
  Sender: TObject; ACloudFile: TTMSFNCCloudFile; const ARequestResult: TTMSFNCCloudBaseRequestResult);
begin
  Label1.Text := 'file upload failed:' + ACloudFile.FilePath;
  ShowMessage(ARequestResult.ResultString);
end;

procedure TForm1.TMSFNCCloudStorageServices1UploadResumableFileFinished(
  Sender: TObject; ACloudFile: TTMSFNCCloudFile; const ARequestResult: TTMSFNCCloudBaseRequestResult);
begin
  Label1.Text := 'file upload finished! ' + ACloudFile.FilePath;
  ProgressBar1.Value := ProgressBar1.Max;
end;

How to resume an upload

Call UploadResumableFile with an existing TTMSFNCCloudFile object as parameter to resume an upload.
During the upload process, the associated upload session and upload progress data are saved in the UploadSessionID and Position properties respectively.
Note that upload sessions generally stay active for a week. If an upload was started longer than a week ago, the upload process will need to be restarted.

begin
  TMSFNCCloudStorageServices1.UploadResumableFile(TMSFNCCloudStorageServices.CloudFiles[0]);
end;


How to persist upload progress

All required data is included in the TTMSFNCCloudFile object. By exporting and importing this data upload progress can be persisted.

Save data

  • Call ToJSON on a TTMSFNCCloudFile oject
  • Save the JSON data in a destination of your choice

Load data

  • Load the saved JSON data from a source of your choice
  • Call FromJSON on a TTMSFNCCloudFile object with the JSON data as parameter

var
  sJSON: String;

//save data
begin
  sJSON := TMSFNCCloudGoogleDrive1.CloudFiles[0].ToJSON;
end;

//load data
begin
  TMSFNCCloudGoogleDrive1.CloudFiles[0].FromJSON(sJSON);
end;


Available Now

The TMS FNC Cloud Pack update with resumable upload support is available now for Delphi & Visual Studio Code (with TMS WEB Core). You can download the latest version and start using the new features right away!



Bart Holvoet




This blog post has received 3 comments.


1. Saturday, June 24, 2023 at 12:02:48 AM

The matching Dropbox component has matching hooks for UploadResumableFile, UploadResumableFileFailed, and UploadResumableFileFinished.

But it does not seem to have a matching CloudFiles.Add method.

That''s shame because the Dropbox component can use the hook for download progress where the TMSFNCCloudStorageServices component does not. I think reporting download and upload progress for very large files is essential.

Leahy Michael


2. Tuesday, June 27, 2023 at 2:56:02 PM

This question was replied to in the TMS Support Center:

https://support.tmssoftware.com/t/how-to-monitor-download-progress/21160

Bart Holvoet

Bart Holvoet


3. Tuesday, June 27, 2023 at 6:21:06 PM

It must have been my coding mistake. I started a new project and the Dropbox component does have the CloudFiles.Add method. Kindly ignore my last comment.

Leahy Michael




Add a new comment

You will receive a confirmation mail with a link to validate your comment, please use a valid email address.
All fields are required.



All Blog Posts  |  Next Post  |  Previous Post