Generic Blob Storage
Introduction
By default, ExFlow stores imported documents in the Business Central database. The documents are mainly XML and PDF files, but they can also be other formats such as DOCX or PNG.
The database grows with the number of invoices the customer receives, and because imported documents often contain images, the storage used in Business Central Online can grow quickly. Generic Blob Storage moves these documents to external storage, which is cheaper, and fetches them back automatically when a user needs them.
The supported storage types are:
- Azure Container (Azure Blob Storage)
- SharePoint (using Microsoft Graph)
For step-by-step setup, see OAuth 2.0 Applications and Storage Setup.
General
A blob is the content of a file: a binary sequence of data. The file is described by its metadata, such as name, creation date and content type, which tells the application how to use the content.
To handle blobs efficiently, they are abstracted from the application. When ExFlow asks for a blob, it gets the content back without knowing where it was stored.
The solution uses three layers to isolate functionality.
ExFlow Application
The application that reads, writes, deletes and lists blobs (files).
Storage Management
Handles where, when and how a blob is stored. Caching, transferring blobs to storage and fetching them back are all handled in this layer.
Storage Implementation (Storage Interface)
The implementation of a storage type, such as Azure Container or SharePoint. It implements the most primitive functions, such as Get and Put.
What is Stored Externally
Only Incoming Document Attachments linked to purchase invoices and credit memos approved in ExFlow are moved to external storage. Documents still in the Import Journal or in Approval Status stay in Business Central, as they need immediate access.
When an ExFlow-approved purchase invoice or credit memo is posted, its attachments are added to the Storage Management queue. The next run of the Storage Management job transfers them to external storage.
How Documents are Transferred and Cached
The Storage Management job (codeunit 12026820 Ex Storage Mgmt. NAS) is created automatically when Blob Storage is activated. Each run does two things:
- Transfers new documents to external storage. When a document has been stored successfully, its content is removed from the Incoming Document Attachment in Business Central. The Batch Size setting limits how many documents are transferred per run. Set it to 0 to transfer all of them in one run.
- Clears the cache. A document that has been fetched back from storage stays in Business Central as a cached copy. When it has not been opened for the number of days set in Cache in Days, the copy is removed again. The document itself remains in external storage. Recommended to set 60 days in the ''Cache in Days''- field.
Automatic Restore
Users do not need to do anything to open a document that has been moved to external storage. ExFlow fetches the document back into Business Central automatically when it is needed, for example when the user:
- Downloads an Incoming Document Attachment, or clicks its file name
- Uses Ctrl+I in ExFlow to download the main attachment
- Downloads several documents at once
- Opens the PDF preview in Approval History
- Views the document image through the ExFlow web service (for example in the ExFlow PDF add-in or ExFlow Web)
The restored document is then kept in the cache as described above.
Blob Naming
Each stored document gets a fixed name made of the record's SystemId, table number and field number, for example:
bd7bd0ec-6671-4626-8f14-e95a7c3a5f15_133_8
The Blob Name Structure setting controls the folder structure in the storage:
| Blob Name Structure | Stored as |
|---|---|
| Split String 2 Char 3 Levels (default, recommended) | bd/7b/d0/bd7bd0ec-6671-4626-8f14-e95a7c3a5f15_133_8 |
| As-Is | bd7bd0ec-6671-4626-8f14-e95a7c3a5f15_133_8 |
Splitting the name into folders keeps the number of files per folder low. The Blob Name Structure cannot be changed while Blob Storage is active.
Blob Storage Setup
Go to: ExFlow Setup --> Actions --> Functions --> Blob Storage Mgmt (--> Storage Setup)
The ExFlow Storage Setup is generic. It contains Code, Description, Blob Source (the storage type) and whether the setup is Enabled. A storage setup must be enabled before it can be used.
There can be more than one storage setup per Blob Source.
In the example, the chosen Blob Source is “Azure Container”.
Click Setup Storage to open the setup page for the selected storage type, where the connection details are entered. Each implementation stores its own parameters, in its own table or in isolated storage. For example, the Azure Container Shared Access Key is kept in isolated storage.
Storage Management does not know how the files are physically stored. A call to Get or Put behaves the same whether the storage is Azure Container or SharePoint.
Storage Management Setup
The fields on the ExFlow Storage Management Setup page are described under Blob Storage Setup for SharePoint.
| Actions | Description |
|---|---|
| Storage Setup | Configure one or more storage setups. |
| Enable/Disable Blob Storage | Activate or deactivate Blob Storage. Same as the Active field. |
| Storage Details | Shows the cached entries and the cross references between records and stored documents. |
| Sync Storage | Runs the same process as the Storage Management job: transfers new documents to storage and clears the cache. |
| Manually Add All Posted Documents to Cache | Adds the attachments of all ExFlow posted invoices and credit memos to the queue, so they are transferred at the next sync. |
| Manually Restore All Posted Incoming From Cache | Fetches all stored documents back into Business Central. |
Activating Blob Storage
- Incoming Doc. Storage Code must be filled in before Blob Storage can be activated.
- If posted ExFlow invoices or credit memos already exist, ExFlow asks to add them. Choose Yes to have them transferred at the next sync.
- The recurring Storage Management job is created.
- While Blob Storage is active, Incoming Doc. Storage Code and Blob Name Structure cannot be changed.
Deactivating Blob Storage
When Blob Storage is deactivated, ExFlow asks to restore all stored documents. When confirmed, every document is fetched back into the Incoming Document Attachments in Business Central, and all storage references are removed. This way, it is possible to stop using external storage without losing any documents.
ExFlow Storage Details
Go to: ExFlow Storage Management Setup --> Actions --> Storage Details
The page shows two lists:
| List | Description |
|---|---|
| Cached Entries | Documents currently held in Business Central: documents waiting to be transferred, and documents fetched back from storage. Shows when each entry was stored and last accessed. |
| Cross References | All documents handled by Blob Storage, identified by Table No., Field No. and System Id, with the storage reference (blob name) and the date it was stored externally. |
General Blob Storage (BSApi)
If General Blob Storage Code is set, partners and developers can use the Blob Storage API (codeunit BSApi) to read and write their own files in the storage, with their own names and folder structure. This storage code can be the same as the one used for incoming documents.
The interface that each storage type implements (Ex IBlob Storage v1) contains:
| Function | Description |
|---|---|
| List | List blobs in a path |
| Get | Read a blob as text or as a stream |
| Put | Save a blob from text or from a stream |
| Delete | Delete a blob |
| Exist | Check if a blob exists |
| Setup, DeleteSetup | Open or delete the storage type's own setup |
| Initialize, ClearInternal | Prepare the implementation for a specific storage setup |
| GetDescription | Description shown on the setup pages |
| Enable | Enable or disable the storage |
| Test | Test the connection |