Skip to content

File upload (IoT Hub)

Overview

The platform supports file upload from devices provisioned on Azure IoT Hub. This feature allows devices to upload files directly to Azure Blob Storage through IoT Hub's file upload mechanism, and enables users to list and download these files through the Management API.

Currently, file upload is only available for devices using the IoT Hub communication channel. In the future, this file upload mechanism will be implemented for other device communication channels, including MQTT and HTTP.

How It Works

Device-Side Upload Process

  1. Request Upload Authorization: The device requests a blob Shared Access Signature (SAS) token from IoT Hub, specifying the desired file name. IoT Hub returns authorization details including a SAS token, blob URL, and a correlation ID.

  2. Direct Upload to Blob Storage: The device uploads the file directly to Azure Blob Storage using the provided SAS token and blob URL. This upload bypasses IoT Hub and goes directly to the storage account, improving performance and reducing load on IoT Hub.

  3. Notify Upload Status: After the upload completes (successfully or with failure), the device notifies IoT Hub of the upload status, providing:

    • The correlation ID received in step 1
    • Upload success status
    • HTTP status code
    • Status message

Storage Organization

  • Files are stored in an Azure Blob Storage container (default name: device-uploaded-files)
  • Files are organized by device interface name (the IoT Hub device ID), not the platform device ID
  • The file path structure is: {deviceInterfaceName}/{fileName}
  • Files maintain their original names as specified by the device during upload
  • File names can contain folder paths using forward slashes (e.g., a/b/file.txt or logs/2024/01/error.log). These folder structures are preserved in the blob storage and reflected in the file names returned by the API

API Access

The Management API provides an endpoint to list files uploaded by a device:

  • Endpoint: GET /tenants/{tenantId}/devices/{deviceId}/files
  • Permission Required: Read.Device
  • Returns: Up to 1000 files with download links (SAS tokens valid for 1 hour)

Limitations

Platform Limitations

  1. IoT Hub Only: File upload is exclusively available for devices provisioned on Azure IoT Hub. Devices using other communication channels (e.g., MQTT, HTTP) cannot upload files through this mechanism.

  2. Azure Storage Only: The feature only works when using Azure Storage Account as the storage service. AWS S3 storage does not support device file listing or management.

  3. Device Type Requirement: The device must have a device type that supports the IoT Hub interface. Devices without an IoT Hub channel cannot upload files.

  4. Provisioning Requirement: The device must be provisioned on IoT Hub. Unprovisioned devices cannot upload files.

  5. Pagination Not Supported: The API returns up to 1000 files at once, ordered by name. Azure Blob Storage does not support direct pagination, so if a device has more than 1000 files, only the first 1000 will be returned. This limitation may be addressed in future versions.

  6. Download Link Expiration: Download links (SAS tokens) expire after 1 hour (3600 seconds). Users must request a new list of files to obtain fresh download links.

  7. File Organization: Files are organized by the IoT Hub device interface name, which may differ from the platform device ID. The system attempts to map the device interface name, but falls back to the device ID if the device is not provisioned.

Azure IoT Hub Limitations

The following limitations are inherited from Azure IoT Hub's file upload feature:

  1. File Size: Subject to Azure Blob Storage limits (typically up to 4.75 TB per blob, but practical limits may be lower based on network and timeout constraints).

  2. Upload Retries: The implementation includes retry logic (max 4 tries) with keep-alive disabled, but devices should implement their own retry mechanisms for reliability.

  3. Concurrent Uploads: Multiple concurrent uploads from the same device are supported, but may be subject to Azure Storage throttling limits.

  4. Storage Account Configuration: The IoT Hub file upload feature must be properly configured in the infrastructure with:

  5. A valid storage account connection string
  6. A designated container name (default: device-uploaded-files)

Usage Example

Device Upload (Node.js/TypeScript)

import { DeviceClient } from "azure-iot-device";
import { BlockBlobClient, newPipeline, AnonymousCredential } from "@azure/storage-blob";

async function uploadFile(connectionString: string, file: Buffer, fileName: string) {
    const client = DeviceClient.fromConnectionString(connectionString);

    // Request SAS token from IoT Hub
    const blobInfo = await client.getBlobSharedAccessSignature(fileName);

    // Upload directly to blob storage
    const blobUrl = `https://${blobInfo.hostName}/${blobInfo.containerName}/${blobInfo.blobName}${blobInfo.sasToken}`;
    const blobClient = new BlockBlobClient(blobUrl, newPipeline(new AnonymousCredential()));
    await blobClient.uploadData(file);

    // Notify IoT Hub of successful upload
    await client.notifyBlobUploadStatus(
        blobInfo.correlationId,
        true,
        200,
        "File uploaded successfully"
    );

    await client.close();
}

API Usage

GET /tenants/{tenantId}/devices/{deviceId}/files
Authorization: Bearer {token}

Response: 

[
  {
    "name": "e2e-test/file.txt",
    "downloadLink": "https://...",
    "downloadLinkExpiresAt": "2024-01-01T12:00:00Z",
    "createdAt": "2024-01-01T10:00:00Z",
    "updatedAt": "2024-01-01T10:00:00Z"
  }
]