Today we are delighted to announce a new feature in Azure IoT Hub which allows users to upload files from their devices to the cloud with the assistance of IoT Hub. This is one of the most common feature asks we’ve heard from our customers, especially from customers who are working with existing systems which produce files that are too big to send over the telemetry channel.
Instead of brokering messages through IoT Hub itself, IoT Hub instead acts as a dispatcher. Devices request a storage token from IoT Hub that is specific to the file the device is trying to upload. IoT Hub returns a scoped SAS URI to the device with a short expiry time (1 day). Storage SAS URIs can have extremely long expiry times, but in this case the expiry time is intentionally kept short to limit the security risk to storage if the device were to become compromised. The device uses the SAS URI to upload its file to storage, and when the upload is complete the device sends a notification of completion to IoT Hub. IoT Hub verifies that the file was uploaded and then adds a file upload notification to the new service-side file notification messaging endpoint. IoT Hub keeps track of all file upload requests in flight and can log errors when a SAS URI expires before the hub receives an upload notification from the device. The messages sent to ask the IoT hub for a SAS URI and to notify the hub of a completed file upload count towards the messaging limit and are billed accordingly.
This is all really, really easy if you use the IoT Hub device client SDK (https://github.com/Azure/azure-iot-sdks). Once you’ve associated a storage account with your IoT hub, all you need to do is call the new UploadFromStringAsync API from the device in order to upload a file. Once the upload is complete, you’ll get a notification at the new {iot hub}.azure-devices.net/messages/servicebound/filenotifications endpoint if you’ve chosen to receive file upload notifications for your IoT hub.
You can still upload files using your IoT hub even if you aren’t using the IoT Hub device client SDKs, although there’s a little more work to do. IoT Hub has two new REST endpoints to support file upload, one to get the SAS URI for storage and the other to notify the IoT hub of a completed upload. The device initiates the file upload process by sending a GET to the IoT hub at {iot hub}.azure-devices.net/devices/{deviceId}/files/{filename}. The hub will return a SAS URI specific to the file to be uploaded, as well as a correlation ID to be used once the upload is completed. The device is responsible for uploading the file to storage using the Azure Storage SDKs. Once the upload is complete, the device sends a POST to the IoT hub at {iot hub}.azure-devices.net/devices/{deviceId}/messages/files/notifications/{correlationId} using the correlation ID received from the initial GET.
We updated IoT Hub’s operations monitoring with a new “File upload” category to help customers debug their solution when using this feature. The new category covers errors that occur during the communications with IoT Hub, such as
- A device notifies the hub that a file upload failed
- A SAS URI expired before a device notifies the hub of a completed upload
- A file is not found in storage during IoT Hub notification message creation
- IoT Hub failed to create a SAS URI from the associated storage account (such as incomplete key rollover)
File upload enables IoT solutions that previously required customers to write a bunch of custom code to get their files to the cloud. On top of an easier upload experience, IoT Hub also provides notifications for completed file uploads as well as a new operations monitoring category to help debug. Our documentation now includes details on how to use file upload in your IoT solutions, so go forth and upload files using IoT Hub!