Azure Cosmos DB and multi-tenant systems

In this blog post, we will discuss how to build a multi-tenant system on Azure Cosmos DB. Azure Cosmos DB itself is a multi-tenant PaaS offering on Microsoft Azure. Building a multi-tenant system on another multi-tenant system can be challenging, but Azure provides us all the tools to make our task easy. An example of a multi-tenant system would be a company providing background check services that any other company can use in their HR system. For the purposes of this blog post we are going to use this example and continue from the point of view of company providing background checks as a service. We will refer to this company as “publisher.”

Let’s begin to discuss how you can build a multi-tenant system that will store sensitive user data. Data isolation and security is the most important aspect of any system. We must design the system so that each tenant’s data is isolated from one another. The data stored in any given tenant should be divided into compartments so one tenant breach cannot flow into another tenant. This would be similar to compartmentalizing the hull of a ship to reduce floodability.

To increase the isolation and protection of customer data in a multi-tenant system, we should build the system with only one approved service that can have just in time (JIT) access to tenant data. We need to set up a different system principal for each customer’s data partition so that the scope of access for any principal is segmented by customer. We don’t want a service to have access to all tenant data, that is a big security risk. What we want is the service to get the access permission for one tenant JIT. The benefit of this approach is tenants can rotate their certificates and keys anytime.

Every tenant can manage their data using the publisher’s front-end service (FES), but they cannot directly manipulate their own data in Azure Cosmos DB collections. This isolation will remove the need for every tenant to have access to master and read-only keys. All the data access will happen through a service and no one will access Azure Cosmos DB unless directly on the portal or through code. The publisher application, which manages the customer data, is hosted in a different Azure Active Directory tenant and subscription, which is separate from that of the customer’s tenant and data.

However, the tenant will own all the collections and data without having direct access to the data. This simplifies the billing for all data storage and throughput that the tenant is directly billed, but is a tricky requirement. Let’s see how you can manage this requirement.

The main actors of this solution are Azure Managed Applications, Daemon Application, Azure Cosmos DB, Azure Key Vault and Azure Active Directory (AAD). The following paragraphs will help us understand each of the mentioned solutions.

An Azure Managed Application is like a service catalog in the marketplace, but with one key difference. In a managed application, the resources are provisioned in a resource group that is managed by the publisher of the app. The resource group is present in the consumer’s subscription, but an identity in the publisher’s tenant has access to the resource group in the customer subscription. As the publisher, you specify the cost of ongoing support for the solution.

Managed applications reduce barriers to consumers using your solutions. They do not need expertise in cloud infrastructure to use your solution. Consumers have limited access to the critical resources. They don’t need to worry about making a mistake when managing it. Managed applications enable customers to adopt your solution without needing expertise in the underlying Azure infrastructure.

Managed applications enable you to establish an ongoing relationship with your consumers. You define the terms for managing the application, and all charges are handled through Azure billing.

Although customers deploy these managed applications in their subscriptions, they do not have to maintain, update, or service them. You can ensure that all customers are using approved versions. Customers do not have to develop application specific domain knowledge to manage these applications. They automatically acquire application updates without the need to worry about troubleshooting and diagnosing issues with the applications. The advantages of an Azure Managed Application is billing, separation of data between different tenants, easy maintenance, among other benefits. For more details, read more about Azure Managed Applications.

After deploying a tenant managed application, create a daemon application. Follow the instructions on how to create an AAD application and service principal that can access resources. This daemon application has its own identity and access to tenant subscription. This application is the bridge between the customer tenant application and the service provider (publisher).

It is important to understand a few things. First, user interaction is not possible with a daemon application, which requires the application to have its own identity. An example of a daemon application is a batch job, or an operating system service running in the background. This type of application requests an access token by using its application identity and presenting its application ID, credentials (password or certificate), and application ID URI to AAD. After a successful authentication, the daemon receives an access token which represents the identity of the application from AAD and is then used to call the web API.

The magic of Azure Managed Applications is that the publisher can access the customer subscription resources it managed as if these resources are located within a subscription in the publisher’s AAD tenant. The customer tenant subscription resources are visible to the customer in their own Azure subscription, but are not accessible due to an Azure Resource Lock. Only the publisher has full access to the managed application resources in the customer’s subscription.

After creating the daemon application, you need to register it in the identity and access (IAM) of Azure Cosmos DB instance, which is deployed as a managed resource component of the customer tenant subscription.

The last piece you will develop is the front-end service (FES). This the service used to manage the components in the customer tenant. This service cannot directly access Azure Cosmos DB until it goes through the orchestration of taking the daemon application identity. The following illustrates a step-by-step walkthrough for the FES interaction with the customer’s subscription resources.

FES takes over the daemon application identity at the run time. FES also has its own managed identity (MSI) which is registered in Key Vault for access. At the run time, the FES connects to the Key Vault using the Azure MSI and obtains the certificate credential, which in turn uses a credential to obtain a token from AAD representing the daemon application (Step 1).

Once the FES gets the certificates, it assumes the identity of daemon service by using the client ID and secret certificate. Then it will call AAD to get the access token for the Managed Application (Step 2). This FES uses Azure Active Directory Authorization DLL (AAD DLL). See the FES code snippet below, which helps FES to get the token from AAD.

using Microsoft.Azure.KeyVault;
using Microsoft.Azure.Services.AppAuthentication;
using Microsoft.IdentityModel.Clients.ActiveDirectory;

string secretIdentifier = " key vault secretIdentifier for daemon app goes here ";
var tokenCache = TokenCache.DefaultShared;
string pubTenantId = " publisher’s Azure AD directory id here ";

//// get app key from Key Vault (… let me know if you need a sample for reading a certificate/private key instead of a secret
var azureServiceTokenProvider = new AzureServiceTokenProvider();

var keyVaultClient = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(azureServiceTokenProvider.KeyVaultTokenCallback));
string secret = keyVaultClient.GetSecretAsync(appKey)
.GetAwaiter()
.GetResult();

//// now get a token representing the Daemon app, using the secret credential of the Daemon Azure AD application.  Resource is the appropriate Azure resource Uri.
string daemonAppId = "daemon Azure AD application Id";
string authString = $"https://login.microsoftonline.com/{pubTenantId}";
string daemonAppResourceUri = "https://management.core.windows.net/";
var clientCredential = new ClientCredential(daemonAppId, secret.Value);
var authenticationContext = new AuthenticationContext(authString, false, tokenCache);

var authnResult = authenticationContext.AcquireTokenAsync(resourceUri, clientCredential)
.GetAwaiter()
.GetResult();

string daemonToken = authnResult.AccessToken;

//// alternately, to use a certificate you would substitute the above variables:
////     var clientCredential = new ClientAssertionCertificate(clientId, certificate);

Once the access token is obtained by FES, it calls into Azure Cosmos DB to get the master key (Step 3 and 4). This is accomplished by using the access token of the daemon application. For this FES we pass the AAD token in header.

using Microsoft.Azure.Management.CosmosDB.Fluent;
using Microsoft.Azure.Management.Fluent;
using Microsoft.Azure.Management.ResourceManager.Fluent;
using Microsoft.Azure.Management.ResourceManager.Fluent.Core;

string subscriptionId = " subscribing customer’s subscription id ";
string resourceGroupName = " subscribing customer’s resource group name ";
string databaseAccountName = " subscribing customer’s Cosmos DB account name ";

var credential = new AzureCredentials(new TokenCredentials(daemonToken), pubTenantId, AzureEnvironment.AzureGlobalCloud);

var azure = Azure.Configure()
.WithLogLevel(HttpLoggingDelegatingHandler.Level.Basic)
.Authenticate(credential)
.WithSubscription(subscriptionId);

var cosmosDbAccounts = azure.CosmosDBAccounts;
var readWritekeys = cosmosDbAccounts.ListKeysAsync(resourceGroupName, databaseAccountName)
.GetAwaiter()
.GetResult();

Once it has the master key, it starts accessing the Cosmos DB (Step 5).

using Microsoft.Azure.Documents;
using Microsoft.Azure.Documents.Client;

string cosmosDBendpointUri = $"{databaseAccountName}.documents.azure.com:443/";

string masterKey = readWritekeys.PrimaryMasterKey; //// pick the one you need

var connectionPolicy = new ConnectionPolicy { ConnectionMode = ConnectionMode.Direct, ConnectionProtocol = Protocol.Tcp };

var documentClient = new DocumentClient(new Uri(cosmosDBendpointUri), masterKey, connectionPolicy);

client.OpenAsync()
.GetAwaiter()
.GetResult();

You may wonder why the daemon application identity, rather than an Azure MSI representing the FES, is used to retrieve the Azure Cosmos DB keys. The answer is security isolation, JIT access, getting the daemon application secret from Key Vault, and accessing AAD to get it’s token all help support security isolation. This orchestration makes sure that FES does not have access to all the tenants’ keys. It can get access to keys JIT only by using the daemon identity.

This system has the following advantages:

• No access key is kept with the publisher, in code, or in any configuration files. This method provides the security needed for every tenant.
• One publisher access token cannot access all the tenants.
• Each subscribing customer is provisioned with its own daemon application identity for access to that customer’s resources.
• Only at the run time can FES get the access token by using the daemon application secrets.

Azure Cosmos DB brings many of its advantages to this solution, such as:

• The publisher does not know how much throughput and space is required at the time of onboarding a new tenant.
• Azure Cosmos DB’s elastic nature for storage and throughput keeps this solution very flexible.
• The Azure Managed Applications template defined by the publisher comes with a minimum default Azure Cosmos DB whose request units can be expanded as needed.
• JIT access through the use of daemon applications and Key Vault.

I hope this article provided you with enough pointers to help you get started on your journey to build a multi-tenant system over Azure Cosmos DB.

Special thanks to Terry Carter, Nikisha Reyes-Grange, and Sneha Gunda for their contribution to this blog post.