Azure Databricks pricing

A Databricks unit (“DBU”) is a unit of processing capability per hour, billed on per-second usage.

The Data Engineering workload is defined as a job that both starts and terminates the cluster on which it runs. For example, a workload may be triggered by the Databricks job scheduler, which launches a new Apache Spark cluster solely for the job and automatically terminates the cluster after the job is complete.

The Data Analytics workload is any workload that is not an automated workload, for example, running a command within Databricks notebooks. These commands run on Apache Spark clusters which may persist until manually terminated. Multiple users can share a cluster to perform interactive analysis collaboratively.

Depending on the type of workload your cluster runs, you will either be charged for Data Engineering or Data Analytics workload. For example, if the cluster runs workloads triggered by the Databricks jobs scheduler, you will be charged for the Data Engineering workload. If your cluster runs interactive features such as ad-hoc commands, you will be billed for Data Analytics workload.

• If you run Premium tier cluster for 100 hours in East US 2 with 10 DS13v2 instances, the billing would be the following for Data Analytics workload:
  • VM cost for 10 DS13v2 instances —100 hours x 10 instances x $0.598/hour = $598
  • DBU cost for Data Analytics workload for 10 DS13v2 instances —100 hours x 10 instances x 2 DBU per node x $0.55/DBU = $1100
  • The total cost would therefore be $598 (VM Cost) + $1100 (DBU Cost) = $1698.
• If you run Premium tier cluster for 100 hours in East US 2 with 10 DS13v2 instances, the billing would be the following for Data Engineering workload:
  • VM cost for 10 DS13v2 instances —100 hours x 10 instances x $0.598/hour = $598
  • DBU cost for Data Engineering workload for 10 DS13v2 instances —100 hours x 10 instances x 2 DBU per node x $0.35/DBU = $700
  • The total cost would therefore be $598 (VM Cost) + $700 (DBU Cost) = $1298.

We charge for the number of minutes your cluster is running rounded to the nearest minute, not hour.

Serverless pools are available on Azure Databricks for the Data Analytics workload only. Serverless pools are available on Standard and Premium tiers. A Serverless pool consists of worker nodes and one driver node. Customers can pick any available VM instance as worker nodes for their Serverless pool. In addition, customers will have to specify the minimum and maximum number of workers for each Serverless pool. All worker nodes in a pool must be the same VM instance. Depending on the load on the Serverless pool, Azure Databricks will automatically scale the number of instances in a pool within the specified minimum and maximum worker nodes. On a per minute basis, customers will be billed for each VM running in the pool and the corresponding DBU’s for those VM instances. In addition, the service will pick a VM instance for one driver node for the Serverless pool. Customers will be billed for the VM and the corresponding DBU charges for the driver node.

Let us say you choose a Premium tier Serverless pool of DS13v2 instances (Azure Databricks charges 2 DBU’s per hour for this VM instance) and you set the minimum and maximum worker nodes to be 2 and 10 respectively. Let us assume this Serverless pool runs for 3 hours. Let us also assume that based on the load, Azure Databricks runs 2 worker nodes for the first hour, 8 worker nodes for the second hour and 3 worker nodes for the third hour, before you shut the Serverless pool down. In addition, Azure Databricks will pick a driver node which will be billed through the life time of the Serverless pool. Let us say Databricks picks D13v2 for this example.

Your bill for the Serverless pool looks as follows:

Currently, Azure Databricks Preview offers premium tier by default. You can downgrade from premium to standard by contacting support via the Azure portal.