FAQ

• What is a DBU?

  A Databricks unit (“DBU”) is a unit of processing capability per hour, billed on per-second usage.

• What's the difference between Data Engineering and Data Analytics workloads?

  The Data Engineering workload is defined as a job that both starts and terminates the cluster on which it runs. For example, a workload may be triggered by the Databricks job scheduler, which launches a new Apache Spark cluster solely for the job and automatically terminates the cluster after the job is complete.

  The Data Analytics workload is any workload that is not an automated workload, for example, running a command within Databricks notebooks. These commands run on Apache Spark clusters that may persist until manually terminated. Multiple users can share a cluster to perform interactive analysis collaboratively.

• Could you give me an example on how billing works?

  Depending on the type of workload your cluster runs, you will either be charged for Data Engineering or Data Analytics workload. For example, if the cluster runs workloads triggered by the Databricks jobs scheduler, you will be charged for the Data Engineering workload. If your cluster runs interactive features such as ad-hoc commands, you will be billed for Data Analytics workload.

  • If you run Premium tier cluster for 100 hours in East 2 with 10 DS13v2 instances, the billing would be the following for Data Analytics workload:
    • VM cost for 10 DS13v2 instances —100 hours x 10 instances x ￥6.92/hour = ￥6,920
    • DBU cost for Data Analytics workload for 10 DS13v2 instances —100 hours x 10 instances x 2 DBU per node x ￥3.498/DBU = ￥6,996
    • The total cost would therefore be ￥6,920 (VM Cost) + ￥6,996 (DBU Cost) = ￥13,916
  • If you run Premium tier cluster for 100 hours in East 2 with 10 DS13v2 instances, the billing would be the following for Data Engineering workload:
    • VM cost for 10 DS13v2 instances —100 hours x 10 instances x ￥6.92/hour = ￥6,920
    • DBU cost for Data Engineering workload for 10 DS13v2 instances —100 hours x 10 instances x 2 DBU per node x ￥1.908/DBU = ￥3,816
    • The total cost would therefore be ￥6,920 (VM Cost) + ￥3,816 (DBU Cost) = ￥10,736
  • If you run Premium tier cluster for 100 hours in East 2 with 10 DS13v2 instances, the billing would be the following for Data Engineering Light workload:
    • VM cost for 10 DS13v2 instances —100 hours x 10 instances x ￥6.92/hour = ￥6,920
    • DBU cost for Data Engineering Light workload for 10 DS13v2 instances —100 hours x 10 instances x 2 DBU per node x ￥0.699/DBU = ￥1,398
    • The total cost would therefore be ￥6,920 (VM Cost) + ￥1,398 (DBU Cost) = ￥8,318

  In addition to VM and DBU charges, you may also be charged for managed disks, public IP address, or any other resource such as Azure Storage, Azure Cosmos DB depending on your application.

• If my cluster ran for less than an hour, how much will I get billed?

  We charge for the number of minutes your cluster is running rounded to the nearest minute, not hour.

• How does Databricks high concurrency work?

  High concurrency pools are available on Azure Databricks for the Data Analytics workload only. High concurrency pools are available on Standard and Premium tiers. A High concurrency pool consists of worker nodes and one driver node. Customers can pick any available VM instance as worker nodes for their high concurrency pool. In addition, customers will have to specify the minimum and maximum number of workers for each high concurrency pool. All worker nodes in a pool must be the same VM instance. Depending on the load on the high concurrency pool, Azure Databricks will automatically scale the number of instances in a pool within the specified minimum and maximum worker nodes. On a per minute basis, customers will be billed for each VM running in the pool and the corresponding DBU’s for those VM instances. In addition, the service will pick a VM instance for one driver node for the high concurrency pool. Customers will be billed for the VM and the corresponding DBU charges for the driver node.

• Can you give me an example of how Databricks high concurrency is billed?

  Let’s say you choose a Premium tier high concurrency pool of DS13v2 instances (Azure Databricks charges 2 DBU’s per hour for this VM instance), and you set the minimum and maximum worker nodes to be 2 and 10 respectively. Let’s assume this high concurrency pool runs for 3 hours. Let’s also assume that based on the load, Azure Databricks runs 2 worker nodes for the first hour, 8 worker nodes for the second hour and 3 worker nodes for the third hour, before you shut the high concurrency pool down. In addition, Azure Databricks will pick a driver node that will be billed through the life time of the high concurrency pool. Let’s say Databricks picks D13v2 for this example.

  Your bill for the high concurrency pool looks as follows:

  HOUR / NODES	NUMBER OF VM NODES	VM CHARGE (DS13V2)	DBU CHARGE (PREMIUM DATA ANALYTICS)	TOTAL
  Hour 1 – Worker Nodes	2	2 nodes x 1 hour x ￥6.92/hour = ￥13.84	2 nodes x 1 hour x 2 DBU’s per node x ￥6.996/hour = ￥27.984	￥41.824
  Hour 2 – Worker Nodes	8	8 nodes x 1 hour x ￥6.92/hour = ￥55.36	8 nodes x 1 hour x 2 DBU’s per node x ￥6.996/hour = ￥111.936	￥167.296
  Hour 3 – Worker Nodes	3	3 个节点 x 1 hour x ￥6.92/hour = ￥20.76	3 nodes x 1 hour x 2 DBU’s per node x ￥6.996/hour = ￥41.976	￥62.736
  Hours 1, 2, 3-Driver Node	1	1 nodes x 3 hour x ￥6.92/hour = ￥20.76	1 nodes x 3 hour x 2 DBU’s per node x ￥6.996/hour = ￥41.976	￥62.736
  			Total	￥334.592

  Your total bill for this high concurrency pool is $47.06. If you ran this set up every day of the month, your monthly bill this high concurrency pool will be ~$1,411.68 (assuming a 30-day month).

• How do I switch between standard and premium tiers?

  To learn more about switch between premium and standard tiers, see ourdocumentation.