Databricks Compute and Cluster Cost Analysis
The Clusters page tracks Databricks compute clusters. SQL warehouses are covered on SQL Analysis; jobs and pipelines are covered on Work Units.
All-purpose
Section titled “All-purpose”Interactive and shared clusters. Use this tab to find long-running clusters, low-utilization clusters, user ownership, DBR versions, and clusters that may be better handled as jobs or SQL warehouses.
Job clusters
Section titled “Job clusters”Ephemeral or job-owned clusters used by Databricks Jobs and pipelines. Use this tab to compare cost per run, worker configuration, runtime versions, and workload ownership.
Optimization
Section titled “Optimization”A focused view of cluster hygiene opportunities, including:
- Long auto-termination settings.
- Spot instance candidates.
- Single-node candidates for low-parallelism workloads.
- Overprovisioned workers based on observed CPU and memory utilization.
- Outdated Databricks Runtime versions.
- Missing job timeouts or configuration guardrails.
- Interactive cluster misuse by scheduled workloads.
- Pool candidates and instance-type mismatch.
Cluster list
Section titled “Cluster list”Rows include cost, workspace, cluster type, state, owner, worker configuration, DBR version, CPU utilization, memory utilization, and linked insights. Search and filters help isolate workspace, cluster type, owner, and status.
Cluster detail view
Section titled “Cluster detail view”Open a cluster to review:
- Cost and usage during the selected time range.
- Utilization trends from node timeline data.
- Associated jobs or parent work units when known.
- Recent runs and activity signals.
- Related insights and recommended actions.
How utilization is computed
Section titled “How utilization is computed”LakeSentry uses Databricks system.compute.node_timeline samples. Worker and driver utilization are aggregated over the selected time range, then compared with cost and runtime. Optimization insights require enough observations to avoid recommending changes from a short or inactive sample.
Cost model
Section titled “Cost model”Cluster cost combines Databricks DBU usage from billing data with cloud infrastructure cost when node price information is available. You can switch cost display mode globally to view Databricks cost, cloud cost, or total cost depending on your plan and configuration.