Databricks Compute and Cluster Cost Analysis
The Clusters page tracks Databricks compute clusters. SQL warehouses are covered on SQL Analysis; jobs and pipelines are covered on Work Units.
All-purpose
Section titled “All-purpose”Interactive and shared clusters. Use this tab to find long-running clusters, low-utilization clusters, user ownership, DBR versions, and clusters that may be better handled as jobs or SQL warehouses.
Job clusters
Section titled “Job clusters”Ephemeral or job-owned clusters used by Databricks Jobs and pipelines. Use this tab to compare cost per run, worker configuration, runtime versions, and workload ownership.
Optimization
Section titled “Optimization”A focused view of cluster optimization opportunities:
- Sizing Recommendations: the active worker-sizing findings, with a link to each insight.
- Oversized Drivers: clusters with a large, underused driver node.
- Weekend Spend Analysis: non-production workspaces with high weekend spend compared to weekdays.
All other cluster findings, such as spot candidates, outdated runtimes, and long auto-termination, appear in Insights. See the catalog.
Cluster list
Section titled “Cluster list”Rows include cost, workspace, cluster type, state, owner, worker configuration, DBR version, CPU utilization, memory utilization, and linked insights. Search and filters help isolate workspace, cluster type, owner, and status.
Cluster detail view
Section titled “Cluster detail view”Open a cluster to review:
- Cost and usage during the selected time range.
- Utilization trends from node timeline data.
- Associated jobs or parent work units when known.
- Recent runs and activity signals.
- Related insights and recommended actions.
Configuration history and cost changes
Section titled “Configuration history and cost changes”Open a cluster’s detail view and review Configuration history alongside the cost chart. The table shows recorded configurations overlapping the selected period: worker and driver types, worker count, autoscale bounds, runtime, and auto-termination. It also compares displayed fields with the previous recorded version when available.
- Effective periods are shown in UTC.
- Not recorded and gaps mean the historical value is unknown; the current configuration is not substituted for it.
- Timing unverified means the version does not have a verified Databricks change timestamp.
- Dashed cost-chart markers identify recorded changes with verified timing. Multiple changes on the same UTC day share a marker; inspect the history table for individual changes.
- For a change near a cost spike, compare workload volume and utilization over the same period.
If no history is recorded, check the selected range and extraction of system.compute.clusters. If history fails to load, select Retry in the history card.
How utilization is computed
Section titled “How utilization is computed”LakeSentry uses Databricks system.compute.node_timeline samples. Worker and driver utilization are aggregated over the selected time range, then compared with cost and runtime. Optimization insights require enough observations to avoid recommending changes from a short or inactive sample.
Cost model
Section titled “Cost model”Cluster cost combines Databricks DBU usage from billing data with cloud infrastructure cost when node price information is available. You can switch cost display mode globally to view Databricks cost, cloud cost, or total cost depending on your plan and configuration.