Insight Type Catalog

Use this catalog to understand why a finding appears and what to do next. Each table lists the finding by the name the Insights page shows. The identifier below the name is the type value in the API.

The thresholds are defaults. A finding also needs enough usage, billing, and utilization history, so a new workspace can take days to produce some findings. For the review flow, see Insights & Actions. For automated execution, see Action Plans.

Cost and reliability anomalies

Finding	When it appears	What to do
Cost Spike cost_spike	The average cost of a job or pipeline over the last 48 hours is more than 2.5× its 30-day average, or is a statistical outlier against it. The baseline average must be at least $10 and the increase at least $50.	Compare inputs, run frequency, configuration, and attribution changes before you change capacity.
Duration Spike duration_anomaly	The average run duration over the last 48 hours is a statistical outlier against the 30-day baseline.	Look for slower stages, data skew, data growth, and configuration changes.
Failure Spike failure_rate_spike	The failure rate over the last 48 hours is more than 15 percentage points above the 7-day baseline.	Inspect the failed runs and upstream dependencies before you retry.
Warehouse Spike warehouse_spend_spike	A SQL warehouse is among the 10 largest cost changes over 30 days, with an increase of at least $1,000.	Compare query volume, runtime, and warehouse configuration with the prior period.
Serving Spike serving_spike	A serving endpoint cost at least $500 more in the last 7 days than in the 7 days before. A new endpoint also counts.	Inspect request volume, endpoint configuration, and model changes.
AI Spend Spike ai_workspace_spend_spike	The daily AI and Genie spend of a workspace is at least $50, at least 2.5× the average of the previous 7 days, and at least $50 above it.	Check which AI product drives the spike, then inspect the usage of that product.
App Cost Spike app_cost_spike	A Databricks App cost at least 50% and $50 more in its latest 7 billed days than its weekly average over the previous 28 days. App billing can arrive late, so the window can end a few days before today.	Compare traffic, uptime, resources, and off-hours usage.
Budget Risk budget_risk	The projected spend exceeds a budget by more than 10%. The check starts 7 days into the budget period.	Review the projection, the budget scope, and the largest cost changes.
Attribution Drop attribution_declining	The unattributed share of cost rose by at least 5 percentage points week over week.	Review mappings and missing resource metadata.

Cluster waste

Finding	When it appears	What to do
Idle Cluster idle_cluster	A running cluster has had no job or task activity for at least 2 hours.	Check for interactive use, then terminate the cluster. LakeSentry can terminate it after approval.
Overprovisioned overprovisioned_cluster	The 95th-percentile CPU and memory utilization both stayed below 50% over the last 7 days. Short peaks can be higher.	Evaluate a smaller configuration and test it at peak demand.
Long Auto-Term auto_terminate_too_long	The auto-termination timeout is more than 30 minutes longer than the usual gap between runs on the cluster.	Reduce the timeout to the suggested value.
Fat Driver fat_driver	A cluster with a large driver node has average CPU below 10% or average memory below 30% over the last 7 days.	Evaluate a smaller driver. Consider driver memory needs and peak load.
Long-Running Cluster long_running_cluster	A cluster ran at least 120 hours in the last 7 days.	Check whether it must run continuously. Use auto-termination or job compute where it fits.

Cluster configuration and sizing

Most configuration findings cover clusters that ran in the last 30 days or were created in the last 7 days. Single-Node and the worker-sizing findings use their own utilization checks. A cluster carries one worker-sizing finding at a time: Excess Workers, Enable Autoscaling, Tighten Autoscale Range, or Lower Pipeline Minimum.

Finding	When it appears	What to do
Long Auto-Term long_auto_termination	An interactive cluster has auto-termination disabled or set above 120 minutes.	Set 120 minutes, or a value that fits the workload.
Spot Candidate spot_instance_candidate	A cluster runs its workers on on-demand instances.	Evaluate spot workers with on-demand fallback for interruption-tolerant workloads. Keep the driver on demand.
Single-Node single_node_candidate	A cluster has exactly one worker. LakeSentry suggests a conversion when the average combined driver and worker load is at most 85% CPU and 90% memory. With a higher average, the finding is informational and advises to keep the worker.	Convert only when the finding suggests it, and test the workload first. Single-node mode does not support distributed operations.
Excess Workers overprovisioned_workers	A fixed-size cluster steadily needed fewer workers than configured over the last 7 days. The finding can also say that a reduction is not suggested, or that the measurements are insufficient.	Reduce workers only when the finding suggests it. Review the workload manually when the measurements are insufficient.
Enable Autoscaling autoscaling_candidate	The daily worker demand of a fixed-size cluster varied by at least 2 workers over the last 14 days, and its low demand is below the configured worker count.	Enable autoscaling with the suggested minimum. Keep the maximum at the current worker count.
Tighten Autoscale Range autoscale_range_too_wide	Over the last 14 days, the minimum of an autoscaling cluster was above its low demand. A maximum at least 4 workers above peak demand also counts, but only when the cluster never scaled above its minimum.	Lower the minimum when the finding suggests it. LakeSentry suggests a lower maximum only when the cluster stayed at its minimum without CPU or memory pressure.
Lower Pipeline Minimum continuous_pipeline_overprovisioned	The minimum worker count of a continuous pipeline was above its demand over the last 14 days.	Evaluate a lower minimum with enhanced autoscaling.
Outdated Runtime outdated_runtime	A cluster runs a Databricks Runtime that is behind the latest LTS, out of support, a preview, or not recognized.	Follow the recommendation in the finding. Test runtime upgrades in development first.
Photon Candidate photon_candidate	An all-purpose or job cluster does not use Photon.	Benchmark Photon for the workload. The benefit is not guaranteed.
Interactive Misuse interactive_cluster_misuse	An interactive cluster runs jobs: at least 2 different jobs, or jobs on at least 3 days.	Evaluate job compute for these workloads.
Pool Candidate pool_candidate	A job or pipeline cluster without an instance pool ran at least 5 times in 7 days.	Evaluate an instance pool to cut startup time. Account for the cost of idle pool instances.
Type Mismatch instance_type_mismatch	A memory-optimized cluster averaged below 40% memory, or a compute-optimized cluster averaged below 40% CPU. A storage-optimized cluster counts when both averages are below 40%.	Evaluate a general-purpose instance type. Check both CPU and memory needs first.
First On-Demand missing_first_on_demand	A cluster on spot instances has no on-demand first node, so the cloud provider can reclaim the driver.	Keep the driver on an on-demand instance. Follow the recommendation for your cloud provider.

Jobs and pipelines

Finding	When it appears	What to do
Runaway Job runaway_job	A running job has run more than 6× its average successful duration.	Investigate the run and cancel it if appropriate. LakeSentry can cancel it after approval.
Retry Storm retry_storm	A job failed at least 5 times within one hour.	Fix the underlying failure. Consider pausing the schedule during the repair.
Missing Timeout missing_job_timeout	A job that runs regularly has no timeout that covers all its tasks.	Set a timeout based on the observed duration.
Dev Mode pipeline_dev_mode	A pipeline runs in development mode.	Use production mode for production pipelines.
Preview Channel pipeline_preview_channel	A pipeline uses the PREVIEW channel.	Use the CURRENT channel unless you need preview features.
Multi-Cluster multi_cluster_job	A job used several clusters in at least 3 runs over the last 14 days.	Evaluate a shared job cluster. Keep the isolation that the tasks need.

SQL warehouses

Finding	When it appears	What to do
Idle Warehouse idle_warehouse	A warehouse ran for at least 60 minutes in the last 24 hours with no queries.	Configure auto-stop or reduce the minimum cluster count.
Failing Queries failing_queries	At least 20% of the queries on a warehouse failed over the last 7 days.	Investigate the common errors before you add capacity.
Warehouse Sizing warehouse_sizing	A warehouse averaged fewer than 10 queries per day over the last 7 days, and a smaller size fits.	Evaluate the suggested size against your latency and concurrency needs.
Cold Starts warehouse_cold_starts	More than 30% of the queries waited for the warehouse to start.	Review the wait time and the auto-stop setting. A longer auto-stop lowers latency and raises cost.
Long Auto-Stop warehouse_long_auto_stop	A warehouse has auto-stop disabled, or set above 10 minutes (serverless) or 30 minutes (classic and pro).	Shorten the auto-stop. Weigh it against cold-start latency.
Long-Running Queries long_running_queries	The daily 99th-percentile query duration on a warehouse averaged at least 5 minutes over the last 7 days.	Investigate the slow query patterns before you resize the warehouse.

Tables and queries

These findings rely on Databricks lineage and query history. Check that access tracking covers a table before you archive or delete it.

Finding	When it appears	What to do
Unused Table unused_table	A table has had no recorded use for at least 30 days.	Verify dependencies and retention requirements before you archive or delete it.
Excessive Retention excessive_retention	A table has had no reads for more than 90 days but still receives writes, or no access at all for more than 180 days.	Review retention requirements before you archive it.
Poor Read/Write Ratio poor_read_write_ratio	A table receives writes but no reads, or more than 5 writes per read.	Review its consumers and its refresh frequency.
Duplicate Dataset duplicate_dataset	The matching column names of two tables make up at least 80% of all distinct column names across both tables.	Verify the data and the owners before you consolidate. Similar schemas do not prove duplicate data.
Poor Pruning poor_pruning	Queries on a table skipped fewer than 50% of its files over the last 7 days.	Inspect the query filters and the table layout. Evaluate partitioning or clustering.
Non-Partitioned Table non_partitioned_table	Queries on a table skipped fewer than 20% of its files over the last 14 days. This can mean that the table layout does not match the query filters.	Review the table layout and the query filters. Evaluate clustering or partitioning.
Scanzilla scanzilla	A repeated query scans at least 100 GiB and returns fewer than 1,000 rows.	Inspect full scans, filters, and selected columns. Reduce unnecessary reads.
Unoptimized Storage unoptimized_storage	Predictive optimization ran at least 10 operations on a table, at a cost of at least $5, in the last 30 days.	Review the OPTIMIZE and VACUUM activity and the write patterns. Maintenance cost is not always avoidable.

Model serving, Apps, and workspaces

Finding	When it appears	What to do
Zombie Endpoint zombie_model	A serving endpoint that used to receive requests has had none for at least 30 days, and cost at least $100.	Check that request tracking is complete. Then consider stopping the endpoint.
App Stop Schedule app_stop_schedule	A Databricks App runs outside business hours. Off-hours make up at least 20% and $10 of its cost over 28 days. Without configured business hours, LakeSentry assumes 12 business hours per weekday in UTC.	Check the business hours against the hours you need the App. Stop it outside those hours.
Weekend Waste weekend_waste	A non-production workspace spends at least 1.5× as much per day on weekends as on weekdays, with at least $100 of weekend Databricks cost.	Review weekend schedules and work that must run continuously.

Lakebase

Finding	When it appears	What to do
Lakebase Always-On lakebase_always_on_compute	An endpoint runs almost around the clock with distinct busy and quiet periods, and at least 30% of its cost comes from the quiet periods.	Check Scale to zero. Remove keep-alive traffic where appropriate.
Lakebase Zombie Storage lakebase_zombie_storage	A branch costs at least $10 per month in storage and has had no billed compute linked to it for 30 days.	Check that compute billing links to the right branch. Confirm that the branch is abandoned, then export or archive it before you delete it.
Lakebase History Retention lakebase_branch_history_bloat	History makes up at least 60% of the recent storage of a project and costs at least $10 per month.	Review the History window against your restore needs. The finding shows the history cost, not a savings estimate.
Lakebase Storage Growth lakebase_storage_growth	The storage of a branch grew by at least 25% in each of the last two weeks.	Investigate the growth and the data lifecycle before the cost compounds.
Lakebase Spend Spike lakebase_spend_spike	The weekly spend of a project is at least $50, and at least double and $50 above its usual week. New spend also counts.	Review the compute and storage changes.