Skip to content
LakeSentry LakeSentry LakeSentry Docs

LakeSentry Quick Start for Databricks Cost Monitoring

Prerequisites

Section titled “Prerequisites”

Before starting, make sure you have:

  • Access to your LakeSentry tenant.
  • Databricks account/admin support for service-principal creation and grants.
  • Unity Catalog system tables enabled in the regions you want to monitor.
  • A Databricks workspace URL, a SQL warehouse the LakeSentry service principal can use, and either OAuth M2M credentials or a PAT.

Step 1: Sign in

Section titled “Step 1: Sign in”

Open your LakeSentry tenant URL and sign in. New users may arrive through an invitation or magic-link verification flow. If you do not have an account yet, sign up for free at app.lakesentry.io — see pricing for plan details.

Step 2: Connect Databricks

Section titled “Step 2: Connect Databricks”

Go to Settings → Connector and configure the Databricks connector. OAuth M2M service-principal credentials are preferred; PAT credentials are supported.

See Connecting Your Databricks Account for the full permission checklist.

Step 3: Add additional connectors if needed

Section titled “Step 3: Add additional connectors if needed”

Add another connector for each Databricks region you want to monitor. System tables are regional, so a workspace in eastus does not provide full metadata for westeurope.

Direct Connection is the current self-service setup path. It lets LakeSentry query Databricks system tables through the configured credentials and SQL warehouse. External Connector/collector deployments are reserved for controlled deployments and are marked as coming soon in the setup UI.

Step 4: Wait for initial data

Section titled “Step 4: Wait for initial data”

The first run extracts historical data, ingests it into raw tables, transforms it into the ledger, computes metrics, and generates insights. Large accounts can take longer during the first backfill.

Use Settings → Connector to trigger an immediate sync, review extraction runs, or change the Data Sync schedule. Direct connectors can be paused or scheduled hourly, every 4 hours, or daily at 8:00 AM UTC. Check Data Freshness if dashboards remain empty.

Step 5: Build mappings

Section titled “Step 5: Build mappings”

Open Mappings and map high-spend users and service principals to teams. Then review Cost Allocation to find unattributed spend and add rules where needed.

Step 6: Investigate and optimize

Section titled “Step 6: Investigate and optimize”

Start with: