Connect Databricks to LakeSentry

LakeSentry needs read access to Databricks billing, compute, query, Lakeflow, serving, storage, and governance metadata. It does not read your business table data.

Connection architecture

A LakeSentry connector uses a workspace URL, Databricks credentials, and a SQL warehouse to validate access and extract system-table data. Configure connectors from Settings → Connector.

Extraction can run in these modes:

Mode	Description
Direct Connection	Current self-service path. LakeSentry pulls data through Databricks APIs and the SQL Statement API.
External Connector	Controlled deployment path for private-network or customer-managed extraction. The setup UI currently marks this option as coming soon.

Prerequisites

• Databricks account admin or equivalent help from an admin.
• A service principal for LakeSentry. OAuth M2M is preferred when available; PAT credentials are also supported.
• Unity Catalog and Databricks system tables enabled for the workspaces/regions you want to monitor.
• Permission to grant CAN USE on a SQL warehouse, USE CATALOG on system, USE SCHEMA on required system schemas, and SELECT on required system tables.

Required system table access

Grant read access to the system tables you want LakeSentry to ingest. Current direct extraction uses tables from these schemas:

• system.billing
• system.compute
• system.lakeflow
• system.query
• system.access
• system.information_schema
• system.serving
• system.storage

Some tables may not exist in every cloud, region, workspace, or Databricks SKU. Missing optional tables reduce feature coverage but should not block core billing ingestion.

MLflow source-table extraction is planned, but is not part of the current default direct-extraction registry. The LakeSentry Audit Log feature uses LakeSentry’s own internal audit trail and does not require access to Databricks system.access.audit.

Setup workflow

1. Create or identify the Databricks service principal.
2. Grant account/workspace API permissions needed for discovery and enrichment.
3. Grant SELECT on required system tables.
4. In LakeSentry, open Settings → Connector and enter account credentials.
5. Choose Direct Connection.
6. Run validation and wait for the first extraction/transform cycle.
7. Add additional connectors for other Databricks regions when needed.

Verifying connection health

After setup, check:

• Connector credential validation status.
• Data Sync schedule, last run, and ingestion status.
• Extraction run errors and table-level coverage.
• Data freshness on dashboards.
• Unattributed or missing data warnings in Cost Allocation.