LakeSentry Collector Issues for Databricks
This guide applies to support-enabled External Connector deployments. For Direct Connection, start with Data Freshness and Settings → Connector → Data Sync.
Is the collector the problem?
Section titled “Is the collector the problem?”Compare the latest Databricks collector job run with the connector’s ingestion history. Use the table below to choose the next check.
| Symptom | First check | Next action |
|---|---|---|
| No job run | Job schedule and cluster startup | Re-enable the intended schedule or resolve startup errors |
| Job starts but cannot authenticate | Exact error, configured tenant API endpoint, and token rotation history | Reconfigure against the correct connector; see connection errors |
| Specific tables missing | Table errors and source availability | Verify grants and table availability; see extraction errors |
| Timeout or DNS failure | Connectivity from the job’s actual runtime | Check the configured endpoint and network/proxy settings |
| Push succeeds but dashboard is empty | Date range, filters, and Ledger Load | Follow processing diagnosis |
Permission problems
Section titled “Permission problems”Missing rows alone do not prove a permission failure: a table can be empty, unavailable in the region, or outside the extraction window. Check the exact job error and the permission checklist. After restoring access, run the collector again and verify the affected table result.
Network and connectivity
Section titled “Network and connectivity”Use the API URL from the collector’s configuration, which points to the intended LakeSentry tenant. Do not substitute a generic hostname. Check DNS and outbound HTTPS from the Databricks job runtime; a connection from an administrator’s laptop does not prove the job has access.
See network connectivity for the administrator checklist. A reachable health endpoint does not validate collector authentication or table access.
Collector job not running
Section titled “Collector job not running”Open the collector job in Databricks and inspect the latest run, task output, schedule, and compute startup events. Confirm that its configured wheel and environment file are accessible to the job identity. Prefer a single concurrent run to avoid unnecessary overlapping extraction.
Extraction failures
Section titled “Extraction failures”Use the administrator error reference to distinguish missing grants, unavailable tables, and query failures. Preserve the table name and exact error. A successful run with skipped tables can still leave a feature without data.
Connection string issues
Section titled “Connection string issues”If the token was rotated or the connector was recreated, update the collector configuration using the new connection string. Follow Collector Deployment and verify a single run before resuming the schedule. Never send the connection string to support in logs or screenshots.
Performance concerns
Section titled “Performance concerns”Initial history loads and deliberate backfills can take longer than incremental runs. Duration depends on the queried tables, source volume, SQL warehouse, network, and compute startup. Compare table durations and warehouse queries with previous runs rather than relying on a fixed account-size estimate.
The collector consumes Databricks compute and SQL resources. Choose a schedule after measuring runtime and cost. Source publication delays still apply even with frequent runs; see Data Freshness.
Escalation checklist
Section titled “Escalation checklist”Include the general support details, plus:
- Databricks job and run IDs, wheel version, and sanitized task output.
- Connector identifier and last successful ingestion time, with timezone.
- Affected tables and whether they failed, were skipped, or returned no rows.
- Recent changes to credentials, permissions, network, schedule, or job configuration.
The administrator guide owns the detailed error, log, and checkpoint reference.