2 posts tagged with "sql"

Tutorial 2 in this series showed the query-before-mutation pattern by hand. Query live state, apply a policy gate, mutate what's out of policy, verify convergence. Every step was a SQL statement typed into a shell. The pattern held, but the shape was low-level. One resource at a time. One session at a time. No structure for repeating it across environments or holding it right over time.

stackql-deploy takes that pattern and turns it into a framework. Define your resources once as SQL-anchored files, declare them in a manifest, and let the framework run the exists check, the create-or-update, the state check, and the exports as a single build. Same underlying pattern, now something a team can operate at scale.

This tutorial walks through the tool end to end. What it is, how a project is shaped, how a single resource is defined, and how the whole thing runs against a real Google Cloud project.

If you have access to a Databricks workspace, you can run StackQL queries directly from the Databricks Web Terminal using your Databricks identity.

How It Works​

Download the latest release of stackql, then run the convenience script included (similar scripts are included for other cloud provider terminals - e.g. AWS Cloud Shell).

curl -L https://bit.ly/stackql-zip -O && unzip stackql-zip
sh stackql-databricks-shell.sh

Example Queries​

Here are the sample queries run in the video, just change the deployment_name for your workspace.

User entitlements

SELECT
  deployment_name,
  id,
  userName,
  displayName,
  entitlement
FROM databricks_workspace.iam.vw_user_entitlements
WHERE deployment_name = 'dbc-74aa95f7-8c7e';

All workspace settings

SELECT * FROM
  databricks_workspace.settings.vw_all_settings
WHERE deployment_name = 'dbc-74aa95f7-8c7e';

Tag policies filtered by key prefix

SELECT
  tag_key as key,
  description
FROM databricks_workspace.tags.tag_policies
WHERE deployment_name = 'dbc-74aa95f7-8c7e'
AND tag_key LIKE 'class%';

Catalog count by type

SELECT
  catalog_type,
  COUNT(*) as num_catalogs
FROM databricks_workspace.catalog.catalogs
WHERE deployment_name = 'dbc-74aa95f7-8c7e'
GROUP BY catalog_type;

Provider Coverage​

The databricks_workspace provider covers workspace related services, the databricks_account provider covers account-level operations including provisioning, billing, and account IAM.

The web terminal flow covers workspace-scoped queries using the token of the logged-in user. For account-level queries (provisioning, billing, account IAM), you need a Databricks service principal with account admin rights and OAuth2 credentials:

export DATABRICKS_ACCOUNT_ID="your-account-id"
export DATABRICKS_CLIENT_ID="your-client-id"
export DATABRICKS_CLIENT_SECRET="your-client-secret"

These are the same variables used by the Databricks CLI and Terraform provider, so if you already have those configured the auth story is identical.

Get Started​

Full provider documentation:

• Databricks Workspace Provider for StackQL
• Databricks Account Provider for StackQL

Visti StackQL on GitHub.