2 posts tagged with "stackql-deploy"

Tutorial 2 in this series showed the query-before-mutation pattern by hand. Query live state, apply a policy gate, mutate what's out of policy, verify convergence. Every step was a SQL statement typed into a shell. The pattern held, but the shape was low-level. One resource at a time. One session at a time. No structure for repeating it across environments or holding it right over time.

stackql-deploy takes that pattern and turns it into a framework. Define your resources once as SQL-anchored files, declare them in a manifest, and let the framework run the exists check, the create-or-update, the state check, and the exports as a single build. Same underlying pattern, now something a team can operate at scale.

This tutorial walks through the tool end to end. What it is, how a project is shaped, how a single resource is defined, and how the whole thing runs against a real Google Cloud project.

stackql-deploy 2.0 is a full rewrite in Rust. The Python package (stackql-deploy on PyPi) is archived at 1.9.4. CLI interface and stack file format are unchanged - no migration required.

Why Rust​

The move to Rust was primarily about distribution and operational simplicity. Rust also brings stronger guarantees around performance and memory safety. Running everything in-process without Foreign Function Interface (FFI) boundaries simplifies the architecture while maintaining predictable resource usage.

Embedded Postgres Wire Protocol Server​

The most significant functional change in 2.0 is that stackql-deploy now runs the StackQL engine as an embedded in-process server over a local postgres wire protocol connection rather than shelling out to the StackQL binary as an external process.

There is nothing to start, stop, or configure. The server is lifecycle-managed by stackql-deploy itself and binds to localhost only - no port is exposed on the network, no inbound firewall rules needed in CI.

The previous model spawned a new StackQL process per operation. The embedded server keeps a persistent connection for the duration of a deployment run. For stacks with many resources, the reduction in process spawn overhead is noticeable - particularly on Windows where process creation is expensive.

Additional Features Added​

In addition to added the architectural change to use the embedded server, several other workflow improvements were added including:

• Enabling resource scoped variable exports in /*+ exists */ queries: When an exists query returns a named field (e.g. vpc_id) instead of count, the value is captured as a resource-scoped variable (this.vpc_id) and made available to all subsequent queries for that resource (e.g. statecheck, exports). This eliminates the need for redundant lookups to resolve resource identifiers between query stages.
• Support for capturing RETURNING payloads from DML operations: INSERT, UPDATE, and DELETE statements can include a RETURNING clause. Fields from the response can be mapped to resource-scoped variables via return_vals in the manifest, keyed by operation (create, update, delete). This allows identifiers assigned by the provider during creation to be used immediately without a round-trip query.
• Additional template filters: Including the to_aws_tag_filters filter, which converts global_tags to the AWS Resource Groups Tagging API TagFilters format, and type-preserving YAML-to-JSON serialization that maintains string types through the rendering pipeline.
• Improved logging and exception handling: Enhanced visibility simplifying troubleshooting.

Installation​

curl -L https://get-stackql-deploy.io | tar xzf -

Invoke-WebRequest https://get-stackql-deploy.io -OutFile stackql-deploy.zip
Expand-Archive stackql-deploy.zip -DestinationPath .

jobs:
  deploy:
    runs-on: ubuntu-latest
    env:
      GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}
    steps:
      - uses: actions/checkout@v4
      - uses: stackql/stackql-deploy-action@v2
        with:
          command: 'build'
          stack_dir: 'examples/my-stack'
          stack_env: 'prod'
          env_vars: 'GOOGLE_PROJECT=my-project'

      - name: Deploy Stack
        id: stackql-deploy
        uses: stackql/stackql-deploy-action@v2
        with:
          command: 'build'
          stack_dir: 'examples/my-stack'
          stack_env: 'prod'
          output_file: 'deployment-outputs.json'
          env_vars: 'GOOGLE_PROJECT=my-project'

      - name: Use outputs
        run: |
          echo '${{ steps.stackql-deploy.outputs.deployment_outputs }}' | jq .

cargo install stackql-deploy

Usage​

The CLI interface is unchanged from the Python version:

# deploy a stack
stackql-deploy build my-stack prod \
  --e GOOGLE_PROJECT=${GOOGLE_PROJECT}

# test a stack
stackql-deploy test my-stack prod \
  --e GOOGLE_PROJECT=${GOOGLE_PROJECT}

# tear down a stack
stackql-deploy teardown my-stack prod \
  --e GOOGLE_PROJECT=${GOOGLE_PROJECT}

# dry run
stackql-deploy build my-stack prod \
  --e GOOGLE_PROJECT=${GOOGLE_PROJECT} \
  --dry-run

Stack files and stackql_manifest.yml structure are unaffected - no migration work needed.

Python Package Deprecation​

stackql-deploy 1.9.4 on PyPi is the final Python release. The Python source repository is archived. If you have pip install stackql-deploy in any scripts or CI pipelines, replace it with one of the install methods above. The 1.9.4 package remains on PyPi and installable, but will not receive updates.

Links​

• GitHub: github.com/stackql/stackql-deploy
• crates.io: crates.io/crates/stackql-deploy
• StackQL docs: stackql.io/docs