Skip to main content

11 posts tagged with "aws"

View All Tags

AWS Provider Update - August 2026

· 4 min read
Technologist and Cloud Consultant

We've released an update to the StackQL AWS provider, regenerated from the latest AWS service definitions. Significant additions include:

  • 9 new services
  • Over 200 new resources
  • Over 500 new operations
  • Full support for S3 object level CRUD operations

New Services

Nine services are new in this release:

ServiceDescription
resiliencehubv2The next generation of AWS Resilience Hub - assess and improve the resilience of critical applications at scale, the largest new service in this release
lambda_microvmsCreate, manage and operate AWS Lambda MicroVMs and their associated MicroVM image environments
lambda_coreShared infrastructure resources for Lambda, including network connectors that give MicroVMs access to resources in your VPC
agent_registry_controlManaged catalog for publishing and discovering MCP servers, agents and agent skills - control plane for registries and records
agent_registryData-plane discovery of approved records published to an Agent Registry
account_accessAccount access manager - manage applications and entitlements that grant IAM Identity Center principals access to IAM roles across accounts
supportauthzSupport authorization - cryptographically signed support permits controlling which actions AWS support operators can perform on your resources
pricing_plan_managerFlat-rate pricing subscriptions - create, approve and cancel subscriptions and associate resources with them
sagemakerjobruntimeAgentic RFT runtime - trajectory and transition data for reinforcement fine-tuning jobs

Expanded Coverage in Existing Services

  • ec2 - the largest operation count increase in this release: account-level VPC encryption controls (account_vpc_encryption_controls), application status checks, capacity reservation cancellation quotes, and a major IPAM build-out covering internet registry associations, route origin authorizations, route protection findings and discovered routes
  • acm - public ACME issuance: acme_accounts, acme_endpoints, acme_domain_validations and external account bindings, plus per-domain certificate validation status
  • quicksight - agentic BI: agents, spaces, knowledge_bases, approval policies, DLP settings and OAuth client applications
  • bedrock_agentcore_control - datasets, dataset versions and examples, evaluation harness_endpoints and versions, capacity providers and gateway rate limits
  • wellarchitected - AI-assisted reviews: agent profiles, goals, contexts and agent_recommendations with per-item detail
  • odb (Oracle Database@AWS) - autonomous database coverage: autonomous_databases, backups, clones, peers, wallet details, Exadata VM clusters and Exascale storage vaults
  • iotsitewise - industrial data pipelines: pipelines, pipeline executions, enrichment jobs, dataset export jobs, ad hoc queries and search
  • glue - business catalog additions: assets, asset_types, glossaries and glossary_terms
  • drs - orchestrated disaster recovery: recovery_plans, plan steps and execution tracking
  • billing - credits, credit allocation histories, billing preferences and enterprise support charge summaries
  • appconfig - feature experiments: experiment_definitions, runs and run events
  • healthlake - FHIR data transformation profiles and jobs

Another 180+ services picked up incremental operations and resources, including connect, securityagent, cleanrooms, socialmessaging, devops_agent and artifact.

S3 Object Content Lifecycles

The headline feature in this release: aws.s3.objects now supports the full content lifecycle for text objects. The object body is projected as a single contents column, so you can read, write, overwrite and delete object contents with standard SQL verbs - no SDK code, no presigned URLs.

The motivating case is reading a Terraform state file straight out of S3:

SELECT contents FROM aws.s3.objects
WHERE region = 'ap-southeast-2'
AND bucket = 'my-bucket'
AND key = 'env/terraform.tfstate';

Writing works through the same resource. INSERT creates an object, and since S3 PutObject is create-or-overwrite, REPLACE updates it in place:

-- create an object
INSERT INTO aws.s3.objects(region, bucket, key, contents)
SELECT 'ap-southeast-2', 'my-bucket', 'app/config.json',
'{"feature_flags": {"dark_mode": true}, "log_level": "info"}';

-- overwrite its contents
REPLACE aws.s3.objects
SET contents = '{"feature_flags": {"dark_mode": false}, "log_level": "warn"}'
WHERE region = 'ap-southeast-2'
AND bucket = 'my-bucket'
AND key = 'app/config.json';

-- delete it
DELETE FROM aws.s3.objects
WHERE region = 'ap-southeast-2'
AND bucket = 'my-bucket'
AND key = 'app/config.json';

Listing is unchanged - a WHERE clause with just bucket and region routes to the list operation, adding key routes to the object read:

SELECT key, size, last_modified FROM aws.s3.objects
WHERE region = 'ap-southeast-2' AND bucket = 'my-bucket';

Content round-trips byte-for-byte, including multi-MB objects and objects uploaded via multipart upload. Text objects only for now - binary content is not supported and base64 support is deferred.

Get Started

Pull the latest provider from the public registry:

stackql registry pull aws;

Authenticate with AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, then explore:

SELECT vpc_id, state, cidr_block, is_default
FROM aws.ec2.vpcs
WHERE region = 'us-east-1';

Provider docs, including required parameters and example queries for every resource, are at aws-provider.stackql.io. Visit us on GitHub and let us know how you're using it.

Auditing three clouds without writing three scripts

· 6 min read
Cloud Consultant and AWS Community Builder

If you've audited buckets in more than one cloud, you already know what this is like. Three consoles. Three CLIs. Three auth patterns. And a bit of glue code to pull the outputs together every time someone asks a question that spans all three.

The question is usually simple. "Which buckets are public across all our clouds?" You end up running three separate scripts, then reconciling three different output shapes just to give one answer.

This tutorial doesn't make that faster. It replaces it. One query, one view, all three clouds.

What StackQL is, in three sentences

StackQL is SQL for cloud APIs. You write a normal SQL query, and it makes the API calls to AWS, GCP, Azure, or dozens of other providers to get the answer back as rows and columns.

The important part for this tutorial: it queries the live provider APIs on every run. There's no local database, no cache, no sync job to schedule. What you see is the current state of your cloud, right now.

It exists because someone got tired of writing the same audit logic three times.

What you'll need

You'll need Docker installed and read-only credentials for whichever clouds you want to audit. All three are optional. If you leave a cloud's credentials blank, it gets skipped.

For AWS, GCP, and Azure, the audit needs a service account or role that can list storage buckets and read their configuration. The audit-action repo has the exact permissions per provider here.

Run the audit

The tutorial folder has three files: a docker-compose file, an .env.audit.example template, and a README. Grab the folder from docs/tutorials/preview-bucket-01/ in the stackql/stackql repo.

1. Copy the env template

cp ./audit/.env.audit.example ./audit/.env.audit

Then open .env.audit and fill in credentials for whichever clouds you want. The variables are named for what they are: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, GOOGLE_CREDENTIALS (the service account JSON, on one line), and GOOGLE_ORG_ID.

If you're only auditing one or two clouds, leave the others blank.

2. Pull the StackQL Docker image

docker compose -f docker-compose.bucket.audit.yaml pull

The tutorial pins to stackql/stackql:v0.10.601. The audit uses views under stackql_preview.* which are still evolving, so pinning matters for reproducibility.

3. Run the audit

docker compose -f docker-compose.bucket.audit.yaml run --rm stackql

On a modest account this takes about 30 seconds. Output goes straight to your terminal as a table.

What the output tells you

When the audit finishes, you get a single table printed to your terminal. Something like this (bucket names are placeholders):

Terminal screenshot showing StackQL audit output as a table with columns for provider, bucket name, encryption class, public flag, and HTTPS enforcement. Rows include AWS, GCP, and Azure buckets with a mix of provider-managed and customer-managed encryption, and one public bucket flagged per AWS and GCP.

Example output from the audit. Bucket configuration across three cloud providers in one normalised table.

The point isn't the specific findings. It's the shape.

Three different resource types (S3 buckets, GCS buckets, and Azure Storage Accounts) normalised into one table with the same columns. Encryption class, public flag, HTTPS enforcement. Side by side, for every cloud you provided credentials for.

A few patterns to notice in the table:

  • Public buckets across providers. Filter by public = true and you have the answer to your security team's question, in every cloud, in one place.
  • Encryption side by side. The encryption_class column shows which buckets use provider-managed keys versus customer-managed. Different providers have different naming conventions internally; StackQL normalises them to the same two categories.
  • HTTPS enforcement in one column. Whether it's S3 bucket policies, GCS uniform access, or Azure secure transfer, same column, same values, no translation needed.

This is what SQL for cloud APIs actually delivers. Not just SQL as a query language, but a uniform data model across providers, so you can ask one question and get one answer.

Performance and scope

StackQL runs each provider's query in parallel using the credentials you provided. There's no local database being populated and no background sync; every run hits the live APIs and returns what's there right now.

On the test account this took about 30 seconds. The query is scoped to one AWS region and one GCP organization, which keeps the API footprint bounded and reflects the current preview default. Streaming output and broader audits (across regions, deeper across resource types) are in flight from the StackQL team.

What this covers today

Today, this audit covers storage buckets across AWS, GCP, and Azure. One region at a time for AWS, one organization at a time for GCP. Entitlements, IAM, and other resource types are on the roadmap as separate audits under stackql_preview.* and will get their own tutorials as they land.

Where to go next

github.com/stackql/stackql is the flagship repo. If this tutorial was worth reading, give it a star. It's genuinely the thing that tells the team to build more tutorials like this one, and it helps other engineers with the same three-console problem find the project.

A few natural next steps from here:

  • Run it against your own accounts. The audit works exactly the same way against real credentials. You can start with one cloud and add the others by adding their env vars.
  • Move it into CI. The stackql-audit-action repo has ready-to-use GitHub Actions workflows for all-clouds, single-cloud, deep-audit, and OIDC-authenticated variants. Drop one into your repo, add secrets, and you have a scheduled cross-cloud audit running on every push.
  • Ask questions. The StackQL community Slack is the fastest way. GitHub issues on the flagship repo also work.

More audits are on the way, including entitlements, deeper checks, and larger estates. If there's a specific one you'd find useful, opening an issue is the fastest way to get it on the roadmap.

New Dedicated AWS Cloud Control Provider Released

· 2 min read
Technologist and Cloud Consultant

We've released a new dedicated StackQL AWS Cloud Control provider, providing full CRUDL operations across AWS services via the Cloud Control API including purpose-built resource definitions leveraging Cloud Control's consistent schema.

Resource Naming Convention

Resources follow a clear pattern to differentiate operations:

Resource PatternOperationsUse Case
{resource} (e.g., s3.buckets)SELECT, INSERT, UPDATE, DELETEFull CRUD with complete resource properties
{resource}_list_only (e.g., s3.buckets_list_only)SELECTFast enumeration of resource identifiers

This separation means listing thousands of resources won't trigger rate limits from individual GET calls:

-- Fast enumeration (list operation only)
SELECT bucket_name
FROM awscc.s3.buckets_list_only
WHERE region = 'us-east-1';

-- Full resource details (get operation)
SELECT *
FROM awscc.s3.buckets
WHERE region = 'us-east-1'
AND data__Identifier = 'my-bucket';

Provider Coverage

The awscc provider includes:

  • 237 services and 2371 resources covering the breadth of AWS
  • Full CRUDL support for all Cloud Control compatible resources
  • Consistent schema derived from AWS CloudFormation resource specifications

Example Operations

Create an S3 Bucket

INSERT INTO awscc.s3.buckets (
BucketName,
region
)
SELECT
'my-new-bucket',
'us-east-1';

Query EC2 Instances

SELECT
instance_id,
instance_type,
tags
FROM awscc.ec2.instances
WHERE region = 'ap-southeast-2'
AND data__Identifier = 'i-1234567890abcdef0';

Delete a Resource

DELETE FROM awscc.lambda.functions
WHERE data__Identifier = 'my-function'
AND region = 'us-east-1';

Enhanced Documentation

The provider documentation at awscc.stackql.io now features:

  • Interactive schema explorer with expandable nested property trees
  • Complete field documentation including complex object structures
  • Ready-to-use SQL examples for SELECT, INSERT, and DELETE operations
  • IAM permissions reference for each resource operation

Get Started

Pull the new provider:

stackql registry pull awscc

Query your AWS resources:

stackql shell
>> SELECT region, bucket_name FROM awscc.s3.buckets_list_only WHERE region = 'us-east-1';

Let us know your thoughts! Visit us and give us a star on GitHub.

(Quickly) Identify Old Node Runtimes in AWS Lambda

· 3 min read
Technologist and Cloud Consultant

Have you been sent one of these?

[Action Required] AWS Lambda end of support for Node.js 18 [AWS Account: 824123456789] [EU-CENTRAL-1]

If you are like me and manage AWS accounts with numerous Lambda functions potentially deployed across multiple regions, you need to identify affected resources, in this case, Lambda node runtimes, which will be discontinued later this year.  

With stackql this task is easy...

  1. Open AWS cloud shell in your AWS account (any region - it doesn't matter)
  2. Download stackql
curl -L https://bit.ly/stackql-zip -O && unzip stackql-zip
  1. Open an authenticated stackql command shell
sh stackql-aws-cloud-shell.sh
  1. Run some analytic queries using stackql; here are some examples...

🔍 List all functions and runtimes across regions

Run a stackql query to get the details about functions, runtimes, etc, deployed at any given time across one or more AWS regions.  You can include all 25 AWS regions; each query will be performed asynchronously - speeding up the results.

select
function_name,
region,
runtime
FROM aws.lambda.functions
WHERE region IN ('us-east-1', 'eu-west-1');

📊 Group by runtime and region

Perform an analytic query like a group by aggregate query such as...

select
runtime,
region,
count(*) as num_functions
FROM aws.lambda.functions
WHERE region IN ('us-east-1', 'eu-west-1', 'ap-southeast-2')
GROUP BY runtime, region;
tip

You can easily visualise this data using a notebook; see stackql-codespaces-notebook or stackql-jupyter-demo.

Using StackQL you can:

  • Quickly spot functions running on runtimes like nodejs18.x that are approaching end of support.
  • Plan your upgrades region-by-region with confidence.

⭐ us on GitHub and join our community!

New AWS Provider Available (Jan 2025)

· 2 min read
Technologist and Cloud Consultant
info

To get started with the aws provider for stackql, pull the provider from the registry as follows:  

registry pull aws;

for more detailed provider documentation, see here.

Happy New Year 🎉. The latest AWS provider for StackQL is now available.  The StackQL AWS Provider by the numbers:

  • 230 services
  • 3174 resources
  • 3917 methods

with additional new support for the following services:

  • amazonmq - Managed message broker service for Apache ActiveMQ and RabbitMQ that simplifies setup and operation of open-source message brokers on AWS.
  • applicationsignals - CloudWatch Application Signals automatically provides a correlated view of application performance that includes real user monitoring data and canaries.
  • apptest - AWS mainframe modernization ppplication Testing
  • connectcampaignsv2 - Amazon Connect Outbound Campaigns V2
  • invoicing - Deploy and query invoice units allowing you separate AWS account costs and configures your invoice for each business entity
  • launchwizard - Easily size, configure, and deploy third party applications on AWS
  • pcaconnectorscep - AWS Private CA Connector for SCEP
  • pcs - AWS Parallel Computing Service, easily run HPC workloads at virtually any scale
  • rbin - Recycle Bin is a resource recovery feature that enables you to restore accidentally deleted snapshots and EBS-backed AMIs.
  • s3tables - Amazon S3 Tables enabling Tabular Data Storage At Scale
  • ssmquicksetup - AWS Systems Manager Quick Setup

And 150 new resources with some notable additions including:

  • aws.apigateway.domain_name_access_associations
  • aws.appconfig.deployments, aws.appconfig.deployment_strategies
  • aws.batch.job_definitions
  • aws.bedrock.flows, aws.bedrock.prompts
  • aws.chatbot.custom_actions
  • aws.cloudformation.guard_hooks, aws.cloudformation.lambda_hooks
  • aws.cloudfront.anycast_ip_lists
  • aws.cloudtrail.dashboards, aws.cloudwatch.dashboards
  • aws.codepipeline.pipelines
  • aws.cognito.user_pool_identity_providers
  • aws.ec2.security_group_vpc_associations, aws.ec2.vpc_block_public_access_exclusions, aws.ec2.vpc_block_public_access_options
  • aws.glue.crawlers, aws.glue.databases, aws.glue.jobs, aws.glue.triggers
  • aws.guardduty.malware_protection_plans
  • aws.iot.commands
  • aws.memorydb.multi_region_clusters
  • aws.rds.db_shard_groups
  • aws.redshift.integrations
  • aws.sagemaker.clusters, aws.sagemaker.endpoints
  • aws.secretsmanager.resource_policies, aws.secretsmanager.rotation_schedules, aws.secretsmanager.secret_target_attachments
  • aws.workspaces.workspaces_pools
  • aws.wisdom.ai_agents, aws.wisdom.ai_prompts, aws.wisdom.ai_guardrails, aws.wisdom.message_templates
  • and much more!

⭐ us on GitHub and join our community!