stackql is a dev tool that allows you to query and manage cloud and SaaS resources using SQL, which developers and analysts can use for CSPM, assurance, user access management reporting, IaC, XOps and more.
An exciting new feature to cap off 2023! Parallel query execution in StackQL. With the latest release of StackQL, parameters in WHERE IN clauses are fetched asyncronously.
This query for example, queries lambda functions across 17 AWS regions in under 1.5 sec, technically these are 17 parallel queries to 17 different endpoints.
SELECT region, function_name
FROM aws.lambda.functions
WHERE region IN ( 'us-east-1','us-east-2','us-west-1','us-west-2','ap-south-1','ap-northeast-3','ap-northeast-2','ap-southeast-1','ap-southeast-2','ap-northeast-1','ca-central-1','eu-central-1','eu-west-1','eu-west-2','eu-west-3','eu-north-1','sa-east-1'
)
You could do something similar for other hyperscalars, for example querying resources across projects in GCP asynchronously, or querying across resource groups in Azure asynchronously.
This capability was previously available using the pystackql package, as discussed in the Query Resources Across AWS Regions Asynchronously blog post, but is now available natively in the StackQL query optimizer.
You just need to add --execution.concurrency.limit=-1 to your stackql exec or stackql shell commands or when starting a StackQL Server using stackql srv. More query optimizations coming! Happy New Year! 🎉 🎉 🎉
Let us know your thoughts! Visit us and give us a ⭐ on GitHub