StackQL is now available for ARM64-based Linux systems. To download the ARM64 binary, please visit our downloads page, where you can find the appropriate version for your system.
Let us know what you think! ⭐ us on GitHub.
stackql-deploy is now available in the GitHub Actions Marketplace.
stackql-deploy is a declarative, stateless (and state file-less) infrastructure-as-code and test framework, driven by stackql queries. stackql-deploy is capable of provisioning, updating, de-provisioning and testing cloud and SaaS stacks across all cloud and SaaS providers.
Given this example stackql-deploy stack definition in a GitHub repo, you would simply add the following to your GitHub Actions workflow:
...
jobs:
stackql-actions-test:
name: StackQL Actions Test
runs-on: ubuntu-latest
env:
GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Deploy a Stack
uses: stackql/setup-deploy@v1.0.1
with:
command: build
stack-dir: examples/k8s-the-hard-way
stack-env: dev
env-vars: GOOGLE_PROJECT=stackql-k8s-the-hard-way-demo
Example output is shown here:
Let us know what you think! ⭐ us on GitHub.
AWS creates default VPCs in each region for convenience. However, these default VPCs often contain noncompliant network ACLs and security group rules that do not align with best practices for AWS Config and Security Hub. Deleting these default VPCs is beneficial, especially for regions not used by your organization or architectures that do not utilize a VPC.
This guide demonstrates how to use StackQL to enumerate and delete all default VPCs and their associated resources in all AWS regions.
All you need to do is to install the pystackql package using:
pip install pystackql
Deleting resources will require a privileged AWS IAM user. You can do this from a terminal with the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables set (and optionally AWS_SESSION_TOKEN set if you are using sts assume-role).
You can also use the StackQL Cloud Shell Scripts, from within AWS Cloud Shell, to run authenticated StackQL queries using:
sh stackql-aws-cloud-shell.sh
Default VPCs in AWS regions have a CIDR block of 172.31.0.0/16, before deleting anything, the program ensures it is not in use by using:
SELECT
id
FROM aws.ec2.network_interfaces
WHERE region = '{region}'
AND vpc_id = '{vpc_id}'
If any resources are using the VPC (such as EC2, RDS, ELB/ALB, VPC attached Lambda functions, etc), it will skip these VPCs. Once determined that the VPC is not in use, all of the resources associated with the VPC must be deleted before the VPC itself can be deleted; this includes:
The default route table and default security group are deleted automatically when deleting the VPC
This is done by discovering the resources using StackQL SELECT queries and deleting them using StackQL DELETE queries, like:
DELETE FROM aws.ec2.network_acls
WHERE data__Identifier = '{nacl_id}'
AND region = '{region}'
The following Python program uses StackQL to list and delete default VPCs and their associated resources (subnets, route tables, internet gateways, security groups, and network ACLs) across all AWS regions.
from pystackql import StackQL
stackql = StackQL()
stackql.executeStmt("REGISTRY PULL aws")
def ensure_one_or_zero(resource_list, resource_name, region):
if len(resource_list) > 1:
raise RuntimeError(f"ah snap! multiple default {resource_name} resources found in {region}")
elif len(resource_list) == 0:
print(f"/* no default {resource_name} found in {region} */\n")
return False
return True
regions = [
'us-east-1', 'us-east-2', 'us-west-1', 'us-west-2',
'eu-central-1', 'eu-central-2', 'eu-west-1', 'eu-west-2', 'eu-west-3', 'eu-north-1', 'eu-south-1',
'ap-east-1', 'ap-south-1', 'ap-south-2', 'ap-northeast-1', 'ap-northeast-2', 'ap-northeast-3',
'ap-southeast-1', 'ap-southeast-2', 'ap-southeast-3', 'ap-southeast-4', 'af-south-1',
'ca-central-1', 'me-south-1', 'me-central-1', 'sa-east-1'
]
for region in regions:
vpc_ids = stackql.execute(
f"""
SELECT
vpc_id
FROM aws.ec2.vpcs
WHERE region = '{region}'
AND cidr_block = '172.31.0.0/16'
AND JSON_ARRAY_LENGTH(tags) = 0
"""
)
if not ensure_one_or_zero(vpc_ids, 'VPC', region): continue
vpc_id = vpc_ids[0]['vpc_id']
# check if vpc is in use
network_interfaces = stackql.execute(
f"""
SELECT
id
FROM aws.ec2.network_interfaces
WHERE region = '{region}'
AND vpc_id = '{vpc_id}'
"""
)
if len(network_interfaces) > 0:
print(f"/* skipping deletion of default VPC ({vpc_id}) in {region} because it is in use */\n")
continue
print(f"/* deleting resources for default VPC ({vpc_id}) in {region} */\n")
# get route table
route_table_ids = stackql.execute(
f"""
SELECT
route_table_id
FROM aws.ec2.route_tables
WHERE region = '{region}'
AND vpc_id = '{vpc_id}'
"""
)
ensure_one_or_zero(route_table_ids, 'route table', region)
route_table_id = route_table_ids[0]['route_table_id']
# get inet gateway id
inet_gateway_ids = stackql.execute(
f"""
SELECT gateway_id
FROM aws.ec2.routes
WHERE data__Identifier = '{route_table_id}|0.0.0.0/0'
AND region = '{region}'
"""
)
ensure_one_or_zero(inet_gateway_ids, 'internet gateway', region)
inet_gateway_id = inet_gateway_ids[0]['gateway_id']
# delete routes
print(f"/* deleting default VPC routes in route table ({route_table_id}) in {region} */")
print(f"""
DELETE FROM aws.ec2.routes
WHERE data__Identifier = '{route_table_id}|0.0.0.0/0'
AND region = '{region}';
""")
# detatch inet gateway
print(f"/* detaching default VPC internet gateway ({inet_gateway_id}) in {region} */")
print(f"""
DELETE FROM aws.ec2.vpc_gateway_attachments
WHERE data__Identifier = 'IGW|{vpc_id}'
AND region = '{region}';
""")
# delete inet gateway
print(f"/* deleting default VPC internet gateway ({inet_gateway_id}) in {region} */")
print(f"""
DELETE FROM aws.ec2.internet_gateways
WHERE data__Identifier = '{inet_gateway_id}'
AND region = '{region}';
""")
# delete nacl
nacl_ids = stackql.execute(
f"""
SELECT
id
FROM aws.ec2.network_acls
WHERE vpc_id = '{vpc_id}'
AND region = '{region}'
"""
)
ensure_one_or_zero(nacl_ids, 'network acl', region)
nacl_id = nacl_ids[0]['id']
print(f"/* deleting default VPC NACL ({nacl_id}) in {region} */")
print(f"""
DELETE FROM aws.ec2.network_acls
WHERE data__Identifier = '{nacl_id}'
AND region = '{region}';
""")
# delete subnets
subnet_ids = stackql.execute(
f"""
SELECT
subnet_id
FROM aws.ec2.subnets
WHERE vpc_id = '{vpc_id}'
AND region = '{region}'
"""
)
for subnet_id in subnet_ids:
print(f"/* deleting default VPC subnet ({subnet_id['subnet_id']}) in {region} */")
print(f"""
DELETE FROM aws.ec2.subnets
WHERE data__Identifier = '{subnet_id['subnet_id']}'
AND region = '{region}';
""")
# delete vpc
print(f"/* deleting default VPC ({vpc_id}) in {region} */")
print(f"""
DELETE FROM aws.ec2.vpcs
WHERE data__Identifier = '{vpc_id}'
AND region = '{region}';
""")
run the program using:
python3 delete-all-default-vpcs.py > delete_default_vpcs.iql
to generate a StackQL script you can run as a batch using stackql exec or as individual statements in the stackql shell.
Deleting default VPCs can help improve your AWS security posture by removing potentially noncompliant network configurations. This program leverages StackQL to automate the process, ensuring consistency across all regions.
Let us know what you think! ⭐ us on GitHub.
New versions of the azure providers for stackql are available now in the stackql-provider-registry.
Summary stats for the main azure provider:
| Total Services | 196 |
| Total Resources | 4,020 |
| Total Methods | 11,160 |
New versions of the azure_extras, azure_isv and azure_stack providers are available as well.
New services available include:
Let us know what you think! ⭐ us on GitHub.
Analyzing firewall rules is crucial for maintaining security in your cloud infrastructure. Using StackQL, you can efficiently query and analyze Google Cloud firewall configurations to ensure that your security policies are correctly implemented and that there are no unexpected open ports or protocols that might pose a security risk. Below is a simple query that retrieves important details about the ingress firewall rules for a specific network in a Google Cloud project.
SELECT
name,
source_range,
ip_protocol,
allowed_ports,
direction
FROM (
SELECT
name,
source_ranges.value as source_range,
JSON_EXTRACT(allowed.value, '$.IPProtocol') as ip_protocol,
JSON_EXTRACT(allowed.value, '$.ports') as allowed_ports,
direction
FROM google.compute.firewalls, json_each(sourceRanges) as source_ranges, json_each(allowed) as allowed
WHERE project = 'stackql-k8s-the-hard-way-demo'
AND network = 'https://www.googleapis.com/compute/v1/projects/stackql-k8s-the-hard-way-demo/global/networks/kubernetes-the-hard-way-dev-vpc'
) t
WHERE
source_range = '0.0.0.0/0'
and direction = 'INGRESS';
This query provides a comprehensive list of all ingress firewall rules that apply to any IP address (0.0.0.0/0) within the specified Google Cloud project and network. The results include the firewall rule name, the source IP range, the protocol, the allowed ports, and the direction of the traffic, an example is shown below:
|-----------------------------------------------|--------------|-------------|---------------|-----------|
| name | source_range | ip_protocol | allowed_ports | direction |
|-----------------------------------------------|--------------|-------------|---------------|-----------|
| default-allow-icmp | 0.0.0.0/0 | icmp | null | INGRESS |
|-----------------------------------------------|--------------|-------------|---------------|-----------|
| default-allow-rdp | 0.0.0.0/0 | tcp | ["3389"] | INGRESS |
|-----------------------------------------------|--------------|-------------|---------------|-----------|
| default-allow-ssh | 0.0.0.0/0 | tcp | ["22"] | INGRESS |
|-----------------------------------------------|--------------|-------------|---------------|-----------|
| kubernetes-the-hard-way-dev-allow-external-fw | 0.0.0.0/0 | tcp | ["22"] | INGRESS |
|-----------------------------------------------|--------------|-------------|---------------|-----------|
| kubernetes-the-hard-way-dev-allow-external-fw | 0.0.0.0/0 | tcp | ["6443"] | INGRESS |
|-----------------------------------------------|--------------|-------------|---------------|-----------|
| kubernetes-the-hard-way-dev-allow-external-fw | 0.0.0.0/0 | icmp | null | INGRESS |
|-----------------------------------------------|--------------|-------------|---------------|-----------|
You can use this query to help quickly identify potential security vulnerabilities. Regularly auditing these rules ensures that your cloud environment remains secure and that only the necessary ports and protocols are open to the internet.
Give us your feedback! ⭐ us here!