Skip to main content

29 posts tagged with "cloud security"

View All Tags

· 4 min read

Excited to announce the release of the Netlify provider for StackQL.

StackQL allows you to query and interact with your cloud and SaaS assets using a simple SQL framework

The netlify provider can be used to query, provision, de-provision or update sites, builds, deploys, functions, identities, domain_names and more.

Here are the steps to get started with the Netlify provider:

Setup​

  1. Create a personal access token for Netlify (if you don't have one already), go to https://app.netlify.com/user/applications/personal.

  2. Export the token to a variable and supply this as the provider authentication for StackQL: 

export NETLIFY_TOKEN=your_personal_access_token
AUTH='{ "netlify": { "type": "api_key", "valuePrefix": "Bearer ", "credentialsenvvar": "NETLIFY_TOKEN" } }'
./stackql shell --auth="${AUTH}"

Install the netlify provider​

  1. Pull the Netlify provider (you only need to do this the first time you use the provider or when you are updating), the following command can be run from the StackQL interactive shell (stackql shell) or using stackql exec: 
registry pull netlify v0.1.0;

Show available services in netlify​

  1. (Optional) Show the available services in the Netlify provider: 
stackql >> show services in netlify;

+----------------------+---------------+-----------------------+
|          id          |     name      |       title           |
|----------------------|---------------|-----------------------|
| builds:v0.1.0        | builds        | Netlify site builds   |
|----------------------|---------------|-----------------------|
| deploys:v0.1.0       | deploys       | Netlify site deploys  |
|----------------------|---------------|-----------------------|
| user_accounts:v0.1.0 | user_accounts | Netlify user accounts |
|----------------------|---------------|-----------------------|
| ...                  | ...           | ...                   |
+----------------------+---------------+-----------------------+

Explore resources in netlify​

  1. (Optional) Explore resources in a service in the Netlify provider: 
stackql  >> show resources in netlify.deploys;

+----------------+--------------------------------+
|      name      |               id               |
|----------------|--------------------------------|
| deploy         | netlify.deploys.deploy         |
|----------------|--------------------------------|
| deployKey      | netlify.deploys.deployKey      |
|----------------|--------------------------------|
| deployedBranch | netlify.deploys.deployedBranch |
+----------------+--------------------------------+
  1. (Optional) List the available fields (or properties) of a resource: 
stackql  >> describe netlify.deploys.deploy;

+--------------------+---------+
|        name        |  type   |
|--------------------|---------|
| id                 | string  |
|--------------------|---------|
| name               | string  |
|--------------------|---------|
| url                | string  |
|--------------------|---------|
| review_id          | number  |
|--------------------|---------|
| commit_url         | string  |
|--------------------|---------|
| commit_ref         | string  |
|--------------------|---------|
| ...                | ...     |
+--------------------+---------+
  1. (Optional) Show the available methods in a resource (the stuff you can do with a resource..): 
stackql  >> show methods in netlify.deploys.deploy;

+--------------------+--------------------+
|     MethodName     |   RequiredParams   |
|--------------------|--------------------|
| cancelSiteDeploy   | deploy_id          |
|--------------------|--------------------|
| createSiteDeploy   | site_id            |
|--------------------|--------------------|
| getSiteDeploy      | deploy_id, site_id |
|--------------------|--------------------|
| listSiteDeploys    | site_id            |
|--------------------|--------------------|
| lockDeploy         | deploy_id          |
|--------------------|--------------------|
| ...                | ...                |
+--------------------+--------------------+

::: tip Methods prefixed by list and get are exposed via SELECT verbs, for example: 

SELECT id, name FROM netlify.deploys.deploy
WHERE site_id = 'ad26d902-9cb1-43be-90d9-284e8c7ac687';

this query accesses the listSiteDeploys method :::

Run some queries!​

  1. Run some queries... 
stackql  >> SELECT created_at, name, state, branch FROM netlify.deploys.deploy
>> WHERE site_id = 'ad26d902-9cb1-43be-90d9-284e8c7ac687'
>> ORDER BY created_at DESC LIMIT 2;

+--------------------------+------------+-------+-------------------------+
|        created_at        |    name    | state |         branch          |
|--------------------------|------------|-------|-------------------------|
| 2022-05-04T22:46:43.015Z | stackql-io | ready | main                    |
|--------------------------|------------|-------|-------------------------|
| 2022-05-04T22:39:34.958Z | stackql-io | ready | feature/content-updates |
+--------------------------+------------+-------+-------------------------+

You can also use StackQL to provision resources in Netlify, the methods you saw in step 7 that are prefixed by create or insert, can be accessed using INSERT statements in StackQL, similarly methods prefixed by delete or remove can be accessed using DELETE statements.

more providers coming soon, if there is anything you are interested in specifically, get in contact and let us know.

Welcome your feedback by getting in touch or raising issues at stackql/stackql-provider-registry, give us some ⭐️ love while you are there!

enjoy!

· 4 min read

The GitHub provider for StackQL is now generally available. This can be used to query resources in GitHub Cloud or GitHub Enterprise, including orgs, teams, users, repositories, branches, pull requests, issues, workflows/actions and much more!

See available providers​

You can see the versions of GitHub Provider (and other providers) available using:

stackql registry list

or from the StackQL Command Shell (stackql shell) using:

REGISTRY LIST;

this would return a list of all the providers that are currently available, for example: 

+----------+---------+
| provider | version |
+----------+---------+
| github   | v0.1.0  |
| google   | v0.1.0  |
| okta     | v0.1.0  |
+----------+---------+

Pull the github provider​

To pull v0.1.0 of the github provider use:

stackql registry pull github v0.1.0

or

REGSITRY PULL github v0.1.0;

to see what providers are installed use:

SHOW PROVIDERS;

this would return something like... 

+--------+
|  name  |
+--------+
| github |
+--------+

Explore the github provider and query public resources​

The provider and public objects can be queried without authentication as shown here: 

AUTH='{"github": { "type": "null_auth" }}'
stackql shell --auth="${AUTH}"

you can now enumerate services, resources, attributes and methods in the github provider using the SHOW and DESCRIBE meta commands, for instance:

show services in github from either the StackQL command shell or via stackql exec would return something like... 

+----------------------------+---------------------+------------------------------------------+
|             id             |        name         |                  title                   |
+----------------------------+---------------------+------------------------------------------+
| actions_enterprises:v0.1.0 | actions_enterprises | GitHub v3 REST API - actions_enterprises |
| billing:v0.1.0             | billing             | GitHub v3 REST API - billing             |
| repos:v0.1.0               | repos               | GitHub v3 REST API - repos               |
| ...                        | ...                 | ...                                      |
+----------------------------+---------------------+------------------------------------------+
tip

Use the EXTENDED operator with the SHOW or DESCRIBE commands to get additional information about services, resources, attributes and methods, e.g. DESCRIBE EXTENDED github.repos.repos

show resources in github.repos would return something like... 

+--------------+---------------------------+
|     name     |            id             |
+--------------+---------------------------+
| branches     | github.repos.branches     |
| commits      | github.repos.commits      |
| deployments  | github.repos.deployments  |
| environments | github.repos.environments |
| forks        | github.repos.forks        |
| releases     | github.repos.releases     |
| repos        | github.repos.repos        |
| statistics   | github.repos.statistics   |
| statuses     | github.repos.statuses     |
| traffic      | github.repos.traffic      |
| ...          | ...                       |
+--------------+---------------------------+

to see fields in a resource (which can be queried or updated) use DESCRIBE for example DESCRIBE github.repos.commits would return something like... 

+--------------+--------+
|     name     |  type  |
+--------------+--------+
| files        | array  |
| stats        | object |
| commit       | object |
| url          | string |
| html_url     | string |
| parents      | array  |
| node_id      | string |
| comments_url | string |
| committer    | object |
| sha          | string |
| author       | object |
+--------------+--------+

to see methods available in a resource use the SHOW METHODS command for example SHOW METHODS IN github.repos.commits would return something like... 

+-------------------------------------------+-------------------------+
|                MethodName                 |     RequiredParams      |
+-------------------------------------------+-------------------------+
| compare_commits                           | basehead, owner, repo   |
| get_commit                                | owner, ref, repo        |
| list_branches_for_head_commit             | commit_sha, owner, repo |
| list_commits                              | owner, repo             |
| list_pull_requests_associated_with_commit | commit_sha, owner, repo |
+-------------------------------------------+-------------------------+
tip

Methods beginning with list or get can usually be accessed via SELECT statements. For example, 

SELECT github.repos.commits.sha 
FROM github.repos.commits 
WHERE owner='${owner}' AND repo='${repo}';

Other methods can be accessed using the EXEC command (for more information see EXEC)

Query protected resources​

Accessing protected resources requires authentication using a Personal Access token as shown here: 

export GITHUB_CREDS=$(echo -n 'yourgithubusername:ghp_YOURPERSONALACCESSTOKEN' | base64)
AUTH='{ "github": { "type": "basic", "credentialsenvvar": "GITHUB_CREDS" } }'
stackql shell --auth="${AUTH}"

Now you are able to access protected resources, for example: 

select id, name, private 
from github.repos_orgs.repos_orgs 
where org = 'stackql';

which would return something like... 

+-----------+-------------------------+---------+
|    id     |          name           | private |
+-----------+-------------------------+---------+
| 443987542 | stackql                 | false   |
| 441087132 | stackqlproviderregistry | false   |
| 409393414 | fullstackchronicles.io  | false   |
| 435085734 | stackql.io              | true    |
| 443979486 | releases.stackql.io     | true    |
| 447890554 | stackqldevel            | true    |
|       ... | ...                     | ...     |
+-----------+-------------------------+---------+

Welcome your feedback by getting in touch or raising issues at stackql/stackql-provider-registry, ⭐️ us while you are there!

· 2 min read

Multi cloud visibility, SecOps, FinOps, DevOps made easy

Today marks a significant epoch in the evolution of the InfraQL/StackQL project. The StackQL provider registry allows contributors to add support for different providers (major cloud, alt cloud and SaaS providers) using a no-code approach. Developers simply add extensions to the providers OpenAPI spec using configuration documents (currently supporting yaml and json – with future support for toml and hcl). These extensions allow StackQL to map an ORM to provider services, resources, and methods.

For example, for a future AWS provider you could run discovery commands such as: 

SHOW SERVICES IN aws;
/* shows the available services in AWS */
SHOW RESOURCES IN aws.ec2;
/* shows the available resources in the AWS EC2 service */
DESCRIBE aws.ec2.instances;
/* show available attributes in the aws.ec2.instances resource schema */
SHOW METHODS IN aws.ec2.instances;
/* shows available lifecycle methods – such as start, stop, etc which can be involved using the EXEC command */

Or create a new EC2 instance using: 

INSERT INTO aws.ec2.instances SELECT …;

View and report on instances and their properties using: 

SELECT col(s) FROM aws.ec2.instances WHERE …;

Or clean up resources using: 

DELETE FROM aws.ec2.instances WHERE …;

The StackQL beta version supporting the provider registry is available for Mac (arm and amd) and Linux, with a Windows version coming in the next few weeks.

Providers are currently available for Google and Okta, see StackQL Provider Registry repo and Developer Guide. We are encouraging developers to contribute – we would be happy to assist, just raise an issue or a PR.

· 2 min read

Its easy enough for anyone to deploy a Cloud Storage bucket in google, this can be done through the console, gcloud, terraform or stackql as shown here: Deploying and Querying GCS Buckets using StackQL. It is also easy to inadvertently allow users to set public ACLs on a bucket, therefore making its contents publicly visible by default. There is an easy way to prevent this from happening by Using public access prevention.

Let's work through a real life scenario using StackQL.

Step 1 : Run a query to find buckets which do not have public access prevention enforced​

Run the following StackQL query from the shell or via exec:

SELECT name, 
JSON_EXTRACT(iamConfiguration, '$.publicAccessPrevention') as publicAccessPrevention
FROM  google.storage.buckets
WHERE project = 'myco-terraform';
/* returns
|-------------------|------------------------|
|       name        | publicAccessPrevention |
|-------------------|------------------------|
| myco-tf-nonprod   | unspecified            |
|-------------------|------------------------|
| myco-tf-prod      | enforced               |
|-------------------|------------------------|
*/

We can see from the query results that the myco-tf-nonprod bucket does not have public access prevention enforced, lets fix it...using StackQL.

Step 2 : Configure public access prevention for a bucket​

Run the following StackQL procedure to enforce public access prevention:

EXEC google.storage.buckets.patch 
@bucket = 'myco-tf-nonprod'
@@json = '{
    "iamConfiguration": {
      "publicAccessPrevention": "enforced"
    }
}';

Step 3: Confirm public access prevention is enforced​

Run the first query again, and you should see that the desired result is in place.

SELECT name, 
JSON_EXTRACT(iamConfiguration, '$.publicAccessPrevention') as publicAccessPrevention
FROM  google.storage.buckets
WHERE project = 'myco-terraform';
/* returns
|-------------------|------------------------|
|       name        | publicAccessPrevention |
|-------------------|------------------------|
| myco-tf-nonprod   | enforced               |
|-------------------|------------------------|
| myco-tf-prod      | enforced               |
|-------------------|------------------------|
*/

Easy!