Skip to main content

28 posts tagged with "analytics"

View All Tags

New Okta Provider Released

· 6 min read
Technologist and Cloud Consultant

The latest okta provider for StackQL is available now (v25.09.00341), featuring a comprehensive reorganization of services and resources that better aligns with Okta's API structure. This update improves discoverability, logical grouping, and overall usability when working with Okta resources through StackQL.

What's Changed

The updated Okta provider features a more granular and logical organization of services that mirrors Okta's API architecture more closely. Here is a summary of the services included in the latest okta provider:

ServiceDescription
agentpoolsManages agent pools for on-premises integration and deployment
api_tokensHandles API token creation, management, and access control
appsManages Okta application integration, configuration, and assignments
attack_protectionConfigures security controls to prevent various attack vectors
authenticatorsManages authentication methods, factors, and settings
authorizationserversControls OAuth 2.0 and OIDC authorization servers and policies
behaviorsConfigures end-user behavioral patterns for risk-based authentication
brandsManages customization of Okta UI appearance and branding
captchasConfigures CAPTCHA settings to prevent automated attacks
device_accessControls access policies based on device attributes
device_assurancesManages device trust requirements and verification
device_integrationsConfigures integrations with MDM and endpoint security tools
device_posture_checksVerifies security posture of devices accessing resources
devicesManages device enrollment, lifecycle, and inventory
directoriesHandles directory service connections and synchronization
domainsManages custom domain configuration for Okta tenant
email_domainsControls email domains for user provisioning and validation
email_serversConfigures email server settings for notifications
eventhooksManages webhook subscriptions for Okta event notifications
featuresControls feature flags and preview feature management
first_party_app_settingsConfigures settings for Okta-developed applications
groupsManages group creation, membership, and rule configuration
hook_keysHandles encryption keys for securing hook communications
iamProvides core identity and access management functionality
identity_sourcesManages sources of identity data for user provisioning
idpsConfigures external identity providers for federation
inlinehooksManages customization points within Okta workflows
integrationsConfigures third-party service integrations
logsProvides access to system logs and audit events
logstreamsManages streaming of log data to external systems
mappingsConfigures attribute mappings between systems
metaProvides metadata about the Okta API and environment
oauth2Manages OAuth 2.0 clients, scopes, and tokens
okta_personal_settingsControls user-specific preferences and settings
orgManages organization-level settings and configurations
orgsHandles multi-org deployments and org relationships
policiesConfigures authentication, password, and access policies
principal_rate_limitsManages API rate limits for specific principals
privileged_accessControls privileged access management settings
push_providersConfigures push notification delivery services
rate_limit_settingsManages global API rate limit configurations
realm_assignmentsMaps users and groups to authentication realms
realmsConfigures authentication realms for different user populations
riskManages risk-based authentication settings and policies
rolesControls administrative role assignments and permissions
securityProvides security settings and configurations
security_events_providersManages integrations with security event sources
sessionsControls user session policies and management
ssfConfigures server-side functions for customization
templatesManages templates for emails, notifications, and forms
threatsControls threat intelligence settings and configurations
trustedoriginsManages CORS and redirect configurations
usersHandles user lifecycle, profiles, and credentials
webauthn_registrationManages WebAuthn/FIDO2 credential registration
zonesConfigures network zones for policy application
note

The improved organization including service and resource naming and mapping will introduce changes to queries against the previous provider version (v23.03.00121), you can pin the previous version in stackql-deploy or via registry pull in the interim while you make necessary query modifications.

Enhanced Documentation

The new Okta Provider Docs provide comprehensive documentation on how to use the new okta provider including ready-to-use SQL examples for each resource and method. A standout feature is the copy-paste functionality for all SQL queries, making it incredibly easy to:

  1. Compose Infrastructure-as-Code workflows: Each method documentation includes working SQL examples that can be directly copied into your deployment scripts or CI/CD pipelines. Simply click the copy button next to any example to get production-ready SQL code.

  2. Build analytics dashboards: Create sophisticated cross-service queries by combining examples from different resources. The documentation's consistent query formatting makes it simple to join related data across multiple Okta services.

  3. Develop governance reports: Copy baseline queries and customize them for your specific compliance needs. The pre-formatted SQL provides the perfect starting point for custom reporting.

Getting Started

To start using the updated okta provider, simply pull the latest version from stackql shell or stackql registry command:

registry pull okta;

Then you can begin querying your Okta resources with SQL:

-- Get all users and their group assignments
SELECT
u.id,
JSON_EXTRACT(u.profile, '$.email') as email,
u.status,
JSON_EXTRACT(g.profile, '$.name') as group_name
FROM
okta.user.users u
JOIN
okta.group.users gu ON u.id = gu.user_id
JOIN
okta.group.groups g ON gu.group_id = g.id
WHERE subdomain = 'my-company';

Use Cases for the Okta Provider

  1. Infrastructure as Code: Manage your Okta resources alongside other cloud providers in a unified IaC approach, see stackql-deploy.

  2. Cost Optimization: Identify unused resources and opportunities for cost savings.

  3. Security and Compliance: Audit account roles, permissions, and access patterns to ensure compliance with security policies.

  4. Performance Monitoring: Track query performance, warehouse utilization, and identify optimization opportunities.

  5. Cross-Provider Orchestration: Build workflows that span Okta and other resource providers, enabling sophisticated data and infrastructure pipelines.

  6. Automated Reporting: Create automated reports on Okta usage, performance, and costs.

⭐ us on GitHub and join our community!

Updated Google Providers for StackQL Available

· 2 min read
Technologist and Cloud Consultant

The latest versions of the Google-related providers for StackQL: google, googleadmin, googleworkspace, and firebase are available now. These updates include the latest services, resources and methods available from Google.

What's New

The latest release introduces several new services to the google provider, expanding your ability to manage and query Google Cloud resources:

  • API Hub: Centrally manage and discover APIs across your organization
  • Area Insights: Access location-based insights and analytics
  • Cloud Location Finder: Identify optimal Google Cloud regions for your workloads
  • Gemini Cloud Assist: Leverage Google's AI assistant for cloud operations
  • Managed Kafka: Work with Google's fully-managed Apache Kafka service
  • Observability: Enhanced monitoring and observability services
  • Parallel Store: Interact with Google's high-performance storage solution
  • Parameter Manager: Manage configuration parameters across services
  • SaaS Service Management: Tools for managing SaaS offerings on Google Cloud
  • Secure Source Manager: Google's secure, fully-managed source control service
  • Security Posture: Assess and improve your cloud security posture
  • Storage Batch Operations: Perform batch operations on Cloud Storage resources

Enhanced Documentation

We've also released enhanced user documentation to help you get the most out of these providers. Check out our comprehensive docs:

Getting Started

To start using these updated providers, simply pull the latest version from stackql shell or stackql registry command:

registry pull google;
registry pull googleadmin;
registry pull googleworkspace;
registry pull firebase;

Then you can begin querying your Google resources with SQL:

SELECT name, region, status 
FROM google.compute.instances
WHERE project = 'my-project';

Use Cases for the Google Provider

The Google provider for StackQL opens up numerous possibilities:

  1. Infrastructure as Code: Manage your Google resources alongside other cloud providers in a unified IaC approach, see stackql-deploy.

  2. Cost Optimization: Identify unused resources and opportunities for cost savings.

  3. Security and Compliance: Audit account roles, permissions, and access patterns to ensure compliance with security policies.

  4. Performance Monitoring: Track query performance, warehouse utilization, and identify optimization opportunities.

  5. Cross-Provider Orchestration: Build workflows that span Google and other cloud providers, enabling sophisticated data and infrastructure pipelines.

  6. Automated Reporting: Create automated reports on Google usage, performance, and costs.

⭐ us on GitHub and join our community!

Snowflake Provider for StackQL Released

· 3 min read
Technologist and Cloud Consultant

We're excited to announce the release of the Snowflake provider for StackQL! This new provider enables you to query and interact with your Snowflake resources using familiar SQL syntax, bridging the gap between data analytics and infrastructure management.

The Snowflake provider for StackQL gives you the ability to:

  • Query Snowflake metadata and statistics using SQL
  • Monitor warehouse, database, and query performance
  • Analyze resource usage and optimize costs
  • Integrate Snowflake management with your existing cloud infrastructure
  • Build cross-provider workflows and automation

Full documentation for the Snowflake provider is available here.

Getting Started

Getting started is as easy as...

REGISTRY PULL snowflake;

Example Queries

Let's explore some powerful examples of what you can do with the Snowflake provider for StackQL.

Analyzing Warehouses

SELECT 
size,
count(*) as num_warehouses
FROM
snowflake.warehouse.warehouses
WHERE
endpoint = 'OKXVNMC-VH34026'
GROUP BY
size;

Other fields for the warehouses resource include : name, warehouse_type, state, scaling_policy, auto_suspend, auto_resume, resource_monitor, enable_query_acceleration, query_acceleration_max_scale_factor, max_concurrency_level, owner, warehouse_credit_limit, target_statement_size and more.

Table Analysis

SELECT 
name,
bytes,
data_retention_time_in_days,
table_type
FROM
snowflake.table.tables
WHERE
database_name = 'SNOWFLAKE_SAMPLE_DATA'
AND schema_name = 'TPCH_SF10'
AND endpoint = 'OKXVNMC-VH34026'
ORDER BY
bytes DESC;

Other Services and Resources

Other notable resources which can be provisioned, managed or queried using the snowflake provider for stackql include:

alerts, api_integrations, catalog_integrations, compute_pools, databases, database_roles, dynamic_tables, event_tables, external_volumes, functions, grants, iceberg_tables, image_repositories, network_policies, notebooks, notification_integrations, pipes, procedures, roles, schemas, stages, streams, tasks, users, user_defined_functions, views, and more!

Use Cases for the Snowflake Provider

The Snowflake provider for StackQL opens up numerous possibilities:

  1. Infrastructure as Code: Manage your Snowflake resources alongside other cloud providers in a unified IaC approach, see stackql-deploy.

  2. Cost Optimization: Identify unused resources, inefficient warehouses, and opportunities for cost savings.

  3. Security and Compliance: Audit account roles, permissions, and access patterns to ensure compliance with security policies.

  4. Performance Monitoring: Track query performance, warehouse utilization, and identify optimization opportunities.

  5. Cross-Provider Orchestration: Build workflows that span Snowflake and other cloud providers, enabling sophisticated data and infrastructure pipelines.

  6. Automated Reporting: Create automated reports on Snowflake usage, performance, and costs.

⭐ us on GitHub and join our community!

(Quickly) Identify Old Node Runtimes in AWS Lambda

· 3 min read
Technologist and Cloud Consultant

Have you been sent one of these?

[Action Required] AWS Lambda end of support for Node.js 18 [AWS Account: 824123456789] [EU-CENTRAL-1]

If you are like me and manage AWS accounts with numerous Lambda functions potentially deployed across multiple regions, you need to identify affected resources, in this case, Lambda node runtimes, which will be discontinued later this year.  

With stackql this task is easy...

  1. Open AWS cloud shell in your AWS account (any region - it doesn't matter)
  2. Download stackql
curl -L https://bit.ly/stackql-zip -O && unzip stackql-zip
  1. Open an authenticated stackql command shell
sh stackql-aws-cloud-shell.sh
  1. Run some analytic queries using stackql; here are some examples...

🔍 List all functions and runtimes across regions

Run a stackql query to get the details about functions, runtimes, etc, deployed at any given time across one or more AWS regions.  You can include all 25 AWS regions; each query will be performed asynchronously - speeding up the results.

select 
function_name,
region,
runtime
FROM aws.lambda.functions
WHERE region IN ('us-east-1', 'eu-west-1');

📊 Group by runtime and region

Perform an analytic query like a group by aggregate query such as...

select 
runtime,
region,
count(*) as num_functions
FROM aws.lambda.functions
WHERE region IN ('us-east-1', 'eu-west-1', 'ap-southeast-2')
GROUP BY runtime, region;
tip

You can easily visualise this data using a notebook; see stackql-codespaces-notebook or stackql-jupyter-demo.

Using StackQL you can:

  • Quickly spot functions running on runtimes like nodejs18.x that are approaching end of support.
  • Plan your upgrades region-by-region with confidence.

⭐ us on GitHub and join our community!

New AWS Provider Available (Jan 2025)

· 2 min read
Technologist and Cloud Consultant
info

To get started with the aws provider for stackql, pull the provider from the registry as follows:  

registry pull aws;

for more detailed provider documentation, see here.

Happy New Year 🎉. The latest AWS provider for StackQL is now available.  The StackQL AWS Provider by the numbers:

  • 230 services
  • 3174 resources
  • 3917 methods

with additional new support for the following services:

  • amazonmq - Managed message broker service for Apache ActiveMQ and RabbitMQ that simplifies setup and operation of open-source message brokers on AWS.
  • applicationsignals - CloudWatch Application Signals automatically provides a correlated view of application performance that includes real user monitoring data and canaries.
  • apptest - AWS mainframe modernization ppplication Testing
  • connectcampaignsv2 - Amazon Connect Outbound Campaigns V2
  • invoicing - Deploy and query invoice units allowing you separate AWS account costs and configures your invoice for each business entity
  • launchwizard - Easily size, configure, and deploy third party applications on AWS
  • pcaconnectorscep - AWS Private CA Connector for SCEP
  • pcs - AWS Parallel Computing Service, easily run HPC workloads at virtually any scale
  • rbin - Recycle Bin is a resource recovery feature that enables you to restore accidentally deleted snapshots and EBS-backed AMIs.
  • s3tables - Amazon S3 Tables enabling Tabular Data Storage At Scale
  • ssmquicksetup - AWS Systems Manager Quick Setup

And 150 new resources with some notable additions including:

  • aws.apigateway.domain_name_access_associations
  • aws.appconfig.deployments, aws.appconfig.deployment_strategies
  • aws.batch.job_definitions
  • aws.bedrock.flows, aws.bedrock.prompts
  • aws.chatbot.custom_actions
  • aws.cloudformation.guard_hooks, aws.cloudformation.lambda_hooks
  • aws.cloudfront.anycast_ip_lists
  • aws.cloudtrail.dashboards, aws.cloudwatch.dashboards
  • aws.codepipeline.pipelines
  • aws.cognito.user_pool_identity_providers
  • aws.ec2.security_group_vpc_associations, aws.ec2.vpc_block_public_access_exclusions, aws.ec2.vpc_block_public_access_options
  • aws.glue.crawlers, aws.glue.databases, aws.glue.jobs, aws.glue.triggers
  • aws.guardduty.malware_protection_plans
  • aws.iot.commands
  • aws.memorydb.multi_region_clusters
  • aws.rds.db_shard_groups
  • aws.redshift.integrations
  • aws.sagemaker.clusters, aws.sagemaker.endpoints
  • aws.secretsmanager.resource_policies, aws.secretsmanager.rotation_schedules, aws.secretsmanager.secret_target_attachments
  • aws.workspaces.workspaces_pools
  • aws.wisdom.ai_agents, aws.wisdom.ai_prompts, aws.wisdom.ai_guardrails, aws.wisdom.message_templates
  • and much more!

⭐ us on GitHub and join our community!