Skip to main content
⚡ Calmops
🤖 AI 🏗️ Architecture 🌐 Web 🗃️ Database ⚙️ DevOps 🌍 Network 💻 Programming 🧮 Algorithms 📐 Software Eng. ☁️ Cloud 🔒 Security 📈 Finance More →

Cloud Custodian: Cloud Security and Compliance Automation

Created: February 24, 2026 CalmOps 5 min read

Introduction

Cloud security and compliance are critical concerns for organizations running workloads in the cloud. Manually monitoring and enforcing security policies across multiple cloud providers is unsustainable. Cloud Custodian provides a unified policy-as-code framework for securing and optimizing cloud resources.

This guide covers Cloud Custodian’s architecture, policy development, and practical implementation for multi-cloud environments.

Understanding Cloud Custodian

Cloud Custodian is an open-source rule engine that enables:

  • Resource management: Find, filter, and take actions on cloud resources
  • Security enforcement: Identify and remediate security issues
  • Compliance: Automate policy compliance checks
  • Cost optimization: Identify and clean up unused resources

Multi-Cloud Support

Provider Resources Supported
AWS 200+ resource types
Azure 100+ resource types
GCP 80+ resource types
Kubernetes Namespaces, pods

Installation

# Install via pip
pip install c7n

# Install via brew
brew install cloud-custodian

# Verify installation
custodian version

# AWS setup
pip install c7n-aws
aws configure

# Azure setup
pip install c7n-azure
az login

# GCP setup
pip install c7n-gcp
gcloud auth application-default login

Policy Structure

Basic Policy Format

policies:
  - name: find-unencrypted-s3-buckets
    description: |
      Find S3 buckets that are not encrypted
    resource: aws.s3
    filters:
      - type: value
        key: ServerSideEncryptionConfiguration
        value: absent
    actions:
      - type: mark-for-op
        op: notify
        days: 7

Policy Components

  • name: Unique identifier
  • description: Human-readable explanation
  • resource: Target cloud resource type
  • filters: Conditions to match resources
  • actions: Operations to perform on matched resources

Common Use Cases

Security: Find Public S3 Buckets

policies:
  - name: s3-public-access
    resource: aws.s3
    description: Find S3 buckets with public access
    filters:
      - or:
          - type: bucket-policy
            statement:
              - Effect: Allow
                Principal: "*"
          - type: public-access-block
            settings:
              BlockPublicAcls: false
              BlockPublicPolicy: false
              IgnorePublicAcls: false
              RestrictPublicBuckets: false
    actions:
      - type: notify
        to:
          - [email protected]
        subject: Public S3 Bucket Detected
        template: default

Security: Find Unencrypted EBS Volumes

policies:
  - name: unencrypted-ebs
    resource: aws.ebs
    description: Find unencrypted EBS volumes
    filters:
      - type: value
        key: Encrypted
        value: false
    actions:
      - type: mark-for-op
        op: snapshot
        days: 7
      - type: notify
        template: default
        to:
          - [email protected]

Security: Find Security Groups with Open Ports

policies:
  - name: security-groups-open-ports
    resource: aws.security-group
    description: Find security groups with overly permissive rules
    filters:
      - type: ingress
        Ports:
          - 22
          - 3389
        Cidr: "0.0.0.0/0"
    actions:
      - type: notify
        to:
          - [email protected]
        subject: Open Port Security Group Alert

Cost Optimization

Find Unused EBS Volumes

policies:
  - name: unused-ebs-volumes
    resource: aws.ebs
    description: Find EBS volumes not attached to instances
    filters:
      - type: value
        key: Attachments
        value: []
    actions:
      - type: mark-for-op
        op: delete
        days: 30
      - type: notify
        to:
          - [email protected]
        subject: Unused EBS Volume - Will be deleted

Find Unused Elastic IPs

policies:
  - name: unused-eips
    resource: aws.eip
    description: Find unassigned Elastic IPs
    filters:
      - type: value
        key: Instance
        value: null
    actions:
      - type: release
        force: true

Find Old Snapshots

policies:
  - name: old-snapshots
    resource: aws.ebs-snapshot
    description: Find EBS snapshots older than 90 days
    filters:
      - type: value
        key: StartTime
        value: 90
        op: less-than-days
    actions:
      - type: notify
      - type: delete

Compliance Automation

Check for MFA on Root Account

policies:
  - name: root-account-mfa
    resource: aws.account
    description: Verify root account has MFA enabled
    region: us-east-1
    filters:
      - type: value
        key: PasswordLastUsed
        value: absent
    actions:
      - type: notify
        to:
          - [email protected]
        subject: Root Account MFA Required

Check for CloudTrail Enabled

policies:
  - name: ensure-cloudtrail
    resource: aws.cloudtrail
    description: Ensure CloudTrail is enabled
    filters:
      - type: value
        key: IsMultiRegionTrail
        value: false
    actions:
      - type: notify
      - type: update-trail
        trail:
          IsMultiRegionTrail: true
          EnableLogFileValidation: true

Check for untagged resources

 policies:
  - name: require-tags
    resource: aws.ec2
    description: Ensure required tags on EC2 instances
    filters:
      - type: missing-tag
        tags:
          - Environment
          - Owner
    actions:
      - type: notify
        to:
          - [email protected]

Running Policies

Local Execution

# Validate policy syntax
custodian validate policy.yaml

# Dry run (report only)
custodian run -s /tmp/output policy.yaml

# Apply policies
custodian run --dryrun false -s /tmp/output policy.yaml

# Filter specific resources
custodian run -s /tmp/output policy.yaml --resource aws.ec2

AWS Lambda Deployment

# policy.yaml
policies:
  - name: lambda-policy
    resource: aws.lambda
    mode:
      type: periodic
      schedule: "rate(1 day)"
      timeout: 300
      memory: 512
    filters:
      - type: value
        key: Runtime
        value: python3.8
    actions:
      - type: notify
        to:
          - [email protected]

# Package for Lambda
custodian package -s /tmp/pkg policy.yaml

# Deploy to AWS
custodian publish -s /tmp/pkg

Multi-Account Setup

AWS Organizations

# Run across all accounts
custodian run -s /tmp/output \
  --assume-role-role 'arn:aws:iam::ACCOUNT:role/CustodianRole' \
  policy.yaml

Azure Management Groups

policies:
  - name: azure-policy
    resource: azure.vm
    source: query
    query:
      - type: ResourceGroup
        group: my-resource-group

GCP Organizations

policies:
  - name: gcp-policy
    resource: gcp.instance
    source: organizational
    conditions:
      - type: value
        key: project.name
        value: my-project-*

Notifications

SNS Notification

actions:
  - type: notify
    to:
      - [email protected]
    subject: Policy Violation
    template: default
    transport:
      type: sqs
      queue: https://sqs.us-east-1.amazonaws.com/123456789/notifications

Webhook Notification

actions:
  - type: notify
    to:
      - webhook: https://hooks.slack.com/services/XXX
    subject: Alert

Best Practices

Policy Organization

# policies.yaml
policies:
  # Security policies
  - name: security-*
    resource: aws.*
    mode:
      type: periodic
      schedule: rate(1 hour)
  
  # Cost policies
  - name: cost-*
    resource: aws.*
    mode:
      type: periodic
      schedule: rate(1 day)

  # Compliance policies
  - name: compliance-*
    resource: aws.*
    mode:
      type: periodic
      schedule: rate(12 hours)

Tagging Strategy

policies:
  - name: enforce-tags
    resource: aws.ec2
    filters:
      - type: marked-for-op
        op: tag
        days: 0
    actions:
      - type: tag
        key: ComplianceRequired
        value: "true"

Implementation Checklist

Setup

  • Install Cloud Custodian
  • Configure cloud provider credentials
  • Test connectivity

Policy Development

  • Start with read-only policies
  • Test filters thoroughly
  • Add notification actions
  • Graduate to remediation

Deployment

  • Package policies for Lambda
  • Set up monitoring
  • Configure notifications
  • Document runbooks

Summary

Cloud Custodian enables powerful cloud security and compliance automation:

  1. Policy-as-code: Version-controlled, testable security policies
  2. Multi-cloud: Single tool for AWS, Azure, GCP
  3. Flexible: Filters and actions for any use case
  4. Scalable: Serverless execution via Lambda

Start with simple read-only policies, then add remediation as confidence grows.

External Resources

Comments