Compliance Framework - Cloud Custodian Plugin

The Cloud Custodian Plugin runs Cloud Custodian policies in dry-run mode, builds a full inventory baseline for each configured resource type, converts each resource/policy pair into a standardized per-resource payload, and then executes CCF OPA bundles against those payloads to generate evidence.

Behavior Overview

1. Load Cloud Custodian policy YAML from config.
2. Parse top-level policies and iterate one policy entry per check.
3. Register runtime-derived subject templates for each configured resource type during runner-v2 Init.
4. Run one unfiltered inventory collection for each unique resource type.
5. Run each configured check with:

custodian run --dryrun -s <output-dir> <single-policy-file>

6. Compare each check's matched resources with the inventory baseline.
7. Build one standardized payload per resource per check. Matched resources are marked non_compliant; baseline resources not matched by that check are marked compliant.
8. Evaluate each OPA policy bundle path from agent EvalRequest.policyPaths.
9. Send evidence via the plugin gRPC helper (CreateEvidence).

Safety Model

This plugin always enforces read-only Cloud Custodian execution:

• --dryrun is always used.
• Mutating actions are not applied.
• For AWS checks, the plugin runs with --region all to evaluate across all AWS regions by default unless aws_regions is configured.
• With more than one region (including all), Cloud Custodian writes one resources.json per region (<output>/<region>/<policy>/resources.json). The plugin reads every one of them, for the inventory baseline and for each check.
• On execution failures or timeouts, the plugin includes the tail of any generated custodian-run.log artifacts in the execution error.

Configuration

All plugin config fields are strings (agent gRPC map<string,string> contract).

Key	Required	Description
policies_yaml	Conditionally	Inline Cloud Custodian policy YAML. Preferred over policies_path when both are set.
policies_path	Conditionally	Local path, file://, http://, or https:// location for policy YAML. Used when policies_yaml is empty.
custodian_binary	No	Path/name of Cloud Custodian executable. Default: custodian.
custodian_debug	No	Boolean (true/false) toggle to pass --debug to Cloud Custodian. This increases Cloud Custodian diagnostic output on stderr. Default: false.
custodian_verbose	No	Boolean (true/false) toggle to pass -v to Cloud Custodian. This increases Cloud Custodian diagnostic output on stderr. Default: false.
custodian_aws_api_trace	No	Boolean (true/false) toggle to inject a temporary Python sitecustomize.py into the Custodian child process. Logs botocore API start/end/error events to stderr and custodian-aws-api-trace.jsonl in the check output directory. Default: false.
custodian_network_diagnostics	No	Boolean (true/false) toggle to run Go DNS/TLS preflight probes for relevant AWS service endpoints before Custodian starts and log child process TCP socket snapshots while Custodian is running. DNS/TLS probe failures are logged as warnings and surfaced in evidence as execution.warnings. Concrete aws_regions are narrowed to regions with reachable resource service endpoints before Custodian runs; if none are reachable for a check, that check is skipped, later checks continue, and the accumulated diagnostic warnings are returned after reachable evidence is submitted. If no concrete aws_regions are configured, service-derived probes are skipped unless custodian_network_diagnostic_endpoints is configured. For AWS resource types not mapped to diagnostic services, diagnostics warn and continue unless explicit endpoints are configured. Invalid configured endpoints still fail before execution. Default: false.
custodian_network_diagnostic_endpoints	No	Comma or whitespace separated list of additional endpoint hostnames, host:port values, or HTTPS URLs to DNS/TLS probe when custodian_network_diagnostics is enabled. Non-HTTPS URL schemes are rejected. Use this for AWS VPC endpoint DNS names such as vpce-123.backup.eu-west-1.vpce.amazonaws.com. Default: unset.
custodian_log_tail_during_run	No	Boolean (true/false) toggle to tail discovered custodian-run.log artifacts during the monitor loop, not only after process exit. Default: false.
aws_regions	No	Comma or whitespace separated AWS regions passed as repeated --region flags. Duplicate entries are removed while preserving order. Default: unset, which falls back to --region all for AWS checks.
check_timeout_seconds	No	Per-check timeout in seconds. Default: 300.
policy_labels	No	JSON map of labels merged into generated evidence labels.
resource_identity_fields	No	JSON object mapping Cloud Custodian resource types to ordered identity field paths. Built-in defaults (see Resource Identity) are used after configured fields. Example: {"aws.ec2":["InstanceId","Arn"]}.
debug_dump_payloads	No	Boolean (true/false) toggle to write standardized resource payload JSON files for troubleshooting. Default: false.
debug_payload_output_dir	No	Directory where debug payload JSON files are written. If set, debug dumping is auto-enabled. Default when enabled without explicit path: ccf-custodian-debug-payloads under the OS temp directory ($TMPDIR, else /tmp), e.g. /tmp/ccf-custodian-debug-payloads. Use an absolute path: a relative value is still honoured but resolves against the plugin process working directory (which the agent may set per plugin) and logs a warning. The resolved absolute path is logged at INFO when dumping is enabled.
preserve_execution_artifacts	No	Boolean (true/false) toggle to keep the temporary Cloud Custodian execution root after a check execution failure for postmortem review. Default: false.
aws_accounts	No	JSON (or YAML) array of AWS accounts to scan: {"id", "name", "role_arn", "role_name", "external_id", "regions", "use_ambient_credentials"}, only id required. See Cross-Account Scanning. Default: unset, which scans the account of the plugin's own credentials.
aws_assume_role_name	No	Role name assumed in every account without its own role_arn or role_name. Only used with aws_accounts.
aws_external_id	No	External ID passed when assuming a role in an account without its own external_id. Only used with aws_accounts. Never logged.
aws_partition	No	Partition of the role ARNs built from role names (aws, aws-cn, aws-us-gov, ...). Only used with aws_accounts. Default: derived from the account's regions (cn-* is aws-cn, us-gov-* is aws-us-gov), else aws.
aws_session_name_suffix	No	Suffix of the STS session name, CloudCustodian@<suffix> (C7N_SESSION_SUFFIX). Only used with aws_accounts. Default: ccf.

Validation rules:

• At least one of policies_yaml or policies_path must be provided.
• custodian_binary must resolve on PATH (or as explicit executable path).
• check_timeout_seconds must be a positive integer.
• resource_identity_fields, when set, must be valid JSON and each resource type must include at least one field path.
• Policy YAML must include top-level policies array.
• With aws_accounts: every account id is 12 digits and listed once; each account has a role (role_arn, role_name or aws_assume_role_name) or use_ambient_credentials: true, but not its own role and ambient credentials together, and at most one account uses ambient credentials; a role_arn belongs to its account; an external ID is 2 to 1224 characters from [A-Za-z0-9_+=,.@:/-]; unknown account keys are rejected; every policy is an AWS policy. Without aws_accounts, the other aws_* account keys are ignored with a warning.

Example Agent Config (Inline YAML)

plugins:
  cloud_custodian:
    source: ./dist/plugin
    policies:
      - ./policy-bundle
    config:
      policies_yaml: |
        policies:
          - name: ec2-public-ip-check
            resource: aws.ec2
            filters:
              - type: value
                key: PublicIpAddress
                op: not-null
      custodian_binary: custodian
      check_timeout_seconds: "300"
      policy_labels: '{"team":"cloud-security","environment":"prod"}'

Example Agent Config (Path/URL Fallback)

plugins:
  cloud_custodian:
    source: ./dist/plugin
    policies:
      - ./policy-bundle
    config:
      policies_path: file:///etc/ccf/cloud-custodian.yaml
      custodian_binary: /usr/local/bin/custodian

Cross-Account Scanning

Without aws_accounts the plugin scans the account of its own (ambient) credentials, exactly as before. With aws_accounts, it scans only the listed accounts, one after the other, assuming a role in each through Cloud Custodian's --assume and --external-id. The account the plugin runs in (the hub) is scanned only when it is listed with use_ambient_credentials: true. Its id is checked against the accounts in the ARNs of the resources it finds: if they name another account, the hub fails and submits no evidence (the other accounts still run); if no ARN names an account, the plugin logs a warning that it can't verify the id.

For each account:

• the role is the entry's role_arn, else arn:<partition>:iam::<id>:role/<role_name>, else the same with aws_assume_role_name;
• the external ID is the entry's external_id, else aws_external_id;
• the regions are the entry's regions, else aws_regions, else all.

plugins:
  cloud_custodian:
    source: ghcr.io/compliance-framework/plugin-cloud-custodian:<version>
    policies:
      - ghcr.io/compliance-framework/plugin-cloud-custodian-policies:<version>
    config:
      policies_path: file:///etc/ccf/cloud-custodian.yaml
      aws_regions: "eu-west-1,us-east-1"
      aws_assume_role_name: ccf-audit
      aws_external_id: "${env:CCF_CUSTODIAN_EXTERNAL_ID}"
      aws_accounts: |
        [
          {"id": "111111111111", "name": "hub", "use_ambient_credentials": true},
          {"id": "222222222222", "name": "prod", "role_name": "ccf-prod-audit", "external_id": "${env:CCF_PROD_EXTERNAL_ID}"},
          {"id": "333333333333", "name": "dev", "regions": ["eu-west-1"]},
          {"id": "444444444444", "name": "legacy", "role_arn": "arn:aws:iam::444444444444:role/security/ccf-audit"}
        ]

Here the hub uses the agent's own credentials; prod assumes ccf-prod-audit with its own external ID; dev assumes ccf-audit with the global external ID in eu-west-1 only; legacy assumes the given role with the global external ID. The agent resolves the ${env:...} placeholders before the plugin sees them.

What changes in accounts mode:

• The account_id label is the scanned account, whatever the resource data says, and account_name is added when the account has a name. Both are reserved: policy_labels can't set them.
• The payload gets "account": {"id": "...", "name": "..."}, and resource.account_id is the scanned account.
• account_id is part of the evidence UUID seed and of the resource subject id (cloud-custodian-resource/<account>/<type>/<id>), and the subject templates' identity keys are provider, account_id, resource_type, resource_id. The same resource id in two accounts is two resources. Moving an existing deployment to aws_accounts therefore re-keys its evidence streams once. account_name is not part of the seed, so renaming an account keeps its evidence streams.
• Each account has its own inventory baseline, Cloud Custodian cache file and output directory.

When an account fails (for example its role can't be assumed, which skips the account's remaining runs), the other accounts still run and submit their evidence. The run then fails with one error whose first line names every failed account, followed, per account, by its role, its errors and the tail of Cloud Custodian's output. The agent records that error as the plugin run's error (its log and the agent run evidence).

Regions

With all (no regions and no aws_regions), Cloud Custodian lists the enabled regions with the hub's credentials (ec2:DescribeRegions), not the target account's, so an opt-in region enabled only in a target account is missed. The plugin logs a warning for each such account; list the regions to scan them. On private networks without a route to the global STS endpoint, set C7N_USE_STS_REGIONAL=true in the agent's environment so Cloud Custodian assumes roles through the regional STS endpoint (for example a VPC endpoint).

IAM

• The hub principal (the plugin's own credentials) needs sts:AssumeRole on the target roles. Narrow it to them, for example with an aws:ResourceOrgID condition or a list of the role ARNs, plus ec2:DescribeRegions when any account scans all regions.
• Each target role trusts the hub principal with an sts:ExternalId condition, and can also require the session name:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"AWS": "arn:aws:iam::111111111111:role/ccf-agent"},
    "Action": "sts:AssumeRole",
    "Condition": {
      "StringEquals": {"sts:ExternalId": "<external id>"},
      "StringLike": {"sts:RoleSessionName": "CloudCustodian@ccf*"}
    }
  }]
}

• The target role's permissions are the AWS managed policies SecurityAudit and job-function/ViewOnlyAccess, not ReadOnlyAccess, which can also read data (S3 objects, DynamoDB items, secrets).

Remote configuration

Never list aws_accounts, aws_assume_role_name, aws_external_id, aws_partition or aws_session_name_suffix in the agent's remote_config.overridable_config_flags (and don't use * there for this plugin): a remote overlay could then point the plugin at other accounts or roles without an operator's review.

Standardized Per-Resource OPA Input

Each resource/check iteration produces one payload with this shape:

{
  "schema_version": "v2",
  "source": "cloud-custodian",
  "check": {
    "name": "ec2-public-ip-check",
    "resource": "aws.ec2",
    "provider": "aws",
    "index": 0,
    "metadata": {}
  },
  "resource": {
    "id": "i-1234567890abcdef0",
    "type": "aws.ec2",
    "provider": "aws",
    "account_id": "123456789012",
    "region": "us-east-1",
    "identity_fields": {
      "InstanceId": "i-1234567890abcdef0"
    },
    "data": {"...": "..."}
  },
  "assessment": {
    "status": "non_compliant",
    "matched": true,
    "inventory_status": "baseline",
    "matched_resource_count": 3,
    "artifact_path": "/tmp/ccf-cloud-custodian-123/001-ec2-public-ip-check",
    "resources_path": "/tmp/ccf-cloud-custodian-123/001-ec2-public-ip-check/ec2-public-ip-check/resources.json"
  },
  "execution": {
    "status": "success",
    "dry_run": true,
    "exit_code": 0,
    "started_at": "2026-03-06T12:00:00Z",
    "ended_at": "2026-03-06T12:00:01Z",
    "duration_ms": 1000,
    "stdout": "...",
    "stderr": "",
    "error": "",
    "errors": []
  },
  "raw_policy": {
    "name": "ec2-public-ip-check",
    "resource": "aws.ec2",
    "filters": []
  }
}

Generated evidence labels include resource_id, resource_type, provider, and any available account_id/region (with aws_accounts: the scanned account's account_id, its account_name, and an account object in the payload, see Cross-Account Scanning). A region in the resource data is used as is; otherwise, when Cloud Custodian wrote per-region output, region is the region directory the resource came from. Global resource types (IAM, CloudFront, Route 53) and resources returned identically by every region (S3 buckets, which ListBuckets returns everywhere) are kept once and get no region from the directory. The resource subject includes the resource identifier as a link and a resource_id property. When the AWS resource identifier is an ARN, generated evidence includes an AWS Resource Explorer link using an exact id:<arn> search. AWS S3 bucket names are parsed as arn:aws:s3:::<bucket-name> for Resource Explorer links. AWS Route53 hosted zone identifiers such as /hostedzone/Z123 are normalized to full ARNs such as arn:aws:route53:::hostedzone/Z123.

assessment.inventory_status is baseline for resources found in the unfiltered inventory run. If a policy returns a resource that is not present in the baseline, the plugin still evaluates it as non_compliant and sets inventory_status to missing_from_baseline.

Resource Identity

resource.id (and the resource_id label) is the first non-empty field from, in order:

1. the type's resource_identity_fields, when configured;
2. the built-in field list for the AWS type below: the ARN first, then the natural id, following the resource metadata of Cloud Custodian 0.9.49;
3. the generic fields Arn, ARN, arn, Id, ID, id, InstanceId, instance_id, InstanceID, ResourceId, resource_id, Name, name;
4. a SHA-256 of the whole resource, as a last resort.

Type	Fields
aws.app-elb	LoadBalancerArn
aws.backup-plan	BackupPlanArn, BackupPlanName
aws.backup-vault	BackupVaultArn, BackupVaultName
aws.cache-cluster	ARN, CacheClusterId
aws.distribution	ARN, Id
aws.dynamodb-table	TableArn, TableName
aws.ebs	VolumeId
aws.ec2	InstanceId
aws.ecs-service	serviceArn
aws.efs	FileSystemArn, FileSystemId
aws.eks	arn, name
aws.firewall	FirewallArn, FirewallName
aws.hostedzone	Id (normalized to an ARN)
aws.iam-group	Arn, GroupName
aws.iam-policy	Arn, PolicyId
aws.iam-role	Arn, RoleName
aws.iam-user	Arn, UserName
aws.kms-key	Arn, KeyId
aws.lambda	FunctionArn, FunctionName
aws.log-group	arn, logGroupName
aws.rds	DBInstanceArn, DBInstanceIdentifier
aws.rds-cluster	DBClusterArn, DBClusterIdentifier
aws.s3	Name
aws.secrets-manager	ARN, Name
aws.sns	TopicArn
aws.sqs	QueueArn, QueueUrl
aws.transfer-server	Arn, ServerId
aws.wafv2	ARN, Id

The id never depends on the annotations Cloud Custodian adds to matched resources (c7n:MatchedFilters and the like), so a check's matched resource is the same resource as its inventory baseline record.

provider extraction rule:

• aws.s3 -> aws
• non <provider>.<resource> formats -> unknown

Policy Data

Any policy_data set on this plugin's block in the agent configuration is passed to every policy evaluation and is available to Rego as data.<key>:

plugins:
  cloud_custodian:
    # ...plugin settings as above...
    policy_data:
      allowed_regions:
        - eu-west-1
        - us-east-1

Operational Prerequisites

• Cloud Custodian CLI must be installed and executable.
• Cloud/provider credentials must be available in the plugin process environment (ambient credentials/profile/env vars). With aws_accounts, these are the hub's credentials, used to assume each account's role.

Testing

Run:

go test ./...