*** Install dependencies (with virtualenv) ***
$ sudo apt-get -y install virtualenv or sudo yum install virtualenv
$ virtualenv custodian_env
$ source custodian_env/bin/activate

*** Install AWS CLI and C7N ***
$ pip install awscli c7n

** Configure AWSCLI **
$ aws configure
(Configure with AWS Credentials and Region)

*** Verify AWSCLI Installation with any CLI command ***
$ aws ec2 describe-regions

*** To Install Cloud Custodian Mailer ***
*** Install repository***
$ git clone https://github.com/capitalone/cloud-custodian
$ cd cloud-custodian/tools/c7n_mailer
$ pip install -r requirements.txt
$ python setup.py develop

*** Verify Installation ***
$ c7n-mailer
$ custodian

For more info, check out Cloud Custodian in GitHub

Cloud Custodian must be run within a virtual environment.

$ cd ~
$ source custodian_env/bin/activate
$ cd cloudcustodian_scripts  (this is the folder where all the custodian policies reside)

** Execute/run the Cloud Custodian Policies **

# Validate the configuration
$ custodian validate s3-bucket-public-access.yml

# Dryrun the policies 
$ custodian run --dryrun -s check-public-access s3-bucket-public-access.yml
(Note: Make sure If you get a match (e.g. count > 0), then run the below command)

# Run the policy 
$ custodian run -s check-public-access s3-bucket-public-access.yml

** Invoking c7n Mailer **
# Validate the configuration
$ custodian validate s3-bucket-public-access.yml

# Dryrun the policies 
$ custodian run --dryrun -s check-public-access s3-bucket-public-access.yml
(Note: Make sure If you get a match (e.g. count > 0), then run the below command)

# Run the policy to invoke custodian mailer
$ c7n-mailer --config mailer.yml --update-lambda && custodian run -c s3-bucket-public-access.yml -s .

When we run this policy, Check the AWS console for a new Lambda named `cloud-custodian-mailer`. 
The mailer runs every five minutes, so wait a bit and then look for an email in your inbox. (Orelse manually, edit CWE scheduled time less than 5 mins for the quick response)



 Cloud Custodian will create a log files in the ~/cloudcustodian_scripts/check-public-access/ subdirectory IF there are any matches. 

Steps for Cloud Custodian mailer to ensure S3 Governance and Compliance.
Step 1: Create Mailer file

Step 2: Create Custodian Policy for S3 Public read/write Access - Sends email notification via Simple Email Service (SES) using notify action

$ vim s3-bucket-public-access-check.yml .

Step 3: Run a Command that installs the mailer and run a policy that triggers an email to your inbox.

$ c7n-mailer --config mailer.yml --update-lambda && custodian run -c s3-bucket-public-access-check.yml -s .

Step 4: Check the AWS console for a new Lambda and CWE named "cloud-custodian-mailer" and "custodian-s3-public-access".

Lambda Functions:

CloudWatch Events:

CWE S3 Bucket Logs:

CWE Custodian mailer Logs:

Step 5: Cloud Custodian mailer deployed lambda and sends a customized mail via SES service.

#Which queue should we listen to for messages
queue_url: https://sqs.us-east-1.amazonaws.com/930337447539/c7n_mailer_for_s3_events
#Standard Lambda Function Config
region: us-east-1
role: arn:aws:iam::930337447539:role/lambda-s3-governance
#Default from address
from_address: [email protected]

Note: Based on your use case, additional permissions may be needed.
Cloud Custodian will generate a msg if that is the case after invocation.
AWS IAM Role & policies plays an important role to allows Lambda functions to call AWS services. (Make a note of IAM ARN ex: arn:aws:iam::930337447539:role/S3-GovernanceForLincoln)
Trust relationship:
"Service": "lambda.amazonaws.com"

Reference:
| AWSS3CustomPolicyForLincoln.json
 | A policy defines the AWS permissions that you can assign to a user, group, or role.  |

(custodian_env) [root@localhost custodian_scripts]# custodian schema s3
aws.s3:
actions: [attach-encrypt, auto-tag-user, configure-lifecycle, delete, delete-bucket-notification,
delete-global-grants, encrypt-keys, encryption-policy, invoke-lambda, mark-for-op,
no-op, notify, put-metric, remove-statements, remove-website-hosting, set-bucket-encryption,
set-inventory, set-statements, tag, toggle-logging, toggle-versioning, unmark]
filters: [and, bucket-encryption, bucket-notification, cross-account, data-events,
event, global-grants, has-statement, inventory, is-log-target, marked-for-op,
metrics, missing-policy-statement, missing-statement, no-encryption-statement,
not, or, value]
[ OR ]
** For S3 Schema Filters **
(custodian_env) [root@localhost custodian_scripts]# custodian schema s3.filters
aws.s3:
filters: [and, bucket-encryption, bucket-notification, cross-account, data-events,
event, global-grants, has-statement, inventory, is-log-target, marked-for-op,
metrics, missing-policy-statement, missing-statement, no-encryption-statement,
not, or, value]
** For S3 Schema actions **
(custodian_env) [root@localhost lfg-custodian]# custodian schema s3.actions
aws.s3:
actions: [attach-encrypt, auto-tag-user, configure-lifecycle, delete, delete-bucket-notification,
delete-global-grants, encrypt-keys, encryption-policy, invoke-lambda, mark-for-op,
no-op, notify, put-metric, remove-statements, remove-website-hosting, set-bucket-encryption,
set-inventory, set-statements, tag, toggle-logging, toggle-versioning, unmark]
** To undesrtand a particular filter & action: **
(custodian_env) [root@localhost custodian_scripts]# custodian schema s3.filters.global-grants
Help

Filters for all S3 buckets that have global-grants
:example:
.. code-block:: yaml
    policies:
      - name: s3-delete-global-grants
        resource: s3
        filters:
          - type: global-grants
        actions:
          - delete-global-grants

Schema

{
"additionalProperties": false,
"required": [
"type"
],
"type": "object",
"properties": {
"allow_website": {
"type": "boolean"
},
"operator": {
"enum": [
"or",
"and"
],
"type": "string"
},
"type": {
"enum": [
"global-grants"
]
},
"permissions": {
"items": {
"enum": [
"READ",
"WRITE",
"WRITE_ACP",
"READ",
"READ_ACP"
],
"type": "string"
},
"type": "array"
}
}
}

Use 'custodian validate' to find syntax errors
Check 'name' of policy doesn't contain spaces
Check SQS to see if Custodian payload is entering the queue
Check cloud-custodian-mailer lambda CloudWatch rule schedule (5 minute by default)
Check Lambda error logs (this requires CloudWatch logging)
Check role for lambda(s) have adequate permissions
Remember to update the cloud-custodian-mailer lambda when making changes to a policy that uses notifications
Clear the cache if you encounter errors due to stale information (rm ~/.cache/cloud-custodian.cache)