CloudWatch resource access policy error while creating Amazon Elasticsearch Service via Cloud Formation CloudWatch resource access policy error while creating Amazon Elasticsearch Service via Cloud Formation elasticsearch elasticsearch

CloudWatch resource access policy error while creating Amazon Elasticsearch Service via Cloud Formation


I believe there is some confusion here about what policies should be updated/set to enable ES writing to a log group.

I think you should apply the PolicyDocESIndexSlow policy to CloudWatch Logs.

And this can't be done in CloudFormation from what I remember. You have to use put-resource-policy, corresponding API call, or console as shown in:


The final code would be something like this,

DeployES lambda_function.py

import loggingimport timeimport boto3import jsonfrom crhelper import CfnResourcelogger = logging.getLogger(__name__)helper = CfnResource(json_logging=False, log_level='DEBUG', boto_level='CRITICAL', sleep_on_delete=120)try:    # Init code goes here    passexcept Exception as e:    helper.init_failure(e)@helper.create@helper.updatedef create(event, _):    logger.info("Got Create/Update")    my_log_group_arn = event['ResourceProperties']['MYLOGGROUPArn']    client = boto3.client('logs')    policy_document = dict()    policy_document['Version'] = '2012-10-17'    policy_document['Statement'] = [{        'Sid': 'ESLogsToCloudWatchLogs',        'Effect': 'Allow',        'Principal': {            'Service': [                'es.amazonaws.com'            ]        },        'Action': 'logs:*',    }]    policy_document['Statement'][0]['Resource'] = my_log_group_arn     client.put_resource_policy(policyName='ESIndexSlowPolicy', policyDocument=json.dumps(policy_document))    helper.Data['success'] = True    helper.Data['message'] = 'ES policy deployment successful'    # To return an error to Cloud Formation you raise an exception:    if not helper.Data["success"]:        raise Exception('Error message to cloud formation')    return "MYESIDDEFAULT"@helper.deletedef delete(event, _):    logger.info("Got Delete")    # Delete never returns anything. Should not fail if the underlying resources are already deleted.    # Desired state.    try:        client = boto3.client('logs')        client.delete_resource_policy(policyName='ESIndexSlowPolicy')    except Exception as ex:        logger.critical(f'ES policy delete failed with error [{repr(ex)}]')def lambda_handler(event, context):    helper(event, context)

And some additional components in CF template

 MYLAMBDAROLE:    Type: 'AWS::IAM::Role'    Properties:      AssumeRolePolicyDocument:        Version: 2012-10-17        Statement:          - Effect: Allow            Principal:              Service:                - lambda.amazonaws.com            Action:              - 'sts:AssumeRole'      ManagedPolicyArns:        - 'arn:aws:iam::aws:policy/AWSLambdaFullAccess'        - 'arn:aws:iam::aws:policy/AmazonS3FullAccess'        - 'arn:aws:iam::aws:policy/AmazonESFullAccess'        - 'arn:aws:iam::aws:policy/CloudWatchFullAccess'      RoleName: !Join        - '-'        - - lambda-role          - !Ref 'AWS::Region'  MYLAMBDADEPLOY:    Type: 'AWS::Lambda::Function'    Properties:      Code:        S3Bucket: es-bucket-for-lambda-ta86asdf596        S3Key: es.zip      FunctionName: deploy_es      Handler: lambda_function.lambda_handler      MemorySize: 128      Role: !GetAtt        - MYLAMBDAROLE        - Arn      Runtime: python3.8      Timeout: 60  MYESSETUP:    Type: 'Custom::MYESSETUP'    Properties:      ServiceToken: !GetAtt        - MYLAMBDADEPLOY        - Arn      MYLOGGROUPArn: !GetAtt        - MYLOGGROUP        - Arn    DependsOn:      - MYLAMBDADEPLOY      - MYLOGGROUP

And just add below DependsOn to MYESDOMAIN

DependsOn:  - MYESSETUP