CloudWatch resource access policy error while creating Amazon Elasticsearch Service via Cloud Formation
amazon-web-services
elasticsearch
amazon-cloudformation
amazon-iam
aws-cloudformation-custom-resource
I believe there is some confusion here about what policies should be updated/set to enable ES writing to a log group.
I think you should apply the PolicyDocESIndexSlow
policy to CloudWatch Logs.
And this can't be done in CloudFormation from what I remember. You have to use put-resource-policy, corresponding API call, or console as shown in:
amazon-web-services
elasticsearch
amazon-cloudformation
amazon-iam
aws-cloudformation-custom-resource
The final code would be something like this,
DeployES lambda_function.py
import loggingimport timeimport boto3import jsonfrom crhelper import CfnResourcelogger = logging.getLogger(__name__)helper = CfnResource(json_logging=False, log_level='DEBUG', boto_level='CRITICAL', sleep_on_delete=120)try: # Init code goes here passexcept Exception as e: helper.init_failure(e)@helper.create@helper.updatedef create(event, _): logger.info("Got Create/Update") my_log_group_arn = event['ResourceProperties']['MYLOGGROUPArn'] client = boto3.client('logs') policy_document = dict() policy_document['Version'] = '2012-10-17' policy_document['Statement'] = [{ 'Sid': 'ESLogsToCloudWatchLogs', 'Effect': 'Allow', 'Principal': { 'Service': [ 'es.amazonaws.com' ] }, 'Action': 'logs:*', }] policy_document['Statement'][0]['Resource'] = my_log_group_arn client.put_resource_policy(policyName='ESIndexSlowPolicy', policyDocument=json.dumps(policy_document)) helper.Data['success'] = True helper.Data['message'] = 'ES policy deployment successful' # To return an error to Cloud Formation you raise an exception: if not helper.Data["success"]: raise Exception('Error message to cloud formation') return "MYESIDDEFAULT"@helper.deletedef delete(event, _): logger.info("Got Delete") # Delete never returns anything. Should not fail if the underlying resources are already deleted. # Desired state. try: client = boto3.client('logs') client.delete_resource_policy(policyName='ESIndexSlowPolicy') except Exception as ex: logger.critical(f'ES policy delete failed with error [{repr(ex)}]')def lambda_handler(event, context): helper(event, context)
And some additional components in CF template
MYLAMBDAROLE: Type: 'AWS::IAM::Role' Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - lambda.amazonaws.com Action: - 'sts:AssumeRole' ManagedPolicyArns: - 'arn:aws:iam::aws:policy/AWSLambdaFullAccess' - 'arn:aws:iam::aws:policy/AmazonS3FullAccess' - 'arn:aws:iam::aws:policy/AmazonESFullAccess' - 'arn:aws:iam::aws:policy/CloudWatchFullAccess' RoleName: !Join - '-' - - lambda-role - !Ref 'AWS::Region' MYLAMBDADEPLOY: Type: 'AWS::Lambda::Function' Properties: Code: S3Bucket: es-bucket-for-lambda-ta86asdf596 S3Key: es.zip FunctionName: deploy_es Handler: lambda_function.lambda_handler MemorySize: 128 Role: !GetAtt - MYLAMBDAROLE - Arn Runtime: python3.8 Timeout: 60 MYESSETUP: Type: 'Custom::MYESSETUP' Properties: ServiceToken: !GetAtt - MYLAMBDADEPLOY - Arn MYLOGGROUPArn: !GetAtt - MYLOGGROUP - Arn DependsOn: - MYLAMBDADEPLOY - MYLOGGROUP
And just add below DependsOn
to MYESDOMAIN
DependsOn: - MYESSETUP