Allowing Lambda in a VPC to access an Elasticsearch domain in the same VPC
Noticed two problems in your setup
- You are creating only one subnet in the stack and assigning only one subnet to Lambda. You need to assign multiple subnets to the Lambda
- You need to fix access policy for ES. I manually updated to "Don't require signing requst with IAM credential" for my testing in console.
I updated Cloudformation template to create python based lambda handler to query elastic search from the same vpc. It is not complete but if you make sure of the above mentioned issues then it should work.
AWSTemplateFormatVersion: 2010-09-09Description: The AWS CloudFormation tutorialResources: SomeDeploymentBucket: Type: 'AWS::S3::Bucket' AppLogGroup: Type: 'AWS::Logs::LogGroup' Properties: LogGroupName: /aws/lambda/some-lambda # ========= The Lambda Execution Role ========= IamRoleLambdaExecution: Type: 'AWS::IAM::Role' Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - lambda.amazonaws.com Action: - 'sts:AssumeRole' Policies: - PolicyName: !Join - '-' - - dev - some-app - lambda PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - 's3:*' - 'rds-db:connect' - 'rds:*' - 'es:*' Resource: '*' Path: / RoleName: !Join - '-' - - some-app - dev - eu-west-1 - lambdaRole ManagedPolicyArns: - !Join - '' - - 'arn:' - !Ref 'AWS::Partition' - ':iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole' # ========= The Lambda ========= AppLambdaFunction: Type: AWS::Lambda::Function DependsOn: - AppLogGroup - IamRoleLambdaExecution Properties: FunctionName: some-lambda Handler: index.lambda_handler Runtime: python2.7 Timeout: 60 MemorySize: 1024 Role: !GetAtt - IamRoleLambdaExecution - Arn VpcConfig: SecurityGroupIds: - !Ref xxxVPCSecurityGroup SubnetIds: - !Ref xxxLambdaSubnet1 - !Ref xxxLambdaSubnet2 Code: ZipFile: !Sub | from __future__ import print_function import boto3 iam = boto3.client('iam') def lambda_handler(event, context): print('called lambda_handler') # ========= VPC ========= xxxVPC: Type: 'AWS::EC2::VPC' Properties: CidrBlock: 172.31.0.0/16 InstanceTenancy: default EnableDnsSupport: 'true' EnableDnsHostnames: 'true' xxxVPCSecurityGroup: Type: 'AWS::EC2::SecurityGroup' Properties: GroupName: VPC SG GroupDescription: VPC Security Group VpcId: !Ref xxxVPC xxxLambdaSubnet1: Type: 'AWS::EC2::Subnet' Properties: VpcId: !Ref xxxVPC CidrBlock: 172.31.32.0/20 xxxLambdaSubnet2: Type: 'AWS::EC2::Subnet' Properties: VpcId: !Ref xxxVPC CidrBlock: 172.31.16.0/20 # ========= Elasticsearch ========= xxxESSecurityGroup: Type: 'AWS::EC2::SecurityGroup' Properties: GroupName: ES SG GroupDescription: ES Security group VpcId: !Ref xxxVPC SecurityGroupIngress: - IpProtocol: -1 FromPort: 0 ToPort: 65535 SourceSecurityGroupId: !Ref xxxVPCSecurityGroup xxxElasticSearch: Type: 'AWS::Elasticsearch::Domain' Properties: AccessPolicies: Version: 2012-10-17 Statement: - Action: - 'es:*' - 'ec2:*' - 's3:*' Principal: AWS: - '*' Resource: '*' Effect: Allow DomainName: es-xxx-domain EBSOptions: EBSEnabled: true Iops: 0 VolumeSize: 20 VolumeType: "gp2" AdvancedOptions: rest.action.multi.allow_explicit_index: 'true' ElasticsearchClusterConfig: InstanceCount: 1 InstanceType: m4.large.elasticsearch DedicatedMasterEnabled: 'false' VPCOptions: SecurityGroupIds: - !Ref xxxESSecurityGroup SubnetIds: - !Ref xxxLambdaSubnet1
Updated Lambda function handler code
import urllib2def lambda_handler(event, context): print('called lambda_handler') data = '' url = 'https://vpc-es-xxx-domain-fixthis.es.amazonaws.com' req = urllib2.Request(url, data, {'Content-Type': 'application/json'}) f = urllib2.urlopen(req) for x in f: print(x) f.close()