Proper access policy for Amazon Elastic Search Cluster

You can lock access down to IAM-only, but how will you view Kibana in your browser? You could setup a proxy (see Gist and/or NPM module) or enable both IAM and IP-based access for viewing results.

I was able to get both IAM access IP-restricted access with the following Access Policy. Note the order is important: I could not get it working with the IP-based statement before the IAM statement.

{  "Version": "2012-10-17",  "Statement": [    {      "Effect": "Allow",      "Principal": {        "AWS": "arn:aws:iam::xxxxxxxxxxxx:root"      },      "Action": "es:*",      "Resource": "arn:aws:es:us-west-2:xxxxxxxxxxxx:domain/my-elasticsearch-domain/*"    },    {      "Sid": "",      "Effect": "Allow",      "Principal": {        "AWS": "*"      },      "Action": "es:*",      "Resource": "arn:aws:es:us-west-2:xxxxxxxxxxxx:domain/my-elasticsearch-domain/*",      "Condition": {        "IpAddress": {          "aws:SourceIp": [            "192.168.1.0",            "192.168.1.1"          ]        }      }    }  ]}

My EC2 instance has an instance profile with thearn:aws:iam::aws:policy/AmazonESFullAccesspolicy. Logstash should sign requests using the logstash-output-amazon-es output plugin. Logstash running on my EC2 instance includes an output section like this:

output {    amazon_es {        hosts => ["ELASTICSEARCH_HOST"]        region => "AWS_REGION"    }    # If you need to do some testing & debugging, uncomment this line:    # stdout { codec => rubydebug }}

I can access Kibana from the two IPs in the access policy (192.168.1.0 and 192.168.1.1).

According to AWS doc and as you (and I) just tested, you cannot restrict access to an AWS ES domain to a role/account/user/... and simply cURL it!

Standard clients, such as curl, cannot perform the request signing that is required of identity-based access policies. You must use an IP address-based access policy that allows anonymous access to successfully perform the instructions for this step. (http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg-search.html)

So you have basically two solutions:

• change your access policy and restrict it to IP(s), I think you cannot use private IP because your ES cluster does not seems to belong to your VPC (default or not). Please use the public IP
• sign your request: http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains.html#es-managedomains-signing-service-requests

Signing your request is probably the best solution if you want to keep your access policy as is (which is more flexible than restricting to an IP), but it seems to be a bit more complex. I haven't tried so far and I cannot find any doc to help.

A bit late to the party, but I was able to deal with the exact same issue by adding signature to my requests.

If you use Python (like I do), you can use the following library to make it particularly easy to implement: https://github.com/DavidMuller/aws-requests-auth

It worked perfectly for me.