Unable to add 2 subnets for an ElasticSearch with Terraform
You're missing zone_awareness_enabled
parameter in the cluster_config
which is required when using multi AZ Elasticsearch clusters.
Thanks @ydaetskcoR for pointing out the way.
I'll share the struggles I had with the configuration of availability_zone_count
and subnet_ids
- hopefully it can save some time for others.
Some context to the problem:
A) I tried to create a Multi zone ES cluster.
B) I had 4 subnets for the data tier (contains other types of DBs as well) and wanted the cluster to be splitted between the available AZs in the current region (3 AZs) - so one of the AZs will have 2 subnets and 2 ES Instances.
Be aware that:
1: The availability_zone_count
field under the zone_awareness_config
block should have the exact amount like the available AZs.
2: The subnet_ids
field under the vpc_options
block should contain the same number of AZs you specified under availability_zone_count
.
So, in one sentence: availability_zone_count == (available AZs) == length( subnet_ids)
Below is a code snippet with the relevant parts (follow also the comments - it might also save you some time):
resource "aws_elasticsearch_domain" "staging" { domain_name = ... vpc_options{ subnet_ids = "${local.subnet_ids}" # Instead of: [for s in aws_subnet.data_tier : s.id] which will lead to: Error creating ElasticSearch domain: ValidationException: You must specify exactly three subnets because you’ve set zone count to three. } cluster_config { zone_awareness_enabled = true #If you ignore it you'll get: Error creating ElasticSearch domain: ValidationException: You must specify exactly one subnet #Notice that there is no "=" Below - or you'll visit this thread: https://github.com/terraform-providers/terraform-provider-aws/issues/12365 zone_awareness_config { availability_zone_count = "${length(var.region_azs)}" } } . . }#Take only X number of subnets where X is the number of available AZs)locals { subnet_ids = "${slice(aws_subnet.data_tier.*.id, 0, length(var.region_azs))}"} # Added this also due to: Error creating ElasticSearch domain: ValidationException: Before you can proceed, you must enable a service-linked role to give Amazon ES permissions to access your VPC.# Solved with: https://stackoverflow.com/questions/47229247/validationexception-before-you-can-proceed-you-must-enable-a-service-linked-ro (Terraform related Answer)resource "aws_iam_service_linked_role" "es" { aws_service_name = "es.amazonaws.com" description = "Allows Amazon ES to manage AWS resources for a domain on your behalf."}