Elasticsearch - Count duplicated and unique values Elasticsearch - Count duplicated and unique values elasticsearch elasticsearch

Elasticsearch - Count duplicated and unique values


Well I've made use of several aggregations here. The below are the lists which I've used. The order of the list is the execution order of the aggregation.

For Duplicates

For Non Duplicates

Aggregation Query:

POST <your_index_name>/_search{     "size":0,   "aggs":{        "duplicate_aggs":{           "terms":{              "field":"firstname.keyword",            "min_doc_count":2         }      },      "duplicate_bucketcount":{           "stats_bucket":{              "buckets_path":"duplicate_aggs._count"         }      },      "nonduplicate_aggs":{           "terms":{              "field":"firstname.keyword"         },         "aggs":{              "equal_one":{                 "bucket_selector":{                    "buckets_path":{                       "count":"_count"                  },                  "script":"params.count == 1"               }            }         }      },      "nonduplicate_bucketcount":{           "sum_bucket":{              "buckets_path":"nonduplicate_aggs._count"         }      }   }}

Response

{  "took": 10,  "timed_out": false,  "_shards": {    "total": 5,    "successful": 5,    "skipped": 0,    "failed": 0  },  "hits": {    "total": 8,    "max_score": 0,    "hits": []  },  "aggregations": {    "duplicate_aggs": {      "doc_count_error_upper_bound": 0,      "sum_other_doc_count": 0,      "buckets": [        {          "key": "jane",          "doc_count": 2        },        {          "key": "joe",          "doc_count": 2        },        {          "key": "john",          "doc_count": 2        }      ]    },    "nonduplicate_aggs": {      "doc_count_error_upper_bound": 0,      "sum_other_doc_count": 0,      "buckets": [        {          "key": "jack",          "doc_count": 1        },        {          "key": "steve",          "doc_count": 1        }      ]    },    "duplicate_bucketcount": {      "count": 3,      "min": 2,      "max": 2,      "avg": 2,      "sum": 6    },    "nonduplicate_bucketcount": {      "value": 2    }  }}

Notice that in the above response, we have duplicate_bucketcount.count key whose value 3 is what would display the bucket count which is the number of keys which are duplicates.

Let me know if it helps!