Elasticsearch Nested Filters being inclusive vs. exclusive
I believe here you might need the advantage of a flattened list of values, like an array of values. The major difference between an array and nested objects is that the latter "knows" which value of a nested property corresponds to another value of another property in the same nested object. The array of values, on the other hand will flatten the values of a certain property and you lose the "association" between a client_id
and a name
. Meaning, with arrays you have props.client_id = [null, 2]
and props.name = ["petlover", "premiumshopper"]
.
With your nested
filter you want to match that string to all values for props.name
meaning ALL nested props.name
s of one parent doc needs to match. Well, this doesn't happen with nested objects, because the nested documents are separate and are queried separately. And, if at least one nested document matches then it's considered a match.
In other words, for a query like "query": "props.name:(carlover NOT petlover)"
you basically need to run it against a flattened list of values, just like arrays. You need that query ran against ["carlover", "petlover"].
My suggestion for you is to make your nested documents "include_in_parent": true
(meaning, keep in parent a flattened, array-like list of values) and change a bit the queries:
- for the
query_string
part, use the flattened properties approach to be able to match your query for a combined list of elements, not element by element. - for the
match
(orterm
, see below) andmissing
parts use the nested properties approach because you can havenull
s in there. Amissing
on an array will match only if the whole array is missing, not one value in it, so here one cannot use the same approach as for the query, where the values were flattened in an array. - optional, but for the
query
match
integer I would useterm
, as it's not string but integer and is by defaultnot_analyzed
.
These being said, with the above changes, these are the changes:
{ "mappings" : { ... "props": { "type": "nested", "include_in_parent": true, ...
- should (and does) return zero results
GET /nesting-test/_search?pretty=true{ "query": { "filtered": { "filter": { "and": [ { "query": { "query_string": { "query": "props.name:((carlover AND premiumshopper) NOT petlover)" } } }, { "nested": { "path": "props", "filter": { "or": [ { "query": { "match": { "props.client_id": 1 } } }, { "missing": { "field": "props.client_id" } } ] } } } ] } } }}
- should (and does) return just 1
GET /nesting-test/_search?pretty=true{ "query": { "filtered": { "filter": { "and": [ {"query": {"query_string": { "query": "props.name:(carlover NOT petlover)" } } }, { "nested": { "path": "props", "filter": { "or": [{ "query": { "match": { "props.client_id": 1 } } },{ "missing": { "field": "props.client_id" } } ] } } } ] } } }}
- should (and does) return just 2
GET /nesting-test/_search?pretty=true{ "query": { "filtered": { "filter": { "and": [ { "query": {"query_string": { "query": "props.name:(* NOT carlover)" } } }, { "nested": { "path": "props", "filter": { "or": [{ "query": { "term": { "props.client_id": 1 } } },{ "missing": { "field": "props.client_id" } } ] } } } ] } } }}