How to integrate ElasticSearch 7.0 version with Spring Boot?

UPDATE

Spring Boot 2.3 is integrating spring-data-elasticsearch 4 so it will support ElasticSearch 7.x out of the box. It will be released soon but you can already try it:

plugins {  id 'org.springframework.boot' version '2.3.0.RC1'  id 'io.spring.dependency-management' version '1.0.9.RELEASE'}

I have tested it positively and all of my test scenarios are passing so I would definitely recommend this way. I will keep the answer below for people that for some reasons can't upgrade to 2.3.

OLD WORKAROUND (previous versions original answer)

As we don`t really know when Spring Data Elastic Search 4.x is going to be released I am posting my way of integrating the current Spring Data Elastic Search 4.x and stable Spring Boot 2.1.7. It might work as a temporary workaround for you if you want to work with the Spring Repositories and the newest Elastic Search.

1) Force newest elastic search client in your dependencies (in my case: build.gradle)

dependencies {    //Force spring-data to use the newest elastic-search client    //this should removed as soon as spring-data-elasticsearch:4.0.0 is released!    implementation('org.springframework.data:spring-data-elasticsearch:4.0.0.BUILD-SNAPSHOT') {        exclude group: 'org.elasticsearch'        exclude group: 'org.elasticsearch.plugin'        exclude group: 'org.elasticsearch.client'    }    implementation('org.elasticsearch:elasticsearch:7.3.0') { force = true }    implementation('org.elasticsearch.client:elasticsearch-rest-high-level-client:7.3.0') { force = true }    implementation('org.elasticsearch.client:elasticsearch-rest-client:7.3.0') { force = true }}

2) Disable the Elastic Search auto configuration and health check components as they become incompatible (you may later want to implement your own health check).

@SpringBootApplication(exclude = {ElasticsearchAutoConfiguration.class, ElasticSearchRestHealthIndicatorAutoConfiguration.class})@EnableElasticsearchRepositoriespublic class SpringBootApp {    public static void main(String[] args) {        SpringApplication.run(SpringBootApp.class, args);    }}

3) As we disabled the auto configuration we need to initialize the ElasticsearchRestTemplate ourselves. We also need to do it to provide the custom MappingElasticsearchConverter to avoid class incompatibilities.

/** * Manual configuration to support the newest ElasticSearch that is currently not supported by {@link org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchAutoConfiguration}. * * @author aleksanderlech */@Configuration@EnableConfigurationProperties(ElasticsearchProperties.class)public class ElasticSearchConfiguration {    @Primary    @Bean    public ElasticsearchRestTemplate elasticsearchTemplate(ElasticsearchProperties configuration) {        var nodes =  Stream.of(configuration.getClusterNodes().split(",")).map(HttpHost::create).toArray(HttpHost[]::new);        var client = new RestHighLevelClient(RestClient.builder(nodes));        var converter = new CustomElasticSearchConverter(new SimpleElasticsearchMappingContext(), createConversionService());        return new ElasticsearchRestTemplate(client, converter, new DefaultResultMapper(converter));    }    private DefaultConversionService createConversionService() {        var conversionService = new DefaultConversionService();        conversionService.addConverter(new StringToLocalDateConverter());        return conversionService;    }}

CustomElasticSearchConverter:

/** * Custom version of {@link MappingElasticsearchConverter} to support newest Spring Data Elasticsearch integration that supports ElasticSearch 7. Remove when Spring Data Elasticsearch 4.x is released. */class CustomElasticSearchConverter extends MappingElasticsearchConverter {    private CustomConversions conversions = new ElasticsearchCustomConversions(Collections.emptyList());    CustomElasticSearchConverter(MappingContext<? extends ElasticsearchPersistentEntity<?>, ElasticsearchPersistentProperty> mappingContext) {        super(mappingContext);        setConversions(conversions);    }    CustomElasticSearchConverter(MappingContext<? extends ElasticsearchPersistentEntity<?>, ElasticsearchPersistentProperty> mappingContext, GenericConversionService conversionService) {        super(mappingContext, conversionService);        setConversions(conversions);    }    @Override    protected <R> R readValue(@Nullable Object source, ElasticsearchPersistentProperty property,                              TypeInformation<R> targetType) {        if (source == null) {            return null;        }        if (source instanceof List) {            return readCollectionValue((List) source, property, targetType);        }        return super.readValue(source, property, targetType);    }    private Object readSimpleValue(@Nullable Object value, TypeInformation<?> targetType) {        Class<?> target = targetType.getType();        if (value == null || target == null || ClassUtils.isAssignableValue(target, value)) {            return value;        }        if (conversions.hasCustomReadTarget(value.getClass(), target)) {            return getConversionService().convert(value, target);        }        if (Enum.class.isAssignableFrom(target)) {            return Enum.valueOf((Class<Enum>) target, value.toString());        }        return getConversionService().convert(value, target);    }    private <R> R readCollectionValue(@Nullable List<?> source, ElasticsearchPersistentProperty property,                                      TypeInformation<R> targetType) {        if (source == null) {            return null;        }        Collection<Object> target = createCollectionForValue(targetType, source.size());        for (Object value : source) {            if (isSimpleType(value)) {                target.add(                        readSimpleValue(value, targetType.getComponentType() != null ? targetType.getComponentType() : targetType));            } else {                if (value instanceof List) {                    target.add(readValue(value, property, property.getTypeInformation().getActualType()));                } else {                    target.add(readEntity(computeGenericValueTypeForRead(property, value), (Map) value));                }            }        }        return (R) target;    }    private Collection<Object> createCollectionForValue(TypeInformation<?> collectionTypeInformation, int size) {        Class<?> collectionType = collectionTypeInformation.isCollectionLike()//                ? collectionTypeInformation.getType() //                : List.class;        TypeInformation<?> componentType = collectionTypeInformation.getComponentType() != null //                ? collectionTypeInformation.getComponentType() //                : ClassTypeInformation.OBJECT;        return collectionTypeInformation.getType().isArray() //                ? new ArrayList<>(size) //                : CollectionFactory.createCollection(collectionType, componentType.getType(), size);    }    private ElasticsearchPersistentEntity<?> computeGenericValueTypeForRead(ElasticsearchPersistentProperty property,                                                                            Object value) {        return ClassTypeInformation.OBJECT.equals(property.getTypeInformation().getActualType())                ? getMappingContext().getRequiredPersistentEntity(value.getClass())                : getMappingContext().getRequiredPersistentEntity(property.getTypeInformation().getActualType());    }    private boolean isSimpleType(Object value) {        return isSimpleType(value.getClass());    }    private boolean isSimpleType(Class<?> type) {        return conversions.isSimpleType(type);    }}

If anyone is using Spring Boot 2.1.2 and Kotlin, the following code may help you. I just translated it from @Alexander Lech answer, with some small changes:

First change to Alexanders Answer:

@SpringBootApplication(exclude = [ElasticsearchAutoConfiguration::class, ElasticsearchDataAutoConfiguration::class])

I had to exclude ElasticsearchDataAutoConfiguration, to make it work.

Second: Since we use Kotlin, and the custom converter is a lot of code, perhaps this translation to Kotlin will help somebody:

class CustomElasticSearchConverter(mappingContext: MappingContext<out ElasticsearchPersistentEntity<*>, ElasticsearchPersistentProperty>, customConversionService: GenericConversionService?) : MappingElasticsearchConverter(mappingContext, customConversionService) {    private val conversionsNew = ElasticsearchCustomConversions(emptyList<Any>())    init {        setConversions(conversionsNew)    }    override fun <R : Any?> readValue(source: Any?, property: ElasticsearchPersistentProperty, targetType: TypeInformation<R>): R? {        if (source == null) {            return null        }        if (source is Collection<*>) {            return readCollectionValue(source, property, targetType) as R?;        }        return super.readValue(source, property, targetType);    }    private fun readCollectionValue(source: Collection<*>?, property: ElasticsearchPersistentProperty, targetType: TypeInformation<*>): Any? {        if (source == null) {            return null        }        val target = createCollectionForValue(targetType, source.size)        for (value in source) {            require(value != null) { "value must not be null" }            if (isSimpleType(value)) {                target.add(readSimpleValue(value, if (targetType.componentType != null) targetType.componentType!! else targetType))            } else {                if (value is MutableCollection<*>) {                    target.add(readValue(value, property, property.typeInformation.actualType as TypeInformation<out Any>))                } else {                    @Suppress("UNCHECKED_CAST")                    target.add(readEntity(computeGenericValueTypeForRead(property, value), value as MutableMap<String, Any>?))                }            }        }        return target    }    private fun readSimpleValue(value: Any?, targetType: TypeInformation<*>): Any? {        val target = targetType.type        @Suppress("SENSELESS_COMPARISON")        if (value == null || target == null || ClassUtils.isAssignableValue(target, value)) {            return value        }        if (conversionsNew.hasCustomReadTarget(value.javaClass, target)) {            return conversionService.convert(value, target)        }        @Suppress("UNCHECKED_CAST")        return when {            Enum::class.java.isAssignableFrom(target) -> enumByName(target as Class<Enum<*>>, value.toString())            else -> conversionService.convert(value, target)        }    }    private fun enumByName(target: Class<Enum<*>>, name: String): Enum<*> {        val enumValue = target.enumConstants.find { it.name == name }        require(enumValue != null) { "no enum value found for name $name and targetClass $target" }        return enumValue    }    private fun createCollectionForValue(collectionTypeInformation: TypeInformation<*>, size: Int): MutableCollection<Any?> {        val collectionType = when {            collectionTypeInformation.isCollectionLike -> collectionTypeInformation.type            else -> MutableList::class.java        }        val componentType = when {            collectionTypeInformation.componentType != null -> collectionTypeInformation.componentType            else -> ClassTypeInformation.OBJECT        }        return when {            collectionTypeInformation.type.isArray -> ArrayList(size)            else -> CollectionFactory.createCollection(collectionType, componentType!!.type, size)        }    }    private fun computeGenericValueTypeForRead(property: ElasticsearchPersistentProperty, value: Any): ElasticsearchPersistentEntity<*> {        return when {            ClassTypeInformation.OBJECT == property.typeInformation.actualType -> mappingContext.getRequiredPersistentEntity(value.javaClass)            else -> mappingContext.getRequiredPersistentEntity(property.typeInformation.actualType!!)        }    }    private fun isSimpleType(value: Any): Boolean {        return isSimpleType(value.javaClass)    }    private fun isSimpleType(type: Class<*>): Boolean {        return conversionsNew.isSimpleType(type)    }}

After this, problems with some repository queries where solved. Please also be aware not to use spring-boot-starter-data-elasticsearch but spring-data-elasticsearch:4.0.0.BUILD-SNAPSHOT. (This took me some time).

Yes, the code is ugly, but after spring-data-elasticsearch:4.0.0 is released, you can throw it away.