Insert into ElasticSearch using Hive/Qubole Insert into ElasticSearch using Hive/Qubole elasticsearch elasticsearch

Insert into ElasticSearch using Hive/Qubole


I determined the problem. As I suspected, insertion was skipping over some records that were considered "bad." I was never able to find what records exactly were being skipped, but I tried replacing all non-alphanumeric characters with a space. This solved the problem! The records are no longer being skipped, and all data is uploaded to Elastic Search.

INSERT OVERWRITE TABLE es_temp_tableSELECT dt, REGEXP_REPLACE(description, '[^0-9a-zA-Z]+', ' ')FROM other_table