Sending solr configs to zookeeper from hdfs using zkcli.sh bash through oozie shell action Sending solr configs to zookeeper from hdfs using zkcli.sh bash through oozie shell action hadoop hadoop

Sending solr configs to zookeeper from hdfs using zkcli.sh bash through oozie shell action


If you look at the code of "ZkConfigManager::uploadToZK()", present here: https://github.com/apache/incubator-ranger/blob/master/ranger_solrj/src/main/java/org/apache/solr/common/cloud/ZkConfigManager.java, you can observe that, the exception is happening at:

if (!Files.exists(rootPath))    throw new IOException("Path " + rootPath + " does not exist");

It is expecting the Path to be a java.nio.file.Path and not org.apache.hadoop.fs.Path (hdfs path).

Since, you are passing an hdfs path, the exception is being thrown.

You should pass a path to the local file system (i.e. "/some_address/user/Peter/dir_with_date" should be a local file path).

Also, check this wiki, which gives an example of uploading the configuration using zkcli: https://cwiki.apache.org/confluence/display/solr/Using+ZooKeeper+to+Manage+Configuration+Files


Thank you @Manjunath, that helped. Here is exact solution:

To send solr configs to zookeeper through oozie shell action you have to copy solr config dir to worker executing this bash script by adding this into workflow

 <file>yourpathto/solr_config_dir#$yourpathto/solr_config_dir</file>

And than in bashscript you refer to this folder like this:

CONFIGDIR="`readlink -f solr_config_dir`"

Add CONFIGDIR as confdir parameter to zkcli.sh and done!

Now you have working oozie shell action that will send your solr configs to zookeeper!