How to use hbase with Spring Boot using Java instead of XML?

Well it is not really an expected answer but I want to develop it too much for a comment.

I carefully read the Spring for Apache Hadoop - Reference Documentation in its last released version, and if it does contain examples and details for namespace configuration, I could not find a single line on Java configuration.

My understanding on it is that only namespace configuration is currently supported for Spring for Apache Hadoop. It should certainly be possible to look at the classes supporting the namespace and hack a working configuration to find how to obtain same result with java config, but honestly I think the cost/gain ratio does not justify it. And as it is currently undocumented, you will never be sure that you did not forget something that will break later.

As Spring offers to include xml config files in a Java config Spring application, I would strongly advice you to keep all your present Java config, write the Apache Hadoop part in xml using the supplied namespace, and simply add an @ImportResource annotation to your configuration class. Say the spring hadoop configuration is hadoopContext.xml at the root of classpath, you could write :

@Configuration...@ImportResource("classpath:/hadoopContext.xml")public classConfig {...}

Alternatively you can use a @Configuration wrapper around the xml config that you manage to be scanned by Spring :

@Configuration@ImportResource("classpath:/hadoopContext.xml")public class HadoopConfig {}

Even though a Java-based configuration for HBase is yet not formally available (as recently as Spring Hadoop 2.2.1), there is a clean workaround. Include the relevant ZooKeeper details as properties within a Java-based Spring Hadoop configuration. Then, the hbase-configuration XML tag need not include any configuration details. By convention, it will depend on the hadoopConfiguration bean defined by the Java-based HadoopConfigConfigurer.

@Configuration@EnableHadooppublic class MapReduceConfiguration extends SpringHadoopConfigurerAdapter {    @Override    public void configure(HadoopConfigConfigurer config) throws Exception {        config                .fileSystemUri(myConfigManager.getHadoopFsUri())                .resourceManagerAddress(myConfigManager.getHadoopResourceManagerAddress())                .withProperties()                .property("hbase.zookeeper.quorum", myConfigManager.getZookeeperQuorum())                .property("hbase.zookeeper.property.clientPort", myConfigManager.getZookeeperPort())                .property("hbase.client.scanner.caching", "1");    }}

The remaining XML configuration is agnostic to the deployment environment:

<beans xmlns="http://www.springframework.org/schema/beans"       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"       xmlns:hdp="http://www.springframework.org/schema/hadoop"       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd        http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd">    <hdp:hbase-configuration /></beans>

I wrote a simple demo project for using hbase in spring boot application without xml.

This demo mainly depends spring-data-hadoop and hbase-client. gradle dependencies:

compile('org.springframework.boot:spring-boot-starter-data-rest')compile('org.springframework.boot:spring-boot-starter-web')compile 'org.springframework.data:spring-data-hadoop:2.5.0.RELEASE'compile('org.apache.hbase:hbase-client:1.3.1'){    exclude group :'log4j',module:'log4j'    exclude group :'org.slf4j',module:'slf4j-log4j12'    exclude group: 'javax.servlet', module: 'servlet-api'}compile('org.springframework.boot:spring-boot-configuration-processor')providedRuntime('org.springframework.boot:spring-boot-starter-tomcat')

Configure the hbase connection parameters in spring boot's application.properties (No XML!):

spring.data.hbase.zkQuorum=192.168.0.109:2181spring.data.hbase.zkBasePath=/hbasespring.data.hbase.rootDir=file:///home/hbase-1.2.2

class HbaseProperties.java:

@ConfigurationProperties(prefix = "spring.data.hbase")public class HbaseProperties {    // Addresses of all registered ZK servers.    private String zkQuorum;    // Location of HBase home directory    private String rootDir;    // Root node of this cluster in ZK.    private String zkBasePath;    // getters and setters...}

HbaseConfig.java, inject the configurations into the HbaseTemplate:

import org.apache.hadoop.hbase.HBaseConfiguration;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.boot.context.properties.EnableConfigurationProperties;import org.springframework.context.annotation.Bean;import org.springframework.context.annotation.Configuration;import org.springframework.data.hadoop.hbase.HbaseTemplate;@Configuration@EnableConfigurationProperties(HbaseProperties.class)public class HbaseConfig {    @Autowired    private HbaseProperties hbaseProperties;    @Bean    public HbaseTemplate hbaseTemplate() {        org.apache.hadoop.conf.Configuration configuration = HBaseConfiguration.create();        configuration.set("hbase.zookeeper.quorum", this.hbaseProperties.getZkQuorum());        configuration.set("hbase.rootdir", this.hbaseProperties.getRootDir());        configuration.set("zookeeper.znode.parent", this.hbaseProperties.getZkBasePath());        return new HbaseTemplate(configuration);    }}

Service class, we can use the configured HbaseTemplate now:

import javax.annotation.PostConstruct;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.data.hadoop.hbase.HbaseTemplate;import org.springframework.stereotype.Service;import com.zql.hbasedemo.vo.Quote;@Servicepublic class FeedService {    @Autowired    private HbaseTemplate hbaseTemplate;    @PostConstruct    public void test(){        Quote quote = new Quote();        quote.setEventType("ft");        quote.setHandicap("4");        quote.setMarket("OU");        quote.setMatchId("27350208");        quote.setSelection("OVER");        quote.setPrice("1.93");        saveQuote(quote);    }    public void saveQuote(Quote quote) {        hbaseTemplate.put("quotes", quote.getMatchId(), "data", quote.getMarket() + ":" + quote.getSelection(),            quote.getPrice().getBytes());    }}

Enjoy! : )