How to copy hbase table from hbase-0.94 cluster to hbase-0.98 cluster
These are available options, out of which you can choose.
- You can use
org.apache.hadoop.hbase.mapreduce.Export
tool to export tables to HDFS and then you can usehadoop distcp
to move data to another cluster. When data is place on second cluster you can useorg.apache.hadoop.hbase.mapreduce.Import
tool to import tables. Please look at http://hbase.apache.org/book.html#export. Second option is to us
CopyTable
tool, please look at: http://hbase.apache.org/book.html#copytable Have a look at pivotalThird option is to enable hbase Snapshots, create table snapshots, and then use
ExportSnapshot
tool to move them to second cluster. When snapshots are on second cluster you can clone tables from snapshots. Please look: http://hbase.apache.org/book.html#ops.snapshots
HBase Snapshots allow you to take a snapshot of a table without too much impact on Region Servers. Snapshot, Clone and restore operations don't involve data copying. Also, Exporting the snapshot to another cluster doesn't have impact on the Region Servers
I was using 1 and 3 for moving data between clusters and I in my case 3was better solution.
Run below command on source cluster, make sure you have cross cluster authentication enabled.
/usr/bin/hbase org.apache.hadoop.hbase.mapreduce.CopyTable -Ddfs.nameservices=nameservice1,devnameservice -Ddfs.ha.namenodes.devnameservice=devnn1,devnn2 -Ddfs.namenode.rpc-address.devnameservice.devnn1=<destination_namenode01_host>:<destination_namenode01_port> -Ddfs.namenode.rpc-address.devnameservice.devnn2=<destination_namenode02_host>:<destination_namenode02_port> -Ddfs.client.failover.proxy.provider.devnameservice=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider -Dmapred.map.tasks.speculative.execution=false --peer.adr=<destination_zookeeper host>:<port>:/hbase --versions=<n> <table_name>