why Hbase with Hadoop map reduce performance is slow? why Hbase with Hadoop map reduce performance is slow? hadoop hadoop

why Hbase with Hadoop map reduce performance is slow?


Hadoop, or any other batch processing system for that matter, is not a suitable choice if you have real time needs or if you need performance in ~ms. No matter how good your h/w is and how good your MR job is, there'll always be some initial delay when you run a MR job. And this is unavoidable. The reason being, when you submit a MR job, a lot of things happen before processing actually starts, like checking the input path, creation of splits, creation of map tasks etc etc.

It is correct that HBase provided real-time data access. But it doesn't hold good if you are accessing HBase through MR. If you really need ~ms access, you are better off writing normal Java+HBase API programs. But you won't be able to leverage the parallelism provided by MR then. So, you basically need to think well before you arrive at any decision.

Tools like Impala and Phoenix could be of help if you have real time needs. But they have their own + and -.

I would like to point one thing here. If your plan is to access not-so-big data at a time, you can definitely use HBase with sequential Java programs. But remember, random reads/writes always come with greater costs as compared to sequential acces. So, think well before you act.