How to compile Hadoop for 64-bit Linux machine? How to compile Hadoop for 64-bit Linux machine? hadoop hadoop

How to compile Hadoop for 64-bit Linux machine?


Once you extract the source you would find a super pom in location like below.\hadoop-2.2.0-src.tar\hadoop-2.2.0-src\hadoop-2.2.0-src\pom.xmlThis would build all the modules.You can build using command: mvn clean install

You should notice logs like below.

            [INFO] ------------------------------------------------------------------------            [INFO] Reactor Build Order:            [INFO]            [INFO] Apache Hadoop Main            [INFO] Apache Hadoop Project POM            [INFO] Apache Hadoop Annotations            [INFO] Apache Hadoop Project Dist POM            [INFO] Apache Hadoop Assemblies            [INFO] Apache Hadoop Maven Plugins            [INFO] Apache Hadoop Auth            [INFO] Apache Hadoop Auth Examples            [INFO] Apache Hadoop Common            [INFO] Apache Hadoop NFS            [INFO] Apache Hadoop Common Project            [INFO] Apache Hadoop HDFS            [INFO] Apache Hadoop HttpFS            [INFO] Apache Hadoop HDFS BookKeeper Journal            [INFO] Apache Hadoop HDFS-NFS            [INFO] Apache Hadoop HDFS Project            [INFO] hadoop-yarn            [INFO] hadoop-yarn-api            [INFO] hadoop-yarn-common            [INFO] hadoop-yarn-server            [INFO] hadoop-yarn-server-common            [INFO] hadoop-yarn-server-nodemanager            [INFO] hadoop-yarn-server-web-proxy            [INFO] hadoop-yarn-server-resourcemanager            [INFO] hadoop-yarn-server-tests            [INFO] hadoop-yarn-client            [INFO] hadoop-yarn-applications            [INFO] hadoop-yarn-applications-distributedshell            [INFO] hadoop-mapreduce-client            [INFO] hadoop-mapreduce-client-core            [INFO] hadoop-yarn-applications-unmanaged-am-launcher            [INFO] hadoop-yarn-site            [INFO] hadoop-yarn-project            [INFO] hadoop-mapreduce-client-common            [INFO] hadoop-mapreduce-client-shuffle            [INFO] hadoop-mapreduce-client-app            [INFO] hadoop-mapreduce-client-hs            [INFO] hadoop-mapreduce-client-jobclient            [INFO] hadoop-mapreduce-client-hs-plugins            [INFO] Apache Hadoop MapReduce Examples            [INFO] hadoop-mapreduce

and more......

However this is a long trip approach if you just want to use Hadoop.You should be able to use the existing libraries.Could be some configuration issue.

Other option is Cloudera. I have installed this on RedHat Linux.

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Quick-Start/cdh4qs_topic_3.html

Good luck.


Consider using Apache Bigtop to build Hadoop.

It's IMHO the most convenient way of compiling Hadoop yourself, as long as you only need the "mainsteam" components and stable versions.