Building Trunk on EC2

[Ronald Petty <ro...@gmail.com>]

Hello,

If anyone is interested, here are my notes on how to build trunk on bare
bones AWS EC2 instance.

   1. Create a XL 64-bit AWS EC2 instance (anything smaller fails due to
   lack of RAM)
   2. SSH to EC2 instance
   3. sudo useradd -u 1000 hadoop
   4. sudo yum install svn gcc.x86_64 gcc-c++.x86_64 libtool.x86_64
   make.x86_64 automake.noarch autoconf.noarch java-1.6.0-openjdk-devel.x86_64
   zlib-devel.x86_64
   5. set JAVA_HOME in vi /etc/profile.d/aws-apitools-common.sh to
   /usr/lib/jvm/java  (if you don't the native components will not build due
   to using an incorrect JAVA_HOME even if you set evn JAVA_HOME)
   6. reboot
   7. login
   8. su hadoop
   9. sudo mkdir /data
   10. sudo chown -R hadoop:hadoop /data
   11. cd /data
   12. wget http://apache.petsads.us//forrest/apache-forrest-0.9.tar.gz
   13. wget
   http://mirror.cc.columbia.edu/pub/software/apache//maven/binaries/apache-maven-3.0.3-bin.tar.gz
   14. wget http://protobuf.googlecode.com/files/protobuf-2.4.1.tar.gz
   15. wget
   http://superb-sea2.dl.sourceforge.net/project/findbugs/findbugs/2.0.0/findbugs-2.0.0.tar.gz
   16. gunzip *.gz
   17. tar xf apache-forrest-0.9.tar
   18. tar xf apache-maven-3.0.3-bin.tar
   19. tar xf findbugs-2.0.0.tar
   20. tar xf protobuf-2.4.1.tar
   21. export FORREST_HOME=/data/apache-forrest-0.9
   22. export MAVEN_HOME=/data/apache-maven-3.0.3
   23. export FINDBUGS_HOME=/data/findbugs-2.0.0
   24. cd protobuf-2.4.1
   25. ./configure
   26. make
   27. sudo make install
   28. cd ..
   29. export
   PATH=/usr/local/bin/:/data/apache-maven-3.0.3/bin/:/data/apache-forrest-0.9/bin/:/data/findbugs-2.0.0/bin/:$PATH
   30. svn checkout
http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-trunk
   31. cd hadoop-trunk
   32. mvn package -Pdist,native,docs -DskipTests -Dtar

I have not actually used the constructed tar yet, but wanted to share my
notes.  I did notice two things missing (compared to the regular pre-yarn
releases), there is no conf directory, nor example jars.  I know trunk is
being worked on but I am not sure what the plan is there (or if I simply
missed something.)

If you have any comments, questions, or advice I love to here it.

Kindest regards.

Ron