[Lucene-hadoop Wiki] Update of "Hbase/MapReduce" by stack

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by stack:
http://wiki.apache.org/lucene-hadoop/Hbase/MapReduce

The comment on the change is:
Start of a MR+hbase page

New page:
= Hbase, MapReduce and the CLASSPATH =

An hbase cluster configuration is made up of the hbase particulars found at ''$HBASE_CONF_DIR'' -- default location is ''$HBASE_HOME/conf'' -- and the hadoop configuration in ''$HADOOP_CONF_DIR'', usually ''$HADOOP_HOME/conf''.  When hbase start/stop scripts run, they will read ''$HBASE_CONF_DIR'' content and then that of ''$HADOOP_CONF_DIR''.

M!apReduce job jars deployed to a mapreduce cluster do not usually have access to ''$HBASE_CONF_DIR''.  Any hbase particular configuration not hard-coded into the job jar classes -- e.g. the address of the target hbase master -- needs to be either included explicitly in the job jar, by adding an ''hbase-site.xml'' under a conf subdirectory, or added to a hbase-site.xml under ''$HADOOP_HOME/conf'' and copied across the mapreduce cluster.

The same holds true for any hbase classes referenced by the mapreduce job jar.  By default the hbase classes are not available on the general mapreduce ''CLASSPATH''.  You have a couple of options. Either include the hadoop-X.X.X-hbase.jar in the job jar under the lib subdirectory or copy the hadoop-X.X.X-hbase.jar to $HADOOP_HOME/lib and copy it across the cluster.  But the cleanest means of adding hbase to the cluster CLASSPATH is by uncommenting ''HADOOP_CLASSPATH'' in ''$HADOOP_HOME/conf/hadoop-env.sh'' adding the path to the hbase jar, usually ''$HADOOP_HOME/contrib/hbase/hadoop-X.X.X-hbase.jar'', and then copying the amended configuration across the cluster.  You'll need to restart the mapreduce cluster if you want it to notice the new configuration.

= Hbase as MapReduce job data source and sink =

Hbase can be used as a data source and data sink for 
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableInputFormat.html
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/javadoc/org/apache/hadoop/hbase/mapred/TableOutputFormat.html
   
Good to have lots of reducers

Need to add hbase lib?

Note on how the splits are done and the config. needed.