You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hama.apache.org by Apache Wiki <wi...@apache.org> on 2014/07/02 08:14:11 UTC
[Hama Wiki] Update of "GettingStartedMesos" by JeffFenchel
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.
The "GettingStartedMesos" page has been changed by JeffFenchel:
https://wiki.apache.org/hama/GettingStartedMesos?action=diff&rev1=1&rev2=2
== Configuration ==
- There are several properties that must be set in hama-site.xml:
+ There are several Mesos related properties that must be set in hama-site.xml:
+ ||'''Property'''||'''Recommended Value'''||'''Description'''||
+ ||bsp.master.TaskWorkerManager.class||org.apache.hama.bsp.MesosScheduler||Instructs the scheduler to use Mesos to execute tasks of each job||
+ ||hama.mesos.executor.uri||hdfs://hdfs.name.node:port/hama.tar.gz ||This is the URI of the Hama distribution. Upload this yourself.||
+ ||bsp.tasks.maximum.total||10|| This is an override for the total maximum tasks that may be run. The default behavior is to determine a value based on the available groom servers. However, if using Mesos, the groom servers are not yet allocated. So, a value indicating the maximum number of slots available in the cluster is needed.||
+ ||hama.mesos.master|| || This is the address of the Mesos master instance. If you're using Zookeeper for master election, use the Zookeeper address here (i.e.,zk://zk.apache.org:2181/hadoop/mesos).||
+ ||bsp.child.java.opts||-Xmx1024m||Java opts for the groom server child processes.||
- {{{
- <property>
- <name>bsp.master.TaskWorkerManager.class</name>
- <value>org.apache.hama.bsp.MesosScheduler</value>
- <description>
- This tells the scheduler to use Mesos
- in order to execute tasks of each job
- </description>
- </property>
- <property>
- <name>hama.mesos.executor.uri</name>
- <value>hdfs://hdfs.name.node:port/hama.tar.gz</value>
- <description>
- This is the URI of the Hama distribution. Upload this yourself.
- </description>
- </property>
- <property>
- <name>bsp.tasks.maximum.total</name>
- <value>10</value>
- <description>
- This is an override for the total maximum tasks that may be run.
- The default behavior is to determine a value based on the available groom
- servers. However, if using Mesos, the groom servers are not yet allocated.
- So, a value indicating the maximum number of slots available
- in the cluster is needed.
- </description>
- </property>
- <property>
- <name>hama.mesos.master</name>
- <value>local</value>
- <description>
- This is the address of the Mesos master instance. If you're using
- Zookeeper for master election, use the Zookeeper address here (i.e.,
- zk://zk.apache.org:2181/hadoop/mesos).
- </description>
- </property>
- }}}
-
Hama requires one cpu and memory defined by bsp.child.java.opts for each slot. This means that a cluster with {{{bsp.tasks.maximum.total}}} set to {{{2}}} and {{{bsp.child.jova.opts}}} set to {{{-Xmx1024m}}} will need at least {{{2}}} cpus and and {{{2048m}}} of memory.
Manually distributing the configuration is not necessary. Hama and Mesos will distribute the configuration and provide overrides where necessary.
+
+ Additionally, there are several other properties that will need to be considered when setting up hama for the first time:
+
+ ||'''Property'''||'''Default Value'''||'''Description'''||
+ ||hama.tmp.dir ||/tmp/hama-${user.name} ||Temporary directory on the local filesystem.||
+ ||hama.zookeeper.quorum || ||Comma separated list of servers in the Zookeeper Quorum||
+ ||hama.zookeeper.property.clientPort ||21810 ||The port to which the zookeeper clients connect||
+
== Starting The BSPMaster ==
With Hama on Mesos you only need to set up and start the BSPMaster. After setting the configuration the bsp master may be started: