You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by cr...@apache.org on 2014/12/09 19:12:20 UTC
incubator-samza git commit: SAMZA-491;
support non-default queues in YARN
Repository: incubator-samza
Updated Branches:
refs/heads/master c79033454 -> d382cb6f3
SAMZA-491; support non-default queues in YARN
Project: http://git-wip-us.apache.org/repos/asf/incubator-samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-samza/commit/d382cb6f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-samza/tree/d382cb6f
Diff: http://git-wip-us.apache.org/repos/asf/incubator-samza/diff/d382cb6f
Branch: refs/heads/master
Commit: d382cb6f38ee5d6186f5077080bf45fdb33775c9
Parents: c790334
Author: Jay Bae <me...@gmail.com>
Authored: Tue Dec 9 10:12:09 2014 -0800
Committer: Chris Riccomini <cr...@criccomi-mn.linkedin.biz>
Committed: Tue Dec 9 10:12:09 2014 -0800
----------------------------------------------------------------------
.../documentation/versioned/jobs/configuration-table.html | 10 +++++++++-
docs/learn/documentation/versioned/jobs/yarn-jobs.md | 10 +++++++++-
2 files changed, 18 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-samza/blob/d382cb6f/docs/learn/documentation/versioned/jobs/configuration-table.html
----------------------------------------------------------------------
diff --git a/docs/learn/documentation/versioned/jobs/configuration-table.html b/docs/learn/documentation/versioned/jobs/configuration-table.html
index 12955ad..6a5d209 100644
--- a/docs/learn/documentation/versioned/jobs/configuration-table.html
+++ b/docs/learn/documentation/versioned/jobs/configuration-table.html
@@ -1244,7 +1244,7 @@
<td class="property" id="yarn-am-poll-interval-ms">yarn.am.poll.interval.ms</td>
<td class="default">1000</td>
<td class="description">
- THe Samza ApplicationMaster sends regular heartbeats to the YARN ResourceManager
+ The Samza ApplicationMaster sends regular heartbeats to the YARN ResourceManager
to confirm that it is alive. This property determines the time (in milliseconds)
between heartbeats.
</td>
@@ -1260,6 +1260,14 @@
</tr>
<tr>
+ <td class="property" id="yarn-queue">yarn.queue</td>
+ <td class="default"></td>
+ <td class="description">
+ Determines which YARN queue will be used for Samza job.
+ </td>
+ </tr>
+
+ <tr>
<th colspan="3" class="section" id="metrics"><a href="../container/metrics.html">Metrics</a></th>
</tr>
http://git-wip-us.apache.org/repos/asf/incubator-samza/blob/d382cb6f/docs/learn/documentation/versioned/jobs/yarn-jobs.md
----------------------------------------------------------------------
diff --git a/docs/learn/documentation/versioned/jobs/yarn-jobs.md b/docs/learn/documentation/versioned/jobs/yarn-jobs.md
index 58ca50d..827cc14 100644
--- a/docs/learn/documentation/versioned/jobs/yarn-jobs.md
+++ b/docs/learn/documentation/versioned/jobs/yarn-jobs.md
@@ -19,7 +19,7 @@ title: YARN Jobs
limitations under the License.
-->
-When you define `job.factory.class=org.apache.samza.job.yarn.YarnJobFactory` in your job's configuration, Samza will use YARN to execute your job. The YarnJobFactory will use the YARN_HOME environment variable on the machine that run-job.sh is executed on to get the appropriate YARN configuration, which will define where the YARN resource manager is. The YarnJob will work with the resource manager to get your job started on the YARN cluster.
+When you define `job.factory.class=org.apache.samza.job.yarn.YarnJobFactory` in your job's configuration, Samza will use YARN to execute your job. The YarnJobFactory will use the HADOOP_YARN_HOME environment variable on the machine that run-job.sh is executed on to get the appropriate YARN configuration, which will define where the YARN resource manager is. The YarnJob will work with the resource manager to get your job started on the YARN cluster.
If you want to use YARN to run your Samza job, you'll also need to define the location of your Samza job's package. For example, you might say:
@@ -29,6 +29,14 @@ yarn.package.path=http://my.http.server/jobs/ingraphs-package-0.0.55.tgz
This .tgz file follows the conventions outlined on the [Packaging](packaging.html) page (it has bin/run-am.sh and bin/run-container.sh). YARN NodeManagers will take responsibility for downloading this .tgz file on the appropriate machines, and untar'ing them. From there, YARN will execute run-am.sh or run-container.sh for the Samza Application Master, and SamzaContainer, respectively.
+If you want to run Samza job in the specific YARN queue not the default one, you can set `yarn.queue` property in your job's configuration. For example, with the following property setting,
+
+{% highlight jproperties %}
+yarn.queue=root.adhoc
+{% endhighlight %}
+
+Samza job will run in the root.adhoc queue.
+
<!-- TODO document yarn.container.count and other key configs -->
## [Logging »](logging.html)