You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by ni...@apache.org on 2016/02/02 01:35:55 UTC
[39/50] [abbrv] samza git commit: SAMZA-829: add doc to configure
YARN Node Manager to allow clean shutdown of SamzaContainer w/ large state
stores
SAMZA-829: add doc to configure YARN Node Manager to allow clean shutdown of SamzaContainer w/ large state stores
Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/5d50c083
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/5d50c083
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/5d50c083
Branch: refs/heads/samza-sql
Commit: 5d50c08308b1626be397a11521e3494ce086fb9f
Parents: 20a1403
Author: Yi Pan <ni...@gmail.com>
Authored: Thu Dec 3 15:28:44 2015 -0800
Committer: Yi Pan (Data Infrastructure) <ni...@gmail.com>
Committed: Thu Dec 3 15:28:44 2015 -0800
----------------------------------------------------------------------
.../learn/documentation/versioned/yarn/yarn-host-affinity.md | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/samza/blob/5d50c083/docs/learn/documentation/versioned/yarn/yarn-host-affinity.md
----------------------------------------------------------------------
diff --git a/docs/learn/documentation/versioned/yarn/yarn-host-affinity.md b/docs/learn/documentation/versioned/yarn/yarn-host-affinity.md
index 108dfbc..1d9c29e 100644
--- a/docs/learn/documentation/versioned/yarn/yarn-host-affinity.md
+++ b/docs/learn/documentation/versioned/yarn/yarn-host-affinity.md
@@ -96,7 +96,13 @@ export LOGGED_STORE_BASE_DIR=<path-for-state-stores>
<value>1000*</value> <!-- Should be tuned per requirement -->
</property>
{% endhighlight %}
-
+3. Configure Yarn Node Manager SIGTERM to SIGKILL timeout to be reasonable time s.t. Node Manager will give Samza Container enough time to perform a clean shutdown in yarn-site.xml {% highlight xml %}
+<property>
+ <name>yarn.nodemanager.sleep-delay-before-sigkill.ms</name>
+ <description>No. of ms to wait between sending a SIGTERM and SIGKILL to a container</description>
+ <value>600000</value> <!-- Set it to 10min to allow enough time for clean shutdown of containers -->
+</property>
+{% endhighlight %}
## Configuring a Samza job to use Host Affinity
Any stateful Samza job can leverage this feature to reduce the Mean Time To Restore (MTTR) of it's state stores by setting <code>yarn.samza.host-affinity</code> to true.