[impala] 01/02: IMPALA-11480: Retain all YARN container logs in a Jenkins run

[jo...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git

commit adadec7fd9f93acb1347947f88f31f37c6aa2d59
Author: Fang-Yu Rao <fa...@cloudera.com>
AuthorDate: Sun Aug 7 20:09:19 2022 -0700

    IMPALA-11480: Retain all YARN container logs in a Jenkins run
    
    This patch explicitly sets 'yarn.nodemanager.log.retain-seconds' to a
    large enough value so that all the YARN container log files created in a
    Jenkins run will be archived.
    
    Testing:
     - Verified in a Jenkins run that all the YARN container logs are indeed
       archived.
    
    Change-Id: I10953f5de98d4312dfc563fcace17908c86be2d3
    Reviewed-on: http://gerrit.cloudera.org:8080/18821
    Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
    Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
 .../cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py    | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py b/testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py
index 262caffa2..eb6359365 100644
--- a/testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py
+++ b/testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py
@@ -54,6 +54,10 @@ CONFIG = {
   'yarn.nodemanager.local-dirs': '${NODE_DIR}/var/lib/hadoop-yarn/cache/${USER}/nm-local-dir',
   'yarn.nodemanager.log-dirs': '${NODE_DIR}/var/log/hadoop-yarn/containers',
 
+  # Set it to a large enough value so that the logs of all the containers ever created in
+  # a Jenkins run will be retained.
+  'yarn.nodemanager.log.retain-seconds': 86400,
+
   # Enable the MR shuffle service, which is also used by Tez.
   'yarn.nodemanager.aux-services': 'mapreduce_shuffle',
   'yarn.nodemanager.aux-services.mapreduce_shuffle.class': 'org.apache.hadoop.mapred.ShuffleHandler',