You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Rajesh Balamohan (Jira)" <ji...@apache.org> on 2020/04/13 08:00:00 UTC
[jira] [Created] (TEZ-4143) Provide an option to disable DAG graph
(.dot) generation for latency sensitive jobs
Rajesh Balamohan created TEZ-4143:
-------------------------------------
Summary: Provide an option to disable DAG graph (.dot) generation for latency sensitive jobs
Key: TEZ-4143
URL: https://issues.apache.org/jira/browse/TEZ-4143
Project: Apache Tez
Issue Type: Improvement
Reporter: Rajesh Balamohan
In latency sensitive jobs, sometimes it takes time to submit the DAG due to ".dot" file creation in busy cluster. It would be good to have {{tez.generate.debug.artifacts}} cover for {{DAGAppMaster::generateDAGVizFile}}
Pasting some log snippets for later ref:
{noformat}
syslog_dag_1581571406290_3831_10007_post:2020-04-11 06:36:41,911 [INFO] [IPC Server handler 0 on 38609] |app.DAGAppMaster|: Generating DAG graphviz file, dagId=dag_1581571406290_3831_10008, filePath=/grid/1/hadoop/yarn/log/application_1581571406290_3831/container_e06_1581571406290_3831_01_000001/dag_1581571406290_3831_10008.dot
syslog_dag_1581571406290_3831_10007_post:2020-04-11 06:36:42,744 [INFO] [IPC Server handler 0 on 38609] |common.TezUtilsInternal|: Redirecting log file based on addend: dag_1581571406290_3831_10008
syslog_dag_1581571406290_3831_10008_post:2020-04-11 06:36:45,377 [INFO] [IPC Server handler 1 on 38609] |app.DAGAppMaster|: Generating DAG graphviz file, dagId=dag_1581571406290_3831_10009, filePath=/grid/1/hadoop/yarn/log/application_1581571406290_3831/container_e06_1581571406290_3831_01_000001/dag_1581571406290_3831_10009.dot
syslog_dag_1581571406290_3831_10008_post:2020-04-11 06:36:45,690 [INFO] [IPC Server handler 1 on 38609] |common.TezUtilsInternal|: Redirecting log file based on addend: dag_1581571406290_3831_10009
syslog_dag_1581571406290_3831_10012_post:2020-04-11 06:36:50,451 [INFO] [IPC Server handler 0 on 38609] |app.DAGAppMaster|: Generating DAG graphviz file, dagId=dag_1581571406290_3831_10013, filePath=/grid/1/hadoop/yarn/log/application_1581571406290_3831/container_e06_1581571406290_3831_01_000001/dag_1581571406290_3831_10013.dot
syslog_dag_1581571406290_3831_10012_post:2020-04-11 06:36:51,574 [INFO] [IPC Server handler 0 on 38609] |common.TezUtilsInternal|: Redirecting log file based on addend: dag_1581571406290_3831_10013
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)