You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2020/04/13 09:40:00 UTC
[jira] [Assigned] (TEZ-4143) Provide an option to disable DAG graph
(.dot) generation for latency sensitive jobs
[ https://issues.apache.org/jira/browse/TEZ-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
László Bodor reassigned TEZ-4143:
---------------------------------
Assignee: László Bodor
> Provide an option to disable DAG graph (.dot) generation for latency sensitive jobs
> -----------------------------------------------------------------------------------
>
> Key: TEZ-4143
> URL: https://issues.apache.org/jira/browse/TEZ-4143
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: László Bodor
> Priority: Minor
>
> In latency sensitive jobs, sometimes it takes time to submit the DAG due to ".dot" file creation in busy cluster. It would be good to have {{tez.generate.debug.artifacts}} cover for {{DAGAppMaster::generateDAGVizFile}}
> Pasting some log snippets for later ref:
> {noformat}
> syslog_dag_1581571406290_3831_10007_post:2020-04-11 06:36:41,911 [INFO] [IPC Server handler 0 on 38609] |app.DAGAppMaster|: Generating DAG graphviz file, dagId=dag_1581571406290_3831_10008, filePath=/grid/1/hadoop/yarn/log/application_1581571406290_3831/container_e06_1581571406290_3831_01_000001/dag_1581571406290_3831_10008.dot
> syslog_dag_1581571406290_3831_10007_post:2020-04-11 06:36:42,744 [INFO] [IPC Server handler 0 on 38609] |common.TezUtilsInternal|: Redirecting log file based on addend: dag_1581571406290_3831_10008
> syslog_dag_1581571406290_3831_10008_post:2020-04-11 06:36:45,377 [INFO] [IPC Server handler 1 on 38609] |app.DAGAppMaster|: Generating DAG graphviz file, dagId=dag_1581571406290_3831_10009, filePath=/grid/1/hadoop/yarn/log/application_1581571406290_3831/container_e06_1581571406290_3831_01_000001/dag_1581571406290_3831_10009.dot
> syslog_dag_1581571406290_3831_10008_post:2020-04-11 06:36:45,690 [INFO] [IPC Server handler 1 on 38609] |common.TezUtilsInternal|: Redirecting log file based on addend: dag_1581571406290_3831_10009
> syslog_dag_1581571406290_3831_10012_post:2020-04-11 06:36:50,451 [INFO] [IPC Server handler 0 on 38609] |app.DAGAppMaster|: Generating DAG graphviz file, dagId=dag_1581571406290_3831_10013, filePath=/grid/1/hadoop/yarn/log/application_1581571406290_3831/container_e06_1581571406290_3831_01_000001/dag_1581571406290_3831_10013.dot
> syslog_dag_1581571406290_3831_10012_post:2020-04-11 06:36:51,574 [INFO] [IPC Server handler 0 on 38609] |common.TezUtilsInternal|: Redirecting log file based on addend: dag_1581571406290_3831_10013
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)