[jira] [Resolved] (TEZ-4281) dag_*_priority.dot files should go into a valid log directory

["László Bodor (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-4281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

László Bodor resolved TEZ-4281.
-------------------------------
    Resolution: Fixed

> dag_*_priority.dot files should go into a valid log directory
> -------------------------------------------------------------
>
>                 Key: TEZ-4281
>                 URL: https://issues.apache.org/jira/browse/TEZ-4281
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: László Bodor
>            Assignee: László Bodor
>            Priority: Major
>             Fix For: 0.10.1
>
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> TEZ-4229 handled the case of e.g. tez-local-cache folders and tez-conf.pb files, but seems like the config ("tez.local.cache.root.folder") has no effect on dag_*_priority.dot files.
> I did the same hive unit test as on TEZ-4229 and I found that there are still lots of files there (that somehow I missed last time):
> {code}
>         itests/hive-unit/dag_1612431668191_0001_10_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_11_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_12_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_13_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_14_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_1_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_2_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_3_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_4_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_5_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_6_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_7_priority.dot
>         itests/hive-unit/dag_1612431668191_0001_8_priority.dot
> {code}
> sometimes I also catch another resources on watch git status, like:
> {code}
>         itests/hive-unit/hive-exec-3.1.3000.7.1.7.0-SNAPSHOT-170a3a6a1dffe
> {code}
> UPDATE: it turned out that the problem is that Utils.generateDAGVizFile has a 3 parameters version without having to supply logDirs, and resolves to env variable. This is not sufficient as the user has to take care of that. With a simple change, we can propagate logDirs from DAGAppMaster through DagImpl to this codepath.
> UPDATE: the root cause was TEZ-3958, which introduced a new method to Utils



--
This message was sent by Atlassian Jira
(v8.3.4#803005)