You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tez.apache.org by "László Bodor (Jira)" <ji...@apache.org> on 2024/08/05 12:08:00 UTC

[jira] [Resolved] (TEZ-4547) Add Tez AM JobID to the JobConf

     [ https://issues.apache.org/jira/browse/TEZ-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

László Bodor resolved TEZ-4547.
-------------------------------
    Resolution: Fixed

> Add Tez AM JobID to the JobConf
> -------------------------------
>
>                 Key: TEZ-4547
>                 URL: https://issues.apache.org/jira/browse/TEZ-4547
>             Project: Apache Tez
>          Issue Type: Improvement
>    Affects Versions: 0.10.2
>            Reporter: Venkatasubrahmanian Narayanan
>            Assignee: Venkatasubrahmanian Narayanan
>            Priority: Major
>             Fix For: 0.10.4
>
>          Time Spent: 7h
>  Remaining Estimate: 0h
>
> Tez creates JobIDs for tasks by appending the vertex index to the cluster timestamp to avoid multiple jobs run in a single Tez session sharing a JobID. Hadoop's MagicS3GuardCommitter needs a job-wide UUID to ensure that the task committers and the job committer write to/read from the same paths and can hence actually commit data. Adding the AM's JobID to the Configuration objects allows applications like Hive to pass that as the UUID to the committer.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)