[jira] [Comment Edited] (PIG-5316) Initialize mapred.task.id property for PoS jobs

["Nandor Kollar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16260931#comment-16260931 ] 

Nandor Kollar edited comment on PIG-5316 at 11/21/17 3:45 PM:
--------------------------------------------------------------

[~xuefuz] I attached a patch, however I'm a bit worried that Hive relies on {{mapred.task.id}}. This is a "hidden" dependency, if we change the name of this property (because it is a deprecated Hadoop property, it's going to change), then Pig-HCat interoperability will be broken. Though MRConfiguration is not intended for public use in Pig, should Hive use {{MRConfiguration#TASK_ID}} instead of referring to the taskId as a string?


was (Author: nkollar):
[~xuefuz] I attached a patch, however I'm a bit worried that Hive relies on {{mapred.task.id}}. This is a "hidden" dependency, if we change the name of this property (because it is a deprecated Hadoop property, it's going to change), then Pig-HCat interoperability will be broken. Though MRConfiguration is not intended for public use in Pig, should Hive use MRConfiguration#TASK_ID instead of referring to the taskId as a string?

> Initialize mapred.task.id property for PoS jobs
> -----------------------------------------------
>
>                 Key: PIG-5316
>                 URL: https://issues.apache.org/jira/browse/PIG-5316
>             Project: Pig
>          Issue Type: Improvement
>          Components: spark
>            Reporter: Adam Szita
>            Assignee: Nandor Kollar
>         Attachments: PIG-5316_1.patch
>
>
> Some downstream systems may require the presence of {{mapred.task.id}} property (e.g. HCatalog). This is currently not set when Pig On Spark jobs are started. Let's initialise it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)