You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Gunther Hagleitner (JIRA)" <ji...@apache.org> on 2013/10/04 23:33:42 UTC

[jira] [Created] (TEZ-529) Hive communicates state from RecordReader to Processor via JobConf

Gunther Hagleitner created TEZ-529:
--------------------------------------

             Summary: Hive communicates state from RecordReader to Processor via JobConf
                 Key: TEZ-529
                 URL: https://issues.apache.org/jira/browse/TEZ-529
             Project: Apache Tez
          Issue Type: Bug
    Affects Versions: 0.2.0
            Reporter: Gunther Hagleitner
            Assignee: Siddharth Seth
             Fix For: 0.2.0


Hive currently switches between operator pipelines + partition descriptors via a map.input.file

In the CombineFileInputFormat case Hive relies on the fact that CombineFileRecordReader sets this field every time a new file is processed. This file will then be read in the processor to setup the correct processing pipeline.

After the Tez refactor RecordReader and TezProcessor use different job conf instances. Because of that Hive will fail since map.input.file isn't set and updated in the processor's conf.



--
This message was sent by Atlassian JIRA
(v6.1#6144)