You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Mithun Radhakrishnan (JIRA)" <ji...@apache.org> on 2017/07/27 04:24:00 UTC

[jira] [Created] (HIVE-17181) HCatOutputFormat should expose complete output-schema (including partition-keys) for dynamic-partitioning MR jobs

Mithun Radhakrishnan created HIVE-17181:
-------------------------------------------

             Summary: HCatOutputFormat should expose complete output-schema (including partition-keys) for dynamic-partitioning MR jobs
                 Key: HIVE-17181
                 URL: https://issues.apache.org/jira/browse/HIVE-17181
             Project: Hive
          Issue Type: Bug
          Components: HCatalog
            Reporter: Mithun Radhakrishnan
            Assignee: Mithun Radhakrishnan


Map/Reduce jobs that use HCatalog APIs to write to Hive tables using Dynamic partitioning are expected to call the following API methods:
# {{HCatOutputFormat.setOutput()}} to indicate which table/partitions to write to. This call populates the {{OutputJobInfo}} with details fetched from the Metastore.
# {{HCatOutputFormat.setSchema()}} to indicate the output-schema for the data being written.

It is a common mistake to invoke {{HCatOUtputFormat.setSchema()}} as follows:
{code:java}
HCatOutputFormat.setSchema(conf, HCatOutputFormat.getTableSchema(conf));
{code}

Unfortunately, {{getTableSchema()}} returns only the record-schema, not the entire table's schema. We'll need a better API for use in M/R jobs to get the complete table-schema.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)