[jira] [Resolved] (FLINK-16803) Need to make sure partition inherit table spec when writing to Hive tables

["Jingsong Lee (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-16803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jingsong Lee resolved FLINK-16803.
----------------------------------
    Fix Version/s: 1.11.0
       Resolution: Fixed

master: 5d9e5d46b37bb6e72e7c130323fa84518f691fbb

> Need to make sure partition inherit table spec when writing to Hive tables
> --------------------------------------------------------------------------
>
>                 Key: FLINK-16803
>                 URL: https://issues.apache.org/jira/browse/FLINK-16803
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Connectors / Hive
>            Reporter: Rui Li
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.11.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> When inserting to a partition, Hive will update the partition's SD according to table SD. For example, consider the following use case:
> {code}
> create table foo (x int,y int) partitioned by (p string);
> insert overwrite table foo partition (p='1') select * from ...;
> alter table foo set fileformat rcfile;
> insert overwrite table foo partition (p='1') select * from ...;
> {code}
> The second INSERT will write RC files to the partition and therefore the partition SD needs to be updated to use RC file input format. Otherwise we hit exception when reading from this partition.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)