You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "uday (JIRA)" <ji...@apache.org> on 2016/01/07 19:21:39 UTC
[jira] [Commented] (HIVE-6897) Allow overwrite/append to external
Hive table (with partitions) via HCatStorer
[ https://issues.apache.org/jira/browse/HIVE-6897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15087812#comment-15087812 ]
uday commented on HIVE-6897:
----------------------------
Hi , I am facing the similar issue whenever I am trying to re-write the existing partitions , but it was saying that Caused by: org.apache.hive.hcatalog.common.HCatException : 2002 : Partition already present with given partition key values : Data already exists, duplicate publish not possible. So i am thinking to re-write the existing partition into by creating another folder (current run of that job). For example it will be written location such as
s3n://bucket/hive/warehouse/foo/2016-01-07/2016000107
s3n://bucket/hive/warehouse/foo/2016-01-07/2016000108
will this be possible , I see lot of conversation around re-writing existing partition. But I am not appending the data to the existing partitions , re-writing whole partitioned data into new folder that is current run. Is that possible ?
I appreciated your responses
> Allow overwrite/append to external Hive table (with partitions) via HCatStorer
> ------------------------------------------------------------------------------
>
> Key: HIVE-6897
> URL: https://issues.apache.org/jira/browse/HIVE-6897
> Project: Hive
> Issue Type: Improvement
> Components: HCatalog, HiveServer2
> Affects Versions: 0.12.0
> Reporter: Dip Kharod
>
> I'm using HCatStorer to write to external Hive table with partition from Pig and have the following different use cases:
> 1) Need to overwrite (aka, refresh) data into table: Currently I end up doing this outside (drop partition and delete HDFS folder) of Pig which is very painful and error-prone
> 2) Need to append (aka, add new file) data to the Hive external table/partition: Again, I end up doing this outside of Pig by copying file in appropriate folder
> It would be very productive for the developers to have both options in HCatStorer.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)