You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@orc.apache.org by Ben Johnson <be...@timber.io> on 2017/04/04 16:15:53 UTC
Removing a hive table column maps the data improperly
I wanted to confirm an issue I'm seeing when a Hive table's schema is modified
(columns deleted or re-ordered). It seems strange that ORC wouldn't support
column re-ordering when the column name is written into the file. Possibly I'm
doing something wrong?
For example, if I have a Hive table with columns A, B, and C which contains
existing data. If I alter the Hive table and remove column B. The new query
results will return the old values of B for column C. Column C is no longer
accessible since the table only contains 2 columns now.
Is this a known issue? Are table modifications recommended against as a result?
Thanks!
BenTimber.io -Blog -Github -Twitter
Re: Removing a hive table column maps the data improperly
Posted by Ben Johnson <be...@timber.io>.
Ah, thanks. Looking forward to the update.
BenTimber.io -Blog -Github -Twitter
On Tue, Apr 4, 2017 1:31 PM, Prasanth Jayachandran j.prasanth.j@gmail.com
wrote:
Hi Ben
Currently, column dropping or re-ordering is not supported in hive for ORC.The
upcoming hive release (2.3) should support this via
https://issues.apache.org/jira/browse/ORC-54
Thanks and Regards,Prasanth Jayachandran
On Tue, Apr 4, 2017 at 9:15 AM, Ben Johnson <be...@timber.io> wrote:
I wanted to confirm an issue I'm seeing when a Hive table's schema is modified
(columns deleted or re-ordered). It seems strange that ORC wouldn't support
column re-ordering when the column name is written into the file. Possibly I'm
doing something wrong?
For example, if I have a Hive table with columns A, B, and C which contains
existing data. If I alter the Hive table and remove column B. The new query
results will return the old values of B for column C. Column C is no longer
accessible since the table only contains 2 columns now.
Is this a known issue? Are table modifications recommended against as a result?
Thanks!
BenTimber.io -Blog -Github -Twitter
Re: Removing a hive table column maps the data improperly
Posted by Prasanth Jayachandran <j....@gmail.com>.
Hi Ben
Currently, column dropping or re-ordering is not supported in hive for ORC.
The upcoming hive release (2.3) should support this via
https://issues.apache.org/jira/browse/ORC-54
Thanks and Regards,
Prasanth Jayachandran
On Tue, Apr 4, 2017 at 9:15 AM, Ben Johnson <be...@timber.io> wrote:
> I wanted to confirm an issue I'm seeing when a Hive table's schema is
> modified (columns deleted or re-ordered). It seems strange that ORC
> wouldn't support column re-ordering when the column name is written into
> the file. Possibly I'm doing something wrong?
>
> For example, if I have a Hive table with columns A, B, and C which
> contains existing data. If I alter the Hive table and remove column B. The
> new query results will return the old values of B for column C. Column C is
> no longer accessible since the table only contains 2 columns now.
>
> Is this a known issue? Are table modifications recommended against as a
> result?
>
> Thanks!
>
>
>
> Ben
> Timber.io <https://timber.io/> - Blog <http://blog.timber.io/> - Github
> <https://github.com/timberio> - Twitter <https://twitter.com/timberdotio>
>