You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Panos Garefalakis <pa...@gmail.com> on 2021/01/14 17:33:33 UTC
Upgrade ORC to 1.6
Hello Hive team,
I am happy to announce that as of today Apache HIve precommit tests with
ORC-1.6.7 (snapshot) are passing on master branch!
There were more than a few compatibility and bug fixes required to make the
jump from 1.5 but the version upgrade actually enables (among other things):
- Column-level encryption (ORC-14)
- Timestamps with local timezone (ORC-189)
- Zstd support (ORC-636)
- String stats min/max value trimming (ORC-203)
- Timestamp predicates with nanosecond precision (ORC-611)
Please feel free to share your feedback as part of HIVE-23553
<https://issues.apache.org/jira/browse/HIVE-23553>
Of course, we still maintain a lot of duplicate logic as part of the LLAP
engine that can be optimized down the line -- planning to add a few more
subtasks there.
Cheers,
Panagiotis
Re: Upgrade ORC to 1.6
Posted by Owen O'Malley <ow...@gmail.com>.
This is great news! Thank you for your hard work to make it happen. Getting
ORC 1.6.7 released shouldn't be hard.
.. Owen
On Fri, Jan 15, 2021 at 5:00 PM Dongjoon Hyun <do...@apache.org> wrote:
> Thank you, Panagiotis and Laszio.
>
> This is a big move forward in both Hive and Hive project.
>
> ORC community will start to release 1.6.7 next week.
>
> Bests,
> Dongjoon.
>
> On 2021/01/15 14:03:13, Panos Garefalakis <pa...@gmail.com> wrote:
> > Hey Laszlo,
> >
> > Thanks for taking a look!
> > Yes, releasing ORC-1.6.7 is definitely the way to go (snapshot was only
> > used for testing) and I am planning to start a discussion on the ORC list
> > soon.
> >
> > However, I wanted to get feedback from the Hive community first and
> address
> > any potential concerns for this (significant) change -- so please share
> > your thoughts on the PR :)
> >
> > Cheers,
> > Panagiotis
> >
> > On Fri, Jan 15, 2021 at 2:52 PM László Bodor <bo...@gmail.com>
> > wrote:
> >
> > > Hi!
> > >
> > > Thanks, Panos, that's very cool!
> > > I guess we won't depend on a snapshot, is there a chance that ORC
> > > community can fix issues in the near future and release 1.6.7
> officially to
> > > depend upon?
> > >
> > > Regards,
> > > Laszlo Bodor
> > >
> > >
> > > Panos Garefalakis <pa...@gmail.com> ezt írta (időpont: 2021. jan.
> 14.,
> > > Cs, 18:33):
> > >
> > >> Hello Hive team,
> > >>
> > >> I am happy to announce that as of today Apache HIve precommit tests
> with
> > >> ORC-1.6.7 (snapshot) are passing on master branch!
> > >> There were more than a few compatibility and bug fixes required to
> make
> > >> the
> > >> jump from 1.5 but the version upgrade actually enables (among other
> > >> things):
> > >>
> > >> - Column-level encryption (ORC-14)
> > >> - Timestamps with local timezone (ORC-189)
> > >> - Zstd support (ORC-636)
> > >> - String stats min/max value trimming (ORC-203)
> > >> - Timestamp predicates with nanosecond precision (ORC-611)
> > >>
> > >> Please feel free to share your feedback as part of HIVE-23553
> > >> <https://issues.apache.org/jira/browse/HIVE-23553>
> > >> Of course, we still maintain a lot of duplicate logic as part of the
> LLAP
> > >> engine that can be optimized down the line -- planning to add a few
> more
> > >> subtasks there.
> > >>
> > >> Cheers,
> > >> Panagiotis
> > >>
> > >
> >
>
Re: Upgrade ORC to 1.6
Posted by Dongjoon Hyun <do...@apache.org>.
Thank you, Panagiotis and Laszio.
This is a big move forward in both Hive and Hive project.
ORC community will start to release 1.6.7 next week.
Bests,
Dongjoon.
On 2021/01/15 14:03:13, Panos Garefalakis <pa...@gmail.com> wrote:
> Hey Laszlo,
>
> Thanks for taking a look!
> Yes, releasing ORC-1.6.7 is definitely the way to go (snapshot was only
> used for testing) and I am planning to start a discussion on the ORC list
> soon.
>
> However, I wanted to get feedback from the Hive community first and address
> any potential concerns for this (significant) change -- so please share
> your thoughts on the PR :)
>
> Cheers,
> Panagiotis
>
> On Fri, Jan 15, 2021 at 2:52 PM László Bodor <bo...@gmail.com>
> wrote:
>
> > Hi!
> >
> > Thanks, Panos, that's very cool!
> > I guess we won't depend on a snapshot, is there a chance that ORC
> > community can fix issues in the near future and release 1.6.7 officially to
> > depend upon?
> >
> > Regards,
> > Laszlo Bodor
> >
> >
> > Panos Garefalakis <pa...@gmail.com> ezt írta (időpont: 2021. jan. 14.,
> > Cs, 18:33):
> >
> >> Hello Hive team,
> >>
> >> I am happy to announce that as of today Apache HIve precommit tests with
> >> ORC-1.6.7 (snapshot) are passing on master branch!
> >> There were more than a few compatibility and bug fixes required to make
> >> the
> >> jump from 1.5 but the version upgrade actually enables (among other
> >> things):
> >>
> >> - Column-level encryption (ORC-14)
> >> - Timestamps with local timezone (ORC-189)
> >> - Zstd support (ORC-636)
> >> - String stats min/max value trimming (ORC-203)
> >> - Timestamp predicates with nanosecond precision (ORC-611)
> >>
> >> Please feel free to share your feedback as part of HIVE-23553
> >> <https://issues.apache.org/jira/browse/HIVE-23553>
> >> Of course, we still maintain a lot of duplicate logic as part of the LLAP
> >> engine that can be optimized down the line -- planning to add a few more
> >> subtasks there.
> >>
> >> Cheers,
> >> Panagiotis
> >>
> >
>
Re: Upgrade ORC to 1.6
Posted by Panos Garefalakis <pa...@gmail.com>.
Hey Laszlo,
Thanks for taking a look!
Yes, releasing ORC-1.6.7 is definitely the way to go (snapshot was only
used for testing) and I am planning to start a discussion on the ORC list
soon.
However, I wanted to get feedback from the Hive community first and address
any potential concerns for this (significant) change -- so please share
your thoughts on the PR :)
Cheers,
Panagiotis
On Fri, Jan 15, 2021 at 2:52 PM László Bodor <bo...@gmail.com>
wrote:
> Hi!
>
> Thanks, Panos, that's very cool!
> I guess we won't depend on a snapshot, is there a chance that ORC
> community can fix issues in the near future and release 1.6.7 officially to
> depend upon?
>
> Regards,
> Laszlo Bodor
>
>
> Panos Garefalakis <pa...@gmail.com> ezt írta (időpont: 2021. jan. 14.,
> Cs, 18:33):
>
>> Hello Hive team,
>>
>> I am happy to announce that as of today Apache HIve precommit tests with
>> ORC-1.6.7 (snapshot) are passing on master branch!
>> There were more than a few compatibility and bug fixes required to make
>> the
>> jump from 1.5 but the version upgrade actually enables (among other
>> things):
>>
>> - Column-level encryption (ORC-14)
>> - Timestamps with local timezone (ORC-189)
>> - Zstd support (ORC-636)
>> - String stats min/max value trimming (ORC-203)
>> - Timestamp predicates with nanosecond precision (ORC-611)
>>
>> Please feel free to share your feedback as part of HIVE-23553
>> <https://issues.apache.org/jira/browse/HIVE-23553>
>> Of course, we still maintain a lot of duplicate logic as part of the LLAP
>> engine that can be optimized down the line -- planning to add a few more
>> subtasks there.
>>
>> Cheers,
>> Panagiotis
>>
>
Re: Upgrade ORC to 1.6
Posted by Panos Garefalakis <pa...@gmail.com>.
Hey Laszlo,
Thanks for taking a look!
Yes, releasing ORC-1.6.7 is definitely the way to go (snapshot was only
used for testing) and I am planning to start a discussion on the ORC list
soon.
However, I wanted to get feedback from the Hive community first and address
any potential concerns for this (significant) change -- so please share
your thoughts on the PR :)
Cheers,
Panagiotis
On Fri, Jan 15, 2021 at 2:52 PM László Bodor <bo...@gmail.com>
wrote:
> Hi!
>
> Thanks, Panos, that's very cool!
> I guess we won't depend on a snapshot, is there a chance that ORC
> community can fix issues in the near future and release 1.6.7 officially to
> depend upon?
>
> Regards,
> Laszlo Bodor
>
>
> Panos Garefalakis <pa...@gmail.com> ezt írta (időpont: 2021. jan. 14.,
> Cs, 18:33):
>
>> Hello Hive team,
>>
>> I am happy to announce that as of today Apache HIve precommit tests with
>> ORC-1.6.7 (snapshot) are passing on master branch!
>> There were more than a few compatibility and bug fixes required to make
>> the
>> jump from 1.5 but the version upgrade actually enables (among other
>> things):
>>
>> - Column-level encryption (ORC-14)
>> - Timestamps with local timezone (ORC-189)
>> - Zstd support (ORC-636)
>> - String stats min/max value trimming (ORC-203)
>> - Timestamp predicates with nanosecond precision (ORC-611)
>>
>> Please feel free to share your feedback as part of HIVE-23553
>> <https://issues.apache.org/jira/browse/HIVE-23553>
>> Of course, we still maintain a lot of duplicate logic as part of the LLAP
>> engine that can be optimized down the line -- planning to add a few more
>> subtasks there.
>>
>> Cheers,
>> Panagiotis
>>
>
Re: Upgrade ORC to 1.6
Posted by László Bodor <bo...@gmail.com>.
Hi!
Thanks, Panos, that's very cool!
I guess we won't depend on a snapshot, is there a chance that ORC community
can fix issues in the near future and release 1.6.7 officially to depend
upon?
Regards,
Laszlo Bodor
Panos Garefalakis <pa...@gmail.com> ezt írta (időpont: 2021. jan. 14.,
Cs, 18:33):
> Hello Hive team,
>
> I am happy to announce that as of today Apache HIve precommit tests with
> ORC-1.6.7 (snapshot) are passing on master branch!
> There were more than a few compatibility and bug fixes required to make the
> jump from 1.5 but the version upgrade actually enables (among other
> things):
>
> - Column-level encryption (ORC-14)
> - Timestamps with local timezone (ORC-189)
> - Zstd support (ORC-636)
> - String stats min/max value trimming (ORC-203)
> - Timestamp predicates with nanosecond precision (ORC-611)
>
> Please feel free to share your feedback as part of HIVE-23553
> <https://issues.apache.org/jira/browse/HIVE-23553>
> Of course, we still maintain a lot of duplicate logic as part of the LLAP
> engine that can be optimized down the line -- planning to add a few more
> subtasks there.
>
> Cheers,
> Panagiotis
>
Re: Upgrade ORC to 1.6
Posted by László Bodor <bo...@gmail.com>.
Hi!
Thanks, Panos, that's very cool!
I guess we won't depend on a snapshot, is there a chance that ORC community
can fix issues in the near future and release 1.6.7 officially to depend
upon?
Regards,
Laszlo Bodor
Panos Garefalakis <pa...@gmail.com> ezt írta (időpont: 2021. jan. 14.,
Cs, 18:33):
> Hello Hive team,
>
> I am happy to announce that as of today Apache HIve precommit tests with
> ORC-1.6.7 (snapshot) are passing on master branch!
> There were more than a few compatibility and bug fixes required to make the
> jump from 1.5 but the version upgrade actually enables (among other
> things):
>
> - Column-level encryption (ORC-14)
> - Timestamps with local timezone (ORC-189)
> - Zstd support (ORC-636)
> - String stats min/max value trimming (ORC-203)
> - Timestamp predicates with nanosecond precision (ORC-611)
>
> Please feel free to share your feedback as part of HIVE-23553
> <https://issues.apache.org/jira/browse/HIVE-23553>
> Of course, we still maintain a lot of duplicate logic as part of the LLAP
> engine that can be optimized down the line -- planning to add a few more
> subtasks there.
>
> Cheers,
> Panagiotis
>