You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@orc.apache.org by Owen O'Malley <om...@apache.org> on 2016/06/02 22:21:20 UTC
[VOTE] Apache ORC 1.1.0 RC 1
Ok, I addressed Prasanth's points and have rolled a new RC.
Artifacts: http://home.apache.org/~omalley/orc-1.1.0/
Git tag: https://github.com/apache/orc/releases/tag/release-1.1.0rc1
Voting will conclude in 72 hours.
Thanks,
Owen
Re: [VOTE] Apache ORC 1.1.0 RC 1
Posted by Matt Burgess <ma...@gmail.com>.
For writing I am using a VectorizedRowBatch and all the vectors in
org.apache.hadoop.hive.ql.exec.vector.*. For reading (in my unit
tests) I am using OrcFile.createReader().
https://github.com/apache/nifi/pull/477/files#diff-e6d9b9ac010d3b7a0f50ae90a46136d2R151
The first record has a 10 for the first column (union of integer and
null), the second has a null for the first column. My test right now
just validates the first record. If I check isNull() for the first
column of the second record, it says false.
I haven't had a chance to try RC3 but plan to do so shortly.
Thanks,
Matt
On Wed, Jun 8, 2016 at 12:28 PM, Owen O'Malley <om...@apache.org> wrote:
> Matt,
> So you were using the org.apache.orc.mapred API? Was the union value
> null or was the contained value null? Can you use orc-contents to dump the
> file's data?
>
> .. Owen
>
> On Thu, Jun 2, 2016 at 3:54 PM, Matt Burgess <ma...@gmail.com> wrote:
>
>> I am out of town through Sunday so I won't be able to give the RC a try
>> before the vote closes, but I've had a good experience with the snapshot so
>> far (the Java part) so I'm looking forward to the release!
>>
>> One thing I did see, but couldn't characterize enough to file a Jira, was
>> the setting of a null value into a union column. While the isNull entry
>> appeared to be set correctly in memory, when i wrote the file then read it
>> back in, the isNull entry was false (when it was true right before
>> writing). I will dig in when I get back; so far all observed errors have
>> been my own :)
>>
>> Regards,
>> Matt
>>
>> > On Jun 2, 2016, at 6:21 PM, Owen O'Malley <om...@apache.org> wrote:
>> >
>> > Ok, I addressed Prasanth's points and have rolled a new RC.
>> >
>> > Artifacts: http://home.apache.org/~omalley/orc-1.1.0/
>> > Git tag: https://github.com/apache/orc/releases/tag/release-1.1.0rc1
>> >
>> > Voting will conclude in 72 hours.
>> >
>> > Thanks,
>> > Owen
>>
Re: [VOTE] Apache ORC 1.1.0 RC 1
Posted by Owen O'Malley <om...@apache.org>.
Matt,
So you were using the org.apache.orc.mapred API? Was the union value
null or was the contained value null? Can you use orc-contents to dump the
file's data?
.. Owen
On Thu, Jun 2, 2016 at 3:54 PM, Matt Burgess <ma...@gmail.com> wrote:
> I am out of town through Sunday so I won't be able to give the RC a try
> before the vote closes, but I've had a good experience with the snapshot so
> far (the Java part) so I'm looking forward to the release!
>
> One thing I did see, but couldn't characterize enough to file a Jira, was
> the setting of a null value into a union column. While the isNull entry
> appeared to be set correctly in memory, when i wrote the file then read it
> back in, the isNull entry was false (when it was true right before
> writing). I will dig in when I get back; so far all observed errors have
> been my own :)
>
> Regards,
> Matt
>
> > On Jun 2, 2016, at 6:21 PM, Owen O'Malley <om...@apache.org> wrote:
> >
> > Ok, I addressed Prasanth's points and have rolled a new RC.
> >
> > Artifacts: http://home.apache.org/~omalley/orc-1.1.0/
> > Git tag: https://github.com/apache/orc/releases/tag/release-1.1.0rc1
> >
> > Voting will conclude in 72 hours.
> >
> > Thanks,
> > Owen
>
Re: [VOTE] Apache ORC 1.1.0 RC 1
Posted by Prasanth J <j....@gmail.com>.
+1 for RC1
- Verified build from src
- Verified signature and checksum
- Ran unit tests
Thanks
Prasanth
> On Jun 3, 2016, at 12:34 PM, Owen O'Malley <om...@apache.org> wrote:
>
> On Fri, Jun 3, 2016 at 10:57 AM, Chris Douglas <cd...@apache.org> wrote:
>
>> Going through the Java dependencies, all the licensing looks good
>> (tukaani:xz is public domain, Objenesis ASLv2, etc.).
>>
>> Should the storage API version be set as *-pre-orc? Just curious.
>> <storage-api.version>2.1.0-pre-orc</storage-api.version>
>>
>
> Yes, that is deliberate. The storage-api module is a mirror of the Hive
> storage-api module on branch-2.1 and so the 2.1.0-pre-orc version string
> was picked so that when Hive releases 2.1.0 maven will automatically pick
> the 2.1.0 version.
>
>
>>
>> Built and ran tests for both C++/Java (all pass), NOTICE/LICENSE look
>> good, except the NOTICE contains the following text:
>> ----
>> This product includes software developed by Hewlett-Packard:
>> (c) Copyright [2014-2015] Hewlett-Packard Development Company, L.P
>>
>> This product includes software developed by Hortonworks:
>> (c) Copyright [2014-2015] Hortonworks, Inc.
>>
>> The site formatting includes software developed by Tom Preston-Werner:
>> (c) Copyright [2008-2015] Tom Preston-Werner
>> ----
>>
>> Are there dependencies that require these? The docs for licensing [1]
>> stipulate that the NOTICE should not contain any text not legally
>> required.
>
>
> I can remove the Hortonworks one, but the HP line was a condition imposed
> by HP
> before moving the C++ code into Apache. I'll file a jira to remove the
> Hortonworks line.
>
> https://issues.apache.org/jira/browse/ORC-66
>
> .. Owen
>
>
>> -C
>>
>> [1] http://www.apache.org/dev/licensing-howto.html#mod-notice
>>
>>
>> On Thu, Jun 2, 2016 at 3:54 PM, Matt Burgess <ma...@gmail.com> wrote:
>>> I am out of town through Sunday so I won't be able to give the RC a try
>> before the vote closes, but I've had a good experience with the snapshot so
>> far (the Java part) so I'm looking forward to the release!
>>>
>>> One thing I did see, but couldn't characterize enough to file a Jira,
>> was the setting of a null value into a union column. While the isNull entry
>> appeared to be set correctly in memory, when i wrote the file then read it
>> back in, the isNull entry was false (when it was true right before
>> writing). I will dig in when I get back; so far all observed errors have
>> been my own :)
>>>
>>> Regards,
>>> Matt
>>>
>>>> On Jun 2, 2016, at 6:21 PM, Owen O'Malley <om...@apache.org> wrote:
>>>>
>>>> Ok, I addressed Prasanth's points and have rolled a new RC.
>>>>
>>>> Artifacts: http://home.apache.org/~omalley/orc-1.1.0/
>>>> Git tag: https://github.com/apache/orc/releases/tag/release-1.1.0rc1
>>>>
>>>> Voting will conclude in 72 hours.
>>>>
>>>> Thanks,
>>>> Owen
Re: [VOTE] Apache ORC 1.1.0 RC 1
Posted by Owen O'Malley <om...@apache.org>.
On Fri, Jun 3, 2016 at 10:57 AM, Chris Douglas <cd...@apache.org> wrote:
> Going through the Java dependencies, all the licensing looks good
> (tukaani:xz is public domain, Objenesis ASLv2, etc.).
>
> Should the storage API version be set as *-pre-orc? Just curious.
> <storage-api.version>2.1.0-pre-orc</storage-api.version>
>
Yes, that is deliberate. The storage-api module is a mirror of the Hive
storage-api module on branch-2.1 and so the 2.1.0-pre-orc version string
was picked so that when Hive releases 2.1.0 maven will automatically pick
the 2.1.0 version.
>
> Built and ran tests for both C++/Java (all pass), NOTICE/LICENSE look
> good, except the NOTICE contains the following text:
> ----
> This product includes software developed by Hewlett-Packard:
> (c) Copyright [2014-2015] Hewlett-Packard Development Company, L.P
>
> This product includes software developed by Hortonworks:
> (c) Copyright [2014-2015] Hortonworks, Inc.
>
> The site formatting includes software developed by Tom Preston-Werner:
> (c) Copyright [2008-2015] Tom Preston-Werner
> ----
>
> Are there dependencies that require these? The docs for licensing [1]
> stipulate that the NOTICE should not contain any text not legally
> required.
I can remove the Hortonworks one, but the HP line was a condition imposed
by HP
before moving the C++ code into Apache. I'll file a jira to remove the
Hortonworks line.
https://issues.apache.org/jira/browse/ORC-66
.. Owen
> -C
>
> [1] http://www.apache.org/dev/licensing-howto.html#mod-notice
>
>
> On Thu, Jun 2, 2016 at 3:54 PM, Matt Burgess <ma...@gmail.com> wrote:
> > I am out of town through Sunday so I won't be able to give the RC a try
> before the vote closes, but I've had a good experience with the snapshot so
> far (the Java part) so I'm looking forward to the release!
> >
> > One thing I did see, but couldn't characterize enough to file a Jira,
> was the setting of a null value into a union column. While the isNull entry
> appeared to be set correctly in memory, when i wrote the file then read it
> back in, the isNull entry was false (when it was true right before
> writing). I will dig in when I get back; so far all observed errors have
> been my own :)
> >
> > Regards,
> > Matt
> >
> >> On Jun 2, 2016, at 6:21 PM, Owen O'Malley <om...@apache.org> wrote:
> >>
> >> Ok, I addressed Prasanth's points and have rolled a new RC.
> >>
> >> Artifacts: http://home.apache.org/~omalley/orc-1.1.0/
> >> Git tag: https://github.com/apache/orc/releases/tag/release-1.1.0rc1
> >>
> >> Voting will conclude in 72 hours.
> >>
> >> Thanks,
> >> Owen
>
Re: [VOTE] Apache ORC 1.1.0 RC 1
Posted by Chris Douglas <cd...@apache.org>.
Going through the Java dependencies, all the licensing looks good
(tukaani:xz is public domain, Objenesis ASLv2, etc.).
Should the storage API version be set as *-pre-orc? Just curious.
<storage-api.version>2.1.0-pre-orc</storage-api.version>
Built and ran tests for both C++/Java (all pass), NOTICE/LICENSE look
good, except the NOTICE contains the following text:
----
This product includes software developed by Hewlett-Packard:
(c) Copyright [2014-2015] Hewlett-Packard Development Company, L.P
This product includes software developed by Hortonworks:
(c) Copyright [2014-2015] Hortonworks, Inc.
The site formatting includes software developed by Tom Preston-Werner:
(c) Copyright [2008-2015] Tom Preston-Werner
----
Are there dependencies that require these? The docs for licensing [1]
stipulate that the NOTICE should not contain any text not legally
required. -C
[1] http://www.apache.org/dev/licensing-howto.html#mod-notice
On Thu, Jun 2, 2016 at 3:54 PM, Matt Burgess <ma...@gmail.com> wrote:
> I am out of town through Sunday so I won't be able to give the RC a try before the vote closes, but I've had a good experience with the snapshot so far (the Java part) so I'm looking forward to the release!
>
> One thing I did see, but couldn't characterize enough to file a Jira, was the setting of a null value into a union column. While the isNull entry appeared to be set correctly in memory, when i wrote the file then read it back in, the isNull entry was false (when it was true right before writing). I will dig in when I get back; so far all observed errors have been my own :)
>
> Regards,
> Matt
>
>> On Jun 2, 2016, at 6:21 PM, Owen O'Malley <om...@apache.org> wrote:
>>
>> Ok, I addressed Prasanth's points and have rolled a new RC.
>>
>> Artifacts: http://home.apache.org/~omalley/orc-1.1.0/
>> Git tag: https://github.com/apache/orc/releases/tag/release-1.1.0rc1
>>
>> Voting will conclude in 72 hours.
>>
>> Thanks,
>> Owen
Re: [VOTE] Apache ORC 1.1.0 RC 1
Posted by Matt Burgess <ma...@gmail.com>.
I am out of town through Sunday so I won't be able to give the RC a try before the vote closes, but I've had a good experience with the snapshot so far (the Java part) so I'm looking forward to the release!
One thing I did see, but couldn't characterize enough to file a Jira, was the setting of a null value into a union column. While the isNull entry appeared to be set correctly in memory, when i wrote the file then read it back in, the isNull entry was false (when it was true right before writing). I will dig in when I get back; so far all observed errors have been my own :)
Regards,
Matt
> On Jun 2, 2016, at 6:21 PM, Owen O'Malley <om...@apache.org> wrote:
>
> Ok, I addressed Prasanth's points and have rolled a new RC.
>
> Artifacts: http://home.apache.org/~omalley/orc-1.1.0/
> Git tag: https://github.com/apache/orc/releases/tag/release-1.1.0rc1
>
> Voting will conclude in 72 hours.
>
> Thanks,
> Owen