You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@orc.apache.org by Owen O'Malley <ow...@gmail.com> on 2019/06/27 03:50:01 UTC
[RESULT][VOTE] Shall we release ORC 1.5.6rc1?
With three +1's the vote passes. I'll finalize the release. Thanks, Alan and Deepak!
.. Owen
> On Jun 26, 2019, at 20:25, Deepak Majeti <ma...@gmail.com> wrote:
>
> +1
> - checked checksum and signature
> - built from tar in release mode
> - ran unit tests
> - ran rat checks
> - was able to generate a package
>
>> On Wed, Jun 26, 2019 at 5:16 PM Alan Gates <al...@gmail.com> wrote:
>>
>> +1. Checked the LICENSE and NOTICE files, did a build, ran the unit
>> tests. On the Java side did a build with a clean repo and ran RAT. (I
>> don't know how to run RAT on the C++ side but it would be good to somehow
>> make the same license checks.)
>>
>> One small nit, the copyright date in NOTICE is way out of date (2015). I
>> already updated it in both master and branch-1.5 so it should be fixed for
>> future releases.
>>
>> Alan.
>>
>> On Fri, Jun 21, 2019 at 9:31 PM Owen O'Malley <ow...@gmail.com>
>> wrote:
>>
>>> All,
>>>
>>> Should we release the following artifacts as ORC 1.5.6?
>>>
>>> tar: http://home.apache.org/~omalley/orc-1.5.6/
>>> tag: https://github.com/apache/orc/releases/tag/release-1.5.6rc1
>>> jiras: https://issues.apache.org/jira/projects/ORC/versions/12345170
>>>
>>> Thanks!
>>>
>>
>
>
> --
> regards,
> Deepak Majeti
Re: [RESULT][VOTE] Shall we release ORC 1.5.6rc1?
Posted by Dongjoon Hyun <do...@gmail.com>.
Got it!
Bests,
Dongjoon.
On Thu, Jun 27, 2019 at 16:25 Owen O'Malley <ow...@gmail.com> wrote:
> Thank you, Dongjoon.
>
> I took sometime this morning and didn't see what was going wrong. I think
> for right now, I'll finalize this release and we can roll a follow up when
> we find and fix the problem. I believe the 1.5.6rc1 is actually better for
> most users than 1.5.5.
>
> On Wed, Jun 26, 2019 at 10:06 PM Dongjoon Hyun <do...@gmail.com>
> wrote:
>
> > Thank you, Owen.
> >
> > Yes, I know that ORC-498 resolved the two NN operations issue and the
> code
> > is directly related to this area.
> >
> > First of all, I filed https://issues.apache.org/jira/browse/ORC-525 for
> > the
> > observed situation as a `Major` issue (not a blocker).
> >
> > I built/installed/tested new ORC with OracleJDK8/AdoptOpenJDK8 on two
> > MacBooks multiple times.
> > (and AdoptOpenJDK11 additionally)
> >
> > The failure consistently happens with Apache Spark master(3.0.0) at
> > `SQLQuerySuite`.
> >
> > Interestingly, the corresponding test suite in Apache Spark
> > branch-2.4(2.4.4) doesn't fail consistently.
> >
> > Instead, ORC 1.5.6 might reopen the flaky issue (SPARK-26427: closed with
> > ORC 1.5.4) instead at `branch-2.4`.
> >
> > Bests,
> > Dongjoon.
> >
> >
> > On Wed, Jun 26, 2019 at 9:05 PM Owen O'Malley <ow...@gmail.com>
> > wrote:
> >
> > > Dongjoon,
> > > If you need more time, I can hold off. For resource leakage, I'd be
> > > worried about ORC-498. I tried to make sure things didn't leak, but
> that
> > > was the change that was sharing a file handle between the ReaderImpl
> and
> > > the RecordReaderImpl so that we didn't end up with two NN operations
> when
> > > reading a file.
> > >
> > > .. Owen
> > >
> > > On Wed, Jun 26, 2019 at 8:51 PM Dongjoon Hyun <dongjoon.hyun@gmail.com
> >
> > > wrote:
> > >
> > > > Hi, All.
> > > >
> > > > Sorry, but Apache ORC 1.5.6-rc1 seems to have a regression.
> > > > Unlike Apache ORC 1.5.5, resource leakage are observed during Apache
> > > Spark
> > > > sql module unit tests.
> > > >
> > > > I've been spending time to retest this to make it sure.
> > > > Could you give me more time?
> > > >
> > > > Bests,
> > > > Dongjoon.
> > > >
> > > > On Wed, Jun 26, 2019 at 8:50 PM Owen O'Malley <
> owen.omalley@gmail.com>
> > > > wrote:
> > > >
> > > > > With three +1's the vote passes. I'll finalize the release. Thanks,
> > > Alan
> > > > > and Deepak!
> > > > >
> > > > > .. Owen
> > > > >
> > > > > > On Jun 26, 2019, at 20:25, Deepak Majeti <
> majeti.deepak@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > +1
> > > > > > - checked checksum and signature
> > > > > > - built from tar in release mode
> > > > > > - ran unit tests
> > > > > > - ran rat checks
> > > > > > - was able to generate a package
> > > > > >
> > > > > >> On Wed, Jun 26, 2019 at 5:16 PM Alan Gates <
> alanfgates@gmail.com>
> > > > > wrote:
> > > > > >>
> > > > > >> +1. Checked the LICENSE and NOTICE files, did a build, ran the
> > unit
> > > > > >> tests. On the Java side did a build with a clean repo and ran
> > RAT.
> > > > (I
> > > > > >> don't know how to run RAT on the C++ side but it would be good
> to
> > > > > somehow
> > > > > >> make the same license checks.)
> > > > > >>
> > > > > >> One small nit, the copyright date in NOTICE is way out of date
> > > (2015).
> > > > > I
> > > > > >> already updated it in both master and branch-1.5 so it should be
> > > fixed
> > > > > for
> > > > > >> future releases.
> > > > > >>
> > > > > >> Alan.
> > > > > >>
> > > > > >> On Fri, Jun 21, 2019 at 9:31 PM Owen O'Malley <
> > > owen.omalley@gmail.com
> > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >>> All,
> > > > > >>>
> > > > > >>> Should we release the following artifacts as ORC 1.5.6?
> > > > > >>>
> > > > > >>> tar: http://home.apache.org/~omalley/orc-1.5.6/
> > > > > >>> tag:
> https://github.com/apache/orc/releases/tag/release-1.5.6rc1
> > > > > >>> jiras:
> > > https://issues.apache.org/jira/projects/ORC/versions/12345170
> > > > > >>>
> > > > > >>> Thanks!
> > > > > >>>
> > > > > >>
> > > > > >
> > > > > >
> > > > > > --
> > > > > > regards,
> > > > > > Deepak Majeti
> > > > >
> > > >
> > >
> >
>
Re: [RESULT][VOTE] Shall we release ORC 1.5.6rc1?
Posted by Owen O'Malley <ow...@gmail.com>.
Thank you, Dongjoon.
I took sometime this morning and didn't see what was going wrong. I think
for right now, I'll finalize this release and we can roll a follow up when
we find and fix the problem. I believe the 1.5.6rc1 is actually better for
most users than 1.5.5.
On Wed, Jun 26, 2019 at 10:06 PM Dongjoon Hyun <do...@gmail.com>
wrote:
> Thank you, Owen.
>
> Yes, I know that ORC-498 resolved the two NN operations issue and the code
> is directly related to this area.
>
> First of all, I filed https://issues.apache.org/jira/browse/ORC-525 for
> the
> observed situation as a `Major` issue (not a blocker).
>
> I built/installed/tested new ORC with OracleJDK8/AdoptOpenJDK8 on two
> MacBooks multiple times.
> (and AdoptOpenJDK11 additionally)
>
> The failure consistently happens with Apache Spark master(3.0.0) at
> `SQLQuerySuite`.
>
> Interestingly, the corresponding test suite in Apache Spark
> branch-2.4(2.4.4) doesn't fail consistently.
>
> Instead, ORC 1.5.6 might reopen the flaky issue (SPARK-26427: closed with
> ORC 1.5.4) instead at `branch-2.4`.
>
> Bests,
> Dongjoon.
>
>
> On Wed, Jun 26, 2019 at 9:05 PM Owen O'Malley <ow...@gmail.com>
> wrote:
>
> > Dongjoon,
> > If you need more time, I can hold off. For resource leakage, I'd be
> > worried about ORC-498. I tried to make sure things didn't leak, but that
> > was the change that was sharing a file handle between the ReaderImpl and
> > the RecordReaderImpl so that we didn't end up with two NN operations when
> > reading a file.
> >
> > .. Owen
> >
> > On Wed, Jun 26, 2019 at 8:51 PM Dongjoon Hyun <do...@gmail.com>
> > wrote:
> >
> > > Hi, All.
> > >
> > > Sorry, but Apache ORC 1.5.6-rc1 seems to have a regression.
> > > Unlike Apache ORC 1.5.5, resource leakage are observed during Apache
> > Spark
> > > sql module unit tests.
> > >
> > > I've been spending time to retest this to make it sure.
> > > Could you give me more time?
> > >
> > > Bests,
> > > Dongjoon.
> > >
> > > On Wed, Jun 26, 2019 at 8:50 PM Owen O'Malley <ow...@gmail.com>
> > > wrote:
> > >
> > > > With three +1's the vote passes. I'll finalize the release. Thanks,
> > Alan
> > > > and Deepak!
> > > >
> > > > .. Owen
> > > >
> > > > > On Jun 26, 2019, at 20:25, Deepak Majeti <ma...@gmail.com>
> > > > wrote:
> > > > >
> > > > > +1
> > > > > - checked checksum and signature
> > > > > - built from tar in release mode
> > > > > - ran unit tests
> > > > > - ran rat checks
> > > > > - was able to generate a package
> > > > >
> > > > >> On Wed, Jun 26, 2019 at 5:16 PM Alan Gates <al...@gmail.com>
> > > > wrote:
> > > > >>
> > > > >> +1. Checked the LICENSE and NOTICE files, did a build, ran the
> unit
> > > > >> tests. On the Java side did a build with a clean repo and ran
> RAT.
> > > (I
> > > > >> don't know how to run RAT on the C++ side but it would be good to
> > > > somehow
> > > > >> make the same license checks.)
> > > > >>
> > > > >> One small nit, the copyright date in NOTICE is way out of date
> > (2015).
> > > > I
> > > > >> already updated it in both master and branch-1.5 so it should be
> > fixed
> > > > for
> > > > >> future releases.
> > > > >>
> > > > >> Alan.
> > > > >>
> > > > >> On Fri, Jun 21, 2019 at 9:31 PM Owen O'Malley <
> > owen.omalley@gmail.com
> > > >
> > > > >> wrote:
> > > > >>
> > > > >>> All,
> > > > >>>
> > > > >>> Should we release the following artifacts as ORC 1.5.6?
> > > > >>>
> > > > >>> tar: http://home.apache.org/~omalley/orc-1.5.6/
> > > > >>> tag: https://github.com/apache/orc/releases/tag/release-1.5.6rc1
> > > > >>> jiras:
> > https://issues.apache.org/jira/projects/ORC/versions/12345170
> > > > >>>
> > > > >>> Thanks!
> > > > >>>
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > regards,
> > > > > Deepak Majeti
> > > >
> > >
> >
>
Re: [RESULT][VOTE] Shall we release ORC 1.5.6rc1?
Posted by Dongjoon Hyun <do...@gmail.com>.
Thank you, Owen.
Yes, I know that ORC-498 resolved the two NN operations issue and the code
is directly related to this area.
First of all, I filed https://issues.apache.org/jira/browse/ORC-525 for the
observed situation as a `Major` issue (not a blocker).
I built/installed/tested new ORC with OracleJDK8/AdoptOpenJDK8 on two
MacBooks multiple times.
(and AdoptOpenJDK11 additionally)
The failure consistently happens with Apache Spark master(3.0.0) at
`SQLQuerySuite`.
Interestingly, the corresponding test suite in Apache Spark
branch-2.4(2.4.4) doesn't fail consistently.
Instead, ORC 1.5.6 might reopen the flaky issue (SPARK-26427: closed with
ORC 1.5.4) instead at `branch-2.4`.
Bests,
Dongjoon.
On Wed, Jun 26, 2019 at 9:05 PM Owen O'Malley <ow...@gmail.com>
wrote:
> Dongjoon,
> If you need more time, I can hold off. For resource leakage, I'd be
> worried about ORC-498. I tried to make sure things didn't leak, but that
> was the change that was sharing a file handle between the ReaderImpl and
> the RecordReaderImpl so that we didn't end up with two NN operations when
> reading a file.
>
> .. Owen
>
> On Wed, Jun 26, 2019 at 8:51 PM Dongjoon Hyun <do...@gmail.com>
> wrote:
>
> > Hi, All.
> >
> > Sorry, but Apache ORC 1.5.6-rc1 seems to have a regression.
> > Unlike Apache ORC 1.5.5, resource leakage are observed during Apache
> Spark
> > sql module unit tests.
> >
> > I've been spending time to retest this to make it sure.
> > Could you give me more time?
> >
> > Bests,
> > Dongjoon.
> >
> > On Wed, Jun 26, 2019 at 8:50 PM Owen O'Malley <ow...@gmail.com>
> > wrote:
> >
> > > With three +1's the vote passes. I'll finalize the release. Thanks,
> Alan
> > > and Deepak!
> > >
> > > .. Owen
> > >
> > > > On Jun 26, 2019, at 20:25, Deepak Majeti <ma...@gmail.com>
> > > wrote:
> > > >
> > > > +1
> > > > - checked checksum and signature
> > > > - built from tar in release mode
> > > > - ran unit tests
> > > > - ran rat checks
> > > > - was able to generate a package
> > > >
> > > >> On Wed, Jun 26, 2019 at 5:16 PM Alan Gates <al...@gmail.com>
> > > wrote:
> > > >>
> > > >> +1. Checked the LICENSE and NOTICE files, did a build, ran the unit
> > > >> tests. On the Java side did a build with a clean repo and ran RAT.
> > (I
> > > >> don't know how to run RAT on the C++ side but it would be good to
> > > somehow
> > > >> make the same license checks.)
> > > >>
> > > >> One small nit, the copyright date in NOTICE is way out of date
> (2015).
> > > I
> > > >> already updated it in both master and branch-1.5 so it should be
> fixed
> > > for
> > > >> future releases.
> > > >>
> > > >> Alan.
> > > >>
> > > >> On Fri, Jun 21, 2019 at 9:31 PM Owen O'Malley <
> owen.omalley@gmail.com
> > >
> > > >> wrote:
> > > >>
> > > >>> All,
> > > >>>
> > > >>> Should we release the following artifacts as ORC 1.5.6?
> > > >>>
> > > >>> tar: http://home.apache.org/~omalley/orc-1.5.6/
> > > >>> tag: https://github.com/apache/orc/releases/tag/release-1.5.6rc1
> > > >>> jiras:
> https://issues.apache.org/jira/projects/ORC/versions/12345170
> > > >>>
> > > >>> Thanks!
> > > >>>
> > > >>
> > > >
> > > >
> > > > --
> > > > regards,
> > > > Deepak Majeti
> > >
> >
>
Re: [RESULT][VOTE] Shall we release ORC 1.5.6rc1?
Posted by Owen O'Malley <ow...@gmail.com>.
Dongjoon,
If you need more time, I can hold off. For resource leakage, I'd be
worried about ORC-498. I tried to make sure things didn't leak, but that
was the change that was sharing a file handle between the ReaderImpl and
the RecordReaderImpl so that we didn't end up with two NN operations when
reading a file.
.. Owen
On Wed, Jun 26, 2019 at 8:51 PM Dongjoon Hyun <do...@gmail.com>
wrote:
> Hi, All.
>
> Sorry, but Apache ORC 1.5.6-rc1 seems to have a regression.
> Unlike Apache ORC 1.5.5, resource leakage are observed during Apache Spark
> sql module unit tests.
>
> I've been spending time to retest this to make it sure.
> Could you give me more time?
>
> Bests,
> Dongjoon.
>
> On Wed, Jun 26, 2019 at 8:50 PM Owen O'Malley <ow...@gmail.com>
> wrote:
>
> > With three +1's the vote passes. I'll finalize the release. Thanks, Alan
> > and Deepak!
> >
> > .. Owen
> >
> > > On Jun 26, 2019, at 20:25, Deepak Majeti <ma...@gmail.com>
> > wrote:
> > >
> > > +1
> > > - checked checksum and signature
> > > - built from tar in release mode
> > > - ran unit tests
> > > - ran rat checks
> > > - was able to generate a package
> > >
> > >> On Wed, Jun 26, 2019 at 5:16 PM Alan Gates <al...@gmail.com>
> > wrote:
> > >>
> > >> +1. Checked the LICENSE and NOTICE files, did a build, ran the unit
> > >> tests. On the Java side did a build with a clean repo and ran RAT.
> (I
> > >> don't know how to run RAT on the C++ side but it would be good to
> > somehow
> > >> make the same license checks.)
> > >>
> > >> One small nit, the copyright date in NOTICE is way out of date (2015).
> > I
> > >> already updated it in both master and branch-1.5 so it should be fixed
> > for
> > >> future releases.
> > >>
> > >> Alan.
> > >>
> > >> On Fri, Jun 21, 2019 at 9:31 PM Owen O'Malley <owen.omalley@gmail.com
> >
> > >> wrote:
> > >>
> > >>> All,
> > >>>
> > >>> Should we release the following artifacts as ORC 1.5.6?
> > >>>
> > >>> tar: http://home.apache.org/~omalley/orc-1.5.6/
> > >>> tag: https://github.com/apache/orc/releases/tag/release-1.5.6rc1
> > >>> jiras: https://issues.apache.org/jira/projects/ORC/versions/12345170
> > >>>
> > >>> Thanks!
> > >>>
> > >>
> > >
> > >
> > > --
> > > regards,
> > > Deepak Majeti
> >
>
Re: [RESULT][VOTE] Shall we release ORC 1.5.6rc1?
Posted by Dongjoon Hyun <do...@gmail.com>.
Hi, All.
Sorry, but Apache ORC 1.5.6-rc1 seems to have a regression.
Unlike Apache ORC 1.5.5, resource leakage are observed during Apache Spark
sql module unit tests.
I've been spending time to retest this to make it sure.
Could you give me more time?
Bests,
Dongjoon.
On Wed, Jun 26, 2019 at 8:50 PM Owen O'Malley <ow...@gmail.com>
wrote:
> With three +1's the vote passes. I'll finalize the release. Thanks, Alan
> and Deepak!
>
> .. Owen
>
> > On Jun 26, 2019, at 20:25, Deepak Majeti <ma...@gmail.com>
> wrote:
> >
> > +1
> > - checked checksum and signature
> > - built from tar in release mode
> > - ran unit tests
> > - ran rat checks
> > - was able to generate a package
> >
> >> On Wed, Jun 26, 2019 at 5:16 PM Alan Gates <al...@gmail.com>
> wrote:
> >>
> >> +1. Checked the LICENSE and NOTICE files, did a build, ran the unit
> >> tests. On the Java side did a build with a clean repo and ran RAT. (I
> >> don't know how to run RAT on the C++ side but it would be good to
> somehow
> >> make the same license checks.)
> >>
> >> One small nit, the copyright date in NOTICE is way out of date (2015).
> I
> >> already updated it in both master and branch-1.5 so it should be fixed
> for
> >> future releases.
> >>
> >> Alan.
> >>
> >> On Fri, Jun 21, 2019 at 9:31 PM Owen O'Malley <ow...@gmail.com>
> >> wrote:
> >>
> >>> All,
> >>>
> >>> Should we release the following artifacts as ORC 1.5.6?
> >>>
> >>> tar: http://home.apache.org/~omalley/orc-1.5.6/
> >>> tag: https://github.com/apache/orc/releases/tag/release-1.5.6rc1
> >>> jiras: https://issues.apache.org/jira/projects/ORC/versions/12345170
> >>>
> >>> Thanks!
> >>>
> >>
> >
> >
> > --
> > regards,
> > Deepak Majeti
>