You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Neal Richardson <ne...@gmail.com> on 2020/10/06 01:41:12 UTC
Re: 2.0.0 release timeline: October 9
Hi folks, checking back in here. Thanks to our collective effort, we're
making good progress towards the 2.0 release, but there's still a bit more
to do. Per
https://cwiki.apache.org/confluence/display/ARROW/Arrow+2.0.0+Release,
we're at 31 unstarted and another 31 in-progress issues left for 2.0.
Blockers are down to 1, which I believe has a pull request up, but there
are still a few nightly builds failing (JIRAs ARROW-10175 to -10178).
We are planning to cut a release candidate on Friday, less than 4 days from
now, so please be working to wrap up any issues you have in flight, and
bump issues to 3.0 that you aren't going to be able to get to in time.
Neal
On Tue, Sep 29, 2020 at 5:46 PM Neal Richardson <ne...@gmail.com>
wrote:
> Thanks Neville. I also just bumped 60 C++/Python issues (new features,
> unassigned, not created recently) from 2.0. We're at 122 now, which still
> needs some pruning. I think something that more folks can do is review the
> issues assigned to them and make sure that they're planning to get all of
> those done for 2.0, otherwise bump to 3.0 or unassign themselves (or both).
>
> Neal
>
> On Tue, Sep 29, 2020 at 5:01 PM Neville Dipale <ne...@gmail.com>
> wrote:
>
>> Hi Neal,
>>
>> I've pruned the Rust backlog a bit, but only changed PRs that I've either
>> opened,
>> or those that I presume nobody is currently working on.
>>
>> There's 2 major features I've been working on:
>>
>> 1. Writing Arrow data to Parquet (separate branch)
>> 2. Integration testing
>>
>> I'll prioritise 2 as that's on the main branch, and we're nearly there.
>> However, with the arrow parquet writer, I'd like to ask the Rust
>> developers
>> if
>> we will want to release the WIP support with 2.0.0, or if we hold back and
>> keep
>> chipping away from a separate branch.
>>
>> There's been a new contributor who's offered to help with the writer,
>> so I think I'll be able to make more progress with her.
>>
>> Neville
>>
>> On Wed, 30 Sep 2020 at 00:34, Neal Richardson <
>> neal.p.richardson@gmail.com>
>> wrote:
>>
>> > Hi folks,
>> > As has been discussed in the biweekly meetings (and in the notes from
>> those
>> > meetings here on the mailing list), we're looking at an October timeline
>> > for our next release since we are going about 3 months between
>> releases. So
>> > that we might get the release voted on and shipped by the middle of the
>> > month, we should aim to be ready to cut our first (and final!) release
>> > candidate by next Friday, October 9.
>> >
>> > According to
>> > https://cwiki.apache.org/confluence/display/ARROW/Arrow+2.0.0+Release,
>> > there are still 178 issues tagged for 2.0 that are not yet started. That
>> > seems... ambitious. Please do go through the backlog and push to the
>> next
>> > release (i.e. 3.0.0) unassigned issues that aren't likely to land in the
>> > next 10 days.
>> >
>> > Likewise, I see that there are a few issues tagged as "blocker". Let's
>> > determine whether those truly should prevent a release candidate from
>> being
>> > made, and if so, let's make sure they get done ASAP.
>> >
>> > Neal
>> >
>>
>
Re: Flight gRPC version and disabling server verification in C++ [was
Re: 2.0.0 release timeline: October 9]
Posted by Micah Kornfield <em...@gmail.com>.
>
> Hence I'll merge it unless there's objections tomorrow afternoon (13 EST).
I left a couple of minor comments.
On Thu, Oct 8, 2020 at 6:42 PM David Li <li...@gmail.com> wrote:
> Wes has taken a look and so have I now, and I think this should be OK.
> I know Krisztián preferred to defer it before, but now it does not
> bump the gRPC version everywhere to pass the tests.
>
> Hence I'll merge it unless there's objections tomorrow afternoon (13 EST).
>
> David
>
> On 10/8/20, Antoine Pitrou <an...@python.org> wrote:
> >
> > I'll get to review the PR on Monday. It may be too late for the release.
> >
> > Regards
> >
> > Antoine.
> >
> >
> > Le 08/10/2020 à 18:43, James Duong a écrit :
> >> Hi,
> >>
> >> I've edited my PR now so that:
> >> 1. The CMakefiles so that we can detect which namespace
> >> TlsCredentialsOptions are in, if any.
> >> 2. Conditionally compile the C++ Flight client to use the namespace to
> >> implement disabling server verification, or compile out the
> >> implementation
> >> and throw an error at runtime if it is not available and the user tries
> >> to
> >> enable it.
> >> 3. The tests are also conditionally compiled.
> >>
> >> I feel this gives us the best of both worlds:
> >> 1. Users of the newer gRPCs (1.27+) can take advantage of this option
> and
> >> the namespace is mapped automatically.
> >> 2. Users of gRPC pre-1.27 do not need to upgrade (but will see an error
> >> if
> >> they use this option).
> >> 3. This is all transparent to someone building gRPC. There's no need to
> >> use
> >> additional macros.
> >>
> >> This has now passed all 36 CI tests, except this travis-ci job that is
> >> still pending:
> >> https://travis-ci.org/github/apache/arrow/builds/734025800
> >>
> >> Are we comfortable getting this into 2.0.0 with this design? I feel it
> is
> >> low risk now, especially since upgrading is not mandatory.
> >>
> >> On Wed, Oct 7, 2020 at 4:15 PM Krisztián Szűcs
> >> <sz...@gmail.com>
> >> wrote:
> >>
> >>> There is still some work left to make the packaging builds pass on the
> >>> PR. Considering how close we are to the release I find it risky to
> >>> include that change to 2.0. So I'm in favor of postponing it to 3.0.
> >>>
> >>>
> >>> On Wed, Oct 7, 2020 at 11:10 PM Micah Kornfield <emkornfield@gmail.com
> >
> >>> wrote:
> >>>>
> >>>> I agree with Antoine that we shouldn't be making changes to dependency
> >>>> versions so close to a release. This is consistent with other types
> of
> >>>> changes that could have a potentially large blast radius
> >>>>
> >>>> I don't have a strong opinion on what version we end up with though
> >>> (would
> >>>> need to do more research on compatibility guarantees)
> >>>>
> >>>> Micah
> >>>>
> >>>> On Wednesday, October 7, 2020, Neal Richardson <
> >>> neal.p.richardson@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> On Wed, Oct 7, 2020 at 1:11 PM Antoine Pitrou <an...@python.org>
> >>> wrote:
> >>>>>
> >>>>>>
> >>>>>> Le 07/10/2020 à 21:55, Neal Richardson a écrit :
> >>>>>>> * The only version that is a requirement is
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516
> >>>>>> ,
> >>>>>>> and so that's the one we're concerned about increasing. If we can
> >>> keep
> >>>>> it
> >>>>>>> low with an #ifdef, great. That said, I have no idea how slow
> >>> people
> >>>>> are
> >>>>>> to
> >>>>>>> update gRPC, or even what constitutes "old", so I can't say how
> >>> much
> >>>>>> extra
> >>>>>>> complication it is worth to support old versions.
> >>>>>>
> >>>>>> Well, the gRPC version provided by Ubuntu 20.04 is 1.16.1.
> >>>>>>
> >>>>>
> >>>>> According to
> >>>>>
> >>>>>
> >>>
> https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2509
> >>>>> ,
> >>>>> we already require 1.17, which is newer than that. And we've required
> >>> that
> >>>>> for the last year:
> >>>>>
> >>>>>
> >>>
> https://github.com/apache/arrow/commit/a70cf783364b140cab172e1851b563295c46e333
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>> * However, provided that the bundled build_grpc cmake macro works
> >>>>> (surely
> >>>>>>> we test that somewhere right?), if someone has
> >>>>>> ARROW_DEPENDENCY_SOURCE=AUTO
> >>>>>>> *and* they have old gRPC on their system, instead of a build
> >>> failure
> >>>>>>> they'll just get a slower build with the bundled grpc included.
> >>> That's
> >>>>>> not
> >>>>>>> a bad experience, and if the user doesn't like it, presumably they
> >>> can
> >>>>>>> upgrade system gRPC and rebuild.
> >>>>>>
> >>>>>> How do you upgrade system gRPC without potentially breaking other
> >>>>>> packages that rely on it? If it's a system library, it's generally
> >>>>>> recommended to follow system-dictated lifecycles.
> >>>>>>
> >>>>>> I am not saying that we should ensure compatibility with antiquated
> >>>>>> versions of gRPC, but being incompatible with the version provided
> by
> >>>>>> Ubuntu 20.04 (a 6-month old distribution) may be exaggerated.
> >>>>>>
> >>>>>> Regards
> >>>>>>
> >>>>>> Antoine.
> >>>>>>
> >>>>>
> >>>
> >>
> >>
> >
>
Re: Flight gRPC version and disabling server verification in C++ [was
Re: 2.0.0 release timeline: October 9]
Posted by David Li <li...@gmail.com>.
Wes has taken a look and so have I now, and I think this should be OK.
I know Krisztián preferred to defer it before, but now it does not
bump the gRPC version everywhere to pass the tests.
Hence I'll merge it unless there's objections tomorrow afternoon (13 EST).
David
On 10/8/20, Antoine Pitrou <an...@python.org> wrote:
>
> I'll get to review the PR on Monday. It may be too late for the release.
>
> Regards
>
> Antoine.
>
>
> Le 08/10/2020 à 18:43, James Duong a écrit :
>> Hi,
>>
>> I've edited my PR now so that:
>> 1. The CMakefiles so that we can detect which namespace
>> TlsCredentialsOptions are in, if any.
>> 2. Conditionally compile the C++ Flight client to use the namespace to
>> implement disabling server verification, or compile out the
>> implementation
>> and throw an error at runtime if it is not available and the user tries
>> to
>> enable it.
>> 3. The tests are also conditionally compiled.
>>
>> I feel this gives us the best of both worlds:
>> 1. Users of the newer gRPCs (1.27+) can take advantage of this option and
>> the namespace is mapped automatically.
>> 2. Users of gRPC pre-1.27 do not need to upgrade (but will see an error
>> if
>> they use this option).
>> 3. This is all transparent to someone building gRPC. There's no need to
>> use
>> additional macros.
>>
>> This has now passed all 36 CI tests, except this travis-ci job that is
>> still pending:
>> https://travis-ci.org/github/apache/arrow/builds/734025800
>>
>> Are we comfortable getting this into 2.0.0 with this design? I feel it is
>> low risk now, especially since upgrading is not mandatory.
>>
>> On Wed, Oct 7, 2020 at 4:15 PM Krisztián Szűcs
>> <sz...@gmail.com>
>> wrote:
>>
>>> There is still some work left to make the packaging builds pass on the
>>> PR. Considering how close we are to the release I find it risky to
>>> include that change to 2.0. So I'm in favor of postponing it to 3.0.
>>>
>>>
>>> On Wed, Oct 7, 2020 at 11:10 PM Micah Kornfield <em...@gmail.com>
>>> wrote:
>>>>
>>>> I agree with Antoine that we shouldn't be making changes to dependency
>>>> versions so close to a release. This is consistent with other types of
>>>> changes that could have a potentially large blast radius
>>>>
>>>> I don't have a strong opinion on what version we end up with though
>>> (would
>>>> need to do more research on compatibility guarantees)
>>>>
>>>> Micah
>>>>
>>>> On Wednesday, October 7, 2020, Neal Richardson <
>>> neal.p.richardson@gmail.com>
>>>> wrote:
>>>>
>>>>> On Wed, Oct 7, 2020 at 1:11 PM Antoine Pitrou <an...@python.org>
>>> wrote:
>>>>>
>>>>>>
>>>>>> Le 07/10/2020 à 21:55, Neal Richardson a écrit :
>>>>>>> * The only version that is a requirement is
>>>>>>>
>>>>>>
>>>>>
>>> https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516
>>>>>> ,
>>>>>>> and so that's the one we're concerned about increasing. If we can
>>> keep
>>>>> it
>>>>>>> low with an #ifdef, great. That said, I have no idea how slow
>>> people
>>>>> are
>>>>>> to
>>>>>>> update gRPC, or even what constitutes "old", so I can't say how
>>> much
>>>>>> extra
>>>>>>> complication it is worth to support old versions.
>>>>>>
>>>>>> Well, the gRPC version provided by Ubuntu 20.04 is 1.16.1.
>>>>>>
>>>>>
>>>>> According to
>>>>>
>>>>>
>>> https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2509
>>>>> ,
>>>>> we already require 1.17, which is newer than that. And we've required
>>> that
>>>>> for the last year:
>>>>>
>>>>>
>>> https://github.com/apache/arrow/commit/a70cf783364b140cab172e1851b563295c46e333
>>>>>
>>>>>
>>>>>>
>>>>>>> * However, provided that the bundled build_grpc cmake macro works
>>>>> (surely
>>>>>>> we test that somewhere right?), if someone has
>>>>>> ARROW_DEPENDENCY_SOURCE=AUTO
>>>>>>> *and* they have old gRPC on their system, instead of a build
>>> failure
>>>>>>> they'll just get a slower build with the bundled grpc included.
>>> That's
>>>>>> not
>>>>>>> a bad experience, and if the user doesn't like it, presumably they
>>> can
>>>>>>> upgrade system gRPC and rebuild.
>>>>>>
>>>>>> How do you upgrade system gRPC without potentially breaking other
>>>>>> packages that rely on it? If it's a system library, it's generally
>>>>>> recommended to follow system-dictated lifecycles.
>>>>>>
>>>>>> I am not saying that we should ensure compatibility with antiquated
>>>>>> versions of gRPC, but being incompatible with the version provided by
>>>>>> Ubuntu 20.04 (a 6-month old distribution) may be exaggerated.
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Antoine.
>>>>>>
>>>>>
>>>
>>
>>
>
Re: Flight gRPC version and disabling server verification in C++ [was
Re: 2.0.0 release timeline: October 9]
Posted by Antoine Pitrou <an...@python.org>.
I'll get to review the PR on Monday. It may be too late for the release.
Regards
Antoine.
Le 08/10/2020 à 18:43, James Duong a écrit :
> Hi,
>
> I've edited my PR now so that:
> 1. The CMakefiles so that we can detect which namespace
> TlsCredentialsOptions are in, if any.
> 2. Conditionally compile the C++ Flight client to use the namespace to
> implement disabling server verification, or compile out the implementation
> and throw an error at runtime if it is not available and the user tries to
> enable it.
> 3. The tests are also conditionally compiled.
>
> I feel this gives us the best of both worlds:
> 1. Users of the newer gRPCs (1.27+) can take advantage of this option and
> the namespace is mapped automatically.
> 2. Users of gRPC pre-1.27 do not need to upgrade (but will see an error if
> they use this option).
> 3. This is all transparent to someone building gRPC. There's no need to use
> additional macros.
>
> This has now passed all 36 CI tests, except this travis-ci job that is
> still pending:
> https://travis-ci.org/github/apache/arrow/builds/734025800
>
> Are we comfortable getting this into 2.0.0 with this design? I feel it is
> low risk now, especially since upgrading is not mandatory.
>
> On Wed, Oct 7, 2020 at 4:15 PM Krisztián Szűcs <sz...@gmail.com>
> wrote:
>
>> There is still some work left to make the packaging builds pass on the
>> PR. Considering how close we are to the release I find it risky to
>> include that change to 2.0. So I'm in favor of postponing it to 3.0.
>>
>>
>> On Wed, Oct 7, 2020 at 11:10 PM Micah Kornfield <em...@gmail.com>
>> wrote:
>>>
>>> I agree with Antoine that we shouldn't be making changes to dependency
>>> versions so close to a release. This is consistent with other types of
>>> changes that could have a potentially large blast radius
>>>
>>> I don't have a strong opinion on what version we end up with though
>> (would
>>> need to do more research on compatibility guarantees)
>>>
>>> Micah
>>>
>>> On Wednesday, October 7, 2020, Neal Richardson <
>> neal.p.richardson@gmail.com>
>>> wrote:
>>>
>>>> On Wed, Oct 7, 2020 at 1:11 PM Antoine Pitrou <an...@python.org>
>> wrote:
>>>>
>>>>>
>>>>> Le 07/10/2020 à 21:55, Neal Richardson a écrit :
>>>>>> * The only version that is a requirement is
>>>>>>
>>>>>
>>>>
>> https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516
>>>>> ,
>>>>>> and so that's the one we're concerned about increasing. If we can
>> keep
>>>> it
>>>>>> low with an #ifdef, great. That said, I have no idea how slow
>> people
>>>> are
>>>>> to
>>>>>> update gRPC, or even what constitutes "old", so I can't say how
>> much
>>>>> extra
>>>>>> complication it is worth to support old versions.
>>>>>
>>>>> Well, the gRPC version provided by Ubuntu 20.04 is 1.16.1.
>>>>>
>>>>
>>>> According to
>>>>
>>>>
>> https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2509
>>>> ,
>>>> we already require 1.17, which is newer than that. And we've required
>> that
>>>> for the last year:
>>>>
>>>>
>> https://github.com/apache/arrow/commit/a70cf783364b140cab172e1851b563295c46e333
>>>>
>>>>
>>>>>
>>>>>> * However, provided that the bundled build_grpc cmake macro works
>>>> (surely
>>>>>> we test that somewhere right?), if someone has
>>>>> ARROW_DEPENDENCY_SOURCE=AUTO
>>>>>> *and* they have old gRPC on their system, instead of a build
>> failure
>>>>>> they'll just get a slower build with the bundled grpc included.
>> That's
>>>>> not
>>>>>> a bad experience, and if the user doesn't like it, presumably they
>> can
>>>>>> upgrade system gRPC and rebuild.
>>>>>
>>>>> How do you upgrade system gRPC without potentially breaking other
>>>>> packages that rely on it? If it's a system library, it's generally
>>>>> recommended to follow system-dictated lifecycles.
>>>>>
>>>>> I am not saying that we should ensure compatibility with antiquated
>>>>> versions of gRPC, but being incompatible with the version provided by
>>>>> Ubuntu 20.04 (a 6-month old distribution) may be exaggerated.
>>>>>
>>>>> Regards
>>>>>
>>>>> Antoine.
>>>>>
>>>>
>>
>
>
Flight gRPC version and disabling server verification in C++ [was Re:
2.0.0 release timeline: October 9]
Posted by James Duong <ja...@bitquilltech.com>.
Hi,
I've edited my PR now so that:
1. The CMakefiles so that we can detect which namespace
TlsCredentialsOptions are in, if any.
2. Conditionally compile the C++ Flight client to use the namespace to
implement disabling server verification, or compile out the implementation
and throw an error at runtime if it is not available and the user tries to
enable it.
3. The tests are also conditionally compiled.
I feel this gives us the best of both worlds:
1. Users of the newer gRPCs (1.27+) can take advantage of this option and
the namespace is mapped automatically.
2. Users of gRPC pre-1.27 do not need to upgrade (but will see an error if
they use this option).
3. This is all transparent to someone building gRPC. There's no need to use
additional macros.
This has now passed all 36 CI tests, except this travis-ci job that is
still pending:
https://travis-ci.org/github/apache/arrow/builds/734025800
Are we comfortable getting this into 2.0.0 with this design? I feel it is
low risk now, especially since upgrading is not mandatory.
On Wed, Oct 7, 2020 at 4:15 PM Krisztián Szűcs <sz...@gmail.com>
wrote:
> There is still some work left to make the packaging builds pass on the
> PR. Considering how close we are to the release I find it risky to
> include that change to 2.0. So I'm in favor of postponing it to 3.0.
>
>
> On Wed, Oct 7, 2020 at 11:10 PM Micah Kornfield <em...@gmail.com>
> wrote:
> >
> > I agree with Antoine that we shouldn't be making changes to dependency
> > versions so close to a release. This is consistent with other types of
> > changes that could have a potentially large blast radius
> >
> > I don't have a strong opinion on what version we end up with though
> (would
> > need to do more research on compatibility guarantees)
> >
> > Micah
> >
> > On Wednesday, October 7, 2020, Neal Richardson <
> neal.p.richardson@gmail.com>
> > wrote:
> >
> > > On Wed, Oct 7, 2020 at 1:11 PM Antoine Pitrou <an...@python.org>
> wrote:
> > >
> > > >
> > > > Le 07/10/2020 à 21:55, Neal Richardson a écrit :
> > > > > * The only version that is a requirement is
> > > > >
> > > >
> > >
> https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516
> > > > ,
> > > > > and so that's the one we're concerned about increasing. If we can
> keep
> > > it
> > > > > low with an #ifdef, great. That said, I have no idea how slow
> people
> > > are
> > > > to
> > > > > update gRPC, or even what constitutes "old", so I can't say how
> much
> > > > extra
> > > > > complication it is worth to support old versions.
> > > >
> > > > Well, the gRPC version provided by Ubuntu 20.04 is 1.16.1.
> > > >
> > >
> > > According to
> > >
> > >
> https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2509
> > > ,
> > > we already require 1.17, which is newer than that. And we've required
> that
> > > for the last year:
> > >
> > >
> https://github.com/apache/arrow/commit/a70cf783364b140cab172e1851b563295c46e333
> > >
> > >
> > > >
> > > > > * However, provided that the bundled build_grpc cmake macro works
> > > (surely
> > > > > we test that somewhere right?), if someone has
> > > > ARROW_DEPENDENCY_SOURCE=AUTO
> > > > > *and* they have old gRPC on their system, instead of a build
> failure
> > > > > they'll just get a slower build with the bundled grpc included.
> That's
> > > > not
> > > > > a bad experience, and if the user doesn't like it, presumably they
> can
> > > > > upgrade system gRPC and rebuild.
> > > >
> > > > How do you upgrade system gRPC without potentially breaking other
> > > > packages that rely on it? If it's a system library, it's generally
> > > > recommended to follow system-dictated lifecycles.
> > > >
> > > > I am not saying that we should ensure compatibility with antiquated
> > > > versions of gRPC, but being incompatible with the version provided by
> > > > Ubuntu 20.04 (a 6-month old distribution) may be exaggerated.
> > > >
> > > > Regards
> > > >
> > > > Antoine.
> > > >
> > >
>
--
*James Duong*
Lead Software Developer
Bit Quill Technologies Inc.
Direct: +1.604.562.6082 | jamesd@bitquilltech.com
https://www.bitquilltech.com
This email message is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure, or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy
all copies of the original message. Thank you.
Re: 2.0.0 release timeline: October 9
Posted by Krisztián Szűcs <sz...@gmail.com>.
On Fri, Oct 9, 2020 at 2:49 AM Neal Richardson
<ne...@gmail.com> wrote:
>
> Hi folks,
> We're almost there.
> https://cwiki.apache.org/confluence/display/ARROW/Arrow+2.0.0+Release shows
> 11 open and 17 in progress issues for 2.0. No blockers, but there are still
> two nightly build failures ticketed (ARROW-10175 (pyarrow/hdfs),
> ARROW-10177 (gandiva xenial)) and perhaps others showing up in the most
> recent nightly build report.
>
> I suspect we will need most of tomorrow to resolve many of those issues.
> Given that, I doubt we'll have a first release candidate built and up for a
> vote tomorrow, but I'm hopeful that we'll be release-ready by the end of
> the day.
>
> Thoughts, everyone? Krisztián (who has bravely agreed to be the release
> manager again): what do you think?
I agree. It's likely that we're going to need this day to resolve the
remaining issues.
I'm ready to cut the release on the weekend as well.
>
> Neal
>
> On Wed, Oct 7, 2020 at 4:15 PM Krisztián Szűcs <sz...@gmail.com>
> wrote:
>
> > There is still some work left to make the packaging builds pass on the
> > PR. Considering how close we are to the release I find it risky to
> > include that change to 2.0. So I'm in favor of postponing it to 3.0.
> >
> >
> > On Wed, Oct 7, 2020 at 11:10 PM Micah Kornfield <em...@gmail.com>
> > wrote:
> > >
> > > I agree with Antoine that we shouldn't be making changes to dependency
> > > versions so close to a release. This is consistent with other types of
> > > changes that could have a potentially large blast radius
> > >
> > > I don't have a strong opinion on what version we end up with though
> > (would
> > > need to do more research on compatibility guarantees)
> > >
> > > Micah
> > >
> > > On Wednesday, October 7, 2020, Neal Richardson <
> > neal.p.richardson@gmail.com>
> > > wrote:
> > >
> > > > On Wed, Oct 7, 2020 at 1:11 PM Antoine Pitrou <an...@python.org>
> > wrote:
> > > >
> > > > >
> > > > > Le 07/10/2020 à 21:55, Neal Richardson a écrit :
> > > > > > * The only version that is a requirement is
> > > > > >
> > > > >
> > > >
> > https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516
> > > > > ,
> > > > > > and so that's the one we're concerned about increasing. If we can
> > keep
> > > > it
> > > > > > low with an #ifdef, great. That said, I have no idea how slow
> > people
> > > > are
> > > > > to
> > > > > > update gRPC, or even what constitutes "old", so I can't say how
> > much
> > > > > extra
> > > > > > complication it is worth to support old versions.
> > > > >
> > > > > Well, the gRPC version provided by Ubuntu 20.04 is 1.16.1.
> > > > >
> > > >
> > > > According to
> > > >
> > > >
> > https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2509
> > > > ,
> > > > we already require 1.17, which is newer than that. And we've required
> > that
> > > > for the last year:
> > > >
> > > >
> > https://github.com/apache/arrow/commit/a70cf783364b140cab172e1851b563295c46e333
> > > >
> > > >
> > > > >
> > > > > > * However, provided that the bundled build_grpc cmake macro works
> > > > (surely
> > > > > > we test that somewhere right?), if someone has
> > > > > ARROW_DEPENDENCY_SOURCE=AUTO
> > > > > > *and* they have old gRPC on their system, instead of a build
> > failure
> > > > > > they'll just get a slower build with the bundled grpc included.
> > That's
> > > > > not
> > > > > > a bad experience, and if the user doesn't like it, presumably they
> > can
> > > > > > upgrade system gRPC and rebuild.
> > > > >
> > > > > How do you upgrade system gRPC without potentially breaking other
> > > > > packages that rely on it? If it's a system library, it's generally
> > > > > recommended to follow system-dictated lifecycles.
> > > > >
> > > > > I am not saying that we should ensure compatibility with antiquated
> > > > > versions of gRPC, but being incompatible with the version provided by
> > > > > Ubuntu 20.04 (a 6-month old distribution) may be exaggerated.
> > > > >
> > > > > Regards
> > > > >
> > > > > Antoine.
> > > > >
> > > >
> >
Re: 2.0.0 release timeline: October 9
Posted by Neal Richardson <ne...@gmail.com>.
Hi folks,
We're almost there.
https://cwiki.apache.org/confluence/display/ARROW/Arrow+2.0.0+Release shows
11 open and 17 in progress issues for 2.0. No blockers, but there are still
two nightly build failures ticketed (ARROW-10175 (pyarrow/hdfs),
ARROW-10177 (gandiva xenial)) and perhaps others showing up in the most
recent nightly build report.
I suspect we will need most of tomorrow to resolve many of those issues.
Given that, I doubt we'll have a first release candidate built and up for a
vote tomorrow, but I'm hopeful that we'll be release-ready by the end of
the day.
Thoughts, everyone? Krisztián (who has bravely agreed to be the release
manager again): what do you think?
Neal
On Wed, Oct 7, 2020 at 4:15 PM Krisztián Szűcs <sz...@gmail.com>
wrote:
> There is still some work left to make the packaging builds pass on the
> PR. Considering how close we are to the release I find it risky to
> include that change to 2.0. So I'm in favor of postponing it to 3.0.
>
>
> On Wed, Oct 7, 2020 at 11:10 PM Micah Kornfield <em...@gmail.com>
> wrote:
> >
> > I agree with Antoine that we shouldn't be making changes to dependency
> > versions so close to a release. This is consistent with other types of
> > changes that could have a potentially large blast radius
> >
> > I don't have a strong opinion on what version we end up with though
> (would
> > need to do more research on compatibility guarantees)
> >
> > Micah
> >
> > On Wednesday, October 7, 2020, Neal Richardson <
> neal.p.richardson@gmail.com>
> > wrote:
> >
> > > On Wed, Oct 7, 2020 at 1:11 PM Antoine Pitrou <an...@python.org>
> wrote:
> > >
> > > >
> > > > Le 07/10/2020 à 21:55, Neal Richardson a écrit :
> > > > > * The only version that is a requirement is
> > > > >
> > > >
> > >
> https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516
> > > > ,
> > > > > and so that's the one we're concerned about increasing. If we can
> keep
> > > it
> > > > > low with an #ifdef, great. That said, I have no idea how slow
> people
> > > are
> > > > to
> > > > > update gRPC, or even what constitutes "old", so I can't say how
> much
> > > > extra
> > > > > complication it is worth to support old versions.
> > > >
> > > > Well, the gRPC version provided by Ubuntu 20.04 is 1.16.1.
> > > >
> > >
> > > According to
> > >
> > >
> https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2509
> > > ,
> > > we already require 1.17, which is newer than that. And we've required
> that
> > > for the last year:
> > >
> > >
> https://github.com/apache/arrow/commit/a70cf783364b140cab172e1851b563295c46e333
> > >
> > >
> > > >
> > > > > * However, provided that the bundled build_grpc cmake macro works
> > > (surely
> > > > > we test that somewhere right?), if someone has
> > > > ARROW_DEPENDENCY_SOURCE=AUTO
> > > > > *and* they have old gRPC on their system, instead of a build
> failure
> > > > > they'll just get a slower build with the bundled grpc included.
> That's
> > > > not
> > > > > a bad experience, and if the user doesn't like it, presumably they
> can
> > > > > upgrade system gRPC and rebuild.
> > > >
> > > > How do you upgrade system gRPC without potentially breaking other
> > > > packages that rely on it? If it's a system library, it's generally
> > > > recommended to follow system-dictated lifecycles.
> > > >
> > > > I am not saying that we should ensure compatibility with antiquated
> > > > versions of gRPC, but being incompatible with the version provided by
> > > > Ubuntu 20.04 (a 6-month old distribution) may be exaggerated.
> > > >
> > > > Regards
> > > >
> > > > Antoine.
> > > >
> > >
>
Re: 2.0.0 release timeline: October 9
Posted by Krisztián Szűcs <sz...@gmail.com>.
There is still some work left to make the packaging builds pass on the
PR. Considering how close we are to the release I find it risky to
include that change to 2.0. So I'm in favor of postponing it to 3.0.
On Wed, Oct 7, 2020 at 11:10 PM Micah Kornfield <em...@gmail.com> wrote:
>
> I agree with Antoine that we shouldn't be making changes to dependency
> versions so close to a release. This is consistent with other types of
> changes that could have a potentially large blast radius
>
> I don't have a strong opinion on what version we end up with though (would
> need to do more research on compatibility guarantees)
>
> Micah
>
> On Wednesday, October 7, 2020, Neal Richardson <ne...@gmail.com>
> wrote:
>
> > On Wed, Oct 7, 2020 at 1:11 PM Antoine Pitrou <an...@python.org> wrote:
> >
> > >
> > > Le 07/10/2020 à 21:55, Neal Richardson a écrit :
> > > > * The only version that is a requirement is
> > > >
> > >
> > https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516
> > > ,
> > > > and so that's the one we're concerned about increasing. If we can keep
> > it
> > > > low with an #ifdef, great. That said, I have no idea how slow people
> > are
> > > to
> > > > update gRPC, or even what constitutes "old", so I can't say how much
> > > extra
> > > > complication it is worth to support old versions.
> > >
> > > Well, the gRPC version provided by Ubuntu 20.04 is 1.16.1.
> > >
> >
> > According to
> >
> > https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2509
> > ,
> > we already require 1.17, which is newer than that. And we've required that
> > for the last year:
> >
> > https://github.com/apache/arrow/commit/a70cf783364b140cab172e1851b563295c46e333
> >
> >
> > >
> > > > * However, provided that the bundled build_grpc cmake macro works
> > (surely
> > > > we test that somewhere right?), if someone has
> > > ARROW_DEPENDENCY_SOURCE=AUTO
> > > > *and* they have old gRPC on their system, instead of a build failure
> > > > they'll just get a slower build with the bundled grpc included. That's
> > > not
> > > > a bad experience, and if the user doesn't like it, presumably they can
> > > > upgrade system gRPC and rebuild.
> > >
> > > How do you upgrade system gRPC without potentially breaking other
> > > packages that rely on it? If it's a system library, it's generally
> > > recommended to follow system-dictated lifecycles.
> > >
> > > I am not saying that we should ensure compatibility with antiquated
> > > versions of gRPC, but being incompatible with the version provided by
> > > Ubuntu 20.04 (a 6-month old distribution) may be exaggerated.
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> >
Re: 2.0.0 release timeline: October 9
Posted by Micah Kornfield <em...@gmail.com>.
I agree with Antoine that we shouldn't be making changes to dependency
versions so close to a release. This is consistent with other types of
changes that could have a potentially large blast radius
I don't have a strong opinion on what version we end up with though (would
need to do more research on compatibility guarantees)
Micah
On Wednesday, October 7, 2020, Neal Richardson <ne...@gmail.com>
wrote:
> On Wed, Oct 7, 2020 at 1:11 PM Antoine Pitrou <an...@python.org> wrote:
>
> >
> > Le 07/10/2020 à 21:55, Neal Richardson a écrit :
> > > * The only version that is a requirement is
> > >
> >
> https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516
> > ,
> > > and so that's the one we're concerned about increasing. If we can keep
> it
> > > low with an #ifdef, great. That said, I have no idea how slow people
> are
> > to
> > > update gRPC, or even what constitutes "old", so I can't say how much
> > extra
> > > complication it is worth to support old versions.
> >
> > Well, the gRPC version provided by Ubuntu 20.04 is 1.16.1.
> >
>
> According to
>
> https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2509
> ,
> we already require 1.17, which is newer than that. And we've required that
> for the last year:
>
> https://github.com/apache/arrow/commit/a70cf783364b140cab172e1851b563295c46e333
>
>
> >
> > > * However, provided that the bundled build_grpc cmake macro works
> (surely
> > > we test that somewhere right?), if someone has
> > ARROW_DEPENDENCY_SOURCE=AUTO
> > > *and* they have old gRPC on their system, instead of a build failure
> > > they'll just get a slower build with the bundled grpc included. That's
> > not
> > > a bad experience, and if the user doesn't like it, presumably they can
> > > upgrade system gRPC and rebuild.
> >
> > How do you upgrade system gRPC without potentially breaking other
> > packages that rely on it? If it's a system library, it's generally
> > recommended to follow system-dictated lifecycles.
> >
> > I am not saying that we should ensure compatibility with antiquated
> > versions of gRPC, but being incompatible with the version provided by
> > Ubuntu 20.04 (a 6-month old distribution) may be exaggerated.
> >
> > Regards
> >
> > Antoine.
> >
>
Re: 2.0.0 release timeline: October 9
Posted by Neal Richardson <ne...@gmail.com>.
On Wed, Oct 7, 2020 at 1:11 PM Antoine Pitrou <an...@python.org> wrote:
>
> Le 07/10/2020 à 21:55, Neal Richardson a écrit :
> > * The only version that is a requirement is
> >
> https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516
> ,
> > and so that's the one we're concerned about increasing. If we can keep it
> > low with an #ifdef, great. That said, I have no idea how slow people are
> to
> > update gRPC, or even what constitutes "old", so I can't say how much
> extra
> > complication it is worth to support old versions.
>
> Well, the gRPC version provided by Ubuntu 20.04 is 1.16.1.
>
According to
https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake#L2509,
we already require 1.17, which is newer than that. And we've required that
for the last year:
https://github.com/apache/arrow/commit/a70cf783364b140cab172e1851b563295c46e333
>
> > * However, provided that the bundled build_grpc cmake macro works (surely
> > we test that somewhere right?), if someone has
> ARROW_DEPENDENCY_SOURCE=AUTO
> > *and* they have old gRPC on their system, instead of a build failure
> > they'll just get a slower build with the bundled grpc included. That's
> not
> > a bad experience, and if the user doesn't like it, presumably they can
> > upgrade system gRPC and rebuild.
>
> How do you upgrade system gRPC without potentially breaking other
> packages that rely on it? If it's a system library, it's generally
> recommended to follow system-dictated lifecycles.
>
> I am not saying that we should ensure compatibility with antiquated
> versions of gRPC, but being incompatible with the version provided by
> Ubuntu 20.04 (a 6-month old distribution) may be exaggerated.
>
> Regards
>
> Antoine.
>
Re: 2.0.0 release timeline: October 9
Posted by Antoine Pitrou <an...@python.org>.
Le 07/10/2020 à 21:55, Neal Richardson a écrit :
> * The only version that is a requirement is
> https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516,
> and so that's the one we're concerned about increasing. If we can keep it
> low with an #ifdef, great. That said, I have no idea how slow people are to
> update gRPC, or even what constitutes "old", so I can't say how much extra
> complication it is worth to support old versions.
Well, the gRPC version provided by Ubuntu 20.04 is 1.16.1.
> * However, provided that the bundled build_grpc cmake macro works (surely
> we test that somewhere right?), if someone has ARROW_DEPENDENCY_SOURCE=AUTO
> *and* they have old gRPC on their system, instead of a build failure
> they'll just get a slower build with the bundled grpc included. That's not
> a bad experience, and if the user doesn't like it, presumably they can
> upgrade system gRPC and rebuild.
How do you upgrade system gRPC without potentially breaking other
packages that rely on it? If it's a system library, it's generally
recommended to follow system-dictated lifecycles.
I am not saying that we should ensure compatibility with antiquated
versions of gRPC, but being incompatible with the version provided by
Ubuntu 20.04 (a 6-month old distribution) may be exaggerated.
Regards
Antoine.
Re: 2.0.0 release timeline: October 9
Posted by Neal Richardson <ne...@gmail.com>.
I am in no way in charge to make a decision here--I'm just the one nagging
everyone to get their patches merged by Friday ;)--but my personal thoughts
when looking at https://github.com/apache/arrow/pull/8325:
* The code and CI config have gRPC versions in various places. Before a
release is a great time to bump the versions we build and test with, so
that we have confidence that our code works with the latest released
versions of dependencies.
* The only version that is a requirement is
https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516,
and so that's the one we're concerned about increasing. If we can keep it
low with an #ifdef, great. That said, I have no idea how slow people are to
update gRPC, or even what constitutes "old", so I can't say how much extra
complication it is worth to support old versions.
* However, provided that the bundled build_grpc cmake macro works (surely
we test that somewhere right?), if someone has ARROW_DEPENDENCY_SOURCE=AUTO
*and* they have old gRPC on their system, instead of a build failure
they'll just get a slower build with the bundled grpc included. That's not
a bad experience, and if the user doesn't like it, presumably they can
upgrade system gRPC and rebuild.
Neal
On Wed, Oct 7, 2020 at 12:35 PM Antoine Pitrou <an...@python.org> wrote:
>
> Or perhaps we (meaning James :-)) can add an #ifdef-based switch.
>
> There's no need to penalize users of older gRPCs for a rather optional
> feature (if you can call disabling a security verification a feature, of
> course ;-)).
>
> Regards
>
> Antoine.
>
>
> Le 07/10/2020 à 21:33, Wes McKinney a écrit :
> > Given Google's "live at head" mantra, in principle I don't have a
> > problem with requiring a < 1 year old version of gRPC
> >
> > On Wed, Oct 7, 2020 at 2:23 PM Antoine Pitrou <an...@python.org>
> wrote:
> >>
> >>
> >> Le 07/10/2020 à 21:19, James Duong a écrit :
> >>> Hi Neal,
> >>>
> >>> Are you the release manager for 2.0?
> >>> I've been working on the task to disable server verification in Flight
> >>> clients, and it appears we'll need
> >>> to update the minimum gRPC version to at least 1.27 to support this.
> >>>
> >>> Would it be OK to do this for the 2.0. release? It looks like we also
> need
> >>> to update Ursa configurations here:
> >>>
> https://github.com/ursa-labs/ursabot/blob/e958c5f95b31e98108df54cf13596c4fde944c3a/projects/arrow/docker/conda-cpp.txt#L19
> >>
> >> IMHO that implies a lot of potential issues to watch for, at the last
> >> minute before a release. Personally, I'd rather see this in 3.0.
> >>
> >> Regards
> >>
> >> Antoine.
>
Re: 2.0.0 release timeline: October 9
Posted by Antoine Pitrou <so...@pitrou.net>.
On Wed, 7 Oct 2020 12:46:22 -0700
James Duong <ja...@bitquilltech.com> wrote:
> I could add a #ifdef around this, however gRPC itself doesn't appear to
> provide a version macro.
Hmm, can you please report an issue upstream? This could be more
generally useful.
> We also need a macro to define what namespace we get the new API from -- it
> changes in gRPC 1.32
> and above.
Note you can perhaps get around those version issues using template
hackery. We already did that here:
https://github.com/apache/arrow/blob/master/cpp/src/arrow/filesystem/s3fs.cc#L462-L477
Not pretty and annoying to get right, but better than nothing :-)
> How about I go ahead and put a few commits on top of my current PR to make
> this a bit more concrete.
Please do!
Regards
Antoine.
Re: 2.0.0 release timeline: October 9
Posted by James Duong <ja...@bitquilltech.com>.
I could add a #ifdef around this, however gRPC itself doesn't appear to
provide a version macro.
So the person building flight would have to define this option themselves.
We also need a macro to define what namespace we get the new API from -- it
changes in gRPC 1.32
and above.
How about I go ahead and put a few commits on top of my current PR to make
this a bit more concrete.
On Wed, Oct 7, 2020 at 12:35 PM Antoine Pitrou <an...@python.org> wrote:
>
> Or perhaps we (meaning James :-)) can add an #ifdef-based switch.
>
> There's no need to penalize users of older gRPCs for a rather optional
> feature (if you can call disabling a security verification a feature, of
> course ;-)).
>
> Regards
>
> Antoine.
>
>
> Le 07/10/2020 à 21:33, Wes McKinney a écrit :
> > Given Google's "live at head" mantra, in principle I don't have a
> > problem with requiring a < 1 year old version of gRPC
> >
> > On Wed, Oct 7, 2020 at 2:23 PM Antoine Pitrou <an...@python.org>
> wrote:
> >>
> >>
> >> Le 07/10/2020 à 21:19, James Duong a écrit :
> >>> Hi Neal,
> >>>
> >>> Are you the release manager for 2.0?
> >>> I've been working on the task to disable server verification in Flight
> >>> clients, and it appears we'll need
> >>> to update the minimum gRPC version to at least 1.27 to support this.
> >>>
> >>> Would it be OK to do this for the 2.0. release? It looks like we also
> need
> >>> to update Ursa configurations here:
> >>>
> https://github.com/ursa-labs/ursabot/blob/e958c5f95b31e98108df54cf13596c4fde944c3a/projects/arrow/docker/conda-cpp.txt#L19
> >>
> >> IMHO that implies a lot of potential issues to watch for, at the last
> >> minute before a release. Personally, I'd rather see this in 3.0.
> >>
> >> Regards
> >>
> >> Antoine.
>
--
*James Duong*
Lead Software Developer
Bit Quill Technologies Inc.
Direct: +1.604.562.6082 | jamesd@bitquilltech.com
https://www.bitquilltech.com
This email message is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure, or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy
all copies of the original message. Thank you.
Re: 2.0.0 release timeline: October 9
Posted by Antoine Pitrou <an...@python.org>.
Or perhaps we (meaning James :-)) can add an #ifdef-based switch.
There's no need to penalize users of older gRPCs for a rather optional
feature (if you can call disabling a security verification a feature, of
course ;-)).
Regards
Antoine.
Le 07/10/2020 à 21:33, Wes McKinney a écrit :
> Given Google's "live at head" mantra, in principle I don't have a
> problem with requiring a < 1 year old version of gRPC
>
> On Wed, Oct 7, 2020 at 2:23 PM Antoine Pitrou <an...@python.org> wrote:
>>
>>
>> Le 07/10/2020 à 21:19, James Duong a écrit :
>>> Hi Neal,
>>>
>>> Are you the release manager for 2.0?
>>> I've been working on the task to disable server verification in Flight
>>> clients, and it appears we'll need
>>> to update the minimum gRPC version to at least 1.27 to support this.
>>>
>>> Would it be OK to do this for the 2.0. release? It looks like we also need
>>> to update Ursa configurations here:
>>> https://github.com/ursa-labs/ursabot/blob/e958c5f95b31e98108df54cf13596c4fde944c3a/projects/arrow/docker/conda-cpp.txt#L19
>>
>> IMHO that implies a lot of potential issues to watch for, at the last
>> minute before a release. Personally, I'd rather see this in 3.0.
>>
>> Regards
>>
>> Antoine.
Re: 2.0.0 release timeline: October 9
Posted by Wes McKinney <we...@gmail.com>.
Given Google's "live at head" mantra, in principle I don't have a
problem with requiring a < 1 year old version of gRPC
On Wed, Oct 7, 2020 at 2:23 PM Antoine Pitrou <an...@python.org> wrote:
>
>
> Le 07/10/2020 à 21:19, James Duong a écrit :
> > Hi Neal,
> >
> > Are you the release manager for 2.0?
> > I've been working on the task to disable server verification in Flight
> > clients, and it appears we'll need
> > to update the minimum gRPC version to at least 1.27 to support this.
> >
> > Would it be OK to do this for the 2.0. release? It looks like we also need
> > to update Ursa configurations here:
> > https://github.com/ursa-labs/ursabot/blob/e958c5f95b31e98108df54cf13596c4fde944c3a/projects/arrow/docker/conda-cpp.txt#L19
>
> IMHO that implies a lot of potential issues to watch for, at the last
> minute before a release. Personally, I'd rather see this in 3.0.
>
> Regards
>
> Antoine.
Re: 2.0.0 release timeline: October 9
Posted by Antoine Pitrou <an...@python.org>.
Le 07/10/2020 à 21:19, James Duong a écrit :
> Hi Neal,
>
> Are you the release manager for 2.0?
> I've been working on the task to disable server verification in Flight
> clients, and it appears we'll need
> to update the minimum gRPC version to at least 1.27 to support this.
>
> Would it be OK to do this for the 2.0. release? It looks like we also need
> to update Ursa configurations here:
> https://github.com/ursa-labs/ursabot/blob/e958c5f95b31e98108df54cf13596c4fde944c3a/projects/arrow/docker/conda-cpp.txt#L19
IMHO that implies a lot of potential issues to watch for, at the last
minute before a release. Personally, I'd rather see this in 3.0.
Regards
Antoine.
Re: 2.0.0 release timeline: October 9
Posted by James Duong <ja...@bitquilltech.com>.
Hi Neal,
Are you the release manager for 2.0?
I've been working on the task to disable server verification in Flight
clients, and it appears we'll need
to update the minimum gRPC version to at least 1.27 to support this.
Would it be OK to do this for the 2.0. release? It looks like we also need
to update Ursa configurations here:
https://github.com/ursa-labs/ursabot/blob/e958c5f95b31e98108df54cf13596c4fde944c3a/projects/arrow/docker/conda-cpp.txt#L19
The relevant JIRA is:
https://issues.apache.org/jira/browse/ARROW-10105
The Java version of this improvement has already been merged to master.
On Mon, Oct 5, 2020 at 6:41 PM Neal Richardson <ne...@gmail.com>
wrote:
> Hi folks, checking back in here. Thanks to our collective effort, we're
> making good progress towards the 2.0 release, but there's still a bit more
> to do. Per
> https://cwiki.apache.org/confluence/display/ARROW/Arrow+2.0.0+Release,
> we're at 31 unstarted and another 31 in-progress issues left for 2.0.
> Blockers are down to 1, which I believe has a pull request up, but there
> are still a few nightly builds failing (JIRAs ARROW-10175 to -10178).
>
> We are planning to cut a release candidate on Friday, less than 4 days from
> now, so please be working to wrap up any issues you have in flight, and
> bump issues to 3.0 that you aren't going to be able to get to in time.
>
> Neal
>
> On Tue, Sep 29, 2020 at 5:46 PM Neal Richardson <
> neal.p.richardson@gmail.com>
> wrote:
>
> > Thanks Neville. I also just bumped 60 C++/Python issues (new features,
> > unassigned, not created recently) from 2.0. We're at 122 now, which still
> > needs some pruning. I think something that more folks can do is review
> the
> > issues assigned to them and make sure that they're planning to get all of
> > those done for 2.0, otherwise bump to 3.0 or unassign themselves (or
> both).
> >
> > Neal
> >
> > On Tue, Sep 29, 2020 at 5:01 PM Neville Dipale <ne...@gmail.com>
> > wrote:
> >
> >> Hi Neal,
> >>
> >> I've pruned the Rust backlog a bit, but only changed PRs that I've
> either
> >> opened,
> >> or those that I presume nobody is currently working on.
> >>
> >> There's 2 major features I've been working on:
> >>
> >> 1. Writing Arrow data to Parquet (separate branch)
> >> 2. Integration testing
> >>
> >> I'll prioritise 2 as that's on the main branch, and we're nearly there.
> >> However, with the arrow parquet writer, I'd like to ask the Rust
> >> developers
> >> if
> >> we will want to release the WIP support with 2.0.0, or if we hold back
> and
> >> keep
> >> chipping away from a separate branch.
> >>
> >> There's been a new contributor who's offered to help with the writer,
> >> so I think I'll be able to make more progress with her.
> >>
> >> Neville
> >>
> >> On Wed, 30 Sep 2020 at 00:34, Neal Richardson <
> >> neal.p.richardson@gmail.com>
> >> wrote:
> >>
> >> > Hi folks,
> >> > As has been discussed in the biweekly meetings (and in the notes from
> >> those
> >> > meetings here on the mailing list), we're looking at an October
> timeline
> >> > for our next release since we are going about 3 months between
> >> releases. So
> >> > that we might get the release voted on and shipped by the middle of
> the
> >> > month, we should aim to be ready to cut our first (and final!) release
> >> > candidate by next Friday, October 9.
> >> >
> >> > According to
> >> > https://cwiki.apache.org/confluence/display/ARROW/Arrow+2.0.0+Release
> ,
> >> > there are still 178 issues tagged for 2.0 that are not yet started.
> That
> >> > seems... ambitious. Please do go through the backlog and push to the
> >> next
> >> > release (i.e. 3.0.0) unassigned issues that aren't likely to land in
> the
> >> > next 10 days.
> >> >
> >> > Likewise, I see that there are a few issues tagged as "blocker". Let's
> >> > determine whether those truly should prevent a release candidate from
> >> being
> >> > made, and if so, let's make sure they get done ASAP.
> >> >
> >> > Neal
> >> >
> >>
> >
>
--
*James Duong*
Lead Software Developer
Bit Quill Technologies Inc.
Direct: +1.604.562.6082 | jamesd@bitquilltech.com
https://www.bitquilltech.com
This email message is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. Any unauthorized review,
use, disclosure, or distribution is prohibited. If you are not the
intended recipient, please contact the sender by reply email and destroy
all copies of the original message. Thank you.