You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Kouhei Sutou <ko...@clear-code.com> on 2018/07/19 10:12:52 UTC
Re: Need a release manager for Arrow 0.10.0
Hi,
I want to take packaging related tasks but I'm busy between
23th and 26th... I'll do at least the Ruby bindings related
tasks. Because 0.10.0 is the first release of the Ruby
bindings.
I hope that I can do other packaging related tasks.
I have a PyPI account: https://pypi.org/user/kou/
Do I need other accounts to release packages? conda-forge?
I have a GPG key: https://github.com/kou.gpg
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: Need a release manager for Arrow 0.10.0 [was Re: Packaging status update]" on Wed, 18 Jul 2018 15:13:46 -0400,
Wes McKinney <we...@gmail.com> wrote:
> If we could cut the release on the 24th (Tuesday) or 25th (Wednesday)
> that would be ideal. We're approaching T+4 months since 0.9.0 so I
> think we need to call pencils down once the last patches go in on
> Monday, barring any crazy blockers. At this rate I'm going to be
> working part of the weekend on this as we haven't managed to burn down
> the backlog to near zero yet (see
> https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.10.0+Release)
>
> In principle the packaging system should work fine against a tagged
> release because it uses setuptools_scm, so from Step 2 in
> https://github.com/apache/arrow/tree/master/dev/tasks#usage you would
> write something like
>
> git checkout apache-arrow-0.10.0
> python dev/tasks/crossbow.py submit ...
>
> We should probably add a script in
> https://github.com/apache/arrow/tree/master/dev/release that will
> fetch the artifacts, sign them, and upload to the dist system. The tag
> and source release artifact will have already been tagged, built,
> signed, and uploaded by dev/release/02-source.sh. We can make sure
> these details are all documented and reproducible in
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>
> Thanks Krisztian, Kou, Phillip for laboring on the packaging; I'm
> hopeful that this marks the beginning of a period of peace and
> prosperity for Arrow release managers so we can make releases roughly
> every month going forward.
>
> - Wes
>
> On Wed, Jul 18, 2018 at 2:51 PM, Krisztián Szűcs
> <sz...@gmail.com> wrote:
>> I won't be available between 27th and 31st.
>>
>> On Jul 18 2018, at 7:53 pm, Phillip Cloud <cp...@gmail.com> wrote:
>>>
>>> I can certainly help with the release vote. I'm out of town from the 20th
>>> to the 23rd, so it would be after that. Is there a particular target date
>>> we have in mind?
>>>
>>> On Wed, Jul 18, 2018 at 12:24 PM Wes McKinney <we...@gmail.com> wrote:
>>> > hi Li,
>>> > Thanks for volunteering -- we need a PMC to sign the source and binary
>>> > artifacts and upload to the dist system for the release. Realistically
>>> > we should probably break up the tasks related to the release (the
>>> > release vote, and follow up updates to PyPI, conda-forge, etc. ) --
>>> > there's quite a lot of work to do before, during, and after the vote.
>>> > @PhillipC, would you want to assist with the release vote process
>>> > since you've been engaged w/ Krisztian on the packaging work? Since
>>> > you guys work in the same physical location it might make coordination
>>> > easier
>>> >
>>> > - Wes
>>> > On Wed, Jul 18, 2018 at 12:00 PM, Li Jin <ic...@gmail.com> wrote:
>>> > > I am currently on vacation but will be back on 22th and I am not PMC. If
>>> > > these are not issues , I can volunteer to be RM for the release.
>>> > >
>>> > > Li
>>> > > On Wed, Jul 18, 2018 at 4:09 PM Wes McKinney <we...@gmail.com>
>>> >
>>> > wrote:
>>> > >
>>> > > > hi folks,
>>> > > > Any takers? It would be good for this knowledge to be spread to more
>>> > > > people.
>>> > > >
>>> > > > Thanks
>>> > > > Wes
>>> > > >
>>> > > > On Thu, Jul 12, 2018 at 1:11 PM, Wes McKinney <we...@gmail.com>
>>> > wrote:
>>> > > > > Can a member of the Arrow PMC volunteer to be the release manager for
>>> > > > > 0.10.0? There is some prep work to be done to make sure that you can
>>> > > > > produce the binary packages as part of the release vote; if this is
>>> > > > > done at the last minute it will probably delay the release.
>>> > > > >
>>> > > > > Thank you,
>>> > > > > Wes
>>> > > > >
>>> > > > > On Wed, Jul 11, 2018 at 12:10 PM, Wes McKinney <we...@gmail.com>
>>> > > > wrote:
>>> > > > > > Thanks Krisztian for the update and all your work on this.
>>> > > > > >
>>> > > > > > We need a PMC member to step up to be the release manager for 0.10.
>>> > > > > > Ideally they should undertake one or more dry runs of simulating the
>>> > > > > > 0.10 release so we aren't faced with any surprises when we go to cut
>>> > > > > > the release on ~July 23.
>>> > > > > >
>>> > > > > > I moved the release management guide to Confluence:
>>> > > > > >
>>> > > >
>>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>>> > > > > >
>>> > > > > > so we should ensure that there aren't any snowflake-y / undocumented
>>> > > > > > aspects to the release build workflow.
>>> > > > > >
>>> > > > > > - Wes
>>> > > > > > On Wed, Jul 11, 2018 at 11:54 AM, Krisztián Szűcs
>>> > > > > > <sz...@gmail.com> wrote:
>>> > > > > > > Hi All!
>>> > > > > > >
>>> > > > > > > Just want to send a quick summary about the packaging.
>>> > > > > > > What's working:
>>> > > > > > > python wheel builds
>>> > > > > > >
>>> > > > > > > conda package builds (the recent conda forge changes caused some
>>> > > > unexpected problems)
>>> > > > > > >
>>> > > > > > > linux package builds (enriched with ubuntu bionic)
>>> > > > > > > scheduled daily builds (triggered by travis)
>>> > > > > > > query build statuses from the CLI (see output
>>> > https://bit.ly/2ztnSqV (
>>> > > >
>>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/0?redirect=https%3A%2F%2Fbit.ly%2F2ztnSqV&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D
>>> > > > ))
>>> > > > > > >
>>> > > > > > > download and gpg sign the binary artifacts
>>> > > > > > > What's partially working:
>>> > > > > > > binary artifact uploading (most of them are uploaded though)
>>> > > > > > >
>>> > > > > > > We have package name collisions due to uploading multiple tasks'
>>> > > > artifacts to a single github release
>>> > > > > > > (see for example:
>>> > > > > >
>>> > > > >
>>> > > >
>>> > > > https://github.com/kszucs/crossbow/releases/tag/build-160 (
>>> > > >
>>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/1?redirect=https%3A%2F%2Fgithub.com%2Fkszucs%2Fcrossbow%2Freleases%2Ftag%2Fbuild-160&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D)
>>> > ).
>>> > > > We used to postfix
>>> > > > > > > these assets with the particular architecture, but we need a more
>>> > > > > >
>>> > > > >
>>> > > >
>>> > > > robust solution without renaming
>>> > > > > > > anything (either by creating tarballs or uploading to multiple
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>> > release
>>> > > > tags).
>>> > > > > > >
>>> > > > > > > All-in-all it's in quite good shape, I'll work on to refactor the
>>> > > > artifact uploading and finish the
>>> > > > > > > remaining jira tasks. IMHO we should start to test the whole release
>>> > > > > >
>>> > > > >
>>> > > >
>>> > > > procedure from next Monday
>>> > > > > > > and ship a couple of test releases before We reach July 20th.
>>> > > > > > >
>>> > > > > > > - Krisztian
Re: Need a release manager for Arrow 0.10.0
Posted by Kouhei Sutou <ko...@clear-code.com>.
Hi,
Thanks! I've confirmed that I'm a member of pyarrow on PyPI.
I've added my GPG key to both KEYS files.
--
kou
In <15...@webmail.messagingengine.com>
"Re: Need a release manager for Arrow 0.10.0" on Thu, 19 Jul 2018 13:03:06 +0200,
"Uwe L. Korn" <uw...@xhochy.com> wrote:
> Hi,
>
> I've added you to the PyPI admins. For the GPG key, please add it to the KEYS file in https://dist.apache.org/repos/dist/release/arrow/KEYS and https://dist.apache.org/repos/dist/dev/arrow/KEYS (they should be kept in sync). There is a small how-to at the top of the file. Just realised that I also need to do this.
>
> Uwe
>
> On Thu, Jul 19, 2018, at 12:12 PM, Kouhei Sutou wrote:
>> Hi,
>>
>> I want to take packaging related tasks but I'm busy between
>> 23th and 26th... I'll do at least the Ruby bindings related
>> tasks. Because 0.10.0 is the first release of the Ruby
>> bindings.
>>
>> I hope that I can do other packaging related tasks.
>>
>> I have a PyPI account: https://pypi.org/user/kou/
>> Do I need other accounts to release packages? conda-forge?
>>
>> I have a GPG key: https://github.com/kou.gpg
>>
>>
>> Thanks,
>> --
>> kou
>>
>> In <CA...@mail.gmail.com>
>> "Re: Need a release manager for Arrow 0.10.0 [was Re: Packaging status
>> update]" on Wed, 18 Jul 2018 15:13:46 -0400,
>> Wes McKinney <we...@gmail.com> wrote:
>>
>> > If we could cut the release on the 24th (Tuesday) or 25th (Wednesday)
>> > that would be ideal. We're approaching T+4 months since 0.9.0 so I
>> > think we need to call pencils down once the last patches go in on
>> > Monday, barring any crazy blockers. At this rate I'm going to be
>> > working part of the weekend on this as we haven't managed to burn down
>> > the backlog to near zero yet (see
>> > https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.10.0+Release)
>> >
>> > In principle the packaging system should work fine against a tagged
>> > release because it uses setuptools_scm, so from Step 2 in
>> > https://github.com/apache/arrow/tree/master/dev/tasks#usage you would
>> > write something like
>> >
>> > git checkout apache-arrow-0.10.0
>> > python dev/tasks/crossbow.py submit ...
>> >
>> > We should probably add a script in
>> > https://github.com/apache/arrow/tree/master/dev/release that will
>> > fetch the artifacts, sign them, and upload to the dist system. The tag
>> > and source release artifact will have already been tagged, built,
>> > signed, and uploaded by dev/release/02-source.sh. We can make sure
>> > these details are all documented and reproducible in
>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> >
>> > Thanks Krisztian, Kou, Phillip for laboring on the packaging; I'm
>> > hopeful that this marks the beginning of a period of peace and
>> > prosperity for Arrow release managers so we can make releases roughly
>> > every month going forward.
>> >
>> > - Wes
>> >
>> > On Wed, Jul 18, 2018 at 2:51 PM, Krisztián Szűcs
>> > <sz...@gmail.com> wrote:
>> >> I won't be available between 27th and 31st.
>> >>
>> >> On Jul 18 2018, at 7:53 pm, Phillip Cloud <cp...@gmail.com> wrote:
>> >>>
>> >>> I can certainly help with the release vote. I'm out of town from the 20th
>> >>> to the 23rd, so it would be after that. Is there a particular target date
>> >>> we have in mind?
>> >>>
>> >>> On Wed, Jul 18, 2018 at 12:24 PM Wes McKinney <we...@gmail.com> wrote:
>> >>> > hi Li,
>> >>> > Thanks for volunteering -- we need a PMC to sign the source and binary
>> >>> > artifacts and upload to the dist system for the release. Realistically
>> >>> > we should probably break up the tasks related to the release (the
>> >>> > release vote, and follow up updates to PyPI, conda-forge, etc. ) --
>> >>> > there's quite a lot of work to do before, during, and after the vote.
>> >>> > @PhillipC, would you want to assist with the release vote process
>> >>> > since you've been engaged w/ Krisztian on the packaging work? Since
>> >>> > you guys work in the same physical location it might make coordination
>> >>> > easier
>> >>> >
>> >>> > - Wes
>> >>> > On Wed, Jul 18, 2018 at 12:00 PM, Li Jin <ic...@gmail.com> wrote:
>> >>> > > I am currently on vacation but will be back on 22th and I am not PMC. If
>> >>> > > these are not issues , I can volunteer to be RM for the release.
>> >>> > >
>> >>> > > Li
>> >>> > > On Wed, Jul 18, 2018 at 4:09 PM Wes McKinney <we...@gmail.com>
>> >>> >
>> >>> > wrote:
>> >>> > >
>> >>> > > > hi folks,
>> >>> > > > Any takers? It would be good for this knowledge to be spread to more
>> >>> > > > people.
>> >>> > > >
>> >>> > > > Thanks
>> >>> > > > Wes
>> >>> > > >
>> >>> > > > On Thu, Jul 12, 2018 at 1:11 PM, Wes McKinney <we...@gmail.com>
>> >>> > wrote:
>> >>> > > > > Can a member of the Arrow PMC volunteer to be the release manager for
>> >>> > > > > 0.10.0? There is some prep work to be done to make sure that you can
>> >>> > > > > produce the binary packages as part of the release vote; if this is
>> >>> > > > > done at the last minute it will probably delay the release.
>> >>> > > > >
>> >>> > > > > Thank you,
>> >>> > > > > Wes
>> >>> > > > >
>> >>> > > > > On Wed, Jul 11, 2018 at 12:10 PM, Wes McKinney <we...@gmail.com>
>> >>> > > > wrote:
>> >>> > > > > > Thanks Krisztian for the update and all your work on this.
>> >>> > > > > >
>> >>> > > > > > We need a PMC member to step up to be the release manager for 0.10.
>> >>> > > > > > Ideally they should undertake one or more dry runs of simulating the
>> >>> > > > > > 0.10 release so we aren't faced with any surprises when we go to cut
>> >>> > > > > > the release on ~July 23.
>> >>> > > > > >
>> >>> > > > > > I moved the release management guide to Confluence:
>> >>> > > > > >
>> >>> > > >
>> >>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> >>> > > > > >
>> >>> > > > > > so we should ensure that there aren't any snowflake-y / undocumented
>> >>> > > > > > aspects to the release build workflow.
>> >>> > > > > >
>> >>> > > > > > - Wes
>> >>> > > > > > On Wed, Jul 11, 2018 at 11:54 AM, Krisztián Szűcs
>> >>> > > > > > <sz...@gmail.com> wrote:
>> >>> > > > > > > Hi All!
>> >>> > > > > > >
>> >>> > > > > > > Just want to send a quick summary about the packaging.
>> >>> > > > > > > What's working:
>> >>> > > > > > > python wheel builds
>> >>> > > > > > >
>> >>> > > > > > > conda package builds (the recent conda forge changes caused some
>> >>> > > > unexpected problems)
>> >>> > > > > > >
>> >>> > > > > > > linux package builds (enriched with ubuntu bionic)
>> >>> > > > > > > scheduled daily builds (triggered by travis)
>> >>> > > > > > > query build statuses from the CLI (see output
>> >>> > https://bit.ly/2ztnSqV (
>> >>> > > >
>> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/0?redirect=https%3A%2F%2Fbit.ly%2F2ztnSqV&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D
>> >>> > > > ))
>> >>> > > > > > >
>> >>> > > > > > > download and gpg sign the binary artifacts
>> >>> > > > > > > What's partially working:
>> >>> > > > > > > binary artifact uploading (most of them are uploaded though)
>> >>> > > > > > >
>> >>> > > > > > > We have package name collisions due to uploading multiple tasks'
>> >>> > > > artifacts to a single github release
>> >>> > > > > > > (see for example:
>> >>> > > > > >
>> >>> > > > >
>> >>> > > >
>> >>> > > > https://github.com/kszucs/crossbow/releases/tag/build-160 (
>> >>> > > >
>> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/1?redirect=https%3A%2F%2Fgithub.com%2Fkszucs%2Fcrossbow%2Freleases%2Ftag%2Fbuild-160&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D)
>> >>> > ).
>> >>> > > > We used to postfix
>> >>> > > > > > > these assets with the particular architecture, but we need a more
>> >>> > > > > >
>> >>> > > > >
>> >>> > > >
>> >>> > > > robust solution without renaming
>> >>> > > > > > > anything (either by creating tarballs or uploading to multiple
>> >>> > > > > >
>> >>> > > > >
>> >>> > > >
>> >>> > >
>> >>> >
>> >>> > release
>> >>> > > > tags).
>> >>> > > > > > >
>> >>> > > > > > > All-in-all it's in quite good shape, I'll work on to refactor the
>> >>> > > > artifact uploading and finish the
>> >>> > > > > > > remaining jira tasks. IMHO we should start to test the whole release
>> >>> > > > > >
>> >>> > > > >
>> >>> > > >
>> >>> > > > procedure from next Monday
>> >>> > > > > > > and ship a couple of test releases before We reach July 20th.
>> >>> > > > > > >
>> >>> > > > > > > - Krisztian
Re: Need a release manager for Arrow 0.10.0
Posted by Kouhei Sutou <ko...@clear-code.com>.
Hi,
Thanks! I've approved all of them!
--
kou
In <15...@webmail.messagingengine.com>
"Re: Need a release manager for Arrow 0.10.0" on Fri, 20 Jul 2018 06:55:30 +0200,
"Uwe L. Korn" <uw...@xhochy.com> wrote:
> Hello,
>
> I opened also pull requests on conda-forge so that you are listed as a maintainer. Then you can also merge these PRs once the build has passed. CI runtimes for conda-forge vary greatly, so it is better to have enough more people that are able to merge.
>
> Uwe
>
> On Fri, Jul 20, 2018, at 2:58 AM, Kouhei Sutou wrote:
>> Hi,
>>
>> > @kou, the conda-forge repositories in question are:
>> >
>> > https://github.com/conda-forge/arrow-cpp-feedstock
>> > https://github.com/conda-forge/parquet-cpp-feedstock
>> > https://github.com/conda-forge/pyarrow-feedstock
>> >
>> > Updating them is a bit laborious because we have arrow-cpp ->
>> > parquet-cpp -> pyarrow dependency. So the updates must be completed
>> > serially
>>
>> Thanks! I've missed "Updating conda packages" section in
>> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide .
>> Sorry.
>>
>> I'll be able to send pull requests to them.
>>
>> > We should maybe post the voted-on release binaries at
>> > anaconda.org/apache eventually but that doesn't need to happen this
>> > release cycle
>>
>> OK. I'll do it after we have the released 0.10.0 source
>> archive.
>>
>> --
>> kou
>>
>> In <CA...@mail.gmail.com>
>> "Re: Need a release manager for Arrow 0.10.0" on Thu, 19 Jul 2018
>> 09:50:53 -0400,
>> Wes McKinney <we...@gmail.com> wrote:
>>
>> > Thanks Kou!
>> >
>> > @Krisztian, where do we stand with the conda recipes in apache/arrow
>> > vs. the ones in conda-forge? Will the updates there still be manual?
>> >
>> > @kou, the conda-forge repositories in question are:
>> >
>> > https://github.com/conda-forge/arrow-cpp-feedstock
>> > https://github.com/conda-forge/parquet-cpp-feedstock
>> > https://github.com/conda-forge/pyarrow-feedstock
>> >
>> > Updating them is a bit laborious because we have arrow-cpp ->
>> > parquet-cpp -> pyarrow dependency. So the updates must be completed
>> > serially
>> >
>> > We should maybe post the voted-on release binaries at
>> > anaconda.org/apache eventually but that doesn't need to happen this
>> > release cycle
>> >
>> > - Wes
>> >
>> > On Thu, Jul 19, 2018 at 7:03 AM, Uwe L. Korn <uw...@xhochy.com> wrote:
>> >> Hi,
>> >>
>> >> I've added you to the PyPI admins. For the GPG key, please add it to the KEYS file in https://dist.apache.org/repos/dist/release/arrow/KEYS and https://dist.apache.org/repos/dist/dev/arrow/KEYS (they should be kept in sync). There is a small how-to at the top of the file. Just realised that I also need to do this.
>> >>
>> >> Uwe
>> >>
>> >> On Thu, Jul 19, 2018, at 12:12 PM, Kouhei Sutou wrote:
>> >>> Hi,
>> >>>
>> >>> I want to take packaging related tasks but I'm busy between
>> >>> 23th and 26th... I'll do at least the Ruby bindings related
>> >>> tasks. Because 0.10.0 is the first release of the Ruby
>> >>> bindings.
>> >>>
>> >>> I hope that I can do other packaging related tasks.
>> >>>
>> >>> I have a PyPI account: https://pypi.org/user/kou/
>> >>> Do I need other accounts to release packages? conda-forge?
>> >>>
>> >>> I have a GPG key: https://github.com/kou.gpg
>> >>>
>> >>>
>> >>> Thanks,
>> >>> --
>> >>> kou
>> >>>
>> >>> In <CA...@mail.gmail.com>
>> >>> "Re: Need a release manager for Arrow 0.10.0 [was Re: Packaging status
>> >>> update]" on Wed, 18 Jul 2018 15:13:46 -0400,
>> >>> Wes McKinney <we...@gmail.com> wrote:
>> >>>
>> >>> > If we could cut the release on the 24th (Tuesday) or 25th (Wednesday)
>> >>> > that would be ideal. We're approaching T+4 months since 0.9.0 so I
>> >>> > think we need to call pencils down once the last patches go in on
>> >>> > Monday, barring any crazy blockers. At this rate I'm going to be
>> >>> > working part of the weekend on this as we haven't managed to burn down
>> >>> > the backlog to near zero yet (see
>> >>> > https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.10.0+Release)
>> >>> >
>> >>> > In principle the packaging system should work fine against a tagged
>> >>> > release because it uses setuptools_scm, so from Step 2 in
>> >>> > https://github.com/apache/arrow/tree/master/dev/tasks#usage you would
>> >>> > write something like
>> >>> >
>> >>> > git checkout apache-arrow-0.10.0
>> >>> > python dev/tasks/crossbow.py submit ...
>> >>> >
>> >>> > We should probably add a script in
>> >>> > https://github.com/apache/arrow/tree/master/dev/release that will
>> >>> > fetch the artifacts, sign them, and upload to the dist system. The tag
>> >>> > and source release artifact will have already been tagged, built,
>> >>> > signed, and uploaded by dev/release/02-source.sh. We can make sure
>> >>> > these details are all documented and reproducible in
>> >>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> >>> >
>> >>> > Thanks Krisztian, Kou, Phillip for laboring on the packaging; I'm
>> >>> > hopeful that this marks the beginning of a period of peace and
>> >>> > prosperity for Arrow release managers so we can make releases roughly
>> >>> > every month going forward.
>> >>> >
>> >>> > - Wes
>> >>> >
>> >>> > On Wed, Jul 18, 2018 at 2:51 PM, Krisztián Szűcs
>> >>> > <sz...@gmail.com> wrote:
>> >>> >> I won't be available between 27th and 31st.
>> >>> >>
>> >>> >> On Jul 18 2018, at 7:53 pm, Phillip Cloud <cp...@gmail.com> wrote:
>> >>> >>>
>> >>> >>> I can certainly help with the release vote. I'm out of town from the 20th
>> >>> >>> to the 23rd, so it would be after that. Is there a particular target date
>> >>> >>> we have in mind?
>> >>> >>>
>> >>> >>> On Wed, Jul 18, 2018 at 12:24 PM Wes McKinney <we...@gmail.com> wrote:
>> >>> >>> > hi Li,
>> >>> >>> > Thanks for volunteering -- we need a PMC to sign the source and binary
>> >>> >>> > artifacts and upload to the dist system for the release. Realistically
>> >>> >>> > we should probably break up the tasks related to the release (the
>> >>> >>> > release vote, and follow up updates to PyPI, conda-forge, etc. ) --
>> >>> >>> > there's quite a lot of work to do before, during, and after the vote.
>> >>> >>> > @PhillipC, would you want to assist with the release vote process
>> >>> >>> > since you've been engaged w/ Krisztian on the packaging work? Since
>> >>> >>> > you guys work in the same physical location it might make coordination
>> >>> >>> > easier
>> >>> >>> >
>> >>> >>> > - Wes
>> >>> >>> > On Wed, Jul 18, 2018 at 12:00 PM, Li Jin <ic...@gmail.com> wrote:
>> >>> >>> > > I am currently on vacation but will be back on 22th and I am not PMC. If
>> >>> >>> > > these are not issues , I can volunteer to be RM for the release.
>> >>> >>> > >
>> >>> >>> > > Li
>> >>> >>> > > On Wed, Jul 18, 2018 at 4:09 PM Wes McKinney <we...@gmail.com>
>> >>> >>> >
>> >>> >>> > wrote:
>> >>> >>> > >
>> >>> >>> > > > hi folks,
>> >>> >>> > > > Any takers? It would be good for this knowledge to be spread to more
>> >>> >>> > > > people.
>> >>> >>> > > >
>> >>> >>> > > > Thanks
>> >>> >>> > > > Wes
>> >>> >>> > > >
>> >>> >>> > > > On Thu, Jul 12, 2018 at 1:11 PM, Wes McKinney <we...@gmail.com>
>> >>> >>> > wrote:
>> >>> >>> > > > > Can a member of the Arrow PMC volunteer to be the release manager for
>> >>> >>> > > > > 0.10.0? There is some prep work to be done to make sure that you can
>> >>> >>> > > > > produce the binary packages as part of the release vote; if this is
>> >>> >>> > > > > done at the last minute it will probably delay the release.
>> >>> >>> > > > >
>> >>> >>> > > > > Thank you,
>> >>> >>> > > > > Wes
>> >>> >>> > > > >
>> >>> >>> > > > > On Wed, Jul 11, 2018 at 12:10 PM, Wes McKinney <we...@gmail.com>
>> >>> >>> > > > wrote:
>> >>> >>> > > > > > Thanks Krisztian for the update and all your work on this.
>> >>> >>> > > > > >
>> >>> >>> > > > > > We need a PMC member to step up to be the release manager for 0.10.
>> >>> >>> > > > > > Ideally they should undertake one or more dry runs of simulating the
>> >>> >>> > > > > > 0.10 release so we aren't faced with any surprises when we go to cut
>> >>> >>> > > > > > the release on ~July 23.
>> >>> >>> > > > > >
>> >>> >>> > > > > > I moved the release management guide to Confluence:
>> >>> >>> > > > > >
>> >>> >>> > > >
>> >>> >>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> >>> >>> > > > > >
>> >>> >>> > > > > > so we should ensure that there aren't any snowflake-y / undocumented
>> >>> >>> > > > > > aspects to the release build workflow.
>> >>> >>> > > > > >
>> >>> >>> > > > > > - Wes
>> >>> >>> > > > > > On Wed, Jul 11, 2018 at 11:54 AM, Krisztián Szűcs
>> >>> >>> > > > > > <sz...@gmail.com> wrote:
>> >>> >>> > > > > > > Hi All!
>> >>> >>> > > > > > >
>> >>> >>> > > > > > > Just want to send a quick summary about the packaging.
>> >>> >>> > > > > > > What's working:
>> >>> >>> > > > > > > python wheel builds
>> >>> >>> > > > > > >
>> >>> >>> > > > > > > conda package builds (the recent conda forge changes caused some
>> >>> >>> > > > unexpected problems)
>> >>> >>> > > > > > >
>> >>> >>> > > > > > > linux package builds (enriched with ubuntu bionic)
>> >>> >>> > > > > > > scheduled daily builds (triggered by travis)
>> >>> >>> > > > > > > query build statuses from the CLI (see output
>> >>> >>> > https://bit.ly/2ztnSqV (
>> >>> >>> > > >
>> >>> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/0?redirect=https%3A%2F%2Fbit.ly%2F2ztnSqV&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D
>> >>> >>> > > > ))
>> >>> >>> > > > > > >
>> >>> >>> > > > > > > download and gpg sign the binary artifacts
>> >>> >>> > > > > > > What's partially working:
>> >>> >>> > > > > > > binary artifact uploading (most of them are uploaded though)
>> >>> >>> > > > > > >
>> >>> >>> > > > > > > We have package name collisions due to uploading multiple tasks'
>> >>> >>> > > > artifacts to a single github release
>> >>> >>> > > > > > > (see for example:
>> >>> >>> > > > > >
>> >>> >>> > > > >
>> >>> >>> > > >
>> >>> >>> > > > https://github.com/kszucs/crossbow/releases/tag/build-160 (
>> >>> >>> > > >
>> >>> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/1?redirect=https%3A%2F%2Fgithub.com%2Fkszucs%2Fcrossbow%2Freleases%2Ftag%2Fbuild-160&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D)
>> >>> >>> > ).
>> >>> >>> > > > We used to postfix
>> >>> >>> > > > > > > these assets with the particular architecture, but we need a more
>> >>> >>> > > > > >
>> >>> >>> > > > >
>> >>> >>> > > >
>> >>> >>> > > > robust solution without renaming
>> >>> >>> > > > > > > anything (either by creating tarballs or uploading to multiple
>> >>> >>> > > > > >
>> >>> >>> > > > >
>> >>> >>> > > >
>> >>> >>> > >
>> >>> >>> >
>> >>> >>> > release
>> >>> >>> > > > tags).
>> >>> >>> > > > > > >
>> >>> >>> > > > > > > All-in-all it's in quite good shape, I'll work on to refactor the
>> >>> >>> > > > artifact uploading and finish the
>> >>> >>> > > > > > > remaining jira tasks. IMHO we should start to test the whole release
>> >>> >>> > > > > >
>> >>> >>> > > > >
>> >>> >>> > > >
>> >>> >>> > > > procedure from next Monday
>> >>> >>> > > > > > > and ship a couple of test releases before We reach July 20th.
>> >>> >>> > > > > > >
>> >>> >>> > > > > > > - Krisztian
Re: Need a release manager for Arrow 0.10.0
Posted by "Uwe L. Korn" <uw...@xhochy.com>.
Hello,
I opened also pull requests on conda-forge so that you are listed as a maintainer. Then you can also merge these PRs once the build has passed. CI runtimes for conda-forge vary greatly, so it is better to have enough more people that are able to merge.
Uwe
On Fri, Jul 20, 2018, at 2:58 AM, Kouhei Sutou wrote:
> Hi,
>
> > @kou, the conda-forge repositories in question are:
> >
> > https://github.com/conda-forge/arrow-cpp-feedstock
> > https://github.com/conda-forge/parquet-cpp-feedstock
> > https://github.com/conda-forge/pyarrow-feedstock
> >
> > Updating them is a bit laborious because we have arrow-cpp ->
> > parquet-cpp -> pyarrow dependency. So the updates must be completed
> > serially
>
> Thanks! I've missed "Updating conda packages" section in
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide .
> Sorry.
>
> I'll be able to send pull requests to them.
>
> > We should maybe post the voted-on release binaries at
> > anaconda.org/apache eventually but that doesn't need to happen this
> > release cycle
>
> OK. I'll do it after we have the released 0.10.0 source
> archive.
>
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Re: Need a release manager for Arrow 0.10.0" on Thu, 19 Jul 2018
> 09:50:53 -0400,
> Wes McKinney <we...@gmail.com> wrote:
>
> > Thanks Kou!
> >
> > @Krisztian, where do we stand with the conda recipes in apache/arrow
> > vs. the ones in conda-forge? Will the updates there still be manual?
> >
> > @kou, the conda-forge repositories in question are:
> >
> > https://github.com/conda-forge/arrow-cpp-feedstock
> > https://github.com/conda-forge/parquet-cpp-feedstock
> > https://github.com/conda-forge/pyarrow-feedstock
> >
> > Updating them is a bit laborious because we have arrow-cpp ->
> > parquet-cpp -> pyarrow dependency. So the updates must be completed
> > serially
> >
> > We should maybe post the voted-on release binaries at
> > anaconda.org/apache eventually but that doesn't need to happen this
> > release cycle
> >
> > - Wes
> >
> > On Thu, Jul 19, 2018 at 7:03 AM, Uwe L. Korn <uw...@xhochy.com> wrote:
> >> Hi,
> >>
> >> I've added you to the PyPI admins. For the GPG key, please add it to the KEYS file in https://dist.apache.org/repos/dist/release/arrow/KEYS and https://dist.apache.org/repos/dist/dev/arrow/KEYS (they should be kept in sync). There is a small how-to at the top of the file. Just realised that I also need to do this.
> >>
> >> Uwe
> >>
> >> On Thu, Jul 19, 2018, at 12:12 PM, Kouhei Sutou wrote:
> >>> Hi,
> >>>
> >>> I want to take packaging related tasks but I'm busy between
> >>> 23th and 26th... I'll do at least the Ruby bindings related
> >>> tasks. Because 0.10.0 is the first release of the Ruby
> >>> bindings.
> >>>
> >>> I hope that I can do other packaging related tasks.
> >>>
> >>> I have a PyPI account: https://pypi.org/user/kou/
> >>> Do I need other accounts to release packages? conda-forge?
> >>>
> >>> I have a GPG key: https://github.com/kou.gpg
> >>>
> >>>
> >>> Thanks,
> >>> --
> >>> kou
> >>>
> >>> In <CA...@mail.gmail.com>
> >>> "Re: Need a release manager for Arrow 0.10.0 [was Re: Packaging status
> >>> update]" on Wed, 18 Jul 2018 15:13:46 -0400,
> >>> Wes McKinney <we...@gmail.com> wrote:
> >>>
> >>> > If we could cut the release on the 24th (Tuesday) or 25th (Wednesday)
> >>> > that would be ideal. We're approaching T+4 months since 0.9.0 so I
> >>> > think we need to call pencils down once the last patches go in on
> >>> > Monday, barring any crazy blockers. At this rate I'm going to be
> >>> > working part of the weekend on this as we haven't managed to burn down
> >>> > the backlog to near zero yet (see
> >>> > https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.10.0+Release)
> >>> >
> >>> > In principle the packaging system should work fine against a tagged
> >>> > release because it uses setuptools_scm, so from Step 2 in
> >>> > https://github.com/apache/arrow/tree/master/dev/tasks#usage you would
> >>> > write something like
> >>> >
> >>> > git checkout apache-arrow-0.10.0
> >>> > python dev/tasks/crossbow.py submit ...
> >>> >
> >>> > We should probably add a script in
> >>> > https://github.com/apache/arrow/tree/master/dev/release that will
> >>> > fetch the artifacts, sign them, and upload to the dist system. The tag
> >>> > and source release artifact will have already been tagged, built,
> >>> > signed, and uploaded by dev/release/02-source.sh. We can make sure
> >>> > these details are all documented and reproducible in
> >>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> >>> >
> >>> > Thanks Krisztian, Kou, Phillip for laboring on the packaging; I'm
> >>> > hopeful that this marks the beginning of a period of peace and
> >>> > prosperity for Arrow release managers so we can make releases roughly
> >>> > every month going forward.
> >>> >
> >>> > - Wes
> >>> >
> >>> > On Wed, Jul 18, 2018 at 2:51 PM, Krisztián Szűcs
> >>> > <sz...@gmail.com> wrote:
> >>> >> I won't be available between 27th and 31st.
> >>> >>
> >>> >> On Jul 18 2018, at 7:53 pm, Phillip Cloud <cp...@gmail.com> wrote:
> >>> >>>
> >>> >>> I can certainly help with the release vote. I'm out of town from the 20th
> >>> >>> to the 23rd, so it would be after that. Is there a particular target date
> >>> >>> we have in mind?
> >>> >>>
> >>> >>> On Wed, Jul 18, 2018 at 12:24 PM Wes McKinney <we...@gmail.com> wrote:
> >>> >>> > hi Li,
> >>> >>> > Thanks for volunteering -- we need a PMC to sign the source and binary
> >>> >>> > artifacts and upload to the dist system for the release. Realistically
> >>> >>> > we should probably break up the tasks related to the release (the
> >>> >>> > release vote, and follow up updates to PyPI, conda-forge, etc. ) --
> >>> >>> > there's quite a lot of work to do before, during, and after the vote.
> >>> >>> > @PhillipC, would you want to assist with the release vote process
> >>> >>> > since you've been engaged w/ Krisztian on the packaging work? Since
> >>> >>> > you guys work in the same physical location it might make coordination
> >>> >>> > easier
> >>> >>> >
> >>> >>> > - Wes
> >>> >>> > On Wed, Jul 18, 2018 at 12:00 PM, Li Jin <ic...@gmail.com> wrote:
> >>> >>> > > I am currently on vacation but will be back on 22th and I am not PMC. If
> >>> >>> > > these are not issues , I can volunteer to be RM for the release.
> >>> >>> > >
> >>> >>> > > Li
> >>> >>> > > On Wed, Jul 18, 2018 at 4:09 PM Wes McKinney <we...@gmail.com>
> >>> >>> >
> >>> >>> > wrote:
> >>> >>> > >
> >>> >>> > > > hi folks,
> >>> >>> > > > Any takers? It would be good for this knowledge to be spread to more
> >>> >>> > > > people.
> >>> >>> > > >
> >>> >>> > > > Thanks
> >>> >>> > > > Wes
> >>> >>> > > >
> >>> >>> > > > On Thu, Jul 12, 2018 at 1:11 PM, Wes McKinney <we...@gmail.com>
> >>> >>> > wrote:
> >>> >>> > > > > Can a member of the Arrow PMC volunteer to be the release manager for
> >>> >>> > > > > 0.10.0? There is some prep work to be done to make sure that you can
> >>> >>> > > > > produce the binary packages as part of the release vote; if this is
> >>> >>> > > > > done at the last minute it will probably delay the release.
> >>> >>> > > > >
> >>> >>> > > > > Thank you,
> >>> >>> > > > > Wes
> >>> >>> > > > >
> >>> >>> > > > > On Wed, Jul 11, 2018 at 12:10 PM, Wes McKinney <we...@gmail.com>
> >>> >>> > > > wrote:
> >>> >>> > > > > > Thanks Krisztian for the update and all your work on this.
> >>> >>> > > > > >
> >>> >>> > > > > > We need a PMC member to step up to be the release manager for 0.10.
> >>> >>> > > > > > Ideally they should undertake one or more dry runs of simulating the
> >>> >>> > > > > > 0.10 release so we aren't faced with any surprises when we go to cut
> >>> >>> > > > > > the release on ~July 23.
> >>> >>> > > > > >
> >>> >>> > > > > > I moved the release management guide to Confluence:
> >>> >>> > > > > >
> >>> >>> > > >
> >>> >>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> >>> >>> > > > > >
> >>> >>> > > > > > so we should ensure that there aren't any snowflake-y / undocumented
> >>> >>> > > > > > aspects to the release build workflow.
> >>> >>> > > > > >
> >>> >>> > > > > > - Wes
> >>> >>> > > > > > On Wed, Jul 11, 2018 at 11:54 AM, Krisztián Szűcs
> >>> >>> > > > > > <sz...@gmail.com> wrote:
> >>> >>> > > > > > > Hi All!
> >>> >>> > > > > > >
> >>> >>> > > > > > > Just want to send a quick summary about the packaging.
> >>> >>> > > > > > > What's working:
> >>> >>> > > > > > > python wheel builds
> >>> >>> > > > > > >
> >>> >>> > > > > > > conda package builds (the recent conda forge changes caused some
> >>> >>> > > > unexpected problems)
> >>> >>> > > > > > >
> >>> >>> > > > > > > linux package builds (enriched with ubuntu bionic)
> >>> >>> > > > > > > scheduled daily builds (triggered by travis)
> >>> >>> > > > > > > query build statuses from the CLI (see output
> >>> >>> > https://bit.ly/2ztnSqV (
> >>> >>> > > >
> >>> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/0?redirect=https%3A%2F%2Fbit.ly%2F2ztnSqV&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D
> >>> >>> > > > ))
> >>> >>> > > > > > >
> >>> >>> > > > > > > download and gpg sign the binary artifacts
> >>> >>> > > > > > > What's partially working:
> >>> >>> > > > > > > binary artifact uploading (most of them are uploaded though)
> >>> >>> > > > > > >
> >>> >>> > > > > > > We have package name collisions due to uploading multiple tasks'
> >>> >>> > > > artifacts to a single github release
> >>> >>> > > > > > > (see for example:
> >>> >>> > > > > >
> >>> >>> > > > >
> >>> >>> > > >
> >>> >>> > > > https://github.com/kszucs/crossbow/releases/tag/build-160 (
> >>> >>> > > >
> >>> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/1?redirect=https%3A%2F%2Fgithub.com%2Fkszucs%2Fcrossbow%2Freleases%2Ftag%2Fbuild-160&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D)
> >>> >>> > ).
> >>> >>> > > > We used to postfix
> >>> >>> > > > > > > these assets with the particular architecture, but we need a more
> >>> >>> > > > > >
> >>> >>> > > > >
> >>> >>> > > >
> >>> >>> > > > robust solution without renaming
> >>> >>> > > > > > > anything (either by creating tarballs or uploading to multiple
> >>> >>> > > > > >
> >>> >>> > > > >
> >>> >>> > > >
> >>> >>> > >
> >>> >>> >
> >>> >>> > release
> >>> >>> > > > tags).
> >>> >>> > > > > > >
> >>> >>> > > > > > > All-in-all it's in quite good shape, I'll work on to refactor the
> >>> >>> > > > artifact uploading and finish the
> >>> >>> > > > > > > remaining jira tasks. IMHO we should start to test the whole release
> >>> >>> > > > > >
> >>> >>> > > > >
> >>> >>> > > >
> >>> >>> > > > procedure from next Monday
> >>> >>> > > > > > > and ship a couple of test releases before We reach July 20th.
> >>> >>> > > > > > >
> >>> >>> > > > > > > - Krisztian
Re: Need a release manager for Arrow 0.10.0
Posted by Kouhei Sutou <ko...@clear-code.com>.
Hi,
> @kou, the conda-forge repositories in question are:
>
> https://github.com/conda-forge/arrow-cpp-feedstock
> https://github.com/conda-forge/parquet-cpp-feedstock
> https://github.com/conda-forge/pyarrow-feedstock
>
> Updating them is a bit laborious because we have arrow-cpp ->
> parquet-cpp -> pyarrow dependency. So the updates must be completed
> serially
Thanks! I've missed "Updating conda packages" section in
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide .
Sorry.
I'll be able to send pull requests to them.
> We should maybe post the voted-on release binaries at
> anaconda.org/apache eventually but that doesn't need to happen this
> release cycle
OK. I'll do it after we have the released 0.10.0 source
archive.
--
kou
In <CA...@mail.gmail.com>
"Re: Need a release manager for Arrow 0.10.0" on Thu, 19 Jul 2018 09:50:53 -0400,
Wes McKinney <we...@gmail.com> wrote:
> Thanks Kou!
>
> @Krisztian, where do we stand with the conda recipes in apache/arrow
> vs. the ones in conda-forge? Will the updates there still be manual?
>
> @kou, the conda-forge repositories in question are:
>
> https://github.com/conda-forge/arrow-cpp-feedstock
> https://github.com/conda-forge/parquet-cpp-feedstock
> https://github.com/conda-forge/pyarrow-feedstock
>
> Updating them is a bit laborious because we have arrow-cpp ->
> parquet-cpp -> pyarrow dependency. So the updates must be completed
> serially
>
> We should maybe post the voted-on release binaries at
> anaconda.org/apache eventually but that doesn't need to happen this
> release cycle
>
> - Wes
>
> On Thu, Jul 19, 2018 at 7:03 AM, Uwe L. Korn <uw...@xhochy.com> wrote:
>> Hi,
>>
>> I've added you to the PyPI admins. For the GPG key, please add it to the KEYS file in https://dist.apache.org/repos/dist/release/arrow/KEYS and https://dist.apache.org/repos/dist/dev/arrow/KEYS (they should be kept in sync). There is a small how-to at the top of the file. Just realised that I also need to do this.
>>
>> Uwe
>>
>> On Thu, Jul 19, 2018, at 12:12 PM, Kouhei Sutou wrote:
>>> Hi,
>>>
>>> I want to take packaging related tasks but I'm busy between
>>> 23th and 26th... I'll do at least the Ruby bindings related
>>> tasks. Because 0.10.0 is the first release of the Ruby
>>> bindings.
>>>
>>> I hope that I can do other packaging related tasks.
>>>
>>> I have a PyPI account: https://pypi.org/user/kou/
>>> Do I need other accounts to release packages? conda-forge?
>>>
>>> I have a GPG key: https://github.com/kou.gpg
>>>
>>>
>>> Thanks,
>>> --
>>> kou
>>>
>>> In <CA...@mail.gmail.com>
>>> "Re: Need a release manager for Arrow 0.10.0 [was Re: Packaging status
>>> update]" on Wed, 18 Jul 2018 15:13:46 -0400,
>>> Wes McKinney <we...@gmail.com> wrote:
>>>
>>> > If we could cut the release on the 24th (Tuesday) or 25th (Wednesday)
>>> > that would be ideal. We're approaching T+4 months since 0.9.0 so I
>>> > think we need to call pencils down once the last patches go in on
>>> > Monday, barring any crazy blockers. At this rate I'm going to be
>>> > working part of the weekend on this as we haven't managed to burn down
>>> > the backlog to near zero yet (see
>>> > https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.10.0+Release)
>>> >
>>> > In principle the packaging system should work fine against a tagged
>>> > release because it uses setuptools_scm, so from Step 2 in
>>> > https://github.com/apache/arrow/tree/master/dev/tasks#usage you would
>>> > write something like
>>> >
>>> > git checkout apache-arrow-0.10.0
>>> > python dev/tasks/crossbow.py submit ...
>>> >
>>> > We should probably add a script in
>>> > https://github.com/apache/arrow/tree/master/dev/release that will
>>> > fetch the artifacts, sign them, and upload to the dist system. The tag
>>> > and source release artifact will have already been tagged, built,
>>> > signed, and uploaded by dev/release/02-source.sh. We can make sure
>>> > these details are all documented and reproducible in
>>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>>> >
>>> > Thanks Krisztian, Kou, Phillip for laboring on the packaging; I'm
>>> > hopeful that this marks the beginning of a period of peace and
>>> > prosperity for Arrow release managers so we can make releases roughly
>>> > every month going forward.
>>> >
>>> > - Wes
>>> >
>>> > On Wed, Jul 18, 2018 at 2:51 PM, Krisztián Szűcs
>>> > <sz...@gmail.com> wrote:
>>> >> I won't be available between 27th and 31st.
>>> >>
>>> >> On Jul 18 2018, at 7:53 pm, Phillip Cloud <cp...@gmail.com> wrote:
>>> >>>
>>> >>> I can certainly help with the release vote. I'm out of town from the 20th
>>> >>> to the 23rd, so it would be after that. Is there a particular target date
>>> >>> we have in mind?
>>> >>>
>>> >>> On Wed, Jul 18, 2018 at 12:24 PM Wes McKinney <we...@gmail.com> wrote:
>>> >>> > hi Li,
>>> >>> > Thanks for volunteering -- we need a PMC to sign the source and binary
>>> >>> > artifacts and upload to the dist system for the release. Realistically
>>> >>> > we should probably break up the tasks related to the release (the
>>> >>> > release vote, and follow up updates to PyPI, conda-forge, etc. ) --
>>> >>> > there's quite a lot of work to do before, during, and after the vote.
>>> >>> > @PhillipC, would you want to assist with the release vote process
>>> >>> > since you've been engaged w/ Krisztian on the packaging work? Since
>>> >>> > you guys work in the same physical location it might make coordination
>>> >>> > easier
>>> >>> >
>>> >>> > - Wes
>>> >>> > On Wed, Jul 18, 2018 at 12:00 PM, Li Jin <ic...@gmail.com> wrote:
>>> >>> > > I am currently on vacation but will be back on 22th and I am not PMC. If
>>> >>> > > these are not issues , I can volunteer to be RM for the release.
>>> >>> > >
>>> >>> > > Li
>>> >>> > > On Wed, Jul 18, 2018 at 4:09 PM Wes McKinney <we...@gmail.com>
>>> >>> >
>>> >>> > wrote:
>>> >>> > >
>>> >>> > > > hi folks,
>>> >>> > > > Any takers? It would be good for this knowledge to be spread to more
>>> >>> > > > people.
>>> >>> > > >
>>> >>> > > > Thanks
>>> >>> > > > Wes
>>> >>> > > >
>>> >>> > > > On Thu, Jul 12, 2018 at 1:11 PM, Wes McKinney <we...@gmail.com>
>>> >>> > wrote:
>>> >>> > > > > Can a member of the Arrow PMC volunteer to be the release manager for
>>> >>> > > > > 0.10.0? There is some prep work to be done to make sure that you can
>>> >>> > > > > produce the binary packages as part of the release vote; if this is
>>> >>> > > > > done at the last minute it will probably delay the release.
>>> >>> > > > >
>>> >>> > > > > Thank you,
>>> >>> > > > > Wes
>>> >>> > > > >
>>> >>> > > > > On Wed, Jul 11, 2018 at 12:10 PM, Wes McKinney <we...@gmail.com>
>>> >>> > > > wrote:
>>> >>> > > > > > Thanks Krisztian for the update and all your work on this.
>>> >>> > > > > >
>>> >>> > > > > > We need a PMC member to step up to be the release manager for 0.10.
>>> >>> > > > > > Ideally they should undertake one or more dry runs of simulating the
>>> >>> > > > > > 0.10 release so we aren't faced with any surprises when we go to cut
>>> >>> > > > > > the release on ~July 23.
>>> >>> > > > > >
>>> >>> > > > > > I moved the release management guide to Confluence:
>>> >>> > > > > >
>>> >>> > > >
>>> >>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>>> >>> > > > > >
>>> >>> > > > > > so we should ensure that there aren't any snowflake-y / undocumented
>>> >>> > > > > > aspects to the release build workflow.
>>> >>> > > > > >
>>> >>> > > > > > - Wes
>>> >>> > > > > > On Wed, Jul 11, 2018 at 11:54 AM, Krisztián Szűcs
>>> >>> > > > > > <sz...@gmail.com> wrote:
>>> >>> > > > > > > Hi All!
>>> >>> > > > > > >
>>> >>> > > > > > > Just want to send a quick summary about the packaging.
>>> >>> > > > > > > What's working:
>>> >>> > > > > > > python wheel builds
>>> >>> > > > > > >
>>> >>> > > > > > > conda package builds (the recent conda forge changes caused some
>>> >>> > > > unexpected problems)
>>> >>> > > > > > >
>>> >>> > > > > > > linux package builds (enriched with ubuntu bionic)
>>> >>> > > > > > > scheduled daily builds (triggered by travis)
>>> >>> > > > > > > query build statuses from the CLI (see output
>>> >>> > https://bit.ly/2ztnSqV (
>>> >>> > > >
>>> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/0?redirect=https%3A%2F%2Fbit.ly%2F2ztnSqV&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D
>>> >>> > > > ))
>>> >>> > > > > > >
>>> >>> > > > > > > download and gpg sign the binary artifacts
>>> >>> > > > > > > What's partially working:
>>> >>> > > > > > > binary artifact uploading (most of them are uploaded though)
>>> >>> > > > > > >
>>> >>> > > > > > > We have package name collisions due to uploading multiple tasks'
>>> >>> > > > artifacts to a single github release
>>> >>> > > > > > > (see for example:
>>> >>> > > > > >
>>> >>> > > > >
>>> >>> > > >
>>> >>> > > > https://github.com/kszucs/crossbow/releases/tag/build-160 (
>>> >>> > > >
>>> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/1?redirect=https%3A%2F%2Fgithub.com%2Fkszucs%2Fcrossbow%2Freleases%2Ftag%2Fbuild-160&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D)
>>> >>> > ).
>>> >>> > > > We used to postfix
>>> >>> > > > > > > these assets with the particular architecture, but we need a more
>>> >>> > > > > >
>>> >>> > > > >
>>> >>> > > >
>>> >>> > > > robust solution without renaming
>>> >>> > > > > > > anything (either by creating tarballs or uploading to multiple
>>> >>> > > > > >
>>> >>> > > > >
>>> >>> > > >
>>> >>> > >
>>> >>> >
>>> >>> > release
>>> >>> > > > tags).
>>> >>> > > > > > >
>>> >>> > > > > > > All-in-all it's in quite good shape, I'll work on to refactor the
>>> >>> > > > artifact uploading and finish the
>>> >>> > > > > > > remaining jira tasks. IMHO we should start to test the whole release
>>> >>> > > > > >
>>> >>> > > > >
>>> >>> > > >
>>> >>> > > > procedure from next Monday
>>> >>> > > > > > > and ship a couple of test releases before We reach July 20th.
>>> >>> > > > > > >
>>> >>> > > > > > > - Krisztian
Re: Need a release manager for Arrow 0.10.0
Posted by Krisztián Szűcs <sz...@gmail.com>.
On Jul 19 2018, at 3:50 pm, Wes McKinney <we...@gmail.com> wrote:
>
> Thanks Kou!
> @Krisztian, where do we stand with the conda recipes in apache/arrow
> vs. the ones in conda-forge? Will the updates there still be manual?
>
Yes, I'm afraid so. Conda forge has a tightly coupled infrastructure with
conda-smithy and the ci services (including circleci for linux packages
which we cannot depend on - AFAIK). We can figure out something to
keep the recipes/feedstocks in sync, but not sure it's worth the effort.
>
> @kou, the conda-forge repositories in question are:
> https://github.com/conda-forge/arrow-cpp-feedstock
> https://github.com/conda-forge/parquet-cpp-feedstock
> https://github.com/conda-forge/pyarrow-feedstock
>
> Updating them is a bit laborious because we have arrow-cpp ->
> parquet-cpp -> pyarrow dependency. So the updates must be completed
> serially
>
> We should maybe post the voted-on release binaries at
> anaconda.org/apache eventually but that doesn't need to happen this
> release cycle
>
> - Wes
> On Thu, Jul 19, 2018 at 7:03 AM, Uwe L. Korn <uw...@xhochy.com> wrote:
> > Hi,
> >
> > I've added you to the PyPI admins. For the GPG key, please add it to the KEYS file in https://dist.apache.org/repos/dist/release/arrow/KEYS and https://dist.apache.org/repos/dist/dev/arrow/KEYS (they should be kept in sync). There is a small how-to at the top of the file. Just realised that I also need to do this.
> > Uwe
> > On Thu, Jul 19, 2018, at 12:12 PM, Kouhei Sutou wrote:
> > > Hi,
> > >
> > > I want to take packaging related tasks but I'm busy between
> > > 23th and 26th... I'll do at least the Ruby bindings related
> > > tasks. Because 0.10.0 is the first release of the Ruby
> > > bindings.
> > >
> > > I hope that I can do other packaging related tasks.
> > > I have a PyPI account: https://pypi.org/user/kou/
> > > Do I need other accounts to release packages? conda-forge?
> > >
> > > I have a GPG key: https://github.com/kou.gpg
> > >
> > > Thanks,
> > > --
> > > kou
> > >
> > > In <CA...@mail.gmail.com>
> > > "Re: Need a release manager for Arrow 0.10.0 [was Re: Packaging status
> > > update]" on Wed, 18 Jul 2018 15:13:46 -0400,
> > > Wes McKinney <we...@gmail.com> wrote:
> > >
> > > > If we could cut the release on the 24th (Tuesday) or 25th (Wednesday)
> > > > that would be ideal. We're approaching T+4 months since 0.9.0 so I
> > > > think we need to call pencils down once the last patches go in on
> > > > Monday, barring any crazy blockers. At this rate I'm going to be
> > > > working part of the weekend on this as we haven't managed to burn down
> > > > the backlog to near zero yet (see
> > > > https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.10.0+Release)
> > > >
> > > > In principle the packaging system should work fine against a tagged
> > > > release because it uses setuptools_scm, so from Step 2 in
> > > > https://github.com/apache/arrow/tree/master/dev/tasks#usage you would
> > > > write something like
> > > >
> > > > git checkout apache-arrow-0.10.0
> > > > python dev/tasks/crossbow.py submit ...
> > > >
> > > > We should probably add a script in
> > > > https://github.com/apache/arrow/tree/master/dev/release that will
> > > > fetch the artifacts, sign them, and upload to the dist system. The tag
> > > > and source release artifact will have already been tagged, built,
> > > > signed, and uploaded by dev/release/02-source.sh. We can make sure
> > > > these details are all documented and reproducible in
> > > > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> > > >
> > > > Thanks Krisztian, Kou, Phillip for laboring on the packaging; I'm
> > > > hopeful that this marks the beginning of a period of peace and
> > > > prosperity for Arrow release managers so we can make releases roughly
> > > > every month going forward.
> > > >
> > > > - Wes
> > > > On Wed, Jul 18, 2018 at 2:51 PM, Krisztián Szűcs
> > > > <sz...@gmail.com> wrote:
> > > > > I won't be available between 27th and 31st.
> > > > >
> > > > > On Jul 18 2018, at 7:53 pm, Phillip Cloud <cp...@gmail.com> wrote:
> > > > > >
> > > > > > I can certainly help with the release vote. I'm out of town from the 20th
> > > > > > to the 23rd, so it would be after that. Is there a particular target date
> > > > > > we have in mind?
> > > > > >
> > > > > > On Wed, Jul 18, 2018 at 12:24 PM Wes McKinney <we...@gmail.com> wrote:
> > > > > > > hi Li,
> > > > > > > Thanks for volunteering -- we need a PMC to sign the source and binary
> > > > > > > artifacts and upload to the dist system for the release. Realistically
> > > > > > > we should probably break up the tasks related to the release (the
> > > > > > > release vote, and follow up updates to PyPI, conda-forge, etc. ) --
> > > > > > > there's quite a lot of work to do before, during, and after the vote.
> > > > > > > @PhillipC, would you want to assist with the release vote process
> > > > > > > since you've been engaged w/ Krisztian on the packaging work? Since
> > > > > > > you guys work in the same physical location it might make coordination
> > > > > > > easier
> > > > > > >
> > > > > > > - Wes
> > > > > > > On Wed, Jul 18, 2018 at 12:00 PM, Li Jin <ic...@gmail.com> wrote:
> > > > > > > > I am currently on vacation but will be back on 22th and I am not PMC. If
> > > > > > > > these are not issues , I can volunteer to be RM for the release.
> > > > > > > >
> > > > > > > > Li
> > > > > > > > On Wed, Jul 18, 2018 at 4:09 PM Wes McKinney <we...@gmail.com>
> > > > > > >
> > > > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > hi folks,
> > > > > > > > > Any takers? It would be good for this knowledge to be spread to more
> > > > > > > > > people.
> > > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > > Wes
> > > > > > > > >
> > > > > > > > > On Thu, Jul 12, 2018 at 1:11 PM, Wes McKinney <we...@gmail.com>
> > > > > > > wrote:
> > > > > > > > > > Can a member of the Arrow PMC volunteer to be the release manager for
> > > > > > > > > > 0.10.0? There is some prep work to be done to make sure that you can
> > > > > > > > > > produce the binary packages as part of the release vote; if this is
> > > > > > > > > > done at the last minute it will probably delay the release.
> > > > > > > > > >
> > > > > > > > > > Thank you,
> > > > > > > > > > Wes
> > > > > > > > > >
> > > > > > > > > > On Wed, Jul 11, 2018 at 12:10 PM, Wes McKinney <we...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > > > Thanks Krisztian for the update and all your work on this.
> > > > > > > > > > >
> > > > > > > > > > > We need a PMC member to step up to be the release manager for 0.10.
> > > > > > > > > > > Ideally they should undertake one or more dry runs of simulating the
> > > > > > > > > > > 0.10 release so we aren't faced with any surprises when we go to cut
> > > > > > > > > > > the release on ~July 23.
> > > > > > > > > > >
> > > > > > > > > > > I moved the release management guide to Confluence:
> > > > > > > > >
> > > > > > > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> > > > > > > > > > >
> > > > > > > > > > > so we should ensure that there aren't any snowflake-y / undocumented
> > > > > > > > > > > aspects to the release build workflow.
> > > > > > > > > > >
> > > > > > > > > > > - Wes
> > > > > > > > > > > On Wed, Jul 11, 2018 at 11:54 AM, Krisztián Szűcs
> > > > > > > > > > > <sz...@gmail.com> wrote:
> > > > > > > > > > > > Hi All!
> > > > > > > > > > > >
> > > > > > > > > > > > Just want to send a quick summary about the packaging.
> > > > > > > > > > > > What's working:
> > > > > > > > > > > > python wheel builds
> > > > > > > > > > > >
> > > > > > > > > > > > conda package builds (the recent conda forge changes caused some
> > > > > > > > > unexpected problems)
> > > > > > > > > > > >
> > > > > > > > > > > > linux package builds (enriched with ubuntu bionic)
> > > > > > > > > > > > scheduled daily builds (triggered by travis)
> > > > > > > > > > > > query build statuses from the CLI (see output
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > https://bit.ly/2ztnSqV (
> > > > > > > > >
> > > > > > > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/0?redirect=https%3A%2F%2Fbit.ly%2F2ztnSqV&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D
> > > > > > > > > ))
> > > > > > > > > > > >
> > > > > > > > > > > > download and gpg sign the binary artifacts
> > > > > > > > > > > > What's partially working:
> > > > > > > > > > > > binary artifact uploading (most of them are uploaded though)
> > > > > > > > > > > >
> > > > > > > > > > > > We have package name collisions due to uploading multiple tasks'
> > > > > > > > > artifacts to a single github release
> > > > > > > > > > > > (see for example:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > https://github.com/kszucs/crossbow/releases/tag/build-160 (
> > > > > > > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/1?redirect=https%3A%2F%2Fgithub.com%2Fkszucs%2Fcrossbow%2Freleases%2Ftag%2Fbuild-160&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D)
> > > > > > > ).
> > > > > > > > > We used to postfix
> > > > > > > > > > > > these assets with the particular architecture, but we need a more
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > robust solution without renaming
> > > > > > > > > > > > anything (either by creating tarballs or uploading to multiple
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > release
> > > > > > > > > tags).
> > > > > > > > > > > >
> > > > > > > > > > > > All-in-all it's in quite good shape, I'll work on to refactor the
> > > > > > > > > artifact uploading and finish the
> > > > > > > > > > > > remaining jira tasks. IMHO we should start to test the whole release
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > procedure from next Monday
> > > > > > > > > > > > and ship a couple of test releases before We reach July 20th.
> > > > > > > > > > > >
> > > > > > > > > > > > - Krisztian
Re: Need a release manager for Arrow 0.10.0
Posted by Wes McKinney <we...@gmail.com>.
Thanks Kou!
@Krisztian, where do we stand with the conda recipes in apache/arrow
vs. the ones in conda-forge? Will the updates there still be manual?
@kou, the conda-forge repositories in question are:
https://github.com/conda-forge/arrow-cpp-feedstock
https://github.com/conda-forge/parquet-cpp-feedstock
https://github.com/conda-forge/pyarrow-feedstock
Updating them is a bit laborious because we have arrow-cpp ->
parquet-cpp -> pyarrow dependency. So the updates must be completed
serially
We should maybe post the voted-on release binaries at
anaconda.org/apache eventually but that doesn't need to happen this
release cycle
- Wes
On Thu, Jul 19, 2018 at 7:03 AM, Uwe L. Korn <uw...@xhochy.com> wrote:
> Hi,
>
> I've added you to the PyPI admins. For the GPG key, please add it to the KEYS file in https://dist.apache.org/repos/dist/release/arrow/KEYS and https://dist.apache.org/repos/dist/dev/arrow/KEYS (they should be kept in sync). There is a small how-to at the top of the file. Just realised that I also need to do this.
>
> Uwe
>
> On Thu, Jul 19, 2018, at 12:12 PM, Kouhei Sutou wrote:
>> Hi,
>>
>> I want to take packaging related tasks but I'm busy between
>> 23th and 26th... I'll do at least the Ruby bindings related
>> tasks. Because 0.10.0 is the first release of the Ruby
>> bindings.
>>
>> I hope that I can do other packaging related tasks.
>>
>> I have a PyPI account: https://pypi.org/user/kou/
>> Do I need other accounts to release packages? conda-forge?
>>
>> I have a GPG key: https://github.com/kou.gpg
>>
>>
>> Thanks,
>> --
>> kou
>>
>> In <CA...@mail.gmail.com>
>> "Re: Need a release manager for Arrow 0.10.0 [was Re: Packaging status
>> update]" on Wed, 18 Jul 2018 15:13:46 -0400,
>> Wes McKinney <we...@gmail.com> wrote:
>>
>> > If we could cut the release on the 24th (Tuesday) or 25th (Wednesday)
>> > that would be ideal. We're approaching T+4 months since 0.9.0 so I
>> > think we need to call pencils down once the last patches go in on
>> > Monday, barring any crazy blockers. At this rate I'm going to be
>> > working part of the weekend on this as we haven't managed to burn down
>> > the backlog to near zero yet (see
>> > https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.10.0+Release)
>> >
>> > In principle the packaging system should work fine against a tagged
>> > release because it uses setuptools_scm, so from Step 2 in
>> > https://github.com/apache/arrow/tree/master/dev/tasks#usage you would
>> > write something like
>> >
>> > git checkout apache-arrow-0.10.0
>> > python dev/tasks/crossbow.py submit ...
>> >
>> > We should probably add a script in
>> > https://github.com/apache/arrow/tree/master/dev/release that will
>> > fetch the artifacts, sign them, and upload to the dist system. The tag
>> > and source release artifact will have already been tagged, built,
>> > signed, and uploaded by dev/release/02-source.sh. We can make sure
>> > these details are all documented and reproducible in
>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> >
>> > Thanks Krisztian, Kou, Phillip for laboring on the packaging; I'm
>> > hopeful that this marks the beginning of a period of peace and
>> > prosperity for Arrow release managers so we can make releases roughly
>> > every month going forward.
>> >
>> > - Wes
>> >
>> > On Wed, Jul 18, 2018 at 2:51 PM, Krisztián Szűcs
>> > <sz...@gmail.com> wrote:
>> >> I won't be available between 27th and 31st.
>> >>
>> >> On Jul 18 2018, at 7:53 pm, Phillip Cloud <cp...@gmail.com> wrote:
>> >>>
>> >>> I can certainly help with the release vote. I'm out of town from the 20th
>> >>> to the 23rd, so it would be after that. Is there a particular target date
>> >>> we have in mind?
>> >>>
>> >>> On Wed, Jul 18, 2018 at 12:24 PM Wes McKinney <we...@gmail.com> wrote:
>> >>> > hi Li,
>> >>> > Thanks for volunteering -- we need a PMC to sign the source and binary
>> >>> > artifacts and upload to the dist system for the release. Realistically
>> >>> > we should probably break up the tasks related to the release (the
>> >>> > release vote, and follow up updates to PyPI, conda-forge, etc. ) --
>> >>> > there's quite a lot of work to do before, during, and after the vote.
>> >>> > @PhillipC, would you want to assist with the release vote process
>> >>> > since you've been engaged w/ Krisztian on the packaging work? Since
>> >>> > you guys work in the same physical location it might make coordination
>> >>> > easier
>> >>> >
>> >>> > - Wes
>> >>> > On Wed, Jul 18, 2018 at 12:00 PM, Li Jin <ic...@gmail.com> wrote:
>> >>> > > I am currently on vacation but will be back on 22th and I am not PMC. If
>> >>> > > these are not issues , I can volunteer to be RM for the release.
>> >>> > >
>> >>> > > Li
>> >>> > > On Wed, Jul 18, 2018 at 4:09 PM Wes McKinney <we...@gmail.com>
>> >>> >
>> >>> > wrote:
>> >>> > >
>> >>> > > > hi folks,
>> >>> > > > Any takers? It would be good for this knowledge to be spread to more
>> >>> > > > people.
>> >>> > > >
>> >>> > > > Thanks
>> >>> > > > Wes
>> >>> > > >
>> >>> > > > On Thu, Jul 12, 2018 at 1:11 PM, Wes McKinney <we...@gmail.com>
>> >>> > wrote:
>> >>> > > > > Can a member of the Arrow PMC volunteer to be the release manager for
>> >>> > > > > 0.10.0? There is some prep work to be done to make sure that you can
>> >>> > > > > produce the binary packages as part of the release vote; if this is
>> >>> > > > > done at the last minute it will probably delay the release.
>> >>> > > > >
>> >>> > > > > Thank you,
>> >>> > > > > Wes
>> >>> > > > >
>> >>> > > > > On Wed, Jul 11, 2018 at 12:10 PM, Wes McKinney <we...@gmail.com>
>> >>> > > > wrote:
>> >>> > > > > > Thanks Krisztian for the update and all your work on this.
>> >>> > > > > >
>> >>> > > > > > We need a PMC member to step up to be the release manager for 0.10.
>> >>> > > > > > Ideally they should undertake one or more dry runs of simulating the
>> >>> > > > > > 0.10 release so we aren't faced with any surprises when we go to cut
>> >>> > > > > > the release on ~July 23.
>> >>> > > > > >
>> >>> > > > > > I moved the release management guide to Confluence:
>> >>> > > > > >
>> >>> > > >
>> >>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
>> >>> > > > > >
>> >>> > > > > > so we should ensure that there aren't any snowflake-y / undocumented
>> >>> > > > > > aspects to the release build workflow.
>> >>> > > > > >
>> >>> > > > > > - Wes
>> >>> > > > > > On Wed, Jul 11, 2018 at 11:54 AM, Krisztián Szűcs
>> >>> > > > > > <sz...@gmail.com> wrote:
>> >>> > > > > > > Hi All!
>> >>> > > > > > >
>> >>> > > > > > > Just want to send a quick summary about the packaging.
>> >>> > > > > > > What's working:
>> >>> > > > > > > python wheel builds
>> >>> > > > > > >
>> >>> > > > > > > conda package builds (the recent conda forge changes caused some
>> >>> > > > unexpected problems)
>> >>> > > > > > >
>> >>> > > > > > > linux package builds (enriched with ubuntu bionic)
>> >>> > > > > > > scheduled daily builds (triggered by travis)
>> >>> > > > > > > query build statuses from the CLI (see output
>> >>> > https://bit.ly/2ztnSqV (
>> >>> > > >
>> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/0?redirect=https%3A%2F%2Fbit.ly%2F2ztnSqV&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D
>> >>> > > > ))
>> >>> > > > > > >
>> >>> > > > > > > download and gpg sign the binary artifacts
>> >>> > > > > > > What's partially working:
>> >>> > > > > > > binary artifact uploading (most of them are uploaded though)
>> >>> > > > > > >
>> >>> > > > > > > We have package name collisions due to uploading multiple tasks'
>> >>> > > > artifacts to a single github release
>> >>> > > > > > > (see for example:
>> >>> > > > > >
>> >>> > > > >
>> >>> > > >
>> >>> > > > https://github.com/kszucs/crossbow/releases/tag/build-160 (
>> >>> > > >
>> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/1?redirect=https%3A%2F%2Fgithub.com%2Fkszucs%2Fcrossbow%2Freleases%2Ftag%2Fbuild-160&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D)
>> >>> > ).
>> >>> > > > We used to postfix
>> >>> > > > > > > these assets with the particular architecture, but we need a more
>> >>> > > > > >
>> >>> > > > >
>> >>> > > >
>> >>> > > > robust solution without renaming
>> >>> > > > > > > anything (either by creating tarballs or uploading to multiple
>> >>> > > > > >
>> >>> > > > >
>> >>> > > >
>> >>> > >
>> >>> >
>> >>> > release
>> >>> > > > tags).
>> >>> > > > > > >
>> >>> > > > > > > All-in-all it's in quite good shape, I'll work on to refactor the
>> >>> > > > artifact uploading and finish the
>> >>> > > > > > > remaining jira tasks. IMHO we should start to test the whole release
>> >>> > > > > >
>> >>> > > > >
>> >>> > > >
>> >>> > > > procedure from next Monday
>> >>> > > > > > > and ship a couple of test releases before We reach July 20th.
>> >>> > > > > > >
>> >>> > > > > > > - Krisztian
Re: Need a release manager for Arrow 0.10.0
Posted by "Uwe L. Korn" <uw...@xhochy.com>.
Hi,
I've added you to the PyPI admins. For the GPG key, please add it to the KEYS file in https://dist.apache.org/repos/dist/release/arrow/KEYS and https://dist.apache.org/repos/dist/dev/arrow/KEYS (they should be kept in sync). There is a small how-to at the top of the file. Just realised that I also need to do this.
Uwe
On Thu, Jul 19, 2018, at 12:12 PM, Kouhei Sutou wrote:
> Hi,
>
> I want to take packaging related tasks but I'm busy between
> 23th and 26th... I'll do at least the Ruby bindings related
> tasks. Because 0.10.0 is the first release of the Ruby
> bindings.
>
> I hope that I can do other packaging related tasks.
>
> I have a PyPI account: https://pypi.org/user/kou/
> Do I need other accounts to release packages? conda-forge?
>
> I have a GPG key: https://github.com/kou.gpg
>
>
> Thanks,
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Re: Need a release manager for Arrow 0.10.0 [was Re: Packaging status
> update]" on Wed, 18 Jul 2018 15:13:46 -0400,
> Wes McKinney <we...@gmail.com> wrote:
>
> > If we could cut the release on the 24th (Tuesday) or 25th (Wednesday)
> > that would be ideal. We're approaching T+4 months since 0.9.0 so I
> > think we need to call pencils down once the last patches go in on
> > Monday, barring any crazy blockers. At this rate I'm going to be
> > working part of the weekend on this as we haven't managed to burn down
> > the backlog to near zero yet (see
> > https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.10.0+Release)
> >
> > In principle the packaging system should work fine against a tagged
> > release because it uses setuptools_scm, so from Step 2 in
> > https://github.com/apache/arrow/tree/master/dev/tasks#usage you would
> > write something like
> >
> > git checkout apache-arrow-0.10.0
> > python dev/tasks/crossbow.py submit ...
> >
> > We should probably add a script in
> > https://github.com/apache/arrow/tree/master/dev/release that will
> > fetch the artifacts, sign them, and upload to the dist system. The tag
> > and source release artifact will have already been tagged, built,
> > signed, and uploaded by dev/release/02-source.sh. We can make sure
> > these details are all documented and reproducible in
> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> >
> > Thanks Krisztian, Kou, Phillip for laboring on the packaging; I'm
> > hopeful that this marks the beginning of a period of peace and
> > prosperity for Arrow release managers so we can make releases roughly
> > every month going forward.
> >
> > - Wes
> >
> > On Wed, Jul 18, 2018 at 2:51 PM, Krisztián Szűcs
> > <sz...@gmail.com> wrote:
> >> I won't be available between 27th and 31st.
> >>
> >> On Jul 18 2018, at 7:53 pm, Phillip Cloud <cp...@gmail.com> wrote:
> >>>
> >>> I can certainly help with the release vote. I'm out of town from the 20th
> >>> to the 23rd, so it would be after that. Is there a particular target date
> >>> we have in mind?
> >>>
> >>> On Wed, Jul 18, 2018 at 12:24 PM Wes McKinney <we...@gmail.com> wrote:
> >>> > hi Li,
> >>> > Thanks for volunteering -- we need a PMC to sign the source and binary
> >>> > artifacts and upload to the dist system for the release. Realistically
> >>> > we should probably break up the tasks related to the release (the
> >>> > release vote, and follow up updates to PyPI, conda-forge, etc. ) --
> >>> > there's quite a lot of work to do before, during, and after the vote.
> >>> > @PhillipC, would you want to assist with the release vote process
> >>> > since you've been engaged w/ Krisztian on the packaging work? Since
> >>> > you guys work in the same physical location it might make coordination
> >>> > easier
> >>> >
> >>> > - Wes
> >>> > On Wed, Jul 18, 2018 at 12:00 PM, Li Jin <ic...@gmail.com> wrote:
> >>> > > I am currently on vacation but will be back on 22th and I am not PMC. If
> >>> > > these are not issues , I can volunteer to be RM for the release.
> >>> > >
> >>> > > Li
> >>> > > On Wed, Jul 18, 2018 at 4:09 PM Wes McKinney <we...@gmail.com>
> >>> >
> >>> > wrote:
> >>> > >
> >>> > > > hi folks,
> >>> > > > Any takers? It would be good for this knowledge to be spread to more
> >>> > > > people.
> >>> > > >
> >>> > > > Thanks
> >>> > > > Wes
> >>> > > >
> >>> > > > On Thu, Jul 12, 2018 at 1:11 PM, Wes McKinney <we...@gmail.com>
> >>> > wrote:
> >>> > > > > Can a member of the Arrow PMC volunteer to be the release manager for
> >>> > > > > 0.10.0? There is some prep work to be done to make sure that you can
> >>> > > > > produce the binary packages as part of the release vote; if this is
> >>> > > > > done at the last minute it will probably delay the release.
> >>> > > > >
> >>> > > > > Thank you,
> >>> > > > > Wes
> >>> > > > >
> >>> > > > > On Wed, Jul 11, 2018 at 12:10 PM, Wes McKinney <we...@gmail.com>
> >>> > > > wrote:
> >>> > > > > > Thanks Krisztian for the update and all your work on this.
> >>> > > > > >
> >>> > > > > > We need a PMC member to step up to be the release manager for 0.10.
> >>> > > > > > Ideally they should undertake one or more dry runs of simulating the
> >>> > > > > > 0.10 release so we aren't faced with any surprises when we go to cut
> >>> > > > > > the release on ~July 23.
> >>> > > > > >
> >>> > > > > > I moved the release management guide to Confluence:
> >>> > > > > >
> >>> > > >
> >>> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide
> >>> > > > > >
> >>> > > > > > so we should ensure that there aren't any snowflake-y / undocumented
> >>> > > > > > aspects to the release build workflow.
> >>> > > > > >
> >>> > > > > > - Wes
> >>> > > > > > On Wed, Jul 11, 2018 at 11:54 AM, Krisztián Szűcs
> >>> > > > > > <sz...@gmail.com> wrote:
> >>> > > > > > > Hi All!
> >>> > > > > > >
> >>> > > > > > > Just want to send a quick summary about the packaging.
> >>> > > > > > > What's working:
> >>> > > > > > > python wheel builds
> >>> > > > > > >
> >>> > > > > > > conda package builds (the recent conda forge changes caused some
> >>> > > > unexpected problems)
> >>> > > > > > >
> >>> > > > > > > linux package builds (enriched with ubuntu bionic)
> >>> > > > > > > scheduled daily builds (triggered by travis)
> >>> > > > > > > query build statuses from the CLI (see output
> >>> > https://bit.ly/2ztnSqV (
> >>> > > >
> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/0?redirect=https%3A%2F%2Fbit.ly%2F2ztnSqV&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D
> >>> > > > ))
> >>> > > > > > >
> >>> > > > > > > download and gpg sign the binary artifacts
> >>> > > > > > > What's partially working:
> >>> > > > > > > binary artifact uploading (most of them are uploaded though)
> >>> > > > > > >
> >>> > > > > > > We have package name collisions due to uploading multiple tasks'
> >>> > > > artifacts to a single github release
> >>> > > > > > > (see for example:
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > > > https://github.com/kszucs/crossbow/releases/tag/build-160 (
> >>> > > >
> >>> > https://link.getmailspring.com/link/1531321550.local-b501f335-6185-v1.2.2-96fb3a99@getmailspring.com/1?redirect=https%3A%2F%2Fgithub.com%2Fkszucs%2Fcrossbow%2Freleases%2Ftag%2Fbuild-160&recipient=ZGV2QGFycm93LmFwYWNoZS5vcmc%3D)
> >>> > ).
> >>> > > > We used to postfix
> >>> > > > > > > these assets with the particular architecture, but we need a more
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > > > robust solution without renaming
> >>> > > > > > > anything (either by creating tarballs or uploading to multiple
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>> > release
> >>> > > > tags).
> >>> > > > > > >
> >>> > > > > > > All-in-all it's in quite good shape, I'll work on to refactor the
> >>> > > > artifact uploading and finish the
> >>> > > > > > > remaining jira tasks. IMHO we should start to test the whole release
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > > > procedure from next Monday
> >>> > > > > > > and ship a couple of test releases before We reach July 20th.
> >>> > > > > > >
> >>> > > > > > > - Krisztian