You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Wes McKinney <we...@gmail.com> on 2017/07/24 14:55:07 UTC

Packaging automation for Arrow releases

We're accumulating more deployment targets and possible package
artifacts from Arrow releases, such as:

- Source tarball
- Java JARs
- Python wheels (for pip), 3 platforms
- Python conda packages (for conda), 3 platforms
- .deb/.rpm packages for C++, GLib C

Doing all of this manually is a lot of work. Also, because many of the
binary artifacts are being built post-release, they are not being
voted on by the PMC and thus are "unofficial" packages. See
http://arrow.apache.org/install/

Recently the apache/arrow-dist repo was created as a place where we
can automate package builds, with uploads happening to a cloud bucket
someplace:

https://github.com/apache/arrow-dist

Once we've set up the build scripts and various CI services, which
should decide which packages we may want to include as part of a
future release vote.

Any help with packaging and distribution would be much appreciated.

Thanks
Wes

Re: Packaging automation for Arrow releases

Posted by Wes McKinney <we...@gmail.com>.
I created https://issues.apache.org/jira/browse/ARROW-1262

We have some existing prior work that needs to be collected in the
arrow-dist repo. Anyone is welcome to help out with the packaging
automation, it would be particularly helpful for Linux package
managers

- Wes

On Tue, Jul 25, 2017 at 5:36 AM, Antonio Cavallo
<an...@gmail.com> wrote:
> Hi
> I'm interested in arrow but I never had time to play with it.
> I can help with that as I gave some experience in packaging and software
> deploy.
> best Antonio
>
> On 24 Jul 2017 17:30, "Wes McKinney" <we...@gmail.com> wrote:
>
>> Yes, definitely. I'll create a packaging / arrow-dist automation
>> umbrella JIRA and attach tasks to it
>>
>> On Mon, Jul 24, 2017 at 4:17 PM, Jacques Nadeau <ja...@apache.org>
>> wrote:
>> > Hey Wes,
>> >
>> > Does it make sense to create some jiras around this and then maybe some
>> > people can pick them up?
>> >
>> > On Mon, Jul 24, 2017 at 7:55 AM, Wes McKinney <we...@gmail.com>
>> wrote:
>> >
>> >> We're accumulating more deployment targets and possible package
>> >> artifacts from Arrow releases, such as:
>> >>
>> >> - Source tarball
>> >> - Java JARs
>> >> - Python wheels (for pip), 3 platforms
>> >> - Python conda packages (for conda), 3 platforms
>> >> - .deb/.rpm packages for C++, GLib C
>> >>
>> >> Doing all of this manually is a lot of work. Also, because many of the
>> >> binary artifacts are being built post-release, they are not being
>> >> voted on by the PMC and thus are "unofficial" packages. See
>> >> http://arrow.apache.org/install/
>> >>
>> >> Recently the apache/arrow-dist repo was created as a place where we
>> >> can automate package builds, with uploads happening to a cloud bucket
>> >> someplace:
>> >>
>> >> https://github.com/apache/arrow-dist
>> >>
>> >> Once we've set up the build scripts and various CI services, which
>> >> should decide which packages we may want to include as part of a
>> >> future release vote.
>> >>
>> >> Any help with packaging and distribution would be much appreciated.
>> >>
>> >> Thanks
>> >> Wes
>> >>
>>

Re: Packaging automation for Arrow releases

Posted by Antonio Cavallo <an...@gmail.com>.
Hi
I'm interested in arrow but I never had time to play with it.
I can help with that as I gave some experience in packaging and software
deploy.
best Antonio

On 24 Jul 2017 17:30, "Wes McKinney" <we...@gmail.com> wrote:

> Yes, definitely. I'll create a packaging / arrow-dist automation
> umbrella JIRA and attach tasks to it
>
> On Mon, Jul 24, 2017 at 4:17 PM, Jacques Nadeau <ja...@apache.org>
> wrote:
> > Hey Wes,
> >
> > Does it make sense to create some jiras around this and then maybe some
> > people can pick them up?
> >
> > On Mon, Jul 24, 2017 at 7:55 AM, Wes McKinney <we...@gmail.com>
> wrote:
> >
> >> We're accumulating more deployment targets and possible package
> >> artifacts from Arrow releases, such as:
> >>
> >> - Source tarball
> >> - Java JARs
> >> - Python wheels (for pip), 3 platforms
> >> - Python conda packages (for conda), 3 platforms
> >> - .deb/.rpm packages for C++, GLib C
> >>
> >> Doing all of this manually is a lot of work. Also, because many of the
> >> binary artifacts are being built post-release, they are not being
> >> voted on by the PMC and thus are "unofficial" packages. See
> >> http://arrow.apache.org/install/
> >>
> >> Recently the apache/arrow-dist repo was created as a place where we
> >> can automate package builds, with uploads happening to a cloud bucket
> >> someplace:
> >>
> >> https://github.com/apache/arrow-dist
> >>
> >> Once we've set up the build scripts and various CI services, which
> >> should decide which packages we may want to include as part of a
> >> future release vote.
> >>
> >> Any help with packaging and distribution would be much appreciated.
> >>
> >> Thanks
> >> Wes
> >>
>

Re: Packaging automation for Arrow releases

Posted by Wes McKinney <we...@gmail.com>.
Yes, definitely. I'll create a packaging / arrow-dist automation
umbrella JIRA and attach tasks to it

On Mon, Jul 24, 2017 at 4:17 PM, Jacques Nadeau <ja...@apache.org> wrote:
> Hey Wes,
>
> Does it make sense to create some jiras around this and then maybe some
> people can pick them up?
>
> On Mon, Jul 24, 2017 at 7:55 AM, Wes McKinney <we...@gmail.com> wrote:
>
>> We're accumulating more deployment targets and possible package
>> artifacts from Arrow releases, such as:
>>
>> - Source tarball
>> - Java JARs
>> - Python wheels (for pip), 3 platforms
>> - Python conda packages (for conda), 3 platforms
>> - .deb/.rpm packages for C++, GLib C
>>
>> Doing all of this manually is a lot of work. Also, because many of the
>> binary artifacts are being built post-release, they are not being
>> voted on by the PMC and thus are "unofficial" packages. See
>> http://arrow.apache.org/install/
>>
>> Recently the apache/arrow-dist repo was created as a place where we
>> can automate package builds, with uploads happening to a cloud bucket
>> someplace:
>>
>> https://github.com/apache/arrow-dist
>>
>> Once we've set up the build scripts and various CI services, which
>> should decide which packages we may want to include as part of a
>> future release vote.
>>
>> Any help with packaging and distribution would be much appreciated.
>>
>> Thanks
>> Wes
>>

Re: Packaging automation for Arrow releases

Posted by Jacques Nadeau <ja...@apache.org>.
Hey Wes,

Does it make sense to create some jiras around this and then maybe some
people can pick them up?

On Mon, Jul 24, 2017 at 7:55 AM, Wes McKinney <we...@gmail.com> wrote:

> We're accumulating more deployment targets and possible package
> artifacts from Arrow releases, such as:
>
> - Source tarball
> - Java JARs
> - Python wheels (for pip), 3 platforms
> - Python conda packages (for conda), 3 platforms
> - .deb/.rpm packages for C++, GLib C
>
> Doing all of this manually is a lot of work. Also, because many of the
> binary artifacts are being built post-release, they are not being
> voted on by the PMC and thus are "unofficial" packages. See
> http://arrow.apache.org/install/
>
> Recently the apache/arrow-dist repo was created as a place where we
> can automate package builds, with uploads happening to a cloud bucket
> someplace:
>
> https://github.com/apache/arrow-dist
>
> Once we've set up the build scripts and various CI services, which
> should decide which packages we may want to include as part of a
> future release vote.
>
> Any help with packaging and distribution would be much appreciated.
>
> Thanks
> Wes
>