Package tarball as deliverable of apex-core

[Chinmay Kolhatkar <ch...@datatorrent.com>]

Dear Community,

Currently Apache Apex does not have a deliverable tarball which bundles the
dependent jars and apex cli which user can just download and get started
with.

Having this would be also be useful for current integration with Apache
Bigtop as well. Currently in bigtop, the bigtop build downloads the src tar
and builds the packages from it. Ideally the bigtop build should just
download the deliverable tarball (containing packaged dependency for cli
for ready to use).

Also, one of the output of bigtop rpm/deb goes in docker image of apache
apex. The Dockerfile for the same resides in my public repository.

Considering above I have following proposals:
1. Create a new deliverable tarball in apex-core which will bundle all
required and sufficient dependencies for user to quickly get started on an
existing hadoop cluster.

2. Once we have the deliverable available in next apex-core release, we can
update apache/bigtop to point to use that tarball.

3. There is also a docker image for apache/apex, the Dockerfile for which
resides in https://github.com/chinmaykolhatkar/docker-pool. I would prefer
if this is moved to apex-core repository so that the releases of docker and
apex-core can be aligned better and also become part of Apache repo.

Please share your opinion on this.

Thanks,
Chinmay.

Re: Package tarball as deliverable of apex-core

[Thomas Weise <th...@apache.org>]

There was a discussion about this some time ago on this list and there are
already JIRAs created.

I also don't see the need for elaborate packaging solutions, that's solved
by Bigtop.

Moving the docker file may need a bit of research as AFAIK it needs to be
in its own repo do work with docker hub (?) and depends on the Bigtop
release build artifact, which is downstream of an Apex release.

I also don't think that Bigtop build can use a binary package from the Apex
release as they want to build the components from source.

Thanks,
Thomas


On Mon, Aug 7, 2017 at 9:05 PM, Chinmay Kolhatkar <ch...@datatorrent.com>
wrote:

> I'm not sure having rpm or deb is good idea from apex point of view because
> then we'll have to support for all or atleast major operating systems and
> their versions. It would just increase the support matrix.
>
> Most Apache projects support a tarball distribution which makes it
> deployment agnostic. Hence and for previously mentioned reasons suggested
> that approach.
>
> - Chinmay.
>
>
> On 7 Aug 2017 8:51 p.m., "Priyanka Gugale" <pr...@apache.org> wrote:
>
> > Hi Chinmay,
> >
> > How about using linux distributions like deb/rpm, why create one more way
> > of distribution?
> >
> > -Priyanka
> >
> > On Tue, Aug 8, 2017 at 8:34 AM, Chinmay Kolhatkar <
> chinmay@datatorrent.com
> > >
> > wrote:
> >
> > > Dear Community,
> > >
> > > Currently Apache Apex does not have a deliverable tarball which bundles
> > the
> > > dependent jars and apex cli which user can just download and get
> started
> > > with.
> > >
> > > Having this would be also be useful for current integration with Apache
> > > Bigtop as well. Currently in bigtop, the bigtop build downloads the src
> > tar
> > > and builds the packages from it. Ideally the bigtop build should just
> > > download the deliverable tarball (containing packaged dependency for
> cli
> > > for ready to use).
> > >
> > > Also, one of the output of bigtop rpm/deb goes in docker image of
> apache
> > > apex. The Dockerfile for the same resides in my public repository.
> > >
> > > Considering above I have following proposals:
> > > 1. Create a new deliverable tarball in apex-core which will bundle all
> > > required and sufficient dependencies for user to quickly get started on
> > an
> > > existing hadoop cluster.
> > >
> > > 2. Once we have the deliverable available in next apex-core release, we
> > can
> > > update apache/bigtop to point to use that tarball.
> > >
> > > 3. There is also a docker image for apache/apex, the Dockerfile for
> which
> > > resides in https://github.com/chinmaykolhatkar/docker-pool. I would
> > prefer
> > > if this is moved to apex-core repository so that the releases of docker
> > and
> > > apex-core can be aligned better and also become part of Apache repo.
> > >
> > > Please share your opinion on this.
> > >
> > > Thanks,
> > > Chinmay.
> > >
> >
>

Re: Package tarball as deliverable of apex-core

[Chinmay Kolhatkar <ch...@datatorrent.com>]

I'm not sure having rpm or deb is good idea from apex point of view because
then we'll have to support for all or atleast major operating systems and
their versions. It would just increase the support matrix.

Most Apache projects support a tarball distribution which makes it
deployment agnostic. Hence and for previously mentioned reasons suggested
that approach.

- Chinmay.


On 7 Aug 2017 8:51 p.m., "Priyanka Gugale" <pr...@apache.org> wrote:

> Hi Chinmay,
>
> How about using linux distributions like deb/rpm, why create one more way
> of distribution?
>
> -Priyanka
>
> On Tue, Aug 8, 2017 at 8:34 AM, Chinmay Kolhatkar <chinmay@datatorrent.com
> >
> wrote:
>
> > Dear Community,
> >
> > Currently Apache Apex does not have a deliverable tarball which bundles
> the
> > dependent jars and apex cli which user can just download and get started
> > with.
> >
> > Having this would be also be useful for current integration with Apache
> > Bigtop as well. Currently in bigtop, the bigtop build downloads the src
> tar
> > and builds the packages from it. Ideally the bigtop build should just
> > download the deliverable tarball (containing packaged dependency for cli
> > for ready to use).
> >
> > Also, one of the output of bigtop rpm/deb goes in docker image of apache
> > apex. The Dockerfile for the same resides in my public repository.
> >
> > Considering above I have following proposals:
> > 1. Create a new deliverable tarball in apex-core which will bundle all
> > required and sufficient dependencies for user to quickly get started on
> an
> > existing hadoop cluster.
> >
> > 2. Once we have the deliverable available in next apex-core release, we
> can
> > update apache/bigtop to point to use that tarball.
> >
> > 3. There is also a docker image for apache/apex, the Dockerfile for which
> > resides in https://github.com/chinmaykolhatkar/docker-pool. I would
> prefer
> > if this is moved to apex-core repository so that the releases of docker
> and
> > apex-core can be aligned better and also become part of Apache repo.
> >
> > Please share your opinion on this.
> >
> > Thanks,
> > Chinmay.
> >
>

Re: Package tarball as deliverable of apex-core

[Priyanka Gugale <pr...@apache.org>]

Hi Chinmay,

How about using linux distributions like deb/rpm, why create one more way
of distribution?

-Priyanka

On Tue, Aug 8, 2017 at 8:34 AM, Chinmay Kolhatkar <ch...@datatorrent.com>
wrote:

> Dear Community,
>
> Currently Apache Apex does not have a deliverable tarball which bundles the
> dependent jars and apex cli which user can just download and get started
> with.
>
> Having this would be also be useful for current integration with Apache
> Bigtop as well. Currently in bigtop, the bigtop build downloads the src tar
> and builds the packages from it. Ideally the bigtop build should just
> download the deliverable tarball (containing packaged dependency for cli
> for ready to use).
>
> Also, one of the output of bigtop rpm/deb goes in docker image of apache
> apex. The Dockerfile for the same resides in my public repository.
>
> Considering above I have following proposals:
> 1. Create a new deliverable tarball in apex-core which will bundle all
> required and sufficient dependencies for user to quickly get started on an
> existing hadoop cluster.
>
> 2. Once we have the deliverable available in next apex-core release, we can
> update apache/bigtop to point to use that tarball.
>
> 3. There is also a docker image for apache/apex, the Dockerfile for which
> resides in https://github.com/chinmaykolhatkar/docker-pool. I would prefer
> if this is moved to apex-core repository so that the releases of docker and
> apex-core can be aligned better and also become part of Apache repo.
>
> Please share your opinion on this.
>
> Thanks,
> Chinmay.
>