You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Andre Kelpe <ak...@concurrentinc.com> on 2015/05/11 13:19:02 UTC
ci infrastructure and release process for Cascading integration
Hi,
the last couple of days I have been investigating the Cascading integration
for Bigtop. I have navigated the project and created RPM files for two of
our projects. Since we are outside of the ASF, things tend to be a bit
different, but I got it to work. I am currently doing the same for the
debian packages, which should not take too much time.
Now to my actual questions: Bigtop supports a number of distributions and
has CI infrastructure to build the packages. I am wondering how we as the
Cascading community can work with that infrastructure for better
integration, or if we have to replicate all of it. Let me know, what could
work best. I'd love to test and build all packages in an easy way, but
going through JIRA with a commiter reviewing and committing it looks like a
pretty heavy process. OTOH if I replicate the infrastructure I will end up
having to catch up with all infrastructure changes all the time.
I am unsure what your way of maintaining things is, once a Bigtop release
is made. I guess you are not targeting a rolling release, but rather a
stable release to be used for a longer time. How do bugfix releases for
packages go in? Suppose we have a Cascading tool version x.y in Bigtop a.b.
and we make a critical bugfix release x.z, which we want to distribute to
Bigtop users as well. Would that then require a complete Bigtop release
with voting and all that or do you envision a faster track like linux
distros, which update packages for a given release.
Thanks for your answers!
- André
--
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com
Re: ci infrastructure and release process for Cascading integration
Posted by Konstantin Boudnik <co...@apache.org>.
We have discussed it before, really. And the consensus was such that there's
nothing is a limit but ppl cycles to support multiple release trains (and CI
too, but we are solving this right now, per Andrew email below). Any committer
on the project can be an RM for a release: PMC needs to vote on it to make it
an "official ASF release", but that's about it.
So, in your example, if an emergency fix or version update for Cascading is in
order, the patch'd need to be submitted via JIRA, reviewed, committed to the
correct branch(s), and a new release would be cut, if there's a willing RM.
Pretty much like any other software project, really. Now, it might be somewhat
impractical to release a whole stack with only one component being updated. In
this case, the upgrades like that would better be grouped together to deliver
a bigger punch.
The flow below would be a viable alternative: with standardization of the
build environment via containers, building a stack yourself should be a matter
of a couple hours (that is if you need to build the whole thing).
Best,
Cos
On Thu, May 14, 2015 at 04:32PM, Andrew Purtell wrote:
> The Apache notion of release is wholly scoped to source distributions. The
> Bigtop source is the build scripting, package files, Puppet files, etc.
> Generated binary packages are not included.
>
> I think we can be limber here by providing nightly -SNAPSHOT versions of
> the Bigtop source on every day our git tree is updated. As part of the
> process of pushing out the Bigtop source snapshot we can use CI to kick off
> binary package builds, then place the resulting convenience binary packages
> from the snapshot into similarly convenient apt or yum repositories.
> Therefore on day N, there would be for example Cascading 1.2.3 binary
> convenience artifacts in the convenience repositories. After check in of
> Cascading package file updates (and related changes), on day N+1 there
> would be Cascading 1.2.3 and 1.2.4 binary convenience artifacts in those
> repos.
>
> Periodically we can plant a flag and promote a -SNAPSHOT into the next
> official release.
>
> Then, rinse, and repeat.
>
>
> On Tue, May 12, 2015 at 11:26 PM, <na...@reactor8.com> wrote:
>
> > Idea would like to work through with the group is post 1.0, evolving the
> > CI and project process to be able to do real-time, or just-in-time builds.
> >
> > We could have a "bigtop core", which would be components like
> > Hadoop/hdfs/zookeeper/etc. Thinks like hbase, hive, etc would be next
> > level components, then things like pig, cascading, etc would be the real
> > fast movers.
> >
> > Think another area we can investigate is components that want to integrate
> > with bigtop but are not dependent on jvm stack and just the
> > interfaces/api's of the core components. Seems those as well could be fast
> > movers
> >
> > Don’t know if that answers your question or not Andre, hopefully with your
> > guys help we can get some Cascading action going and get the build/release
> > process evolving so no matter how fast you guys are releasing users can
> > deploy the bits integrated fashion to other Bigtop components
> >
> > -----Original Message-----
> > From: Evans Ye [mailto:evansye@apache.org]
> > Sent: Tuesday, May 12, 2015 9:16 AM
> > To: dev@bigtop.apache.org
> > Subject: Re: ci infrastructure and release process for Cascading
> > integration
> >
> > Hi Andre
> >
> > I take you're not asking technical questions but rather than wondering
> > something related to the process or policy.
> >
> > AFAIK, at this moment we don't have a process to have the up-to-date
> > packages available.
> > (The Bigtop CI used to be working, but collapsed when most of the build
> > slaves went out of available)
> >
> > So in your example, when upgrading upstream component version from 1.1 to
> > 2.0, they would need to submit a patch and build the package by themselves.
> >
> > But, thanks to the EMR team sponsoring us EC2 resources, we're now
> > rebuilding our CI infra, which surely would have nightly build(RPM/DEB)
> > available.
> >
> > OTOH, I can recall that Nate is also working on bigtop repo nightly build,
> > when that comes online we can always have SNAPSHOT repos available as well.
> >
> > Evans
> >
> > 2015-05-12 0:27 GMT+08:00 Andre Kelpe <ak...@concurrentinc.com>:
> >
> > > If that is true, what would happen in the case where we want to make a
> > > new major release available? Suppose we have project in 1.x and now as
> > > a upstream we release 2.0. Would such a change still be covered by
> > > this? How would somebody trying to rebuild all packages handle these
> > > changes? If we submit a package update, the bigtop source release will
> > > only have the spec files/debian directory from the time that a bigtop
> > > release was made. Does that still work with the apache rules?
> > >
> > > - André
> > >
> > > On Mon, May 11, 2015 at 3:52 PM, jay vyas
> > > <ja...@gmail.com>
> > > wrote:
> > >
> > > > i think -- that -- since bigtop RPM/DEB packages ARE NOT bound by
> > > > the
> > > ASF
> > > > guidelines, they are just binary convenience artifacts.... we
> > > > probably
> > > dont
> > > > have much to worry about :)
> > > >
> > > > when this happens id suggest : create a jira, folks will gladly
> > > > help you to get your latest binaries published so that people can
> > > > use the latest cascading bits from bigtop yum repos .
> > > >
> > > > just my thoughts, i assume this is correct though.
> > > >
> > > >
> > > > On Mon, May 11, 2015 at 7:19 AM, Andre Kelpe
> > > > <ak...@concurrentinc.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > the last couple of days I have been investigating the Cascading
> > > > integration
> > > > > for Bigtop. I have navigated the project and created RPM files for
> > > > > two
> > > of
> > > > > our projects. Since we are outside of the ASF, things tend to be a
> > > > > bit different, but I got it to work. I am currently doing the same
> > > > > for the debian packages, which should not take too much time.
> > > > >
> > > > > Now to my actual questions: Bigtop supports a number of
> > > > > distributions
> > > and
> > > > > has CI infrastructure to build the packages. I am wondering how we
> > > > > as
> > > the
> > > > > Cascading community can work with that infrastructure for better
> > > > > integration, or if we have to replicate all of it. Let me know,
> > > > > what
> > > > could
> > > > > work best. I'd love to test and build all packages in an easy way,
> > > > > but going through JIRA with a commiter reviewing and committing it
> > > > > looks
> > > > like a
> > > > > pretty heavy process. OTOH if I replicate the infrastructure I
> > > > > will end
> > > > up
> > > > > having to catch up with all infrastructure changes all the time.
> > > > >
> > > > > I am unsure what your way of maintaining things is, once a Bigtop
> > > release
> > > > > is made. I guess you are not targeting a rolling release, but
> > > > > rather a stable release to be used for a longer time. How do
> > > > > bugfix releases for packages go in? Suppose we have a Cascading
> > > > > tool version x.y in Bigtop
> > > > a.b.
> > > > > and we make a critical bugfix release x.z, which we want to
> > > > > distribute
> > > to
> > > > > Bigtop users as well. Would that then require a complete Bigtop
> > > > > release with voting and all that or do you envision a faster track
> > > > > like linux distros, which update packages for a given release.
> > > > >
> > > > > Thanks for your answers!
> > > > >
> > > > > - André
> > > > >
> > > > > --
> > > > > André Kelpe
> > > > > andre@concurrentinc.com
> > > > > http://concurrentinc.com
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > jay vyas
> > > >
> > >
> > >
> > >
> > > --
> > > André Kelpe
> > > andre@concurrentinc.com
> > > http://concurrentinc.com
> > >
> >
> >
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
RE: ci infrastructure and release process for Cascading integration
Posted by na...@reactor8.com.
We also talked briefly about the idea of post 1.0 maybe leveraging semver style where 1.0.1, or 1.1 might be release, and the 1.0.1 could be patch for a component, where maybe a 1.1.0 would be a rev where for instance cascading was revved but the "bigtop core" was the same
-----Original Message-----
From: Konstantin Boudnik [mailto:cos@apache.org]
Sent: Thursday, May 14, 2015 4:49 PM
To: dev@bigtop.apache.org
Subject: Re: ci infrastructure and release process for Cascading integration
All packages include the build stamp that increases monotonically: that guarantees the name uniqueness and ability to tell them apart for update purposes and such.
Cos
On Thu, May 14, 2015 at 04:41PM, Andrew Purtell wrote:
> If we did want to do the below, the next consideration is how we would
> differentiate between artifacts produced by one nightly build versus
> another. Let's say for example we update Hadoop. All components that
> pull in Hadoop as a dependency will produce changed binary artifacts
> at the next build. These need to be versioned differently from earlier built packages.
> I think we would want to introduce a global build identifier (could be
> managed by our CI master), increment the identifier for every nightly,
> and incorporate it as a package version suffix.
>
>
> On Thu, May 14, 2015 at 4:32 PM, Andrew Purtell <ap...@apache.org> wrote:
>
> > The Apache notion of release is wholly scoped to source
> > distributions. The Bigtop source is the build scripting, package files, Puppet files, etc.
> > Generated binary packages are not included.
> >
> > I think we can be limber here by providing nightly -SNAPSHOT
> > versions of the Bigtop source on every day our git tree is updated.
> > As part of the process of pushing out the Bigtop source snapshot we
> > can use CI to kick off binary package builds, then place the
> > resulting convenience binary packages from the snapshot into similarly convenient apt or yum repositories.
> > Therefore on day N, there would be for example Cascading 1.2.3
> > binary convenience artifacts in the convenience repositories. After
> > check in of Cascading package file updates (and related changes), on
> > day N+1 there would be Cascading 1.2.3 and 1.2.4 binary convenience
> > artifacts in those repos.
> >
> > Periodically we can plant a flag and promote a -SNAPSHOT into the
> > next official release.
> >
> > Then, rinse, and repeat.
> >
> >
> > On Tue, May 12, 2015 at 11:26 PM, <na...@reactor8.com> wrote:
> >
> >> Idea would like to work through with the group is post 1.0,
> >> evolving the CI and project process to be able to do real-time, or just-in-time builds.
> >>
> >> We could have a "bigtop core", which would be components like
> >> Hadoop/hdfs/zookeeper/etc. Thinks like hbase, hive, etc would be
> >> next level components, then things like pig, cascading, etc would
> >> be the real fast movers.
> >>
> >> Think another area we can investigate is components that want to
> >> integrate with bigtop but are not dependent on jvm stack and just
> >> the interfaces/api's of the core components. Seems those as well
> >> could be fast movers
> >>
> >> Don’t know if that answers your question or not Andre, hopefully
> >> with your guys help we can get some Cascading action going and get
> >> the build/release process evolving so no matter how fast you guys
> >> are releasing users can deploy the bits integrated fashion to other
> >> Bigtop components
> >>
> >> -----Original Message-----
> >> From: Evans Ye [mailto:evansye@apache.org]
> >> Sent: Tuesday, May 12, 2015 9:16 AM
> >> To: dev@bigtop.apache.org
> >> Subject: Re: ci infrastructure and release process for Cascading
> >> integration
> >>
> >> Hi Andre
> >>
> >> I take you're not asking technical questions but rather than
> >> wondering something related to the process or policy.
> >>
> >> AFAIK, at this moment we don't have a process to have the
> >> up-to-date packages available.
> >> (The Bigtop CI used to be working, but collapsed when most of the
> >> build slaves went out of available)
> >>
> >> So in your example, when upgrading upstream component version from
> >> 1.1 to 2.0, they would need to submit a patch and build the package by themselves.
> >>
> >> But, thanks to the EMR team sponsoring us EC2 resources, we're now
> >> rebuilding our CI infra, which surely would have nightly
> >> build(RPM/DEB) available.
> >>
> >> OTOH, I can recall that Nate is also working on bigtop repo nightly
> >> build, when that comes online we can always have SNAPSHOT repos
> >> available as well.
> >>
> >> Evans
> >>
> >> 2015-05-12 0:27 GMT+08:00 Andre Kelpe <ak...@concurrentinc.com>:
> >>
> >> > If that is true, what would happen in the case where we want to
> >> > make a new major release available? Suppose we have project in
> >> > 1.x and now as a upstream we release 2.0. Would such a change
> >> > still be covered by this? How would somebody trying to rebuild
> >> > all packages handle these changes? If we submit a package update,
> >> > the bigtop source release will only have the spec files/debian
> >> > directory from the time that a bigtop release was made. Does that still work with the apache rules?
> >> >
> >> > - André
> >> >
> >> > On Mon, May 11, 2015 at 3:52 PM, jay vyas
> >> > <ja...@gmail.com>
> >> > wrote:
> >> >
> >> > > i think -- that -- since bigtop RPM/DEB packages ARE NOT bound
> >> > > by the
> >> > ASF
> >> > > guidelines, they are just binary convenience artifacts.... we
> >> > > probably
> >> > dont
> >> > > have much to worry about :)
> >> > >
> >> > > when this happens id suggest : create a jira, folks will
> >> > > gladly help you to get your latest binaries published so that
> >> > > people can use the latest cascading bits from bigtop yum repos .
> >> > >
> >> > > just my thoughts, i assume this is correct though.
> >> > >
> >> > >
> >> > > On Mon, May 11, 2015 at 7:19 AM, Andre Kelpe
> >> > > <ak...@concurrentinc.com>
> >> > > wrote:
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > the last couple of days I have been investigating the
> >> > > > Cascading
> >> > > integration
> >> > > > for Bigtop. I have navigated the project and created RPM
> >> > > > files for two
> >> > of
> >> > > > our projects. Since we are outside of the ASF, things tend to
> >> > > > be a bit different, but I got it to work. I am currently
> >> > > > doing the same for the debian packages, which should not take too much time.
> >> > > >
> >> > > > Now to my actual questions: Bigtop supports a number of
> >> > > > distributions
> >> > and
> >> > > > has CI infrastructure to build the packages. I am wondering
> >> > > > how we as
> >> > the
> >> > > > Cascading community can work with that infrastructure for
> >> > > > better integration, or if we have to replicate all of it. Let
> >> > > > me know, what
> >> > > could
> >> > > > work best. I'd love to test and build all packages in an easy
> >> > > > way, but going through JIRA with a commiter reviewing and
> >> > > > committing it looks
> >> > > like a
> >> > > > pretty heavy process. OTOH if I replicate the infrastructure
> >> > > > I will end
> >> > > up
> >> > > > having to catch up with all infrastructure changes all the time.
> >> > > >
> >> > > > I am unsure what your way of maintaining things is, once a
> >> > > > Bigtop
> >> > release
> >> > > > is made. I guess you are not targeting a rolling release, but
> >> > > > rather a stable release to be used for a longer time. How do
> >> > > > bugfix releases for packages go in? Suppose we have a
> >> > > > Cascading tool version x.y in Bigtop
> >> > > a.b.
> >> > > > and we make a critical bugfix release x.z, which we want to
> >> > > > distribute
> >> > to
> >> > > > Bigtop users as well. Would that then require a complete
> >> > > > Bigtop release with voting and all that or do you envision a
> >> > > > faster track like linux distros, which update packages for a given release.
> >> > > >
> >> > > > Thanks for your answers!
> >> > > >
> >> > > > - André
> >> > > >
> >> > > > --
> >> > > > André Kelpe
> >> > > > andre@concurrentinc.com
> >> > > > http://concurrentinc.com
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > jay vyas
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > André Kelpe
> >> > andre@concurrentinc.com
> >> > http://concurrentinc.com
> >> >
> >>
> >>
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein (via Tom White)
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein (via Tom White)
Re: ci infrastructure and release process for Cascading integration
Posted by Konstantin Boudnik <co...@apache.org>.
All packages include the build stamp that increases monotonically: that
guarantees the name uniqueness and ability to tell them apart for update
purposes and such.
Cos
On Thu, May 14, 2015 at 04:41PM, Andrew Purtell wrote:
> If we did want to do the below, the next consideration is how we would
> differentiate between artifacts produced by one nightly build versus
> another. Let's say for example we update Hadoop. All components that pull
> in Hadoop as a dependency will produce changed binary artifacts at the next
> build. These need to be versioned differently from earlier built packages.
> I think we would want to introduce a global build identifier (could be
> managed by our CI master), increment the identifier for every nightly, and
> incorporate it as a package version suffix.
>
>
> On Thu, May 14, 2015 at 4:32 PM, Andrew Purtell <ap...@apache.org> wrote:
>
> > The Apache notion of release is wholly scoped to source distributions. The
> > Bigtop source is the build scripting, package files, Puppet files, etc.
> > Generated binary packages are not included.
> >
> > I think we can be limber here by providing nightly -SNAPSHOT versions of
> > the Bigtop source on every day our git tree is updated. As part of the
> > process of pushing out the Bigtop source snapshot we can use CI to kick off
> > binary package builds, then place the resulting convenience binary packages
> > from the snapshot into similarly convenient apt or yum repositories.
> > Therefore on day N, there would be for example Cascading 1.2.3 binary
> > convenience artifacts in the convenience repositories. After check in of
> > Cascading package file updates (and related changes), on day N+1 there
> > would be Cascading 1.2.3 and 1.2.4 binary convenience artifacts in those
> > repos.
> >
> > Periodically we can plant a flag and promote a -SNAPSHOT into the next
> > official release.
> >
> > Then, rinse, and repeat.
> >
> >
> > On Tue, May 12, 2015 at 11:26 PM, <na...@reactor8.com> wrote:
> >
> >> Idea would like to work through with the group is post 1.0, evolving the
> >> CI and project process to be able to do real-time, or just-in-time builds.
> >>
> >> We could have a "bigtop core", which would be components like
> >> Hadoop/hdfs/zookeeper/etc. Thinks like hbase, hive, etc would be next
> >> level components, then things like pig, cascading, etc would be the real
> >> fast movers.
> >>
> >> Think another area we can investigate is components that want to
> >> integrate with bigtop but are not dependent on jvm stack and just the
> >> interfaces/api's of the core components. Seems those as well could be fast
> >> movers
> >>
> >> Don’t know if that answers your question or not Andre, hopefully with
> >> your guys help we can get some Cascading action going and get the
> >> build/release process evolving so no matter how fast you guys are releasing
> >> users can deploy the bits integrated fashion to other Bigtop components
> >>
> >> -----Original Message-----
> >> From: Evans Ye [mailto:evansye@apache.org]
> >> Sent: Tuesday, May 12, 2015 9:16 AM
> >> To: dev@bigtop.apache.org
> >> Subject: Re: ci infrastructure and release process for Cascading
> >> integration
> >>
> >> Hi Andre
> >>
> >> I take you're not asking technical questions but rather than wondering
> >> something related to the process or policy.
> >>
> >> AFAIK, at this moment we don't have a process to have the up-to-date
> >> packages available.
> >> (The Bigtop CI used to be working, but collapsed when most of the build
> >> slaves went out of available)
> >>
> >> So in your example, when upgrading upstream component version from 1.1 to
> >> 2.0, they would need to submit a patch and build the package by themselves.
> >>
> >> But, thanks to the EMR team sponsoring us EC2 resources, we're now
> >> rebuilding our CI infra, which surely would have nightly build(RPM/DEB)
> >> available.
> >>
> >> OTOH, I can recall that Nate is also working on bigtop repo nightly
> >> build, when that comes online we can always have SNAPSHOT repos available
> >> as well.
> >>
> >> Evans
> >>
> >> 2015-05-12 0:27 GMT+08:00 Andre Kelpe <ak...@concurrentinc.com>:
> >>
> >> > If that is true, what would happen in the case where we want to make a
> >> > new major release available? Suppose we have project in 1.x and now as
> >> > a upstream we release 2.0. Would such a change still be covered by
> >> > this? How would somebody trying to rebuild all packages handle these
> >> > changes? If we submit a package update, the bigtop source release will
> >> > only have the spec files/debian directory from the time that a bigtop
> >> > release was made. Does that still work with the apache rules?
> >> >
> >> > - André
> >> >
> >> > On Mon, May 11, 2015 at 3:52 PM, jay vyas
> >> > <ja...@gmail.com>
> >> > wrote:
> >> >
> >> > > i think -- that -- since bigtop RPM/DEB packages ARE NOT bound by
> >> > > the
> >> > ASF
> >> > > guidelines, they are just binary convenience artifacts.... we
> >> > > probably
> >> > dont
> >> > > have much to worry about :)
> >> > >
> >> > > when this happens id suggest : create a jira, folks will gladly
> >> > > help you to get your latest binaries published so that people can
> >> > > use the latest cascading bits from bigtop yum repos .
> >> > >
> >> > > just my thoughts, i assume this is correct though.
> >> > >
> >> > >
> >> > > On Mon, May 11, 2015 at 7:19 AM, Andre Kelpe
> >> > > <ak...@concurrentinc.com>
> >> > > wrote:
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > the last couple of days I have been investigating the Cascading
> >> > > integration
> >> > > > for Bigtop. I have navigated the project and created RPM files for
> >> > > > two
> >> > of
> >> > > > our projects. Since we are outside of the ASF, things tend to be a
> >> > > > bit different, but I got it to work. I am currently doing the same
> >> > > > for the debian packages, which should not take too much time.
> >> > > >
> >> > > > Now to my actual questions: Bigtop supports a number of
> >> > > > distributions
> >> > and
> >> > > > has CI infrastructure to build the packages. I am wondering how we
> >> > > > as
> >> > the
> >> > > > Cascading community can work with that infrastructure for better
> >> > > > integration, or if we have to replicate all of it. Let me know,
> >> > > > what
> >> > > could
> >> > > > work best. I'd love to test and build all packages in an easy way,
> >> > > > but going through JIRA with a commiter reviewing and committing it
> >> > > > looks
> >> > > like a
> >> > > > pretty heavy process. OTOH if I replicate the infrastructure I
> >> > > > will end
> >> > > up
> >> > > > having to catch up with all infrastructure changes all the time.
> >> > > >
> >> > > > I am unsure what your way of maintaining things is, once a Bigtop
> >> > release
> >> > > > is made. I guess you are not targeting a rolling release, but
> >> > > > rather a stable release to be used for a longer time. How do
> >> > > > bugfix releases for packages go in? Suppose we have a Cascading
> >> > > > tool version x.y in Bigtop
> >> > > a.b.
> >> > > > and we make a critical bugfix release x.z, which we want to
> >> > > > distribute
> >> > to
> >> > > > Bigtop users as well. Would that then require a complete Bigtop
> >> > > > release with voting and all that or do you envision a faster track
> >> > > > like linux distros, which update packages for a given release.
> >> > > >
> >> > > > Thanks for your answers!
> >> > > >
> >> > > > - André
> >> > > >
> >> > > > --
> >> > > > André Kelpe
> >> > > > andre@concurrentinc.com
> >> > > > http://concurrentinc.com
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > jay vyas
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > André Kelpe
> >> > andre@concurrentinc.com
> >> > http://concurrentinc.com
> >> >
> >>
> >>
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
Re: ci infrastructure and release process for Cascading integration
Posted by Andrew Purtell <ap...@apache.org>.
If we did want to do the below, the next consideration is how we would
differentiate between artifacts produced by one nightly build versus
another. Let's say for example we update Hadoop. All components that pull
in Hadoop as a dependency will produce changed binary artifacts at the next
build. These need to be versioned differently from earlier built packages.
I think we would want to introduce a global build identifier (could be
managed by our CI master), increment the identifier for every nightly, and
incorporate it as a package version suffix.
On Thu, May 14, 2015 at 4:32 PM, Andrew Purtell <ap...@apache.org> wrote:
> The Apache notion of release is wholly scoped to source distributions. The
> Bigtop source is the build scripting, package files, Puppet files, etc.
> Generated binary packages are not included.
>
> I think we can be limber here by providing nightly -SNAPSHOT versions of
> the Bigtop source on every day our git tree is updated. As part of the
> process of pushing out the Bigtop source snapshot we can use CI to kick off
> binary package builds, then place the resulting convenience binary packages
> from the snapshot into similarly convenient apt or yum repositories.
> Therefore on day N, there would be for example Cascading 1.2.3 binary
> convenience artifacts in the convenience repositories. After check in of
> Cascading package file updates (and related changes), on day N+1 there
> would be Cascading 1.2.3 and 1.2.4 binary convenience artifacts in those
> repos.
>
> Periodically we can plant a flag and promote a -SNAPSHOT into the next
> official release.
>
> Then, rinse, and repeat.
>
>
> On Tue, May 12, 2015 at 11:26 PM, <na...@reactor8.com> wrote:
>
>> Idea would like to work through with the group is post 1.0, evolving the
>> CI and project process to be able to do real-time, or just-in-time builds.
>>
>> We could have a "bigtop core", which would be components like
>> Hadoop/hdfs/zookeeper/etc. Thinks like hbase, hive, etc would be next
>> level components, then things like pig, cascading, etc would be the real
>> fast movers.
>>
>> Think another area we can investigate is components that want to
>> integrate with bigtop but are not dependent on jvm stack and just the
>> interfaces/api's of the core components. Seems those as well could be fast
>> movers
>>
>> Don’t know if that answers your question or not Andre, hopefully with
>> your guys help we can get some Cascading action going and get the
>> build/release process evolving so no matter how fast you guys are releasing
>> users can deploy the bits integrated fashion to other Bigtop components
>>
>> -----Original Message-----
>> From: Evans Ye [mailto:evansye@apache.org]
>> Sent: Tuesday, May 12, 2015 9:16 AM
>> To: dev@bigtop.apache.org
>> Subject: Re: ci infrastructure and release process for Cascading
>> integration
>>
>> Hi Andre
>>
>> I take you're not asking technical questions but rather than wondering
>> something related to the process or policy.
>>
>> AFAIK, at this moment we don't have a process to have the up-to-date
>> packages available.
>> (The Bigtop CI used to be working, but collapsed when most of the build
>> slaves went out of available)
>>
>> So in your example, when upgrading upstream component version from 1.1 to
>> 2.0, they would need to submit a patch and build the package by themselves.
>>
>> But, thanks to the EMR team sponsoring us EC2 resources, we're now
>> rebuilding our CI infra, which surely would have nightly build(RPM/DEB)
>> available.
>>
>> OTOH, I can recall that Nate is also working on bigtop repo nightly
>> build, when that comes online we can always have SNAPSHOT repos available
>> as well.
>>
>> Evans
>>
>> 2015-05-12 0:27 GMT+08:00 Andre Kelpe <ak...@concurrentinc.com>:
>>
>> > If that is true, what would happen in the case where we want to make a
>> > new major release available? Suppose we have project in 1.x and now as
>> > a upstream we release 2.0. Would such a change still be covered by
>> > this? How would somebody trying to rebuild all packages handle these
>> > changes? If we submit a package update, the bigtop source release will
>> > only have the spec files/debian directory from the time that a bigtop
>> > release was made. Does that still work with the apache rules?
>> >
>> > - André
>> >
>> > On Mon, May 11, 2015 at 3:52 PM, jay vyas
>> > <ja...@gmail.com>
>> > wrote:
>> >
>> > > i think -- that -- since bigtop RPM/DEB packages ARE NOT bound by
>> > > the
>> > ASF
>> > > guidelines, they are just binary convenience artifacts.... we
>> > > probably
>> > dont
>> > > have much to worry about :)
>> > >
>> > > when this happens id suggest : create a jira, folks will gladly
>> > > help you to get your latest binaries published so that people can
>> > > use the latest cascading bits from bigtop yum repos .
>> > >
>> > > just my thoughts, i assume this is correct though.
>> > >
>> > >
>> > > On Mon, May 11, 2015 at 7:19 AM, Andre Kelpe
>> > > <ak...@concurrentinc.com>
>> > > wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > the last couple of days I have been investigating the Cascading
>> > > integration
>> > > > for Bigtop. I have navigated the project and created RPM files for
>> > > > two
>> > of
>> > > > our projects. Since we are outside of the ASF, things tend to be a
>> > > > bit different, but I got it to work. I am currently doing the same
>> > > > for the debian packages, which should not take too much time.
>> > > >
>> > > > Now to my actual questions: Bigtop supports a number of
>> > > > distributions
>> > and
>> > > > has CI infrastructure to build the packages. I am wondering how we
>> > > > as
>> > the
>> > > > Cascading community can work with that infrastructure for better
>> > > > integration, or if we have to replicate all of it. Let me know,
>> > > > what
>> > > could
>> > > > work best. I'd love to test and build all packages in an easy way,
>> > > > but going through JIRA with a commiter reviewing and committing it
>> > > > looks
>> > > like a
>> > > > pretty heavy process. OTOH if I replicate the infrastructure I
>> > > > will end
>> > > up
>> > > > having to catch up with all infrastructure changes all the time.
>> > > >
>> > > > I am unsure what your way of maintaining things is, once a Bigtop
>> > release
>> > > > is made. I guess you are not targeting a rolling release, but
>> > > > rather a stable release to be used for a longer time. How do
>> > > > bugfix releases for packages go in? Suppose we have a Cascading
>> > > > tool version x.y in Bigtop
>> > > a.b.
>> > > > and we make a critical bugfix release x.z, which we want to
>> > > > distribute
>> > to
>> > > > Bigtop users as well. Would that then require a complete Bigtop
>> > > > release with voting and all that or do you envision a faster track
>> > > > like linux distros, which update packages for a given release.
>> > > >
>> > > > Thanks for your answers!
>> > > >
>> > > > - André
>> > > >
>> > > > --
>> > > > André Kelpe
>> > > > andre@concurrentinc.com
>> > > > http://concurrentinc.com
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > jay vyas
>> > >
>> >
>> >
>> >
>> > --
>> > André Kelpe
>> > andre@concurrentinc.com
>> > http://concurrentinc.com
>> >
>>
>>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: ci infrastructure and release process for Cascading integration
Posted by Andrew Purtell <ap...@apache.org>.
The Apache notion of release is wholly scoped to source distributions. The
Bigtop source is the build scripting, package files, Puppet files, etc.
Generated binary packages are not included.
I think we can be limber here by providing nightly -SNAPSHOT versions of
the Bigtop source on every day our git tree is updated. As part of the
process of pushing out the Bigtop source snapshot we can use CI to kick off
binary package builds, then place the resulting convenience binary packages
from the snapshot into similarly convenient apt or yum repositories.
Therefore on day N, there would be for example Cascading 1.2.3 binary
convenience artifacts in the convenience repositories. After check in of
Cascading package file updates (and related changes), on day N+1 there
would be Cascading 1.2.3 and 1.2.4 binary convenience artifacts in those
repos.
Periodically we can plant a flag and promote a -SNAPSHOT into the next
official release.
Then, rinse, and repeat.
On Tue, May 12, 2015 at 11:26 PM, <na...@reactor8.com> wrote:
> Idea would like to work through with the group is post 1.0, evolving the
> CI and project process to be able to do real-time, or just-in-time builds.
>
> We could have a "bigtop core", which would be components like
> Hadoop/hdfs/zookeeper/etc. Thinks like hbase, hive, etc would be next
> level components, then things like pig, cascading, etc would be the real
> fast movers.
>
> Think another area we can investigate is components that want to integrate
> with bigtop but are not dependent on jvm stack and just the
> interfaces/api's of the core components. Seems those as well could be fast
> movers
>
> Don’t know if that answers your question or not Andre, hopefully with your
> guys help we can get some Cascading action going and get the build/release
> process evolving so no matter how fast you guys are releasing users can
> deploy the bits integrated fashion to other Bigtop components
>
> -----Original Message-----
> From: Evans Ye [mailto:evansye@apache.org]
> Sent: Tuesday, May 12, 2015 9:16 AM
> To: dev@bigtop.apache.org
> Subject: Re: ci infrastructure and release process for Cascading
> integration
>
> Hi Andre
>
> I take you're not asking technical questions but rather than wondering
> something related to the process or policy.
>
> AFAIK, at this moment we don't have a process to have the up-to-date
> packages available.
> (The Bigtop CI used to be working, but collapsed when most of the build
> slaves went out of available)
>
> So in your example, when upgrading upstream component version from 1.1 to
> 2.0, they would need to submit a patch and build the package by themselves.
>
> But, thanks to the EMR team sponsoring us EC2 resources, we're now
> rebuilding our CI infra, which surely would have nightly build(RPM/DEB)
> available.
>
> OTOH, I can recall that Nate is also working on bigtop repo nightly build,
> when that comes online we can always have SNAPSHOT repos available as well.
>
> Evans
>
> 2015-05-12 0:27 GMT+08:00 Andre Kelpe <ak...@concurrentinc.com>:
>
> > If that is true, what would happen in the case where we want to make a
> > new major release available? Suppose we have project in 1.x and now as
> > a upstream we release 2.0. Would such a change still be covered by
> > this? How would somebody trying to rebuild all packages handle these
> > changes? If we submit a package update, the bigtop source release will
> > only have the spec files/debian directory from the time that a bigtop
> > release was made. Does that still work with the apache rules?
> >
> > - André
> >
> > On Mon, May 11, 2015 at 3:52 PM, jay vyas
> > <ja...@gmail.com>
> > wrote:
> >
> > > i think -- that -- since bigtop RPM/DEB packages ARE NOT bound by
> > > the
> > ASF
> > > guidelines, they are just binary convenience artifacts.... we
> > > probably
> > dont
> > > have much to worry about :)
> > >
> > > when this happens id suggest : create a jira, folks will gladly
> > > help you to get your latest binaries published so that people can
> > > use the latest cascading bits from bigtop yum repos .
> > >
> > > just my thoughts, i assume this is correct though.
> > >
> > >
> > > On Mon, May 11, 2015 at 7:19 AM, Andre Kelpe
> > > <ak...@concurrentinc.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > the last couple of days I have been investigating the Cascading
> > > integration
> > > > for Bigtop. I have navigated the project and created RPM files for
> > > > two
> > of
> > > > our projects. Since we are outside of the ASF, things tend to be a
> > > > bit different, but I got it to work. I am currently doing the same
> > > > for the debian packages, which should not take too much time.
> > > >
> > > > Now to my actual questions: Bigtop supports a number of
> > > > distributions
> > and
> > > > has CI infrastructure to build the packages. I am wondering how we
> > > > as
> > the
> > > > Cascading community can work with that infrastructure for better
> > > > integration, or if we have to replicate all of it. Let me know,
> > > > what
> > > could
> > > > work best. I'd love to test and build all packages in an easy way,
> > > > but going through JIRA with a commiter reviewing and committing it
> > > > looks
> > > like a
> > > > pretty heavy process. OTOH if I replicate the infrastructure I
> > > > will end
> > > up
> > > > having to catch up with all infrastructure changes all the time.
> > > >
> > > > I am unsure what your way of maintaining things is, once a Bigtop
> > release
> > > > is made. I guess you are not targeting a rolling release, but
> > > > rather a stable release to be used for a longer time. How do
> > > > bugfix releases for packages go in? Suppose we have a Cascading
> > > > tool version x.y in Bigtop
> > > a.b.
> > > > and we make a critical bugfix release x.z, which we want to
> > > > distribute
> > to
> > > > Bigtop users as well. Would that then require a complete Bigtop
> > > > release with voting and all that or do you envision a faster track
> > > > like linux distros, which update packages for a given release.
> > > >
> > > > Thanks for your answers!
> > > >
> > > > - André
> > > >
> > > > --
> > > > André Kelpe
> > > > andre@concurrentinc.com
> > > > http://concurrentinc.com
> > > >
> > >
> > >
> > >
> > > --
> > > jay vyas
> > >
> >
> >
> >
> > --
> > André Kelpe
> > andre@concurrentinc.com
> > http://concurrentinc.com
> >
>
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
RE: ci infrastructure and release process for Cascading integration
Posted by na...@reactor8.com.
Idea would like to work through with the group is post 1.0, evolving the CI and project process to be able to do real-time, or just-in-time builds.
We could have a "bigtop core", which would be components like Hadoop/hdfs/zookeeper/etc. Thinks like hbase, hive, etc would be next level components, then things like pig, cascading, etc would be the real fast movers.
Think another area we can investigate is components that want to integrate with bigtop but are not dependent on jvm stack and just the interfaces/api's of the core components. Seems those as well could be fast movers
Don’t know if that answers your question or not Andre, hopefully with your guys help we can get some Cascading action going and get the build/release process evolving so no matter how fast you guys are releasing users can deploy the bits integrated fashion to other Bigtop components
-----Original Message-----
From: Evans Ye [mailto:evansye@apache.org]
Sent: Tuesday, May 12, 2015 9:16 AM
To: dev@bigtop.apache.org
Subject: Re: ci infrastructure and release process for Cascading integration
Hi Andre
I take you're not asking technical questions but rather than wondering something related to the process or policy.
AFAIK, at this moment we don't have a process to have the up-to-date packages available.
(The Bigtop CI used to be working, but collapsed when most of the build slaves went out of available)
So in your example, when upgrading upstream component version from 1.1 to 2.0, they would need to submit a patch and build the package by themselves.
But, thanks to the EMR team sponsoring us EC2 resources, we're now rebuilding our CI infra, which surely would have nightly build(RPM/DEB) available.
OTOH, I can recall that Nate is also working on bigtop repo nightly build, when that comes online we can always have SNAPSHOT repos available as well.
Evans
2015-05-12 0:27 GMT+08:00 Andre Kelpe <ak...@concurrentinc.com>:
> If that is true, what would happen in the case where we want to make a
> new major release available? Suppose we have project in 1.x and now as
> a upstream we release 2.0. Would such a change still be covered by
> this? How would somebody trying to rebuild all packages handle these
> changes? If we submit a package update, the bigtop source release will
> only have the spec files/debian directory from the time that a bigtop
> release was made. Does that still work with the apache rules?
>
> - André
>
> On Mon, May 11, 2015 at 3:52 PM, jay vyas
> <ja...@gmail.com>
> wrote:
>
> > i think -- that -- since bigtop RPM/DEB packages ARE NOT bound by
> > the
> ASF
> > guidelines, they are just binary convenience artifacts.... we
> > probably
> dont
> > have much to worry about :)
> >
> > when this happens id suggest : create a jira, folks will gladly
> > help you to get your latest binaries published so that people can
> > use the latest cascading bits from bigtop yum repos .
> >
> > just my thoughts, i assume this is correct though.
> >
> >
> > On Mon, May 11, 2015 at 7:19 AM, Andre Kelpe
> > <ak...@concurrentinc.com>
> > wrote:
> >
> > > Hi,
> > >
> > > the last couple of days I have been investigating the Cascading
> > integration
> > > for Bigtop. I have navigated the project and created RPM files for
> > > two
> of
> > > our projects. Since we are outside of the ASF, things tend to be a
> > > bit different, but I got it to work. I am currently doing the same
> > > for the debian packages, which should not take too much time.
> > >
> > > Now to my actual questions: Bigtop supports a number of
> > > distributions
> and
> > > has CI infrastructure to build the packages. I am wondering how we
> > > as
> the
> > > Cascading community can work with that infrastructure for better
> > > integration, or if we have to replicate all of it. Let me know,
> > > what
> > could
> > > work best. I'd love to test and build all packages in an easy way,
> > > but going through JIRA with a commiter reviewing and committing it
> > > looks
> > like a
> > > pretty heavy process. OTOH if I replicate the infrastructure I
> > > will end
> > up
> > > having to catch up with all infrastructure changes all the time.
> > >
> > > I am unsure what your way of maintaining things is, once a Bigtop
> release
> > > is made. I guess you are not targeting a rolling release, but
> > > rather a stable release to be used for a longer time. How do
> > > bugfix releases for packages go in? Suppose we have a Cascading
> > > tool version x.y in Bigtop
> > a.b.
> > > and we make a critical bugfix release x.z, which we want to
> > > distribute
> to
> > > Bigtop users as well. Would that then require a complete Bigtop
> > > release with voting and all that or do you envision a faster track
> > > like linux distros, which update packages for a given release.
> > >
> > > Thanks for your answers!
> > >
> > > - André
> > >
> > > --
> > > André Kelpe
> > > andre@concurrentinc.com
> > > http://concurrentinc.com
> > >
> >
> >
> >
> > --
> > jay vyas
> >
>
>
>
> --
> André Kelpe
> andre@concurrentinc.com
> http://concurrentinc.com
>
Re: ci infrastructure and release process for Cascading integration
Posted by Evans Ye <ev...@apache.org>.
Hi Andre
I take you're not asking technical questions but rather than wondering
something related to the process or policy.
AFAIK, at this moment we don't have a process to have the up-to-date
packages available.
(The Bigtop CI used to be working, but collapsed when most of the build
slaves went out of available)
So in your example, when upgrading upstream component version from 1.1 to
2.0, they would need to submit a patch and build the package by themselves.
But, thanks to the EMR team sponsoring us EC2 resources, we're now
rebuilding our CI infra, which surely would have nightly build(RPM/DEB)
available.
OTOH, I can recall that Nate is also working on bigtop repo nightly build,
when that comes online we can always have SNAPSHOT repos available as well.
Evans
2015-05-12 0:27 GMT+08:00 Andre Kelpe <ak...@concurrentinc.com>:
> If that is true, what would happen in the case where we want to make a new
> major release available? Suppose we have project in 1.x and now as a
> upstream we release 2.0. Would such a change still be covered by this? How
> would somebody trying to rebuild all packages handle these changes? If we
> submit a package update, the bigtop source release will only have the spec
> files/debian directory from the time that a bigtop release was made. Does
> that still work with the apache rules?
>
> - André
>
> On Mon, May 11, 2015 at 3:52 PM, jay vyas <ja...@gmail.com>
> wrote:
>
> > i think -- that -- since bigtop RPM/DEB packages ARE NOT bound by the
> ASF
> > guidelines, they are just binary convenience artifacts.... we probably
> dont
> > have much to worry about :)
> >
> > when this happens id suggest : create a jira, folks will gladly help you
> > to get your latest binaries published so that people can use the latest
> > cascading bits from bigtop yum repos .
> >
> > just my thoughts, i assume this is correct though.
> >
> >
> > On Mon, May 11, 2015 at 7:19 AM, Andre Kelpe <ak...@concurrentinc.com>
> > wrote:
> >
> > > Hi,
> > >
> > > the last couple of days I have been investigating the Cascading
> > integration
> > > for Bigtop. I have navigated the project and created RPM files for two
> of
> > > our projects. Since we are outside of the ASF, things tend to be a bit
> > > different, but I got it to work. I am currently doing the same for the
> > > debian packages, which should not take too much time.
> > >
> > > Now to my actual questions: Bigtop supports a number of distributions
> and
> > > has CI infrastructure to build the packages. I am wondering how we as
> the
> > > Cascading community can work with that infrastructure for better
> > > integration, or if we have to replicate all of it. Let me know, what
> > could
> > > work best. I'd love to test and build all packages in an easy way, but
> > > going through JIRA with a commiter reviewing and committing it looks
> > like a
> > > pretty heavy process. OTOH if I replicate the infrastructure I will end
> > up
> > > having to catch up with all infrastructure changes all the time.
> > >
> > > I am unsure what your way of maintaining things is, once a Bigtop
> release
> > > is made. I guess you are not targeting a rolling release, but rather a
> > > stable release to be used for a longer time. How do bugfix releases for
> > > packages go in? Suppose we have a Cascading tool version x.y in Bigtop
> > a.b.
> > > and we make a critical bugfix release x.z, which we want to distribute
> to
> > > Bigtop users as well. Would that then require a complete Bigtop release
> > > with voting and all that or do you envision a faster track like linux
> > > distros, which update packages for a given release.
> > >
> > > Thanks for your answers!
> > >
> > > - André
> > >
> > > --
> > > André Kelpe
> > > andre@concurrentinc.com
> > > http://concurrentinc.com
> > >
> >
> >
> >
> > --
> > jay vyas
> >
>
>
>
> --
> André Kelpe
> andre@concurrentinc.com
> http://concurrentinc.com
>
Re: ci infrastructure and release process for Cascading integration
Posted by Andre Kelpe <ak...@concurrentinc.com>.
If that is true, what would happen in the case where we want to make a new
major release available? Suppose we have project in 1.x and now as a
upstream we release 2.0. Would such a change still be covered by this? How
would somebody trying to rebuild all packages handle these changes? If we
submit a package update, the bigtop source release will only have the spec
files/debian directory from the time that a bigtop release was made. Does
that still work with the apache rules?
- André
On Mon, May 11, 2015 at 3:52 PM, jay vyas <ja...@gmail.com>
wrote:
> i think -- that -- since bigtop RPM/DEB packages ARE NOT bound by the ASF
> guidelines, they are just binary convenience artifacts.... we probably dont
> have much to worry about :)
>
> when this happens id suggest : create a jira, folks will gladly help you
> to get your latest binaries published so that people can use the latest
> cascading bits from bigtop yum repos .
>
> just my thoughts, i assume this is correct though.
>
>
> On Mon, May 11, 2015 at 7:19 AM, Andre Kelpe <ak...@concurrentinc.com>
> wrote:
>
> > Hi,
> >
> > the last couple of days I have been investigating the Cascading
> integration
> > for Bigtop. I have navigated the project and created RPM files for two of
> > our projects. Since we are outside of the ASF, things tend to be a bit
> > different, but I got it to work. I am currently doing the same for the
> > debian packages, which should not take too much time.
> >
> > Now to my actual questions: Bigtop supports a number of distributions and
> > has CI infrastructure to build the packages. I am wondering how we as the
> > Cascading community can work with that infrastructure for better
> > integration, or if we have to replicate all of it. Let me know, what
> could
> > work best. I'd love to test and build all packages in an easy way, but
> > going through JIRA with a commiter reviewing and committing it looks
> like a
> > pretty heavy process. OTOH if I replicate the infrastructure I will end
> up
> > having to catch up with all infrastructure changes all the time.
> >
> > I am unsure what your way of maintaining things is, once a Bigtop release
> > is made. I guess you are not targeting a rolling release, but rather a
> > stable release to be used for a longer time. How do bugfix releases for
> > packages go in? Suppose we have a Cascading tool version x.y in Bigtop
> a.b.
> > and we make a critical bugfix release x.z, which we want to distribute to
> > Bigtop users as well. Would that then require a complete Bigtop release
> > with voting and all that or do you envision a faster track like linux
> > distros, which update packages for a given release.
> >
> > Thanks for your answers!
> >
> > - André
> >
> > --
> > André Kelpe
> > andre@concurrentinc.com
> > http://concurrentinc.com
> >
>
>
>
> --
> jay vyas
>
--
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com
Re: ci infrastructure and release process for Cascading integration
Posted by jay vyas <ja...@gmail.com>.
i think -- that -- since bigtop RPM/DEB packages ARE NOT bound by the ASF
guidelines, they are just binary convenience artifacts.... we probably dont
have much to worry about :)
when this happens id suggest : create a jira, folks will gladly help you
to get your latest binaries published so that people can use the latest
cascading bits from bigtop yum repos .
just my thoughts, i assume this is correct though.
On Mon, May 11, 2015 at 7:19 AM, Andre Kelpe <ak...@concurrentinc.com>
wrote:
> Hi,
>
> the last couple of days I have been investigating the Cascading integration
> for Bigtop. I have navigated the project and created RPM files for two of
> our projects. Since we are outside of the ASF, things tend to be a bit
> different, but I got it to work. I am currently doing the same for the
> debian packages, which should not take too much time.
>
> Now to my actual questions: Bigtop supports a number of distributions and
> has CI infrastructure to build the packages. I am wondering how we as the
> Cascading community can work with that infrastructure for better
> integration, or if we have to replicate all of it. Let me know, what could
> work best. I'd love to test and build all packages in an easy way, but
> going through JIRA with a commiter reviewing and committing it looks like a
> pretty heavy process. OTOH if I replicate the infrastructure I will end up
> having to catch up with all infrastructure changes all the time.
>
> I am unsure what your way of maintaining things is, once a Bigtop release
> is made. I guess you are not targeting a rolling release, but rather a
> stable release to be used for a longer time. How do bugfix releases for
> packages go in? Suppose we have a Cascading tool version x.y in Bigtop a.b.
> and we make a critical bugfix release x.z, which we want to distribute to
> Bigtop users as well. Would that then require a complete Bigtop release
> with voting and all that or do you envision a faster track like linux
> distros, which update packages for a given release.
>
> Thanks for your answers!
>
> - André
>
> --
> André Kelpe
> andre@concurrentinc.com
> http://concurrentinc.com
>
--
jay vyas