You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@toree.apache.org by Chip Senkbeil <ch...@gmail.com> on 2016/01/08 18:54:01 UTC
Continuous Integration Tooling
I know that Apache Spark uses Jenkins with a plugin in Github to allow them
to manually invoke Jenkins testing against PRs.
Do we want something similar (or do we have it already) for Toree?
Currently, we use Travis CI, which is free for open source projects. I
don't know off the top of my head if Travis allows you to manually invoke
builds/tests againsts PRs.
Is there something specific that Apache requires us to use? Does anyone
have a preference in how we handle PRs?
Also, I was reading over (https://www.apache.org/dev/release.html). There
are multiple release types including nightly builds. While we don't
advertise nightly builds on the main site, it would be useful to have
nightly builds for those interested in trying bug fixes, getting new
features, etc. This is especially important considering we currently ask
all users to build from source; so, moving to a release model changes our
flow a little bit.
Is there available Apache tooling for nightly builds? Do we need to set
something up ourselves?
Re: Continuous Integration Tooling
Posted by Chip Senkbeil <ch...@gmail.com>.
Opened a new JIRA regarding gaining access to enable Travis CI and the
associated webhook. Luciano Resende, please check out this issue and
include anything I missed or correct anything that I got incorrect.
https://issues.apache.org/jira/browse/INFRA-11134
On Mon, Jan 25, 2016 at 1:29 PM Gino Bustelo <gi...@bustelos.com> wrote:
> We need to be able to get to the settings of the github mirror and add it
> as a Service. I do not have permissions to go to Settings of
> https://github.com/apache/incubator-toree. Also, we would need a user-id
> to
> Travis and Key. From
> https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci, it
> seems that it is a shared apache account.
>
> On Mon, Jan 25, 2016 at 11:24 AM, Luciano Resende <lu...@gmail.com>
> wrote:
>
> > On Mon, Jan 25, 2016 at 9:18 AM, Chip Senkbeil <ch...@gmail.com>
> > wrote:
> >
> > > So, I'm trying to set up our incubator project to use Travis CI. I'm
> > basing
> > > it on the slides I found here:
> > >
> > >
> >
> http://www.slideshare.net/jukka/apache-development-with-github-and-travis-ci
> > >
> > > Looks like I start by adding myself to the Github team:
> > > https://svn.apache.org/repos/private/committers/docs/github_team.txt
> > >
> > > I checked Apache Parquet as a reference, but I don't see anything
> special
> > > in their travis.yml file:
> > > https://github.com/apache/parquet-mr/blob/master/.travis.yml
> > >
> > > I'm assuming this gives me access to Travis CI to enable our incubator
> > > project, but I wanted to double-check with the mentors here. Can anyone
> > > confirm that this is the way to enable our project on Travis CI? Is
> there
> > > something else that needs to be done?\
> > >
> >
> >
> > Because others are already using TravisCI, I believe that the Apache
> github
> > organization already trusts TravisCI so I assume nothing else on the ACL
> > side needs to be done, and we just need to configure our yml file. If you
> > are having any specific issues, please contact infrastructure mailing
> list
> > for help.
> >
> >
> > --
> > Luciano Resende
> > http://people.apache.org/~lresende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
> >
>
Re: Continuous Integration Tooling
Posted by Gino Bustelo <gi...@bustelos.com>.
We need to be able to get to the settings of the github mirror and add it
as a Service. I do not have permissions to go to Settings of
https://github.com/apache/incubator-toree. Also, we would need a user-id to
Travis and Key. From
https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci, it
seems that it is a shared apache account.
On Mon, Jan 25, 2016 at 11:24 AM, Luciano Resende <lu...@gmail.com>
wrote:
> On Mon, Jan 25, 2016 at 9:18 AM, Chip Senkbeil <ch...@gmail.com>
> wrote:
>
> > So, I'm trying to set up our incubator project to use Travis CI. I'm
> basing
> > it on the slides I found here:
> >
> >
> http://www.slideshare.net/jukka/apache-development-with-github-and-travis-ci
> >
> > Looks like I start by adding myself to the Github team:
> > https://svn.apache.org/repos/private/committers/docs/github_team.txt
> >
> > I checked Apache Parquet as a reference, but I don't see anything special
> > in their travis.yml file:
> > https://github.com/apache/parquet-mr/blob/master/.travis.yml
> >
> > I'm assuming this gives me access to Travis CI to enable our incubator
> > project, but I wanted to double-check with the mentors here. Can anyone
> > confirm that this is the way to enable our project on Travis CI? Is there
> > something else that needs to be done?\
> >
>
>
> Because others are already using TravisCI, I believe that the Apache github
> organization already trusts TravisCI so I assume nothing else on the ACL
> side needs to be done, and we just need to configure our yml file. If you
> are having any specific issues, please contact infrastructure mailing list
> for help.
>
>
> --
> Luciano Resende
> http://people.apache.org/~lresende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>
Re: Continuous Integration Tooling
Posted by Luciano Resende <lu...@gmail.com>.
On Mon, Jan 25, 2016 at 9:18 AM, Chip Senkbeil <ch...@gmail.com>
wrote:
> So, I'm trying to set up our incubator project to use Travis CI. I'm basing
> it on the slides I found here:
>
> http://www.slideshare.net/jukka/apache-development-with-github-and-travis-ci
>
> Looks like I start by adding myself to the Github team:
> https://svn.apache.org/repos/private/committers/docs/github_team.txt
>
> I checked Apache Parquet as a reference, but I don't see anything special
> in their travis.yml file:
> https://github.com/apache/parquet-mr/blob/master/.travis.yml
>
> I'm assuming this gives me access to Travis CI to enable our incubator
> project, but I wanted to double-check with the mentors here. Can anyone
> confirm that this is the way to enable our project on Travis CI? Is there
> something else that needs to be done?\
>
Because others are already using TravisCI, I believe that the Apache github
organization already trusts TravisCI so I assume nothing else on the ACL
side needs to be done, and we just need to configure our yml file. If you
are having any specific issues, please contact infrastructure mailing list
for help.
--
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/
Re: Continuous Integration Tooling
Posted by Chip Senkbeil <ch...@gmail.com>.
So, I'm trying to set up our incubator project to use Travis CI. I'm basing
it on the slides I found here:
http://www.slideshare.net/jukka/apache-development-with-github-and-travis-ci
Looks like I start by adding myself to the Github team:
https://svn.apache.org/repos/private/committers/docs/github_team.txt
I checked Apache Parquet as a reference, but I don't see anything special
in their travis.yml file:
https://github.com/apache/parquet-mr/blob/master/.travis.yml
I'm assuming this gives me access to Travis CI to enable our incubator
project, but I wanted to double-check with the mentors here. Can anyone
confirm that this is the way to enable our project on Travis CI? Is there
something else that needs to be done?
On Mon, Jan 11, 2016 at 12:19 PM Luciano Resende <lu...@gmail.com>
wrote:
> TravisCI or Jenkins on apache infrastructure is fine.
>
> You might have more flexibility configuring integration if you use Apache
> infrastructure, but you can make the hard decision when you start getting
> blocked on infra related issues.
>
> On Fri, Jan 8, 2016 at 3:42 PM, Julien Le Dem <ju...@ledem.net> wrote:
>
> > FYI, Apache Parquet uses github and the travis-ci integration.
> > When moving to apache I did notice a slower turnaround on the travis
> > builds.
> > Possibly because there are a lot of projects under the apache github
> > account.
> > The main difference is you can not use the PR merge button in github.
> > (as the apache github is read only replicated from the apache git repo.)
> >
> > > On Jan 8, 2016, at 2:45 PM, Chip Senkbeil <ch...@gmail.com>
> > wrote:
> > >
> > > From there, I see there is better integration with Github that can be
> > used
> > > if you opt-in:
> > >
> >
> https://blogs.apache.org/infra/entry/improved_integration_between_apache_and
> > >
> > > On Fri, Jan 8, 2016 at 4:15 PM Gino Bustelo <gi...@bustelos.com> wrote:
> > >
> > >> Based on
> > >>
> https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci,
> > it
> > >> seems that we could continue to use travisci. I honestly prefer Travis
> > >> since the configuration is all contained within our project.
> > >>
> > >> On Fri, Jan 8, 2016 at 3:55 PM, Reynold Xin <rx...@databricks.com>
> > wrote:
> > >>
> > >>> ASF does have a Jenkins infrastructure. You can use that as an
> option.
> > >>>
> > >>> https://builds.apache.org/
> > >>>
> > >>> On Fri, Jan 8, 2016 at 9:54 AM, Chip Senkbeil <
> chip.senkbeil@gmail.com
> > >
> > >>> wrote:
> > >>>
> > >>>> I know that Apache Spark uses Jenkins with a plugin in Github to
> allow
> > >>> them
> > >>>> to manually invoke Jenkins testing against PRs.
> > >>>>
> > >>>> Do we want something similar (or do we have it already) for Toree?
> > >>>> Currently, we use Travis CI, which is free for open source
> projects. I
> > >>>> don't know off the top of my head if Travis allows you to manually
> > >> invoke
> > >>>> builds/tests againsts PRs.
> > >>>>
> > >>>> Is there something specific that Apache requires us to use? Does
> > anyone
> > >>>> have a preference in how we handle PRs?
> > >>>>
> > >>>> Also, I was reading over (https://www.apache.org/dev/release.html).
> > >>> There
> > >>>> are multiple release types including nightly builds. While we don't
> > >>>> advertise nightly builds on the main site, it would be useful to
> have
> > >>>> nightly builds for those interested in trying bug fixes, getting new
> > >>>> features, etc. This is especially important considering we currently
> > >> ask
> > >>>> all users to build from source; so, moving to a release model
> changes
> > >> our
> > >>>> flow a little bit.
> > >>>>
> > >>>> Is there available Apache tooling for nightly builds? Do we need to
> > set
> > >>>> something up ourselves?
> > >>>>
> > >>>
> > >>
> >
> >
>
>
> --
> Luciano Resende
> http://people.apache.org/~lresende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>
Re: Continuous Integration Tooling
Posted by Luciano Resende <lu...@gmail.com>.
TravisCI or Jenkins on apache infrastructure is fine.
You might have more flexibility configuring integration if you use Apache
infrastructure, but you can make the hard decision when you start getting
blocked on infra related issues.
On Fri, Jan 8, 2016 at 3:42 PM, Julien Le Dem <ju...@ledem.net> wrote:
> FYI, Apache Parquet uses github and the travis-ci integration.
> When moving to apache I did notice a slower turnaround on the travis
> builds.
> Possibly because there are a lot of projects under the apache github
> account.
> The main difference is you can not use the PR merge button in github.
> (as the apache github is read only replicated from the apache git repo.)
>
> > On Jan 8, 2016, at 2:45 PM, Chip Senkbeil <ch...@gmail.com>
> wrote:
> >
> > From there, I see there is better integration with Github that can be
> used
> > if you opt-in:
> >
> https://blogs.apache.org/infra/entry/improved_integration_between_apache_and
> >
> > On Fri, Jan 8, 2016 at 4:15 PM Gino Bustelo <gi...@bustelos.com> wrote:
> >
> >> Based on
> >> https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci,
> it
> >> seems that we could continue to use travisci. I honestly prefer Travis
> >> since the configuration is all contained within our project.
> >>
> >> On Fri, Jan 8, 2016 at 3:55 PM, Reynold Xin <rx...@databricks.com>
> wrote:
> >>
> >>> ASF does have a Jenkins infrastructure. You can use that as an option.
> >>>
> >>> https://builds.apache.org/
> >>>
> >>> On Fri, Jan 8, 2016 at 9:54 AM, Chip Senkbeil <chip.senkbeil@gmail.com
> >
> >>> wrote:
> >>>
> >>>> I know that Apache Spark uses Jenkins with a plugin in Github to allow
> >>> them
> >>>> to manually invoke Jenkins testing against PRs.
> >>>>
> >>>> Do we want something similar (or do we have it already) for Toree?
> >>>> Currently, we use Travis CI, which is free for open source projects. I
> >>>> don't know off the top of my head if Travis allows you to manually
> >> invoke
> >>>> builds/tests againsts PRs.
> >>>>
> >>>> Is there something specific that Apache requires us to use? Does
> anyone
> >>>> have a preference in how we handle PRs?
> >>>>
> >>>> Also, I was reading over (https://www.apache.org/dev/release.html).
> >>> There
> >>>> are multiple release types including nightly builds. While we don't
> >>>> advertise nightly builds on the main site, it would be useful to have
> >>>> nightly builds for those interested in trying bug fixes, getting new
> >>>> features, etc. This is especially important considering we currently
> >> ask
> >>>> all users to build from source; so, moving to a release model changes
> >> our
> >>>> flow a little bit.
> >>>>
> >>>> Is there available Apache tooling for nightly builds? Do we need to
> set
> >>>> something up ourselves?
> >>>>
> >>>
> >>
>
>
--
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/
Re: Continuous Integration Tooling
Posted by Julien Le Dem <ju...@ledem.net>.
FYI, Apache Parquet uses github and the travis-ci integration.
When moving to apache I did notice a slower turnaround on the travis builds.
Possibly because there are a lot of projects under the apache github account.
The main difference is you can not use the PR merge button in github.
(as the apache github is read only replicated from the apache git repo.)
> On Jan 8, 2016, at 2:45 PM, Chip Senkbeil <ch...@gmail.com> wrote:
>
> From there, I see there is better integration with Github that can be used
> if you opt-in:
> https://blogs.apache.org/infra/entry/improved_integration_between_apache_and
>
> On Fri, Jan 8, 2016 at 4:15 PM Gino Bustelo <gi...@bustelos.com> wrote:
>
>> Based on
>> https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci, it
>> seems that we could continue to use travisci. I honestly prefer Travis
>> since the configuration is all contained within our project.
>>
>> On Fri, Jan 8, 2016 at 3:55 PM, Reynold Xin <rx...@databricks.com> wrote:
>>
>>> ASF does have a Jenkins infrastructure. You can use that as an option.
>>>
>>> https://builds.apache.org/
>>>
>>> On Fri, Jan 8, 2016 at 9:54 AM, Chip Senkbeil <ch...@gmail.com>
>>> wrote:
>>>
>>>> I know that Apache Spark uses Jenkins with a plugin in Github to allow
>>> them
>>>> to manually invoke Jenkins testing against PRs.
>>>>
>>>> Do we want something similar (or do we have it already) for Toree?
>>>> Currently, we use Travis CI, which is free for open source projects. I
>>>> don't know off the top of my head if Travis allows you to manually
>> invoke
>>>> builds/tests againsts PRs.
>>>>
>>>> Is there something specific that Apache requires us to use? Does anyone
>>>> have a preference in how we handle PRs?
>>>>
>>>> Also, I was reading over (https://www.apache.org/dev/release.html).
>>> There
>>>> are multiple release types including nightly builds. While we don't
>>>> advertise nightly builds on the main site, it would be useful to have
>>>> nightly builds for those interested in trying bug fixes, getting new
>>>> features, etc. This is especially important considering we currently
>> ask
>>>> all users to build from source; so, moving to a release model changes
>> our
>>>> flow a little bit.
>>>>
>>>> Is there available Apache tooling for nightly builds? Do we need to set
>>>> something up ourselves?
>>>>
>>>
>>
Re: Continuous Integration Tooling
Posted by Chip Senkbeil <ch...@gmail.com>.
>From there, I see there is better integration with Github that can be used
if you opt-in:
https://blogs.apache.org/infra/entry/improved_integration_between_apache_and
On Fri, Jan 8, 2016 at 4:15 PM Gino Bustelo <gi...@bustelos.com> wrote:
> Based on
> https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci, it
> seems that we could continue to use travisci. I honestly prefer Travis
> since the configuration is all contained within our project.
>
> On Fri, Jan 8, 2016 at 3:55 PM, Reynold Xin <rx...@databricks.com> wrote:
>
> > ASF does have a Jenkins infrastructure. You can use that as an option.
> >
> > https://builds.apache.org/
> >
> > On Fri, Jan 8, 2016 at 9:54 AM, Chip Senkbeil <ch...@gmail.com>
> > wrote:
> >
> > > I know that Apache Spark uses Jenkins with a plugin in Github to allow
> > them
> > > to manually invoke Jenkins testing against PRs.
> > >
> > > Do we want something similar (or do we have it already) for Toree?
> > > Currently, we use Travis CI, which is free for open source projects. I
> > > don't know off the top of my head if Travis allows you to manually
> invoke
> > > builds/tests againsts PRs.
> > >
> > > Is there something specific that Apache requires us to use? Does anyone
> > > have a preference in how we handle PRs?
> > >
> > > Also, I was reading over (https://www.apache.org/dev/release.html).
> > There
> > > are multiple release types including nightly builds. While we don't
> > > advertise nightly builds on the main site, it would be useful to have
> > > nightly builds for those interested in trying bug fixes, getting new
> > > features, etc. This is especially important considering we currently
> ask
> > > all users to build from source; so, moving to a release model changes
> our
> > > flow a little bit.
> > >
> > > Is there available Apache tooling for nightly builds? Do we need to set
> > > something up ourselves?
> > >
> >
>
Re: Continuous Integration Tooling
Posted by Gino Bustelo <gi...@bustelos.com>.
Based on
https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci, it
seems that we could continue to use travisci. I honestly prefer Travis
since the configuration is all contained within our project.
On Fri, Jan 8, 2016 at 3:55 PM, Reynold Xin <rx...@databricks.com> wrote:
> ASF does have a Jenkins infrastructure. You can use that as an option.
>
> https://builds.apache.org/
>
> On Fri, Jan 8, 2016 at 9:54 AM, Chip Senkbeil <ch...@gmail.com>
> wrote:
>
> > I know that Apache Spark uses Jenkins with a plugin in Github to allow
> them
> > to manually invoke Jenkins testing against PRs.
> >
> > Do we want something similar (or do we have it already) for Toree?
> > Currently, we use Travis CI, which is free for open source projects. I
> > don't know off the top of my head if Travis allows you to manually invoke
> > builds/tests againsts PRs.
> >
> > Is there something specific that Apache requires us to use? Does anyone
> > have a preference in how we handle PRs?
> >
> > Also, I was reading over (https://www.apache.org/dev/release.html).
> There
> > are multiple release types including nightly builds. While we don't
> > advertise nightly builds on the main site, it would be useful to have
> > nightly builds for those interested in trying bug fixes, getting new
> > features, etc. This is especially important considering we currently ask
> > all users to build from source; so, moving to a release model changes our
> > flow a little bit.
> >
> > Is there available Apache tooling for nightly builds? Do we need to set
> > something up ourselves?
> >
>
Re: Continuous Integration Tooling
Posted by Reynold Xin <rx...@databricks.com>.
ASF does have a Jenkins infrastructure. You can use that as an option.
https://builds.apache.org/
On Fri, Jan 8, 2016 at 9:54 AM, Chip Senkbeil <ch...@gmail.com>
wrote:
> I know that Apache Spark uses Jenkins with a plugin in Github to allow them
> to manually invoke Jenkins testing against PRs.
>
> Do we want something similar (or do we have it already) for Toree?
> Currently, we use Travis CI, which is free for open source projects. I
> don't know off the top of my head if Travis allows you to manually invoke
> builds/tests againsts PRs.
>
> Is there something specific that Apache requires us to use? Does anyone
> have a preference in how we handle PRs?
>
> Also, I was reading over (https://www.apache.org/dev/release.html). There
> are multiple release types including nightly builds. While we don't
> advertise nightly builds on the main site, it would be useful to have
> nightly builds for those interested in trying bug fixes, getting new
> features, etc. This is especially important considering we currently ask
> all users to build from source; so, moving to a release model changes our
> flow a little bit.
>
> Is there available Apache tooling for nightly builds? Do we need to set
> something up ourselves?
>