You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Michael Mior <mm...@apache.org> on 2019/04/01 12:10:56 UTC
Re: [DISCUSS] Automated website builds for Calcite projects
I'm not sure how the builds were set up initially, but AFAIK all the
configuration is in the control panel nd not in the repo. Would be
great to have this automated :)
--
Michael Mior
mmior@apache.org
Le dim. 31 mars 2019 à 19:07, Francis Chuang
<fr...@apache.org> a écrit :
>
> While moving the website repo from SVN to Git a while back, I briefly
> mentioned the possibility of automating our website builds, so that we
> do not need to manually push the built website assets.
>
> I have done some research regarding ASF's build infrastructure and it
> turns out there is a git-website jenkins node for doing this.
> Unfortunately, there isn't much documentation for this, and the howto
> for this appears to be work in progress:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75964385
>
> Before reaching out to infra and builds I'd like to clarify my
> understanding of how Jenkins builds are currently set up for Calcite,
> before asking builds and infra for further assistance:
>
> - How were the Jenkins builds set up initially? Did INFRA set this up
> for us or was there a self-service system?
>
> - Are the ASF builds configured using exclusively using the control
> panel at https://builds.apache.org/job/Calcite-Master/ or do we have
> other configurations in the repository using configuration files similar
> to .travis.yml?
>
> Francis
Re: [DISCUSS] Automated website builds for Calcite projects
Posted by Francis Chuang <fr...@apache.org>.
Thanks for the pointers, Josh! The HBase job looks like a good guide for
us to model our build on.
On 5/04/2019 1:26 am, Josh Elser wrote:
> Yep to all of the below: Jenkins is self-service and we have the ability
> to automate this stuff. FWIW, HBase has some automation around
> auto-publishing website content that you could look at
> https://builds.apache.org/job/hbase_generate_website/
>
> Some more info: Jenkins has the ability to run jobs via a "Jenkinsfile":
> https://jenkins.io/doc/book/pipeline/jenkinsfile/. This might give you
> more control over exactly what you want to do, than a free-form job in
> the Jenkins job creation via your web browser.
>
> I would guess that figuring out the credentials you can use to push the
> built content to the asf-site branch will be the most obtuse thing. The
> HBase job might have some hints for you about how this could work --
> most of the time, I find that Jenkins has the necessary credentials
> saved/setup by Infra already.
>
> On 4/1/19 7:06 PM, Francis Chuang wrote:
>> Thanks, Kevin and Michael! I'll look at our existing Jenkins
>> configuration and see if I can get this working.
>>
>> On 2/04/2019 12:51 am, Kevin Risden wrote:
>>> As far as I know Jenkins builds.apache.org is self service. You
>>> should be
>>> able to login with your Apache LDAP credentials. Can definitely
>>> modify jobs
>>> there for other projects and create new ones.
>>>
>>> The Calcite jobs on builds.apache.org are configured in the Jenkins UI.
>>> There is no Jenkinsfile for Calcite projects currently. Some other
>>> projects
>>> have looked at using Jenkinsfiles. Some details are here [1].
>>>
>>> [1]
>>> https://cwiki.apache.org/confluence/display/INFRA/Multibranch+Pipeline+recipies
>>>
>>>
>>> Kevin Risden
>>>
>>>
>>> On Mon, Apr 1, 2019 at 8:11 AM Michael Mior <mm...@apache.org> wrote:
>>>
>>>> I'm not sure how the builds were set up initially, but AFAIK all the
>>>> configuration is in the control panel nd not in the repo. Would be
>>>> great to have this automated :)
>>>>
>>>> --
>>>> Michael Mior
>>>> mmior@apache.org
>>>>
>>>> Le dim. 31 mars 2019 à 19:07, Francis Chuang
>>>> <fr...@apache.org> a écrit :
>>>>>
>>>>> While moving the website repo from SVN to Git a while back, I briefly
>>>>> mentioned the possibility of automating our website builds, so that we
>>>>> do not need to manually push the built website assets.
>>>>>
>>>>> I have done some research regarding ASF's build infrastructure and it
>>>>> turns out there is a git-website jenkins node for doing this.
>>>>> Unfortunately, there isn't much documentation for this, and the howto
>>>>> for this appears to be work in progress:
>>>>>
>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75964385
>>>>
>>>>>
>>>>> Before reaching out to infra and builds I'd like to clarify my
>>>>> understanding of how Jenkins builds are currently set up for Calcite,
>>>>> before asking builds and infra for further assistance:
>>>>>
>>>>> - How were the Jenkins builds set up initially? Did INFRA set this up
>>>>> for us or was there a self-service system?
>>>>>
>>>>> - Are the ASF builds configured using exclusively using the control
>>>>> panel at https://builds.apache.org/job/Calcite-Master/ or do we have
>>>>> other configurations in the repository using configuration files
>>>>> similar
>>>>> to .travis.yml?
>>>>>
>>>>> Francis
>>>>
>>>
Re: [DISCUSS] Automated website builds for Calcite projects
Posted by Josh Elser <el...@apache.org>.
Yep to all of the below: Jenkins is self-service and we have the ability
to automate this stuff. FWIW, HBase has some automation around
auto-publishing website content that you could look at
https://builds.apache.org/job/hbase_generate_website/
Some more info: Jenkins has the ability to run jobs via a "Jenkinsfile":
https://jenkins.io/doc/book/pipeline/jenkinsfile/. This might give you
more control over exactly what you want to do, than a free-form job in
the Jenkins job creation via your web browser.
I would guess that figuring out the credentials you can use to push the
built content to the asf-site branch will be the most obtuse thing. The
HBase job might have some hints for you about how this could work --
most of the time, I find that Jenkins has the necessary credentials
saved/setup by Infra already.
On 4/1/19 7:06 PM, Francis Chuang wrote:
> Thanks, Kevin and Michael! I'll look at our existing Jenkins
> configuration and see if I can get this working.
>
> On 2/04/2019 12:51 am, Kevin Risden wrote:
>> As far as I know Jenkins builds.apache.org is self service. You should be
>> able to login with your Apache LDAP credentials. Can definitely modify
>> jobs
>> there for other projects and create new ones.
>>
>> The Calcite jobs on builds.apache.org are configured in the Jenkins UI.
>> There is no Jenkinsfile for Calcite projects currently. Some other
>> projects
>> have looked at using Jenkinsfiles. Some details are here [1].
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/INFRA/Multibranch+Pipeline+recipies
>>
>>
>> Kevin Risden
>>
>>
>> On Mon, Apr 1, 2019 at 8:11 AM Michael Mior <mm...@apache.org> wrote:
>>
>>> I'm not sure how the builds were set up initially, but AFAIK all the
>>> configuration is in the control panel nd not in the repo. Would be
>>> great to have this automated :)
>>>
>>> --
>>> Michael Mior
>>> mmior@apache.org
>>>
>>> Le dim. 31 mars 2019 à 19:07, Francis Chuang
>>> <fr...@apache.org> a écrit :
>>>>
>>>> While moving the website repo from SVN to Git a while back, I briefly
>>>> mentioned the possibility of automating our website builds, so that we
>>>> do not need to manually push the built website assets.
>>>>
>>>> I have done some research regarding ASF's build infrastructure and it
>>>> turns out there is a git-website jenkins node for doing this.
>>>> Unfortunately, there isn't much documentation for this, and the howto
>>>> for this appears to be work in progress:
>>>>
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75964385
>>>
>>>>
>>>> Before reaching out to infra and builds I'd like to clarify my
>>>> understanding of how Jenkins builds are currently set up for Calcite,
>>>> before asking builds and infra for further assistance:
>>>>
>>>> - How were the Jenkins builds set up initially? Did INFRA set this up
>>>> for us or was there a self-service system?
>>>>
>>>> - Are the ASF builds configured using exclusively using the control
>>>> panel at https://builds.apache.org/job/Calcite-Master/ or do we have
>>>> other configurations in the repository using configuration files
>>>> similar
>>>> to .travis.yml?
>>>>
>>>> Francis
>>>
>>
Re: [DISCUSS] Automated website builds for Calcite projects
Posted by Francis Chuang <fr...@apache.org>.
Thanks, Kevin and Michael! I'll look at our existing Jenkins
configuration and see if I can get this working.
On 2/04/2019 12:51 am, Kevin Risden wrote:
> As far as I know Jenkins builds.apache.org is self service. You should be
> able to login with your Apache LDAP credentials. Can definitely modify jobs
> there for other projects and create new ones.
>
> The Calcite jobs on builds.apache.org are configured in the Jenkins UI.
> There is no Jenkinsfile for Calcite projects currently. Some other projects
> have looked at using Jenkinsfiles. Some details are here [1].
>
> [1]
> https://cwiki.apache.org/confluence/display/INFRA/Multibranch+Pipeline+recipies
>
> Kevin Risden
>
>
> On Mon, Apr 1, 2019 at 8:11 AM Michael Mior <mm...@apache.org> wrote:
>
>> I'm not sure how the builds were set up initially, but AFAIK all the
>> configuration is in the control panel nd not in the repo. Would be
>> great to have this automated :)
>>
>> --
>> Michael Mior
>> mmior@apache.org
>>
>> Le dim. 31 mars 2019 à 19:07, Francis Chuang
>> <fr...@apache.org> a écrit :
>>>
>>> While moving the website repo from SVN to Git a while back, I briefly
>>> mentioned the possibility of automating our website builds, so that we
>>> do not need to manually push the built website assets.
>>>
>>> I have done some research regarding ASF's build infrastructure and it
>>> turns out there is a git-website jenkins node for doing this.
>>> Unfortunately, there isn't much documentation for this, and the howto
>>> for this appears to be work in progress:
>>>
>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75964385
>>>
>>> Before reaching out to infra and builds I'd like to clarify my
>>> understanding of how Jenkins builds are currently set up for Calcite,
>>> before asking builds and infra for further assistance:
>>>
>>> - How were the Jenkins builds set up initially? Did INFRA set this up
>>> for us or was there a self-service system?
>>>
>>> - Are the ASF builds configured using exclusively using the control
>>> panel at https://builds.apache.org/job/Calcite-Master/ or do we have
>>> other configurations in the repository using configuration files similar
>>> to .travis.yml?
>>>
>>> Francis
>>
>
Re: [DISCUSS] Automated website builds for Calcite projects
Posted by Kevin Risden <kr...@apache.org>.
As far as I know Jenkins builds.apache.org is self service. You should be
able to login with your Apache LDAP credentials. Can definitely modify jobs
there for other projects and create new ones.
The Calcite jobs on builds.apache.org are configured in the Jenkins UI.
There is no Jenkinsfile for Calcite projects currently. Some other projects
have looked at using Jenkinsfiles. Some details are here [1].
[1]
https://cwiki.apache.org/confluence/display/INFRA/Multibranch+Pipeline+recipies
Kevin Risden
On Mon, Apr 1, 2019 at 8:11 AM Michael Mior <mm...@apache.org> wrote:
> I'm not sure how the builds were set up initially, but AFAIK all the
> configuration is in the control panel nd not in the repo. Would be
> great to have this automated :)
>
> --
> Michael Mior
> mmior@apache.org
>
> Le dim. 31 mars 2019 à 19:07, Francis Chuang
> <fr...@apache.org> a écrit :
> >
> > While moving the website repo from SVN to Git a while back, I briefly
> > mentioned the possibility of automating our website builds, so that we
> > do not need to manually push the built website assets.
> >
> > I have done some research regarding ASF's build infrastructure and it
> > turns out there is a git-website jenkins node for doing this.
> > Unfortunately, there isn't much documentation for this, and the howto
> > for this appears to be work in progress:
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75964385
> >
> > Before reaching out to infra and builds I'd like to clarify my
> > understanding of how Jenkins builds are currently set up for Calcite,
> > before asking builds and infra for further assistance:
> >
> > - How were the Jenkins builds set up initially? Did INFRA set this up
> > for us or was there a self-service system?
> >
> > - Are the ASF builds configured using exclusively using the control
> > panel at https://builds.apache.org/job/Calcite-Master/ or do we have
> > other configurations in the repository using configuration files similar
> > to .travis.yml?
> >
> > Francis
>