You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Francis Chuang <fr...@apache.org> on 2019/08/11 22:15:35 UTC
Github Actions
Github Actions recently added support for CI/CD and has been in public
beta for a while. It's also pretty well documented [1] and supports
running tests in multiple docker containers using docker-compose.
I think there are quite a few ASF projects using gitbox and Github and
this would be a very good complement or replacement for Travis, appvoyer
and other CI/CD platforms currently in use.
Is there any interest from the ASF to enable this for all Gitbox
projects when it becomes fully public?
I am also interested in being able to push to our website automatically
using Github Actions. If the git token that can push to a particular
website repository is added as a secret [2] to Github Actions, this
would be pretty easy to use for projects to automate the building of
their websites.
What do you guys think about his?
Francis
[1]
https://help.github.com/en/categories/automating-your-workflow-with-github-actions
[2]
https://help.github.com/en/articles/virtual-environments-for-github-actions#creating-and-using-secrets-encrypted-variables
Re: Github Actions
Posted by Francis Chuang <fr...@apache.org>.
For the most part, I've found Github actions to work really well. I've
migrated Calcite's Avatica Go Driver [1] from Travis to Github Actions.
I haven't had time to look into the Calcite and Calcite Avatica
repositories, but I think it should be doable.
The blocker for us is using Actions to automate the publishing of our
website as mentioned previously. Has infra worked out a way/process to
enable this? For us, it would simply be a token to push to the
calcite-site repository in Gitbox (I'd imagine the same token being used
on the git-website node would suffice) being added as a Github Actions
secret for us to enable this.
The automation [2] to do this using Github Actions has been tested and
validated, but we are just missing the credentials to push to the
calcite-site repo.
Francis
[1] https://github.com/apache/calcite-avatica-go
[2] https://github.com/apache/calcite/tree/test-site/.github/workflows
On 17/10/2019 7:28 pm, Greg Stein wrote:
> It has been a couple months. Have GitHub Actions worked out for you? Can
> you share? Are there things needed from Infra?
>
> I would like to know if there is a path for projects to move *off* Jenkins,
> Travis, Circle, etc .. and *onto* GitHub Actions. Can GHA solve projects'
> needs for CD/CI??
>
> thx,
> -g
>
>
> On Sun, Aug 11, 2019 at 5:31 PM Greg Stein <gs...@gmail.com> wrote:
>
>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <fr...@apache.org>
>> wrote:
>>> ...
>>
>>> I think there are quite a few ASF projects using gitbox and Github and
>>> this would be a very good complement or replacement for Travis, appvoyer
>>> and other CI/CD platforms currently in use.
>>>
>>> Is there any interest from the ASF to enable this for all Gitbox
>>> projects when it becomes fully public?
>>>
>>
>> Absolutely. The Infrastructure team would love to see groups try this out,
>> and share the experiences here.
>>
>> If there are any hurdles, then share them and we'll try to knock them down.
>>
>> I am also interested in being able to push to our website automatically
>>> using Github Actions. If the git token that can push to a particular
>>> website repository is added as a secret [2] to Github Actions, this
>>> would be pretty easy to use for projects to automate the building of
>>> their websites.
>>>
>>
>> Should be possible. Again, comes back to groups trying this and reporting
>> back how well it went.
>>
>> Cheers,
>> Greg Stein
>> Infrastructure Administrator, ASF
>>
>>
>
Re: Github Actions
Posted by Greg Stein <gs...@gmail.com>.
It has been a couple months. Have GitHub Actions worked out for you? Can
you share? Are there things needed from Infra?
I would like to know if there is a path for projects to move *off* Jenkins,
Travis, Circle, etc .. and *onto* GitHub Actions. Can GHA solve projects'
needs for CD/CI??
thx,
-g
On Sun, Aug 11, 2019 at 5:31 PM Greg Stein <gs...@gmail.com> wrote:
> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <fr...@apache.org>
> wrote:
> >...
>
>> I think there are quite a few ASF projects using gitbox and Github and
>> this would be a very good complement or replacement for Travis, appvoyer
>> and other CI/CD platforms currently in use.
>>
>> Is there any interest from the ASF to enable this for all Gitbox
>> projects when it becomes fully public?
>>
>
> Absolutely. The Infrastructure team would love to see groups try this out,
> and share the experiences here.
>
> If there are any hurdles, then share them and we'll try to knock them down.
>
> I am also interested in being able to push to our website automatically
>> using Github Actions. If the git token that can push to a particular
>> website repository is added as a secret [2] to Github Actions, this
>> would be pretty easy to use for projects to automate the building of
>> their websites.
>>
>
> Should be possible. Again, comes back to groups trying this and reporting
> back how well it went.
>
> Cheers,
> Greg Stein
> Infrastructure Administrator, ASF
>
>
Re: Github Actions
Posted by Francis Chuang <fr...@apache.org>.
Thanks, Zoran!
In our case, our site lives in a separate repository, as we have code in
3 different repositories that pushes to this site repository.
I've been playing around with Github Actions, but haven't gotten to the
point where I am ready to open a request with INFRA to see if it's
possible for the token to be added as a Github Actions secret. Currently
I see the following methods being possible candidates:
1. Use a Github token for the ASF github org. This gives write access to
all repos under the ASF org, which might be undesirable as there is no
way to scope Github tokens to individual repos.
2. Use the Gitbox token that is currently used by the git-websites node
which has write access to the asf-site branch of all repos. I think this
is probably the more secure way, but it would be nice if the token can
be scoped to individual repos.
On 13/08/2019 5:21 pm, Zoran Regvart wrote:
> Hi Francis & Builders,
>
> On Mon, Aug 12, 2019 at 12:39 AM Francis Chuang
> <fr...@apache.org> wrote:
>> Regarding automated website builds: Currently, this is only possible on
>> the ASF git-websites jenkins node as it holds the correct git
>> credentials to push to the website branch for ASF git repositories. Is
>> there an official process/workflow for this token to be added as a
>> secret in Github Actions?
>
> From my early experiments with GitHub actions, a GitHub action has
> full access to the GitHub repository, that includes pushing to the
> `asf-site` branch.
>
> zoran
>
Re: Github Actions
Posted by Zoran Regvart <zo...@regvart.com>.
Hi Francis & Builders,
On Mon, Aug 12, 2019 at 12:39 AM Francis Chuang
<fr...@apache.org> wrote:
> Regarding automated website builds: Currently, this is only possible on
> the ASF git-websites jenkins node as it holds the correct git
> credentials to push to the website branch for ASF git repositories. Is
> there an official process/workflow for this token to be added as a
> secret in Github Actions?
From my early experiments with GitHub actions, a GitHub action has
full access to the GitHub repository, that includes pushing to the
`asf-site` branch.
zoran
--
Zoran Regvart
Re: Github Actions
Posted by Jarek Potiuk <Ja...@polidea.com>.
We are going to try it in Airflow in parallel to our tests for GitLab CI +
Kubernetes Cluster on GKE. For now the only problem is that you cannot have
runners run on your own infrastructure (but we will have to see how it will
work with Actions and whether Airflow's requirements for our test
environments are not too tough for Actions) .
J.
On Sun, Aug 11, 2019 at 7:39 PM Matt Sicker <bo...@gmail.com> wrote:
> This service certainly sounds more useful than Travis at this point. Fairly
> useful for basic CI, especially projects that control their builds through
> proper tooling (like Maven) and don’t rely on a bunch of Jenkins plugins.
>
> On Sun, Aug 11, 2019 at 18:24, Greg Stein <gs...@gmail.com> wrote:
>
> > On Sun, Aug 11, 2019 at 5:39 PM Francis Chuang <francischuang@apache.org
> >
> > wrote:
> >
> > > Hi Greg,
> > >
> > > Should we/are we allowed to use Github Actions for ASF projects on
> > > Github right now? Do we need permission from infra or ASF itself?
> > >
> >
> > Try it. We don't know if/what barriers might exist.
> >
> > Infra's position is that we want to enable this for all of our projects.
> So
> > "permission granted" conceptually. Is there something more to effect
> that?
> > Don't know.
> >
> >
> > > Regarding automated website builds: Currently, this is only possible on
> > > the ASF git-websites jenkins node as it holds the correct git
> > > credentials to push to the website branch for ASF git repositories. Is
> > > there an official process/workflow for this token to be added as a
> > > secret in Github Actions?
> > >
> >
> > Again: figure out how to add a secret, and let us know.
> >
> > One concern is whether that secret is visible to the project, or only the
> > org admins. We don't know. Go find out and report back :)
> >
> > Really: the short answer is that Infra is not going to do the
> *exploration*
> > at this time. But if/when you hit a roadblock, *then* we can fix it. Even
> > better, figuring it out and telling Infra "please do steps 1, 2, and 3."
> >
> > Thanks,
> > -g
> > InfraAdmin, ASF
> >
> --
> Matt Sicker <bo...@gmail.com>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: Github Actions
Posted by Matt Sicker <bo...@gmail.com>.
This service certainly sounds more useful than Travis at this point. Fairly
useful for basic CI, especially projects that control their builds through
proper tooling (like Maven) and don’t rely on a bunch of Jenkins plugins.
On Sun, Aug 11, 2019 at 18:24, Greg Stein <gs...@gmail.com> wrote:
> On Sun, Aug 11, 2019 at 5:39 PM Francis Chuang <fr...@apache.org>
> wrote:
>
> > Hi Greg,
> >
> > Should we/are we allowed to use Github Actions for ASF projects on
> > Github right now? Do we need permission from infra or ASF itself?
> >
>
> Try it. We don't know if/what barriers might exist.
>
> Infra's position is that we want to enable this for all of our projects. So
> "permission granted" conceptually. Is there something more to effect that?
> Don't know.
>
>
> > Regarding automated website builds: Currently, this is only possible on
> > the ASF git-websites jenkins node as it holds the correct git
> > credentials to push to the website branch for ASF git repositories. Is
> > there an official process/workflow for this token to be added as a
> > secret in Github Actions?
> >
>
> Again: figure out how to add a secret, and let us know.
>
> One concern is whether that secret is visible to the project, or only the
> org admins. We don't know. Go find out and report back :)
>
> Really: the short answer is that Infra is not going to do the *exploration*
> at this time. But if/when you hit a roadblock, *then* we can fix it. Even
> better, figuring it out and telling Infra "please do steps 1, 2, and 3."
>
> Thanks,
> -g
> InfraAdmin, ASF
>
--
Matt Sicker <bo...@gmail.com>
Re: Github Actions
Posted by Greg Stein <gs...@gmail.com>.
On Sun, Aug 11, 2019 at 5:39 PM Francis Chuang <fr...@apache.org>
wrote:
> Hi Greg,
>
> Should we/are we allowed to use Github Actions for ASF projects on
> Github right now? Do we need permission from infra or ASF itself?
>
Try it. We don't know if/what barriers might exist.
Infra's position is that we want to enable this for all of our projects. So
"permission granted" conceptually. Is there something more to effect that?
Don't know.
> Regarding automated website builds: Currently, this is only possible on
> the ASF git-websites jenkins node as it holds the correct git
> credentials to push to the website branch for ASF git repositories. Is
> there an official process/workflow for this token to be added as a
> secret in Github Actions?
>
Again: figure out how to add a secret, and let us know.
One concern is whether that secret is visible to the project, or only the
org admins. We don't know. Go find out and report back :)
Really: the short answer is that Infra is not going to do the *exploration*
at this time. But if/when you hit a roadblock, *then* we can fix it. Even
better, figuring it out and telling Infra "please do steps 1, 2, and 3."
Thanks,
-g
InfraAdmin, ASF
Re: Github Actions
Posted by Francis Chuang <fr...@apache.org>.
Hi Greg,
Should we/are we allowed to use Github Actions for ASF projects on
Github right now? Do we need permission from infra or ASF itself?
Regarding automated website builds: Currently, this is only possible on
the ASF git-websites jenkins node as it holds the correct git
credentials to push to the website branch for ASF git repositories. Is
there an official process/workflow for this token to be added as a
secret in Github Actions?
Cheers,
Francis
On 12/08/2019 8:31 am, Greg Stein wrote:
> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <fr...@apache.org>
> wrote:
>> ...
>
>> I think there are quite a few ASF projects using gitbox and Github and
>> this would be a very good complement or replacement for Travis, appvoyer
>> and other CI/CD platforms currently in use.
>>
>> Is there any interest from the ASF to enable this for all Gitbox
>> projects when it becomes fully public?
>>
>
> Absolutely. The Infrastructure team would love to see groups try this out,
> and share the experiences here.
>
> If there are any hurdles, then share them and we'll try to knock them down.
>
> I am also interested in being able to push to our website automatically
>> using Github Actions. If the git token that can push to a particular
>> website repository is added as a secret [2] to Github Actions, this
>> would be pretty easy to use for projects to automate the building of
>> their websites.
>>
>
> Should be possible. Again, comes back to groups trying this and reporting
> back how well it went.
>
> Cheers,
> Greg Stein
> Infrastructure Administrator, ASF
>
Re: Github Actions
Posted by Francis Chuang <fr...@apache.org>.
Many thanks for the example, Zoran.
I think I see the problem now. For camel-website you're running Github
actions in camel-website, so I think it implicitly allows you to push to
the same repo.
In my case, I have actions in calcite, calcite-avatica and
calcite-avatica-go pushing to calcite-site. I was thinking I could
reverse the relationship and store the actions in calcite-site (thus
avoiding the need to authenticate), but I don't think GH actions allow
triggering on events that originate from other repositories.
Francis
On 5/09/2019 4:31 pm, Zoran Regvart wrote:
> Hi Francis,
> either that little loophole was closed or you're missing `user.email`
> and `user.name` in your git config. Here's[1] how we used to do it for
> the Camel website.
>
> zoran
>
> [1] https://github.com/apache/camel-website/blob/master/.github/action-website/publish#L11-L12
>
> On Thu, Sep 5, 2019 at 3:02 AM Francis Chuang <fr...@apache.org> wrote:
>>
>> Hi Zoran,
>>
>> I tired cloning from github.com, but pushing to it is still denied.
>>
>> If I don't set any credentials in git, it complains that there is no
>> username for github.com:
>> https://github.com/apache/calcite/commit/d55e389310dcd024ae96623692a1e0a67274873c/checks
>>
>> If I set the GITHUB_TOKEN as the credential, I get a 403:
>> https://github.com/apache/calcite/runs/212573263
>>
>> Francis
>>
>> On 4/09/2019 10:04 pm, Zoran Regvart wrote:
>>> Hi Francis,
>>>
>>> On Tue, Aug 27, 2019 at 12:19 PM Francis Chuang
>>> <fr...@apache.org> wrote:
>>>>
>>>> I have implemented the ability to generate the website and javadoc for
>>>> Calcite using Github Actions. See:
>>>> https://github.com/apache/calcite/tree/test-site/.github/workflows
>>>>
>>>> The missing piece is that we need the token to publish to our
>>>> calcite-site repository to be added as a secret in Github Actions and
>>>> there is currently no clear process as to whether this is allowed or how
>>>> to get this done.
>>>
>>> In your GitHub actions try cloning from github.com rather than
>>> gitbox.apache.org, I think you'll find that you can push to a
>>> github.com repository without authentication from a GitHub action.
>>>
>>> zoran
>>>
>
>
>
Re: Github Actions
Posted by Zoran Regvart <zo...@regvart.com>.
Hi Francis,
either that little loophole was closed or you're missing `user.email`
and `user.name` in your git config. Here's[1] how we used to do it for
the Camel website.
zoran
[1] https://github.com/apache/camel-website/blob/master/.github/action-website/publish#L11-L12
On Thu, Sep 5, 2019 at 3:02 AM Francis Chuang <fr...@apache.org> wrote:
>
> Hi Zoran,
>
> I tired cloning from github.com, but pushing to it is still denied.
>
> If I don't set any credentials in git, it complains that there is no
> username for github.com:
> https://github.com/apache/calcite/commit/d55e389310dcd024ae96623692a1e0a67274873c/checks
>
> If I set the GITHUB_TOKEN as the credential, I get a 403:
> https://github.com/apache/calcite/runs/212573263
>
> Francis
>
> On 4/09/2019 10:04 pm, Zoran Regvart wrote:
> > Hi Francis,
> >
> > On Tue, Aug 27, 2019 at 12:19 PM Francis Chuang
> > <fr...@apache.org> wrote:
> >>
> >> I have implemented the ability to generate the website and javadoc for
> >> Calcite using Github Actions. See:
> >> https://github.com/apache/calcite/tree/test-site/.github/workflows
> >>
> >> The missing piece is that we need the token to publish to our
> >> calcite-site repository to be added as a secret in Github Actions and
> >> there is currently no clear process as to whether this is allowed or how
> >> to get this done.
> >
> > In your GitHub actions try cloning from github.com rather than
> > gitbox.apache.org, I think you'll find that you can push to a
> > github.com repository without authentication from a GitHub action.
> >
> > zoran
> >
--
Zoran Regvart
Re: Github Actions
Posted by Francis Chuang <fr...@apache.org>.
Hi Zoran,
I tired cloning from github.com, but pushing to it is still denied.
If I don't set any credentials in git, it complains that there is no
username for github.com:
https://github.com/apache/calcite/commit/d55e389310dcd024ae96623692a1e0a67274873c/checks
If I set the GITHUB_TOKEN as the credential, I get a 403:
https://github.com/apache/calcite/runs/212573263
Francis
On 4/09/2019 10:04 pm, Zoran Regvart wrote:
> Hi Francis,
>
> On Tue, Aug 27, 2019 at 12:19 PM Francis Chuang
> <fr...@apache.org> wrote:
>>
>> I have implemented the ability to generate the website and javadoc for
>> Calcite using Github Actions. See:
>> https://github.com/apache/calcite/tree/test-site/.github/workflows
>>
>> The missing piece is that we need the token to publish to our
>> calcite-site repository to be added as a secret in Github Actions and
>> there is currently no clear process as to whether this is allowed or how
>> to get this done.
>
> In your GitHub actions try cloning from github.com rather than
> gitbox.apache.org, I think you'll find that you can push to a
> github.com repository without authentication from a GitHub action.
>
> zoran
>
Re: Github Actions
Posted by Zoran Regvart <zo...@regvart.com>.
Hi Francis,
On Tue, Aug 27, 2019 at 12:19 PM Francis Chuang
<fr...@apache.org> wrote:
>
> I have implemented the ability to generate the website and javadoc for
> Calcite using Github Actions. See:
> https://github.com/apache/calcite/tree/test-site/.github/workflows
>
> The missing piece is that we need the token to publish to our
> calcite-site repository to be added as a secret in Github Actions and
> there is currently no clear process as to whether this is allowed or how
> to get this done.
In your GitHub actions try cloning from github.com rather than
gitbox.apache.org, I think you'll find that you can push to a
github.com repository without authentication from a GitHub action.
zoran
--
Zoran Regvart
Re: Github Actions
Posted by Joan Touzet <wo...@apache.org>.
Continuing the top-post trend...
I'd rather see full audit logs kept ~forever for any use of credentials,
including the code that was executed.
If we can't stop the leak, we can at least keep the paper trail.
Right now, with our aggressive build cleanup steps, I don't think this
is happening. Archiving that data somewhere else for legal purposes
might be a good idea.
-Joan "just an idea" Touzet
On 2019-08-28 5:02, sebb wrote:
> I think the pre-verified code could run on a separate system with
> restricted access.
> That's how self-service works for creating mailing lists, for example.
>
> In this case, there would need to be a separate host with read access
> to Jenkins.
> It could accept publish requests from Jenkins, and route them accordingly.
>
> This would require a bit of effort to set up, but could be used for
> multiple projects.
>
> On Wed, 28 Aug 2019 at 03:26, Greg Stein <gs...@gmail.com> wrote:
>>
>> Yeah. FIgured as much, hoped that I was missing something :)
>>
>> (note: we have the same issue with buildbot and jenkins: we simply trust
>> the communities to not exfil that data)
>>
>> On Tue, Aug 27, 2019 at 9:16 PM Matt Sicker <bo...@gmail.com> wrote:
>>
>>> How to avoid leaking secrets: only way to do that is via pre-verified code
>>> that executes something with that secret. Otherwise, there’s literally
>>> infinite ways to leak it being a Turing machine and all. This applies to
>>> all CICD tools.
>>>
>>> On Tue, Aug 27, 2019 at 20:32, Greg Stein <gs...@gmail.com> wrote:
>>>
>>>> Hi Francis,
>>>>
>>>> Is the token needed to push from calcite to calcite-site? Is that an
>>> oauth
>>>> token or something? And are you able to use the repository settings to
>>> add
>>>> secrets, but you don't have the right token? Or you cannot add secrets at
>>>> all? (I can't tell since I have superpowers)
>>>>
>>>> I've added GSTEIN_TEST_SECRET to Calcite. See if you can extract/print
>>> that
>>>> into your build/action log. If so, then we can try to figure out the
>>>> security here (ie. how do we avoid Actions exfiltrating the token?)
>>>>
>>>> Thanks,
>>>> -g
>>>>
>>>> On Tue, Aug 27, 2019 at 5:19 AM Francis Chuang <francischuang@apache.org
>>>>
>>>> wrote:
>>>>
>>>>> I have implemented the ability to generate the website and javadoc for
>>>>> Calcite using Github Actions. See:
>>>>> https://github.com/apache/calcite/tree/test-site/.github/workflows
>>>>>
>>>>> The missing piece is that we need the token to publish to our
>>>>> calcite-site repository to be added as a secret in Github Actions and
>>>>> there is currently no clear process as to whether this is allowed or
>>> how
>>>>> to get this done.
>>>>>
>>>>> See:
>>>>> https://issues.apache.org/jira/browse/INFRA-18874
>>>>> https://issues.apache.org/jira/browse/INFRA-18875
>>>>>
>>>>> Francis
>>>>>
>>>>> On 27/08/2019 7:52 pm, Greg Stein wrote:
>>>>>> Have you had an opportunity to make progress on this, to share with
>>> us?
>>>>>>
>>>>>> Anybody else with news?
>>>>>>
>>>>>> Thanks!
>>>>>> -g
>>>>>> InfraAdmin, ASF
>>>>>>
>>>>>>
>>>>>> On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise <
>>> khmarbaise@gmx.de
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I've made a simple PoC for the Apache Maven Dependency Plugin on a
>>>>>>> separate branch.
>>>>>>>
>>>>>>> I will try within the next days more features for example Mac OS
>>>> builds
>>>>>>> etc.
>>>>>>>
>>>>>>>
>>>>>>> Currently I simply push my changes via gitbox ..
>>>>>>>
>>>>>>> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
>>>>>>> origin
>>>> https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
>>>>>>> (fetch)
>>>>>>> origin
>>>> https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
>>>>>>> (push)
>>>>>>>
>>>>>>>
>>>>>>> Also I'm interested to use SonarCloud related with GitHub Actions..?
>>>>>>>
>>>>>>>
>>>>>>> Kind regards
>>>>>>> Karl Heinz Marbaise
>>>>>>> Apache Maven PMC
>>>>>>>
>>>>>>> [1]:
>>> https://github.com/apache/maven-dependency-plugin/runs/192633340
>>>>>>> [2]:
>>>>>>>
>>>>>>>
>>>>>
>>>>
>>> https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
>>>>>>>
>>>>>>> On 12.08.19 00:31, Greg Stein wrote:
>>>>>>>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <
>>>>> francischuang@apache.org
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>> ...
>>>>>>>>
>>>>>>>>> I think there are quite a few ASF projects using gitbox and Github
>>>> and
>>>>>>>>> this would be a very good complement or replacement for Travis,
>>>>> appvoyer
>>>>>>>>> and other CI/CD platforms currently in use.
>>>>>>>>>
>>>>>>>>> Is there any interest from the ASF to enable this for all Gitbox
>>>>>>>>> projects when it becomes fully public?
>>>>>>>>>
>>>>>>>>
>>>>>>>> Absolutely. The Infrastructure team would love to see groups try
>>> this
>>>>>>> out,
>>>>>>>> and share the experiences here.
>>>>>>>>
>>>>>>>> If there are any hurdles, then share them and we'll try to knock
>>> them
>>>>>>> down.
>>>>>>>>
>>>>>>>> I am also interested in being able to push to our website
>>>> automatically
>>>>>>>>> using Github Actions. If the git token that can push to a
>>> particular
>>>>>>>>> website repository is added as a secret [2] to Github Actions,
>>> this
>>>>>>>>> would be pretty easy to use for projects to automate the building
>>> of
>>>>>>>>> their websites.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Should be possible. Again, comes back to groups trying this and
>>>>> reporting
>>>>>>>> back how well it went.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Greg Stein
>>>>>>>> Infrastructure Administrator, ASF
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>> --
>>> Matt Sicker <bo...@gmail.com>
>>>
Re: Github Actions
Posted by sebb <se...@gmail.com>.
I think the pre-verified code could run on a separate system with
restricted access.
That's how self-service works for creating mailing lists, for example.
In this case, there would need to be a separate host with read access
to Jenkins.
It could accept publish requests from Jenkins, and route them accordingly.
This would require a bit of effort to set up, but could be used for
multiple projects.
On Wed, 28 Aug 2019 at 03:26, Greg Stein <gs...@gmail.com> wrote:
>
> Yeah. FIgured as much, hoped that I was missing something :)
>
> (note: we have the same issue with buildbot and jenkins: we simply trust
> the communities to not exfil that data)
>
> On Tue, Aug 27, 2019 at 9:16 PM Matt Sicker <bo...@gmail.com> wrote:
>
> > How to avoid leaking secrets: only way to do that is via pre-verified code
> > that executes something with that secret. Otherwise, there’s literally
> > infinite ways to leak it being a Turing machine and all. This applies to
> > all CICD tools.
> >
> > On Tue, Aug 27, 2019 at 20:32, Greg Stein <gs...@gmail.com> wrote:
> >
> > > Hi Francis,
> > >
> > > Is the token needed to push from calcite to calcite-site? Is that an
> > oauth
> > > token or something? And are you able to use the repository settings to
> > add
> > > secrets, but you don't have the right token? Or you cannot add secrets at
> > > all? (I can't tell since I have superpowers)
> > >
> > > I've added GSTEIN_TEST_SECRET to Calcite. See if you can extract/print
> > that
> > > into your build/action log. If so, then we can try to figure out the
> > > security here (ie. how do we avoid Actions exfiltrating the token?)
> > >
> > > Thanks,
> > > -g
> > >
> > > On Tue, Aug 27, 2019 at 5:19 AM Francis Chuang <francischuang@apache.org
> > >
> > > wrote:
> > >
> > > > I have implemented the ability to generate the website and javadoc for
> > > > Calcite using Github Actions. See:
> > > > https://github.com/apache/calcite/tree/test-site/.github/workflows
> > > >
> > > > The missing piece is that we need the token to publish to our
> > > > calcite-site repository to be added as a secret in Github Actions and
> > > > there is currently no clear process as to whether this is allowed or
> > how
> > > > to get this done.
> > > >
> > > > See:
> > > > https://issues.apache.org/jira/browse/INFRA-18874
> > > > https://issues.apache.org/jira/browse/INFRA-18875
> > > >
> > > > Francis
> > > >
> > > > On 27/08/2019 7:52 pm, Greg Stein wrote:
> > > > > Have you had an opportunity to make progress on this, to share with
> > us?
> > > > >
> > > > > Anybody else with news?
> > > > >
> > > > > Thanks!
> > > > > -g
> > > > > InfraAdmin, ASF
> > > > >
> > > > >
> > > > > On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise <
> > khmarbaise@gmx.de
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> I've made a simple PoC for the Apache Maven Dependency Plugin on a
> > > > >> separate branch.
> > > > >>
> > > > >> I will try within the next days more features for example Mac OS
> > > builds
> > > > >> etc.
> > > > >>
> > > > >>
> > > > >> Currently I simply push my changes via gitbox ..
> > > > >>
> > > > >> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
> > > > >> origin
> > > https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> > > > >> (fetch)
> > > > >> origin
> > > https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> > > > >> (push)
> > > > >>
> > > > >>
> > > > >> Also I'm interested to use SonarCloud related with GitHub Actions..?
> > > > >>
> > > > >>
> > > > >> Kind regards
> > > > >> Karl Heinz Marbaise
> > > > >> Apache Maven PMC
> > > > >>
> > > > >> [1]:
> > https://github.com/apache/maven-dependency-plugin/runs/192633340
> > > > >> [2]:
> > > > >>
> > > > >>
> > > >
> > >
> > https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
> > > > >>
> > > > >> On 12.08.19 00:31, Greg Stein wrote:
> > > > >>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <
> > > > francischuang@apache.org
> > > > >>>
> > > > >>> wrote:
> > > > >>>> ...
> > > > >>>
> > > > >>>> I think there are quite a few ASF projects using gitbox and Github
> > > and
> > > > >>>> this would be a very good complement or replacement for Travis,
> > > > appvoyer
> > > > >>>> and other CI/CD platforms currently in use.
> > > > >>>>
> > > > >>>> Is there any interest from the ASF to enable this for all Gitbox
> > > > >>>> projects when it becomes fully public?
> > > > >>>>
> > > > >>>
> > > > >>> Absolutely. The Infrastructure team would love to see groups try
> > this
> > > > >> out,
> > > > >>> and share the experiences here.
> > > > >>>
> > > > >>> If there are any hurdles, then share them and we'll try to knock
> > them
> > > > >> down.
> > > > >>>
> > > > >>> I am also interested in being able to push to our website
> > > automatically
> > > > >>>> using Github Actions. If the git token that can push to a
> > particular
> > > > >>>> website repository is added as a secret [2] to Github Actions,
> > this
> > > > >>>> would be pretty easy to use for projects to automate the building
> > of
> > > > >>>> their websites.
> > > > >>>>
> > > > >>>
> > > > >>> Should be possible. Again, comes back to groups trying this and
> > > > reporting
> > > > >>> back how well it went.
> > > > >>>
> > > > >>> Cheers,
> > > > >>> Greg Stein
> > > > >>> Infrastructure Administrator, ASF
> > > > >>>
> > > > >>
> > > > >
> > > >
> > >
> > --
> > Matt Sicker <bo...@gmail.com>
> >
Re: Github Actions
Posted by Greg Stein <gs...@gmail.com>.
Yeah. FIgured as much, hoped that I was missing something :)
(note: we have the same issue with buildbot and jenkins: we simply trust
the communities to not exfil that data)
On Tue, Aug 27, 2019 at 9:16 PM Matt Sicker <bo...@gmail.com> wrote:
> How to avoid leaking secrets: only way to do that is via pre-verified code
> that executes something with that secret. Otherwise, there’s literally
> infinite ways to leak it being a Turing machine and all. This applies to
> all CICD tools.
>
> On Tue, Aug 27, 2019 at 20:32, Greg Stein <gs...@gmail.com> wrote:
>
> > Hi Francis,
> >
> > Is the token needed to push from calcite to calcite-site? Is that an
> oauth
> > token or something? And are you able to use the repository settings to
> add
> > secrets, but you don't have the right token? Or you cannot add secrets at
> > all? (I can't tell since I have superpowers)
> >
> > I've added GSTEIN_TEST_SECRET to Calcite. See if you can extract/print
> that
> > into your build/action log. If so, then we can try to figure out the
> > security here (ie. how do we avoid Actions exfiltrating the token?)
> >
> > Thanks,
> > -g
> >
> > On Tue, Aug 27, 2019 at 5:19 AM Francis Chuang <francischuang@apache.org
> >
> > wrote:
> >
> > > I have implemented the ability to generate the website and javadoc for
> > > Calcite using Github Actions. See:
> > > https://github.com/apache/calcite/tree/test-site/.github/workflows
> > >
> > > The missing piece is that we need the token to publish to our
> > > calcite-site repository to be added as a secret in Github Actions and
> > > there is currently no clear process as to whether this is allowed or
> how
> > > to get this done.
> > >
> > > See:
> > > https://issues.apache.org/jira/browse/INFRA-18874
> > > https://issues.apache.org/jira/browse/INFRA-18875
> > >
> > > Francis
> > >
> > > On 27/08/2019 7:52 pm, Greg Stein wrote:
> > > > Have you had an opportunity to make progress on this, to share with
> us?
> > > >
> > > > Anybody else with news?
> > > >
> > > > Thanks!
> > > > -g
> > > > InfraAdmin, ASF
> > > >
> > > >
> > > > On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise <
> khmarbaise@gmx.de
> > >
> > > > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> I've made a simple PoC for the Apache Maven Dependency Plugin on a
> > > >> separate branch.
> > > >>
> > > >> I will try within the next days more features for example Mac OS
> > builds
> > > >> etc.
> > > >>
> > > >>
> > > >> Currently I simply push my changes via gitbox ..
> > > >>
> > > >> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
> > > >> origin
> > https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> > > >> (fetch)
> > > >> origin
> > https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> > > >> (push)
> > > >>
> > > >>
> > > >> Also I'm interested to use SonarCloud related with GitHub Actions..?
> > > >>
> > > >>
> > > >> Kind regards
> > > >> Karl Heinz Marbaise
> > > >> Apache Maven PMC
> > > >>
> > > >> [1]:
> https://github.com/apache/maven-dependency-plugin/runs/192633340
> > > >> [2]:
> > > >>
> > > >>
> > >
> >
> https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
> > > >>
> > > >> On 12.08.19 00:31, Greg Stein wrote:
> > > >>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <
> > > francischuang@apache.org
> > > >>>
> > > >>> wrote:
> > > >>>> ...
> > > >>>
> > > >>>> I think there are quite a few ASF projects using gitbox and Github
> > and
> > > >>>> this would be a very good complement or replacement for Travis,
> > > appvoyer
> > > >>>> and other CI/CD platforms currently in use.
> > > >>>>
> > > >>>> Is there any interest from the ASF to enable this for all Gitbox
> > > >>>> projects when it becomes fully public?
> > > >>>>
> > > >>>
> > > >>> Absolutely. The Infrastructure team would love to see groups try
> this
> > > >> out,
> > > >>> and share the experiences here.
> > > >>>
> > > >>> If there are any hurdles, then share them and we'll try to knock
> them
> > > >> down.
> > > >>>
> > > >>> I am also interested in being able to push to our website
> > automatically
> > > >>>> using Github Actions. If the git token that can push to a
> particular
> > > >>>> website repository is added as a secret [2] to Github Actions,
> this
> > > >>>> would be pretty easy to use for projects to automate the building
> of
> > > >>>> their websites.
> > > >>>>
> > > >>>
> > > >>> Should be possible. Again, comes back to groups trying this and
> > > reporting
> > > >>> back how well it went.
> > > >>>
> > > >>> Cheers,
> > > >>> Greg Stein
> > > >>> Infrastructure Administrator, ASF
> > > >>>
> > > >>
> > > >
> > >
> >
> --
> Matt Sicker <bo...@gmail.com>
>
Re: Github Actions
Posted by Matt Sicker <bo...@gmail.com>.
How to avoid leaking secrets: only way to do that is via pre-verified code
that executes something with that secret. Otherwise, there’s literally
infinite ways to leak it being a Turing machine and all. This applies to
all CICD tools.
On Tue, Aug 27, 2019 at 20:32, Greg Stein <gs...@gmail.com> wrote:
> Hi Francis,
>
> Is the token needed to push from calcite to calcite-site? Is that an oauth
> token or something? And are you able to use the repository settings to add
> secrets, but you don't have the right token? Or you cannot add secrets at
> all? (I can't tell since I have superpowers)
>
> I've added GSTEIN_TEST_SECRET to Calcite. See if you can extract/print that
> into your build/action log. If so, then we can try to figure out the
> security here (ie. how do we avoid Actions exfiltrating the token?)
>
> Thanks,
> -g
>
> On Tue, Aug 27, 2019 at 5:19 AM Francis Chuang <fr...@apache.org>
> wrote:
>
> > I have implemented the ability to generate the website and javadoc for
> > Calcite using Github Actions. See:
> > https://github.com/apache/calcite/tree/test-site/.github/workflows
> >
> > The missing piece is that we need the token to publish to our
> > calcite-site repository to be added as a secret in Github Actions and
> > there is currently no clear process as to whether this is allowed or how
> > to get this done.
> >
> > See:
> > https://issues.apache.org/jira/browse/INFRA-18874
> > https://issues.apache.org/jira/browse/INFRA-18875
> >
> > Francis
> >
> > On 27/08/2019 7:52 pm, Greg Stein wrote:
> > > Have you had an opportunity to make progress on this, to share with us?
> > >
> > > Anybody else with news?
> > >
> > > Thanks!
> > > -g
> > > InfraAdmin, ASF
> > >
> > >
> > > On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise <khmarbaise@gmx.de
> >
> > > wrote:
> > >
> > >> Hi,
> > >>
> > >> I've made a simple PoC for the Apache Maven Dependency Plugin on a
> > >> separate branch.
> > >>
> > >> I will try within the next days more features for example Mac OS
> builds
> > >> etc.
> > >>
> > >>
> > >> Currently I simply push my changes via gitbox ..
> > >>
> > >> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
> > >> origin
> https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> > >> (fetch)
> > >> origin
> https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> > >> (push)
> > >>
> > >>
> > >> Also I'm interested to use SonarCloud related with GitHub Actions..?
> > >>
> > >>
> > >> Kind regards
> > >> Karl Heinz Marbaise
> > >> Apache Maven PMC
> > >>
> > >> [1]: https://github.com/apache/maven-dependency-plugin/runs/192633340
> > >> [2]:
> > >>
> > >>
> >
> https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
> > >>
> > >> On 12.08.19 00:31, Greg Stein wrote:
> > >>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <
> > francischuang@apache.org
> > >>>
> > >>> wrote:
> > >>>> ...
> > >>>
> > >>>> I think there are quite a few ASF projects using gitbox and Github
> and
> > >>>> this would be a very good complement or replacement for Travis,
> > appvoyer
> > >>>> and other CI/CD platforms currently in use.
> > >>>>
> > >>>> Is there any interest from the ASF to enable this for all Gitbox
> > >>>> projects when it becomes fully public?
> > >>>>
> > >>>
> > >>> Absolutely. The Infrastructure team would love to see groups try this
> > >> out,
> > >>> and share the experiences here.
> > >>>
> > >>> If there are any hurdles, then share them and we'll try to knock them
> > >> down.
> > >>>
> > >>> I am also interested in being able to push to our website
> automatically
> > >>>> using Github Actions. If the git token that can push to a particular
> > >>>> website repository is added as a secret [2] to Github Actions, this
> > >>>> would be pretty easy to use for projects to automate the building of
> > >>>> their websites.
> > >>>>
> > >>>
> > >>> Should be possible. Again, comes back to groups trying this and
> > reporting
> > >>> back how well it went.
> > >>>
> > >>> Cheers,
> > >>> Greg Stein
> > >>> Infrastructure Administrator, ASF
> > >>>
> > >>
> > >
> >
>
--
Matt Sicker <bo...@gmail.com>
Re: Github Actions
Posted by Matt Sicker <bo...@gmail.com>.
Secrets masking is another thing that only works for basic cases. Don’t try
encoding it and printing it, for example. Just a different character set
can throw off some tools let alone base 64.
On Wed, Aug 28, 2019 at 04:23, Francis Chuang <fr...@apache.org>
wrote:
> Hi Greg,
>
> Yes, the token is needed to push to calcite-site as we are running the
> job in the calcite repository. In terms of the token, if we are pushing
> to the calcite-site using Gitbox, then the token would be the one that
> is currently used in the git-websites node on Jekins. If we want to push
> directly to the Github calcite-site repository, then the token would
> probably be a Personal Access Token for the Apache org on Github (not
> 100% sure on this, but I think this is the case to the best of my
> understanding).
>
> We currently don't have ability to add secrets to Github Actions at all.
> However, this isn't a huge problem for us at the moment. We currently
> just need the Gitbox token on git-websites (which we do not have direct
> access to) or a Github Personal Access Token added to our repositories.
>
> I have added an action to print GSTEIN_TEST_SECRET. See this run here:
> https://github.com/apache/calcite/runs/205292835
>
> I don't know the original value of the secret, but it prints as "***".
>
> Francis
>
> On 28/08/2019 11:32 am, Greg Stein wrote:
> > Hi Francis,
> >
> > Is the token needed to push from calcite to calcite-site? Is that an
> > oauth token or something? And are you able to use the repository
> > settings to add secrets, but you don't have the right token? Or you
> > cannot add secrets at all? (I can't tell since I have superpowers)
> >
> > I've added GSTEIN_TEST_SECRET to Calcite. See if you can extract/print
> > that into your build/action log. If so, then we can try to figure out
> > the security here (ie. how do we avoid Actions exfiltrating the token?)
> >
> > Thanks,
> > -g
> >
> > On Tue, Aug 27, 2019 at 5:19 AM Francis Chuang <francischuang@apache.org
> > <ma...@apache.org>> wrote:
> >
> > I have implemented the ability to generate the website and javadoc
> for
> > Calcite using Github Actions. See:
> > https://github.com/apache/calcite/tree/test-site/.github/workflows
> >
> > The missing piece is that we need the token to publish to our
> > calcite-site repository to be added as a secret in Github Actions and
> > there is currently no clear process as to whether this is allowed or
> > how
> > to get this done.
> >
> > See:
> > https://issues.apache.org/jira/browse/INFRA-18874
> > https://issues.apache.org/jira/browse/INFRA-18875
> >
> > Francis
> >
> > On 27/08/2019 7:52 pm, Greg Stein wrote:
> > > Have you had an opportunity to make progress on this, to share
> > with us?
> > >
> > > Anybody else with news?
> > >
> > > Thanks!
> > > -g
> > > InfraAdmin, ASF
> > >
> > >
> > > On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise
> > <khmarbaise@gmx.de <ma...@gmx.de>>
> > > wrote:
> > >
> > >> Hi,
> > >>
> > >> I've made a simple PoC for the Apache Maven Dependency Plugin on
> a
> > >> separate branch.
> > >>
> > >> I will try within the next days more features for example Mac OS
> > builds
> > >> etc.
> > >>
> > >>
> > >> Currently I simply push my changes via gitbox ..
> > >>
> > >> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
> > >> origin
> > https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> > >> (fetch)
> > >> origin
> > https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> > >> (push)
> > >>
> > >>
> > >> Also I'm interested to use SonarCloud related with GitHub
> Actions..?
> > >>
> > >>
> > >> Kind regards
> > >> Karl Heinz Marbaise
> > >> Apache Maven PMC
> > >>
> > >> [1]:
> > https://github.com/apache/maven-dependency-plugin/runs/192633340
> > >> [2]:
> > >>
> > >>
> >
> https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
> > >>
> > >> On 12.08.19 00:31, Greg Stein wrote:
> > >>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang
> > <francischuang@apache.org <ma...@apache.org>
> > >>>
> > >>> wrote:
> > >>>> ...
> > >>>
> > >>>> I think there are quite a few ASF projects using gitbox and
> > Github and
> > >>>> this would be a very good complement or replacement for
> > Travis, appvoyer
> > >>>> and other CI/CD platforms currently in use.
> > >>>>
> > >>>> Is there any interest from the ASF to enable this for all
> Gitbox
> > >>>> projects when it becomes fully public?
> > >>>>
> > >>>
> > >>> Absolutely. The Infrastructure team would love to see groups
> > try this
> > >> out,
> > >>> and share the experiences here.
> > >>>
> > >>> If there are any hurdles, then share them and we'll try to
> > knock them
> > >> down.
> > >>>
> > >>> I am also interested in being able to push to our website
> > automatically
> > >>>> using Github Actions. If the git token that can push to a
> > particular
> > >>>> website repository is added as a secret [2] to Github Actions,
> > this
> > >>>> would be pretty easy to use for projects to automate the
> > building of
> > >>>> their websites.
> > >>>>
> > >>>
> > >>> Should be possible. Again, comes back to groups trying this and
> > reporting
> > >>> back how well it went.
> > >>>
> > >>> Cheers,
> > >>> Greg Stein
> > >>> Infrastructure Administrator, ASF
> > >>>
> > >>
> > >
> >
>
--
Matt Sicker <bo...@gmail.com>
Re: Github Actions
Posted by Francis Chuang <fr...@apache.org>.
Hi Greg,
Yes, the token is needed to push to calcite-site as we are running the
job in the calcite repository. In terms of the token, if we are pushing
to the calcite-site using Gitbox, then the token would be the one that
is currently used in the git-websites node on Jekins. If we want to push
directly to the Github calcite-site repository, then the token would
probably be a Personal Access Token for the Apache org on Github (not
100% sure on this, but I think this is the case to the best of my
understanding).
We currently don't have ability to add secrets to Github Actions at all.
However, this isn't a huge problem for us at the moment. We currently
just need the Gitbox token on git-websites (which we do not have direct
access to) or a Github Personal Access Token added to our repositories.
I have added an action to print GSTEIN_TEST_SECRET. See this run here:
https://github.com/apache/calcite/runs/205292835
I don't know the original value of the secret, but it prints as "***".
Francis
On 28/08/2019 11:32 am, Greg Stein wrote:
> Hi Francis,
>
> Is the token needed to push from calcite to calcite-site? Is that an
> oauth token or something? And are you able to use the repository
> settings to add secrets, but you don't have the right token? Or you
> cannot add secrets at all? (I can't tell since I have superpowers)
>
> I've added GSTEIN_TEST_SECRET to Calcite. See if you can extract/print
> that into your build/action log. If so, then we can try to figure out
> the security here (ie. how do we avoid Actions exfiltrating the token?)
>
> Thanks,
> -g
>
> On Tue, Aug 27, 2019 at 5:19 AM Francis Chuang <francischuang@apache.org
> <ma...@apache.org>> wrote:
>
> I have implemented the ability to generate the website and javadoc for
> Calcite using Github Actions. See:
> https://github.com/apache/calcite/tree/test-site/.github/workflows
>
> The missing piece is that we need the token to publish to our
> calcite-site repository to be added as a secret in Github Actions and
> there is currently no clear process as to whether this is allowed or
> how
> to get this done.
>
> See:
> https://issues.apache.org/jira/browse/INFRA-18874
> https://issues.apache.org/jira/browse/INFRA-18875
>
> Francis
>
> On 27/08/2019 7:52 pm, Greg Stein wrote:
> > Have you had an opportunity to make progress on this, to share
> with us?
> >
> > Anybody else with news?
> >
> > Thanks!
> > -g
> > InfraAdmin, ASF
> >
> >
> > On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise
> <khmarbaise@gmx.de <ma...@gmx.de>>
> > wrote:
> >
> >> Hi,
> >>
> >> I've made a simple PoC for the Apache Maven Dependency Plugin on a
> >> separate branch.
> >>
> >> I will try within the next days more features for example Mac OS
> builds
> >> etc.
> >>
> >>
> >> Currently I simply push my changes via gitbox ..
> >>
> >> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
> >> origin
> https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> >> (fetch)
> >> origin
> https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> >> (push)
> >>
> >>
> >> Also I'm interested to use SonarCloud related with GitHub Actions..?
> >>
> >>
> >> Kind regards
> >> Karl Heinz Marbaise
> >> Apache Maven PMC
> >>
> >> [1]:
> https://github.com/apache/maven-dependency-plugin/runs/192633340
> >> [2]:
> >>
> >>
> https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
> >>
> >> On 12.08.19 00:31, Greg Stein wrote:
> >>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang
> <francischuang@apache.org <ma...@apache.org>
> >>>
> >>> wrote:
> >>>> ...
> >>>
> >>>> I think there are quite a few ASF projects using gitbox and
> Github and
> >>>> this would be a very good complement or replacement for
> Travis, appvoyer
> >>>> and other CI/CD platforms currently in use.
> >>>>
> >>>> Is there any interest from the ASF to enable this for all Gitbox
> >>>> projects when it becomes fully public?
> >>>>
> >>>
> >>> Absolutely. The Infrastructure team would love to see groups
> try this
> >> out,
> >>> and share the experiences here.
> >>>
> >>> If there are any hurdles, then share them and we'll try to
> knock them
> >> down.
> >>>
> >>> I am also interested in being able to push to our website
> automatically
> >>>> using Github Actions. If the git token that can push to a
> particular
> >>>> website repository is added as a secret [2] to Github Actions,
> this
> >>>> would be pretty easy to use for projects to automate the
> building of
> >>>> their websites.
> >>>>
> >>>
> >>> Should be possible. Again, comes back to groups trying this and
> reporting
> >>> back how well it went.
> >>>
> >>> Cheers,
> >>> Greg Stein
> >>> Infrastructure Administrator, ASF
> >>>
> >>
> >
>
Re: Github Actions
Posted by Greg Stein <gs...@gmail.com>.
Hi Francis,
Is the token needed to push from calcite to calcite-site? Is that an oauth
token or something? And are you able to use the repository settings to add
secrets, but you don't have the right token? Or you cannot add secrets at
all? (I can't tell since I have superpowers)
I've added GSTEIN_TEST_SECRET to Calcite. See if you can extract/print that
into your build/action log. If so, then we can try to figure out the
security here (ie. how do we avoid Actions exfiltrating the token?)
Thanks,
-g
On Tue, Aug 27, 2019 at 5:19 AM Francis Chuang <fr...@apache.org>
wrote:
> I have implemented the ability to generate the website and javadoc for
> Calcite using Github Actions. See:
> https://github.com/apache/calcite/tree/test-site/.github/workflows
>
> The missing piece is that we need the token to publish to our
> calcite-site repository to be added as a secret in Github Actions and
> there is currently no clear process as to whether this is allowed or how
> to get this done.
>
> See:
> https://issues.apache.org/jira/browse/INFRA-18874
> https://issues.apache.org/jira/browse/INFRA-18875
>
> Francis
>
> On 27/08/2019 7:52 pm, Greg Stein wrote:
> > Have you had an opportunity to make progress on this, to share with us?
> >
> > Anybody else with news?
> >
> > Thanks!
> > -g
> > InfraAdmin, ASF
> >
> >
> > On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise <kh...@gmx.de>
> > wrote:
> >
> >> Hi,
> >>
> >> I've made a simple PoC for the Apache Maven Dependency Plugin on a
> >> separate branch.
> >>
> >> I will try within the next days more features for example Mac OS builds
> >> etc.
> >>
> >>
> >> Currently I simply push my changes via gitbox ..
> >>
> >> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
> >> origin https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> >> (fetch)
> >> origin https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> >> (push)
> >>
> >>
> >> Also I'm interested to use SonarCloud related with GitHub Actions..?
> >>
> >>
> >> Kind regards
> >> Karl Heinz Marbaise
> >> Apache Maven PMC
> >>
> >> [1]: https://github.com/apache/maven-dependency-plugin/runs/192633340
> >> [2]:
> >>
> >>
> https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
> >>
> >> On 12.08.19 00:31, Greg Stein wrote:
> >>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <
> francischuang@apache.org
> >>>
> >>> wrote:
> >>>> ...
> >>>
> >>>> I think there are quite a few ASF projects using gitbox and Github and
> >>>> this would be a very good complement or replacement for Travis,
> appvoyer
> >>>> and other CI/CD platforms currently in use.
> >>>>
> >>>> Is there any interest from the ASF to enable this for all Gitbox
> >>>> projects when it becomes fully public?
> >>>>
> >>>
> >>> Absolutely. The Infrastructure team would love to see groups try this
> >> out,
> >>> and share the experiences here.
> >>>
> >>> If there are any hurdles, then share them and we'll try to knock them
> >> down.
> >>>
> >>> I am also interested in being able to push to our website automatically
> >>>> using Github Actions. If the git token that can push to a particular
> >>>> website repository is added as a secret [2] to Github Actions, this
> >>>> would be pretty easy to use for projects to automate the building of
> >>>> their websites.
> >>>>
> >>>
> >>> Should be possible. Again, comes back to groups trying this and
> reporting
> >>> back how well it went.
> >>>
> >>> Cheers,
> >>> Greg Stein
> >>> Infrastructure Administrator, ASF
> >>>
> >>
> >
>
Re: Github Actions
Posted by Francis Chuang <fr...@apache.org>.
I have implemented the ability to generate the website and javadoc for
Calcite using Github Actions. See:
https://github.com/apache/calcite/tree/test-site/.github/workflows
The missing piece is that we need the token to publish to our
calcite-site repository to be added as a secret in Github Actions and
there is currently no clear process as to whether this is allowed or how
to get this done.
See:
https://issues.apache.org/jira/browse/INFRA-18874
https://issues.apache.org/jira/browse/INFRA-18875
Francis
On 27/08/2019 7:52 pm, Greg Stein wrote:
> Have you had an opportunity to make progress on this, to share with us?
>
> Anybody else with news?
>
> Thanks!
> -g
> InfraAdmin, ASF
>
>
> On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise <kh...@gmx.de>
> wrote:
>
>> Hi,
>>
>> I've made a simple PoC for the Apache Maven Dependency Plugin on a
>> separate branch.
>>
>> I will try within the next days more features for example Mac OS builds
>> etc.
>>
>>
>> Currently I simply push my changes via gitbox ..
>>
>> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
>> origin https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
>> (fetch)
>> origin https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
>> (push)
>>
>>
>> Also I'm interested to use SonarCloud related with GitHub Actions..?
>>
>>
>> Kind regards
>> Karl Heinz Marbaise
>> Apache Maven PMC
>>
>> [1]: https://github.com/apache/maven-dependency-plugin/runs/192633340
>> [2]:
>>
>> https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
>>
>> On 12.08.19 00:31, Greg Stein wrote:
>>> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <francischuang@apache.org
>>>
>>> wrote:
>>>> ...
>>>
>>>> I think there are quite a few ASF projects using gitbox and Github and
>>>> this would be a very good complement or replacement for Travis, appvoyer
>>>> and other CI/CD platforms currently in use.
>>>>
>>>> Is there any interest from the ASF to enable this for all Gitbox
>>>> projects when it becomes fully public?
>>>>
>>>
>>> Absolutely. The Infrastructure team would love to see groups try this
>> out,
>>> and share the experiences here.
>>>
>>> If there are any hurdles, then share them and we'll try to knock them
>> down.
>>>
>>> I am also interested in being able to push to our website automatically
>>>> using Github Actions. If the git token that can push to a particular
>>>> website repository is added as a secret [2] to Github Actions, this
>>>> would be pretty easy to use for projects to automate the building of
>>>> their websites.
>>>>
>>>
>>> Should be possible. Again, comes back to groups trying this and reporting
>>> back how well it went.
>>>
>>> Cheers,
>>> Greg Stein
>>> Infrastructure Administrator, ASF
>>>
>>
>
Re: Github Actions
Posted by Greg Stein <gs...@gmail.com>.
Have you had an opportunity to make progress on this, to share with us?
Anybody else with news?
Thanks!
-g
InfraAdmin, ASF
On Tue, Aug 13, 2019 at 3:59 PM Karl Heinz Marbaise <kh...@gmx.de>
wrote:
> Hi,
>
> I've made a simple PoC for the Apache Maven Dependency Plugin on a
> separate branch.
>
> I will try within the next days more features for example Mac OS builds
> etc.
>
>
> Currently I simply push my changes via gitbox ..
>
> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
> origin https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> (fetch)
> origin https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> (push)
>
>
> Also I'm interested to use SonarCloud related with GitHub Actions..?
>
>
> Kind regards
> Karl Heinz Marbaise
> Apache Maven PMC
>
> [1]: https://github.com/apache/maven-dependency-plugin/runs/192633340
> [2]:
>
> https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
>
> On 12.08.19 00:31, Greg Stein wrote:
> > On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <francischuang@apache.org
> >
> > wrote:
> >> ...
> >
> >> I think there are quite a few ASF projects using gitbox and Github and
> >> this would be a very good complement or replacement for Travis, appvoyer
> >> and other CI/CD platforms currently in use.
> >>
> >> Is there any interest from the ASF to enable this for all Gitbox
> >> projects when it becomes fully public?
> >>
> >
> > Absolutely. The Infrastructure team would love to see groups try this
> out,
> > and share the experiences here.
> >
> > If there are any hurdles, then share them and we'll try to knock them
> down.
> >
> > I am also interested in being able to push to our website automatically
> >> using Github Actions. If the git token that can push to a particular
> >> website repository is added as a secret [2] to Github Actions, this
> >> would be pretty easy to use for projects to automate the building of
> >> their websites.
> >>
> >
> > Should be possible. Again, comes back to groups trying this and reporting
> > back how well it went.
> >
> > Cheers,
> > Greg Stein
> > Infrastructure Administrator, ASF
> >
>
Re: Github Actions
Posted by Robert Munteanu <ro...@apache.org>.
Hi Karl Heinz,
Do you plan on investigating s setup similar to your gitbox/Jenkins
one, where you manage all your builds using a common Jenkinsfile? I was
curious whether Github actions support such a pattern.
Thanks,
Robert
On Tue, 2019-08-13 at 22:59 +0200, Karl Heinz Marbaise wrote:
> Hi,
>
> I've made a simple PoC for the Apache Maven Dependency Plugin on a
> separate branch.
>
> I will try within the next days more features for example Mac OS
> builds etc.
>
>
> Currently I simply push my changes via gitbox ..
>
> maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
> origin
> https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> (fetch)
> origin
> https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
> (push)
>
>
> Also I'm interested to use SonarCloud related with GitHub Actions..?
>
>
> Kind regards
> Karl Heinz Marbaise
> Apache Maven PMC
>
> [1]: https://github.com/apache/maven-dependency-plugin/runs/192633340
> [2]:
> https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
>
> On 12.08.19 00:31, Greg Stein wrote:
> > On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <
> > francischuang@apache.org>
> > wrote:
> > > ...
> > > I think there are quite a few ASF projects using gitbox and
> > > Github and
> > > this would be a very good complement or replacement for Travis,
> > > appvoyer
> > > and other CI/CD platforms currently in use.
> > >
> > > Is there any interest from the ASF to enable this for all Gitbox
> > > projects when it becomes fully public?
> > >
> >
> > Absolutely. The Infrastructure team would love to see groups try
> > this out,
> > and share the experiences here.
> >
> > If there are any hurdles, then share them and we'll try to knock
> > them down.
> >
> > I am also interested in being able to push to our website
> > automatically
> > > using Github Actions. If the git token that can push to a
> > > particular
> > > website repository is added as a secret [2] to Github Actions,
> > > this
> > > would be pretty easy to use for projects to automate the building
> > > of
> > > their websites.
> > >
> >
> > Should be possible. Again, comes back to groups trying this and
> > reporting
> > back how well it went.
> >
> > Cheers,
> > Greg Stein
> > Infrastructure Administrator, ASF
> >
Re: Github Actions
Posted by Karl Heinz Marbaise <kh...@gmx.de>.
Hi,
I've made a simple PoC for the Apache Maven Dependency Plugin on a
separate branch.
I will try within the next days more features for example Mac OS builds etc.
Currently I simply push my changes via gitbox ..
maven-dependency-plugin (GITHUB_ACTIONS)$ git remote -v
origin https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
(fetch)
origin https://gitbox.apache.org/repos/asf/maven-dependency-plugin.git
(push)
Also I'm interested to use SonarCloud related with GitHub Actions..?
Kind regards
Karl Heinz Marbaise
Apache Maven PMC
[1]: https://github.com/apache/maven-dependency-plugin/runs/192633340
[2]:
https://github.com/apache/maven-dependency-plugin/blob/66435b225e7885f44b25207e025469f6d5237107/.github/workflows/maven.yml
On 12.08.19 00:31, Greg Stein wrote:
> On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <fr...@apache.org>
> wrote:
>> ...
>
>> I think there are quite a few ASF projects using gitbox and Github and
>> this would be a very good complement or replacement for Travis, appvoyer
>> and other CI/CD platforms currently in use.
>>
>> Is there any interest from the ASF to enable this for all Gitbox
>> projects when it becomes fully public?
>>
>
> Absolutely. The Infrastructure team would love to see groups try this out,
> and share the experiences here.
>
> If there are any hurdles, then share them and we'll try to knock them down.
>
> I am also interested in being able to push to our website automatically
>> using Github Actions. If the git token that can push to a particular
>> website repository is added as a secret [2] to Github Actions, this
>> would be pretty easy to use for projects to automate the building of
>> their websites.
>>
>
> Should be possible. Again, comes back to groups trying this and reporting
> back how well it went.
>
> Cheers,
> Greg Stein
> Infrastructure Administrator, ASF
>
Re: Github Actions
Posted by Greg Stein <gs...@gmail.com>.
On Sun, Aug 11, 2019 at 5:15 PM Francis Chuang <fr...@apache.org>
wrote:
>...
> I think there are quite a few ASF projects using gitbox and Github and
> this would be a very good complement or replacement for Travis, appvoyer
> and other CI/CD platforms currently in use.
>
> Is there any interest from the ASF to enable this for all Gitbox
> projects when it becomes fully public?
>
Absolutely. The Infrastructure team would love to see groups try this out,
and share the experiences here.
If there are any hurdles, then share them and we'll try to knock them down.
I am also interested in being able to push to our website automatically
> using Github Actions. If the git token that can push to a particular
> website repository is added as a secret [2] to Github Actions, this
> would be pretty easy to use for projects to automate the building of
> their websites.
>
Should be possible. Again, comes back to groups trying this and reporting
back how well it went.
Cheers,
Greg Stein
Infrastructure Administrator, ASF