You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesatee.apache.org by Yu Ding <di...@apache.org> on 2019/10/21 01:23:05 UTC
MesaTEE CI migration
Currently MesaTEE and rust-sgx-sdk depends on a customized drone CI system.
We need to migrate to the Jenkins hosted by ASF.
The challenge is that MesaTEE and rust-sgx-sdk needs some specific hardware
to finish the build and tests:- Only Intel E3 series CPU which supports
Intel SGX can finish the tests. Other Intel CPUs can only do simulation
tests. AMD cpu cannot do any tests.- In future, we need TPM hardware to
run TPM-based tests.
And currently the only CI which supports peripheral devices is drone (
https://drone.io/).
To migrate the CI pipelines to ASF's Jenkins, I think we can make it work
together with our current drone service:
- Jenkin's Build stage simply builds the project as usual.
- Test stage triggers the current drone services to run tests based on the
output of Build stage.
In this way we can test MesaTEE and rust-sgx-sdk in hardware mode. To
enable this, we may need to write some scripts to trigger Drone and collect
the results. Drone would help on load balancing.
Comments are welcomed.
Yu
Re: MesaTEE CI migration
Posted by Mingshen Sun <ms...@apache.org>.
Hi all, since the infra team is still discussing the possibility of using
GitHub Actions (
https://issues.apache.org/jira/browse/INFRA-19381?focusedCommentId=16976743&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16976743).
We need to manually check all PRs. Thanks.
On Tue, Nov 12, 2019 at 4:37 PM Mingshen Sun <ms...@apache.org> wrote:
> Submitting the request in Jira (
> https://issues.apache.org/jira/browse/INFRA-19381?focusedCommentId=16972919&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16972919).
> Hopefully, we can get it ready in this week since there is a blocking PR
> right now.
>
> On Tue, Nov 12, 2019 at 4:25 PM Mingshen Sun <ms...@apache.org> wrote:
>
>> After some investigation on GitHub Action (self-hosted) and Jenkins, both
>> of them can fit our requirements. However, I think GitHub Action is a
>> better option. Though Jenkins is more flexible, the the maintenance efforts
>> is very expensive. I'll try to contact the infra team and try to quickly
>> have the GitHub Action (self-hosted) ready.
>>
>> On Tue, Nov 12, 2019 at 12:00 PM Mingshen Sun <ms...@apache.org> wrote:
>>
>>> For the CI migration, I found that our previous CI system needs to
>>> install an self-created OAuth app. I guess this is not acceptable for the
>>> Apache infra. I'm checking with the team right now (
>>> https://issues.apache.org/jira/browse/INFRA-19381?focusedCommentId=16972727&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16972727
>>> ).
>>>
>>> In the meantime, we tried the GitHub Action. Seems that the self-hosted
>>> GitHub Action (
>>> https://github.blog/2019-11-05-self-hosted-runners-for-github-actions-is-now-in-beta/)
>>> could be an alternative, which may be more comfortable.
>>>
>>> On Sun, Oct 20, 2019 at 7:55 PM Luciano Resende <lu...@gmail.com>
>>> wrote:
>>>
>>>> On Sun, Oct 20, 2019 at 6:23 PM Yu Ding <di...@apache.org> wrote:
>>>> >
>>>> > Currently MesaTEE and rust-sgx-sdk depends on a customized drone CI
>>>> system.
>>>> > We need to migrate to the Jenkins hosted by ASF.
>>>> >
>>>> > The challenge is that MesaTEE and rust-sgx-sdk needs some specific
>>>> hardware
>>>> > to finish the build and tests:- Only Intel E3 series CPU which
>>>> supports
>>>> > Intel SGX can finish the tests. Other Intel CPUs can only do
>>>> simulation
>>>> > tests. AMD cpu cannot do any tests.- In future, we need TPM hardware
>>>> to
>>>> > run TPM-based tests.
>>>> >
>>>> > And currently the only CI which supports peripheral devices is drone (
>>>> > https://drone.io/).
>>>> >
>>>> > To migrate the CI pipelines to ASF's Jenkins, I think we can make it
>>>> work
>>>> > together with our current drone service:
>>>> > - Jenkin's Build stage simply builds the project as usual.
>>>> > - Test stage triggers the current drone services to run tests based
>>>> on the
>>>> > output of Build stage.
>>>> >
>>>> > In this way we can test MesaTEE and rust-sgx-sdk in hardware mode. To
>>>> > enable this, we may need to write some scripts to trigger Drone and
>>>> collect
>>>> > the results. Drone would help on load balancing.
>>>> >
>>>> > Comments are welcomed.
>>>> >
>>>> > Yu
>>>>
>>>> Well, if Drone IO provides public visibility for the actual tests, and
>>>> is currently serving the community, there is no need to force moving
>>>> to Apache Jenkins hosted by ASF. There are several Apache projects
>>>> that are currently using other CI environments, such as Travis CI,
>>>> self hosted (but publicly available for status checking), etc.
>>>>
>>>> --
>>>> Luciano Resende
>>>> http://twitter.com/lresende1975
>>>> http://lresende.blogspot.com/
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@mesatee.apache.org
>>>> For additional commands, e-mail: dev-help@mesatee.apache.org
>>>>
>>>
Re: MesaTEE CI migration
Posted by Mingshen Sun <ms...@apache.org>.
Submitting the request in Jira (
https://issues.apache.org/jira/browse/INFRA-19381?focusedCommentId=16972919&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16972919).
Hopefully, we can get it ready in this week since there is a blocking PR
right now.
On Tue, Nov 12, 2019 at 4:25 PM Mingshen Sun <ms...@apache.org> wrote:
> After some investigation on GitHub Action (self-hosted) and Jenkins, both
> of them can fit our requirements. However, I think GitHub Action is a
> better option. Though Jenkins is more flexible, the the maintenance efforts
> is very expensive. I'll try to contact the infra team and try to quickly
> have the GitHub Action (self-hosted) ready.
>
> On Tue, Nov 12, 2019 at 12:00 PM Mingshen Sun <ms...@apache.org> wrote:
>
>> For the CI migration, I found that our previous CI system needs to
>> install an self-created OAuth app. I guess this is not acceptable for the
>> Apache infra. I'm checking with the team right now (
>> https://issues.apache.org/jira/browse/INFRA-19381?focusedCommentId=16972727&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16972727
>> ).
>>
>> In the meantime, we tried the GitHub Action. Seems that the self-hosted
>> GitHub Action (
>> https://github.blog/2019-11-05-self-hosted-runners-for-github-actions-is-now-in-beta/)
>> could be an alternative, which may be more comfortable.
>>
>> On Sun, Oct 20, 2019 at 7:55 PM Luciano Resende <lu...@gmail.com>
>> wrote:
>>
>>> On Sun, Oct 20, 2019 at 6:23 PM Yu Ding <di...@apache.org> wrote:
>>> >
>>> > Currently MesaTEE and rust-sgx-sdk depends on a customized drone CI
>>> system.
>>> > We need to migrate to the Jenkins hosted by ASF.
>>> >
>>> > The challenge is that MesaTEE and rust-sgx-sdk needs some specific
>>> hardware
>>> > to finish the build and tests:- Only Intel E3 series CPU which supports
>>> > Intel SGX can finish the tests. Other Intel CPUs can only do simulation
>>> > tests. AMD cpu cannot do any tests.- In future, we need TPM hardware
>>> to
>>> > run TPM-based tests.
>>> >
>>> > And currently the only CI which supports peripheral devices is drone (
>>> > https://drone.io/).
>>> >
>>> > To migrate the CI pipelines to ASF's Jenkins, I think we can make it
>>> work
>>> > together with our current drone service:
>>> > - Jenkin's Build stage simply builds the project as usual.
>>> > - Test stage triggers the current drone services to run tests based on
>>> the
>>> > output of Build stage.
>>> >
>>> > In this way we can test MesaTEE and rust-sgx-sdk in hardware mode. To
>>> > enable this, we may need to write some scripts to trigger Drone and
>>> collect
>>> > the results. Drone would help on load balancing.
>>> >
>>> > Comments are welcomed.
>>> >
>>> > Yu
>>>
>>> Well, if Drone IO provides public visibility for the actual tests, and
>>> is currently serving the community, there is no need to force moving
>>> to Apache Jenkins hosted by ASF. There are several Apache projects
>>> that are currently using other CI environments, such as Travis CI,
>>> self hosted (but publicly available for status checking), etc.
>>>
>>> --
>>> Luciano Resende
>>> http://twitter.com/lresende1975
>>> http://lresende.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@mesatee.apache.org
>>> For additional commands, e-mail: dev-help@mesatee.apache.org
>>>
>>
Re: MesaTEE CI migration
Posted by Mingshen Sun <ms...@apache.org>.
After some investigation on GitHub Action (self-hosted) and Jenkins, both
of them can fit our requirements. However, I think GitHub Action is a
better option. Though Jenkins is more flexible, the the maintenance efforts
is very expensive. I'll try to contact the infra team and try to quickly
have the GitHub Action (self-hosted) ready.
On Tue, Nov 12, 2019 at 12:00 PM Mingshen Sun <ms...@apache.org> wrote:
> For the CI migration, I found that our previous CI system needs to install
> an self-created OAuth app. I guess this is not acceptable for the Apache
> infra. I'm checking with the team right now (
> https://issues.apache.org/jira/browse/INFRA-19381?focusedCommentId=16972727&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16972727
> ).
>
> In the meantime, we tried the GitHub Action. Seems that the self-hosted
> GitHub Action (
> https://github.blog/2019-11-05-self-hosted-runners-for-github-actions-is-now-in-beta/)
> could be an alternative, which may be more comfortable.
>
> On Sun, Oct 20, 2019 at 7:55 PM Luciano Resende <lu...@gmail.com>
> wrote:
>
>> On Sun, Oct 20, 2019 at 6:23 PM Yu Ding <di...@apache.org> wrote:
>> >
>> > Currently MesaTEE and rust-sgx-sdk depends on a customized drone CI
>> system.
>> > We need to migrate to the Jenkins hosted by ASF.
>> >
>> > The challenge is that MesaTEE and rust-sgx-sdk needs some specific
>> hardware
>> > to finish the build and tests:- Only Intel E3 series CPU which supports
>> > Intel SGX can finish the tests. Other Intel CPUs can only do simulation
>> > tests. AMD cpu cannot do any tests.- In future, we need TPM hardware to
>> > run TPM-based tests.
>> >
>> > And currently the only CI which supports peripheral devices is drone (
>> > https://drone.io/).
>> >
>> > To migrate the CI pipelines to ASF's Jenkins, I think we can make it
>> work
>> > together with our current drone service:
>> > - Jenkin's Build stage simply builds the project as usual.
>> > - Test stage triggers the current drone services to run tests based on
>> the
>> > output of Build stage.
>> >
>> > In this way we can test MesaTEE and rust-sgx-sdk in hardware mode. To
>> > enable this, we may need to write some scripts to trigger Drone and
>> collect
>> > the results. Drone would help on load balancing.
>> >
>> > Comments are welcomed.
>> >
>> > Yu
>>
>> Well, if Drone IO provides public visibility for the actual tests, and
>> is currently serving the community, there is no need to force moving
>> to Apache Jenkins hosted by ASF. There are several Apache projects
>> that are currently using other CI environments, such as Travis CI,
>> self hosted (but publicly available for status checking), etc.
>>
>> --
>> Luciano Resende
>> http://twitter.com/lresende1975
>> http://lresende.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@mesatee.apache.org
>> For additional commands, e-mail: dev-help@mesatee.apache.org
>>
>
Re: MesaTEE CI migration
Posted by Mingshen Sun <ms...@apache.org>.
For the CI migration, I found that our previous CI system needs to install
an self-created OAuth app. I guess this is not acceptable for the Apache
infra. I'm checking with the team right now (
https://issues.apache.org/jira/browse/INFRA-19381?focusedCommentId=16972727&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16972727
).
In the meantime, we tried the GitHub Action. Seems that the self-hosted
GitHub Action (
https://github.blog/2019-11-05-self-hosted-runners-for-github-actions-is-now-in-beta/)
could be an alternative, which may be more comfortable.
On Sun, Oct 20, 2019 at 7:55 PM Luciano Resende <lu...@gmail.com>
wrote:
> On Sun, Oct 20, 2019 at 6:23 PM Yu Ding <di...@apache.org> wrote:
> >
> > Currently MesaTEE and rust-sgx-sdk depends on a customized drone CI
> system.
> > We need to migrate to the Jenkins hosted by ASF.
> >
> > The challenge is that MesaTEE and rust-sgx-sdk needs some specific
> hardware
> > to finish the build and tests:- Only Intel E3 series CPU which supports
> > Intel SGX can finish the tests. Other Intel CPUs can only do simulation
> > tests. AMD cpu cannot do any tests.- In future, we need TPM hardware to
> > run TPM-based tests.
> >
> > And currently the only CI which supports peripheral devices is drone (
> > https://drone.io/).
> >
> > To migrate the CI pipelines to ASF's Jenkins, I think we can make it work
> > together with our current drone service:
> > - Jenkin's Build stage simply builds the project as usual.
> > - Test stage triggers the current drone services to run tests based on
> the
> > output of Build stage.
> >
> > In this way we can test MesaTEE and rust-sgx-sdk in hardware mode. To
> > enable this, we may need to write some scripts to trigger Drone and
> collect
> > the results. Drone would help on load balancing.
> >
> > Comments are welcomed.
> >
> > Yu
>
> Well, if Drone IO provides public visibility for the actual tests, and
> is currently serving the community, there is no need to force moving
> to Apache Jenkins hosted by ASF. There are several Apache projects
> that are currently using other CI environments, such as Travis CI,
> self hosted (but publicly available for status checking), etc.
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@mesatee.apache.org
> For additional commands, e-mail: dev-help@mesatee.apache.org
>
Re: MesaTEE CI migration
Posted by Luciano Resende <lu...@gmail.com>.
On Sun, Oct 20, 2019 at 6:23 PM Yu Ding <di...@apache.org> wrote:
>
> Currently MesaTEE and rust-sgx-sdk depends on a customized drone CI system.
> We need to migrate to the Jenkins hosted by ASF.
>
> The challenge is that MesaTEE and rust-sgx-sdk needs some specific hardware
> to finish the build and tests:- Only Intel E3 series CPU which supports
> Intel SGX can finish the tests. Other Intel CPUs can only do simulation
> tests. AMD cpu cannot do any tests.- In future, we need TPM hardware to
> run TPM-based tests.
>
> And currently the only CI which supports peripheral devices is drone (
> https://drone.io/).
>
> To migrate the CI pipelines to ASF's Jenkins, I think we can make it work
> together with our current drone service:
> - Jenkin's Build stage simply builds the project as usual.
> - Test stage triggers the current drone services to run tests based on the
> output of Build stage.
>
> In this way we can test MesaTEE and rust-sgx-sdk in hardware mode. To
> enable this, we may need to write some scripts to trigger Drone and collect
> the results. Drone would help on load balancing.
>
> Comments are welcomed.
>
> Yu
Well, if Drone IO provides public visibility for the actual tests, and
is currently serving the community, there is no need to force moving
to Apache Jenkins hosted by ASF. There are several Apache projects
that are currently using other CI environments, such as Travis CI,
self hosted (but publicly available for status checking), etc.
--
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@mesatee.apache.org
For additional commands, e-mail: dev-help@mesatee.apache.org