You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@allura.apache.org by "Vrinda A M (RBEI/EMT2)" <Vr...@in.bosch.com.INVALID> on 2020/11/11 12:49:50 UTC
CI for Allura
Hello,
I would like to setup continuous integration, testing and deployment of Allura in Jenkins. Can you please give me some hints on what needs to be configured in Jenkins?
Also what should be configured in the webhook of Allura project so that a build is triggered each time a push happens on the master branch?
Can a docker image be created after the build goes through? If so, do I need to make any configurations in the docker-compose.yml file?
Regards,
Vrinda.
RE: CI for Allura
Posted by "Vrinda A M (RBEI/EMT2)" <Vr...@in.bosch.com.INVALID>.
Thank you!
We are able to generate the report now :)
Regards,
Vrinda.
-----Original Message-----
From: Dave Brondsema <da...@brondsema.net>
Sent: Friday, December 4, 2020 10:44 PM
To: dev@allura.apache.org; Vrinda A M (RBEI/EMT2) <Vr...@in.bosch.com>
Subject: Re: CI for Allura
Hello,
When running the tests with --with-xunitmp (or --with-xunit if not doing
parallel) it should create some nosetests.xml output files.
Then in Jenkins, set up a Publish JUnit Reports step, with a file pattern of "**/nosetests.xml" Then it should produce those tests results in jenkins.
-Dave
On 12/3/20 1:02 PM, Vrinda A M (RBEI/EMT2) wrote:
> Hi Dave,
>
> Thank you! We are able to now run the build and tests with the configurations you have provided.
>
> One point that is not clear is the configuration of the 'Latest Test Result' with the consolidated report of all tests run on the platform.
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbuil
> ds.apache.org%2Fjob%2FAllura%2Fjob%2FAllura%2FlastCompletedBuild%2Ftes
> tReport%2F&data=04%7C01%7CVrinda.A%40in.bosch.com%7Cf3a233840fd443
> 27f5aa08d89878001c%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C1%7C637426
> 989044716979%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luM
> zIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mXfm9yNrXGL3GkRyuEE
> g1buOtFEiR1zFT4yOGQxIkWs%3D&reserved=0
> or
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbuil
> ds.apache.org%2Fjob%2FAllura%2Fjob%2FAllura%2F31%2FtestReport%2F&d
> ata=04%7C01%7CVrinda.A%40in.bosch.com%7Cf3a233840fd44327f5aa08d8987800
> 1c%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C1%7C637426989044726973%7CU
> nknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> WwiLCJXVCI6Mn0%3D%7C3000&sdata=qhOFyt2a26HnCT%2F48Cj6nF0mFGJ9toagc
> 1aySeQbQlI%3D&reserved=0
>
> Can you please let me know how this test report can be configured?
>
>
> Regards,
> Vrinda
>
>
> -----Original Message-----
> From: Dave Brondsema <da...@brondsema.net>
> Sent: Thursday, November 12, 2020 1:25 AM
> To: dev@allura.apache.org; Vrinda A M (RBEI/EMT2)
> <Vr...@in.bosch.com>
> Subject: Re: CI for Allura
>
> Hi,
>
> Here's basically what is configured on
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fci-b
> uilds.apache.org%2Fjob%2FAllura%2Fjob%2FAllura%2F&data=04%7C01%7CV
> rinda.A%40in.bosch.com%7Cf3a233840fd44327f5aa08d89878001c%7C0ae51e1907
> c84e4bbb6d648ee58410f4%7C0%7C1%7C637426989044726973%7CUnknown%7CTWFpbG
> Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> 3D%7C3000&sdata=DFdrd5OqdB3F%2FOGwF6d0J5PWFU6%2FjHPvPZUimvkKZe8%3D
> &reserved=0
>
> Have an "execute shell" step with:
>
> #!/bin/bash
> # need to specify the shell, else Jenkins will quit after the first non-zero exit rm -rf ".allura-venv"
> virtualenv .allura-venv
> . .allura-venv/bin/activate
> git clean -f -x # remove test.log, nosetest.xml etc (don't use -d
> since it'd remove our venv dir) pip install -r requirements.txt pip
> install nose-xunitmp ./rebuild-all.bash npm install
> LANG=en_US.UTF-8 ./run_tests --with-xunitmp retcode=$?
> exit $retcode
>
> And a publish junit reports step, with file pattern "**/nosetests.xml"
>
> I am not sure about a webhook exactly, the jenkins configs I've set up just poll SCM. But you could probably find some online examples of how to do it, since that is a general git + jenkins thing, not limited to Allura.
>
> The Docker config for allura is designed for development or a production setup with local source code files. That is, right now the Dockerfile won't build an image that includes the code. I think it would need an "ADD" line, might not be too hard to do. Let us know if you try it out. But also allura needs multiple docker images to fully run. Web & taskd maybe could run off of the same image.
> But there's also solr, mongo and likely mail handling, so I'm not sure how useful a Docker image of just allura would be.
>
> -Dave
>
> On 11/11/20 7:49 AM, Vrinda A M (RBEI/EMT2) wrote:
>> Hello,
>>
>> I would like to setup continuous integration, testing and deployment of Allura in Jenkins. Can you please give me some hints on what needs to be configured in Jenkins?
>> Also what should be configured in the webhook of Allura project so that a build is triggered each time a push happens on the master branch?
>>
>> Can a docker image be created after the build goes through? If so, do I need to make any configurations in the docker-compose.yml file?
>>
>>
>> Regards,
>> Vrinda.
>>
>>
>
>
> --
> Dave Brondsema : dave@brondsema.net
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.b
> rondsema.net%2F&data=04%7C01%7CVrinda.A%40in.bosch.com%7Cf3a233840
> fd44327f5aa08d89878001c%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C1%7C6
> 37426989044726973%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=2DCvH%2FuukhEE
> nxh2IeB23ZkYexgdbJpP3oRzIaVtnYU%3D&reserved=0 : personal
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.splike.com%2F&data=04%7C01%7CVrinda.A%40in.bosch.com%7Cf3a233840fd44327f5aa08d89878001c%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C1%7C637426989044726973%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=9ciIQiI%2FO4ahCmeISEjJUwlIeH2Y0iW7z6l6QUshz7A%3D&reserved=0 : programming
> <><
>
--
Dave Brondsema : dave@brondsema.net
https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.brondsema.net%2F&data=04%7C01%7CVrinda.A%40in.bosch.com%7Cf3a233840fd44327f5aa08d89878001c%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C1%7C637426989044726973%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=2DCvH%2FuukhEEnxh2IeB23ZkYexgdbJpP3oRzIaVtnYU%3D&reserved=0 : personal
https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.splike.com%2F&data=04%7C01%7CVrinda.A%40in.bosch.com%7Cf3a233840fd44327f5aa08d89878001c%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C1%7C637426989044726973%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=9ciIQiI%2FO4ahCmeISEjJUwlIeH2Y0iW7z6l6QUshz7A%3D&reserved=0 : programming
<><
Re: CI for Allura
Posted by Dave Brondsema <da...@brondsema.net>.
Hello,
When running the tests with --with-xunitmp (or --with-xunit if not doing
parallel) it should create some nosetests.xml output files.
Then in Jenkins, set up a Publish JUnit Reports step, with a file pattern of
"**/nosetests.xml" Then it should produce those tests results in jenkins.
-Dave
On 12/3/20 1:02 PM, Vrinda A M (RBEI/EMT2) wrote:
> Hi Dave,
>
> Thank you! We are able to now run the build and tests with the configurations you have provided.
>
> One point that is not clear is the configuration of the 'Latest Test Result' with the consolidated report of all tests run on the platform.
> https://builds.apache.org/job/Allura/job/Allura/lastCompletedBuild/testReport/
> or
> https://builds.apache.org/job/Allura/job/Allura/31/testReport/
>
> Can you please let me know how this test report can be configured?
>
>
> Regards,
> Vrinda
>
>
> -----Original Message-----
> From: Dave Brondsema <da...@brondsema.net>
> Sent: Thursday, November 12, 2020 1:25 AM
> To: dev@allura.apache.org; Vrinda A M (RBEI/EMT2) <Vr...@in.bosch.com>
> Subject: Re: CI for Allura
>
> Hi,
>
> Here's basically what is configured on
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fci-builds.apache.org%2Fjob%2FAllura%2Fjob%2FAllura%2F&data=04%7C01%7CVrinda.A%40in.bosch.com%7Cdeefa8c02eeb4064792b08d8867b9c43%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C637407213189460217%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=E9CyUsICy%2BW3pgiaTDpD5jIK09x%2F8nL19cxGUDjRkiI%3D&reserved=0
>
> Have an "execute shell" step with:
>
> #!/bin/bash
> # need to specify the shell, else Jenkins will quit after the first non-zero exit rm -rf ".allura-venv"
> virtualenv .allura-venv
> . .allura-venv/bin/activate
> git clean -f -x # remove test.log, nosetest.xml etc (don't use -d since it'd remove our venv dir) pip install -r requirements.txt pip install nose-xunitmp ./rebuild-all.bash npm install
> LANG=en_US.UTF-8 ./run_tests --with-xunitmp retcode=$?
> exit $retcode
>
> And a publish junit reports step, with file pattern "**/nosetests.xml"
>
> I am not sure about a webhook exactly, the jenkins configs I've set up just poll SCM. But you could probably find some online examples of how to do it, since that is a general git + jenkins thing, not limited to Allura.
>
> The Docker config for allura is designed for development or a production setup with local source code files. That is, right now the Dockerfile won't build an image that includes the code. I think it would need an "ADD" line, might not be too hard to do. Let us know if you try it out. But also allura needs multiple docker images to fully run. Web & taskd maybe could run off of the same image.
> But there's also solr, mongo and likely mail handling, so I'm not sure how useful a Docker image of just allura would be.
>
> -Dave
>
> On 11/11/20 7:49 AM, Vrinda A M (RBEI/EMT2) wrote:
>> Hello,
>>
>> I would like to setup continuous integration, testing and deployment of Allura in Jenkins. Can you please give me some hints on what needs to be configured in Jenkins?
>> Also what should be configured in the webhook of Allura project so that a build is triggered each time a push happens on the master branch?
>>
>> Can a docker image be created after the build goes through? If so, do I need to make any configurations in the docker-compose.yml file?
>>
>>
>> Regards,
>> Vrinda.
>>
>>
>
>
> --
> Dave Brondsema : dave@brondsema.net
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.brondsema.net%2F&data=04%7C01%7CVrinda.A%40in.bosch.com%7Cdeefa8c02eeb4064792b08d8867b9c43%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C637407213189460217%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=2OddTqv6dHJupWw4GPqMuEaw0WfSAWy4mF5govNL3oA%3D&reserved=0 : personal
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.splike.com%2F&data=04%7C01%7CVrinda.A%40in.bosch.com%7Cdeefa8c02eeb4064792b08d8867b9c43%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C637407213189460217%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6fKZ%2FTYZ8yu9rYcv1RTR8%2Ffjos9QpYWn7ZIuV%2BqGy8A%3D&reserved=0 : programming
> <><
>
--
Dave Brondsema : dave@brondsema.net
http://www.brondsema.net : personal
http://www.splike.com : programming
<><
RE: CI for Allura
Posted by "Vrinda A M (RBEI/EMT2)" <Vr...@in.bosch.com.INVALID>.
Hi Dave,
Thank you! We are able to now run the build and tests with the configurations you have provided.
One point that is not clear is the configuration of the 'Latest Test Result' with the consolidated report of all tests run on the platform.
https://builds.apache.org/job/Allura/job/Allura/lastCompletedBuild/testReport/
or
https://builds.apache.org/job/Allura/job/Allura/31/testReport/
Can you please let me know how this test report can be configured?
Regards,
Vrinda
-----Original Message-----
From: Dave Brondsema <da...@brondsema.net>
Sent: Thursday, November 12, 2020 1:25 AM
To: dev@allura.apache.org; Vrinda A M (RBEI/EMT2) <Vr...@in.bosch.com>
Subject: Re: CI for Allura
Hi,
Here's basically what is configured on
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fci-builds.apache.org%2Fjob%2FAllura%2Fjob%2FAllura%2F&data=04%7C01%7CVrinda.A%40in.bosch.com%7Cdeefa8c02eeb4064792b08d8867b9c43%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C637407213189460217%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=E9CyUsICy%2BW3pgiaTDpD5jIK09x%2F8nL19cxGUDjRkiI%3D&reserved=0
Have an "execute shell" step with:
#!/bin/bash
# need to specify the shell, else Jenkins will quit after the first non-zero exit rm -rf ".allura-venv"
virtualenv .allura-venv
. .allura-venv/bin/activate
git clean -f -x # remove test.log, nosetest.xml etc (don't use -d since it'd remove our venv dir) pip install -r requirements.txt pip install nose-xunitmp ./rebuild-all.bash npm install
LANG=en_US.UTF-8 ./run_tests --with-xunitmp retcode=$?
exit $retcode
And a publish junit reports step, with file pattern "**/nosetests.xml"
I am not sure about a webhook exactly, the jenkins configs I've set up just poll SCM. But you could probably find some online examples of how to do it, since that is a general git + jenkins thing, not limited to Allura.
The Docker config for allura is designed for development or a production setup with local source code files. That is, right now the Dockerfile won't build an image that includes the code. I think it would need an "ADD" line, might not be too hard to do. Let us know if you try it out. But also allura needs multiple docker images to fully run. Web & taskd maybe could run off of the same image.
But there's also solr, mongo and likely mail handling, so I'm not sure how useful a Docker image of just allura would be.
-Dave
On 11/11/20 7:49 AM, Vrinda A M (RBEI/EMT2) wrote:
> Hello,
>
> I would like to setup continuous integration, testing and deployment of Allura in Jenkins. Can you please give me some hints on what needs to be configured in Jenkins?
> Also what should be configured in the webhook of Allura project so that a build is triggered each time a push happens on the master branch?
>
> Can a docker image be created after the build goes through? If so, do I need to make any configurations in the docker-compose.yml file?
>
>
> Regards,
> Vrinda.
>
>
--
Dave Brondsema : dave@brondsema.net
https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.brondsema.net%2F&data=04%7C01%7CVrinda.A%40in.bosch.com%7Cdeefa8c02eeb4064792b08d8867b9c43%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C637407213189460217%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=2OddTqv6dHJupWw4GPqMuEaw0WfSAWy4mF5govNL3oA%3D&reserved=0 : personal
https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.splike.com%2F&data=04%7C01%7CVrinda.A%40in.bosch.com%7Cdeefa8c02eeb4064792b08d8867b9c43%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C637407213189460217%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6fKZ%2FTYZ8yu9rYcv1RTR8%2Ffjos9QpYWn7ZIuV%2BqGy8A%3D&reserved=0 : programming
<><
Re: CI for Allura
Posted by Dave Brondsema <da...@brondsema.net>.
Hi,
Here's basically what is configured on
https://ci-builds.apache.org/job/Allura/job/Allura/
Have an "execute shell" step with:
#!/bin/bash
# need to specify the shell, else Jenkins will quit after the first non-zero exit
rm -rf ".allura-venv"
virtualenv .allura-venv
. .allura-venv/bin/activate
git clean -f -x # remove test.log, nosetest.xml etc (don't use -d since it'd
remove our venv dir)
pip install -r requirements.txt
pip install nose-xunitmp
./rebuild-all.bash
npm install
LANG=en_US.UTF-8 ./run_tests --with-xunitmp
retcode=$?
exit $retcode
And a publish junit reports step, with file pattern "**/nosetests.xml"
I am not sure about a webhook exactly, the jenkins configs I've set up just poll
SCM. But you could probably find some online examples of how to do it, since
that is a general git + jenkins thing, not limited to Allura.
The Docker config for allura is designed for development or a production setup
with local source code files. That is, right now the Dockerfile won't build an
image that includes the code. I think it would need an "ADD" line, might not be
too hard to do. Let us know if you try it out. But also allura needs multiple
docker images to fully run. Web & taskd maybe could run off of the same image.
But there's also solr, mongo and likely mail handling, so I'm not sure how
useful a Docker image of just allura would be.
-Dave
On 11/11/20 7:49 AM, Vrinda A M (RBEI/EMT2) wrote:
> Hello,
>
> I would like to setup continuous integration, testing and deployment of Allura in Jenkins. Can you please give me some hints on what needs to be configured in Jenkins?
> Also what should be configured in the webhook of Allura project so that a build is triggered each time a push happens on the master branch?
>
> Can a docker image be created after the build goes through? If so, do I need to make any configurations in the docker-compose.yml file?
>
>
> Regards,
> Vrinda.
>
>
--
Dave Brondsema : dave@brondsema.net
http://www.brondsema.net : personal
http://www.splike.com : programming
<><