[GitHub] [arrow-rs] alamb commented on issue #165: [CI] Try out buildkite

[GitBox <gi...@apache.org>]

alamb commented on issue #165:
URL: https://github.com/apache/arrow-rs/issues/165#issuecomment-826805929


   Comment from Jorge Leitão(jorgecarleitao) @ 2021-01-06T06:07:55.582+0000:
   <pre>[~wesm_impala_7e40], would it be possible to add a new pipe with a single command,
   
   {{buildkite-agent pipeline upload .buildkite/rust.yml}}
   
   with the default github integration?</pre>
   
   Comment from Jorge Leitão(jorgecarleitao) @ 2021-01-06T06:16:00.080+0000:
   <pre>The main blocker I see atm is that buildkite does not support starting pipelines only if some part of the code changes: [https://github.com/buildkite/feedback/issues/243]</pre>
   
   Comment from Jorge Leitão(jorgecarleitao) @ 2021-01-06T06:26:09.316+0000:
   <pre>Example of the pipeline running on a mac: [https://buildkite.com/test-811/example/builds/36#c19196dc-72dd-4c3c-868e-893dd392a63a]
   
   can't make cache working. can't make artifacts working.</pre>
   
   Comment from Antoine Pitrou(apitrou) @ 2021-01-06T09:39:03.248+0000:
   <pre>You can still adapt the {{ci/detect_changes.py}} script to limit certain builds depending on the changed files.</pre>
   
   Comment from Jorge Leitão(jorgecarleitao) @ 2021-01-06T19:17:45.515+0000:
   <pre>[~apitrou], thanks! [~kszucs] solved this in a smarter way within buildkite. I was also able to address the caching / artifacts, which unblocks this.
   
   I have pushed the latest version to github. [~kszucs], it assumes that there are 3 queues:
    * {{docker}}, any machine that can run a docker job using the docker plugin (basically can run docker)
    * {{macos}}, macos machine with rust installed
    * {{windows}}, windows machine with rust installed
   
   If / when we have the runners, the pipeline should reproduce the same jobs that we have on github for Rust.
   
   I do not have strong opinions here, I just used the same setup that we have on github wrt to the runners. It looks as follows on my repo / account, where I have one (macos) agent: [https://buildkite.com/test-811/example/builds?branch=buildkite]
   
    </pre>
   
   Comment from Jorge Leitão(jorgecarleitao) @ 2021-01-06T19:20:05.869+0000:
   <pre>I cannot proceed further without the runners on apache account and therefore will park for now. Thanks a lot for your help!</pre>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org