You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/11/20 15:18:10 UTC
[GitHub] [airflow] turbaszek opened a new issue #12509: Add support for using Cloud Build in breeze build-image
turbaszek opened a new issue #12509:
URL: https://github.com/apache/airflow/issues/12509
**Description**
I would like to build airflow images using external services like Google Cloud Build. In this way I don't have to run the build on local machine and have to design custom CI pipelines to build it (especially for dev purposes).
**Use case / motivation**
Building production image can take time and running this on notebooks sounds like not an optimal way of doing it. We can take advantage of system dedicated to do this task like Google Cloud Build.
**Related Issues**
N/A
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12509: Add support for using Cloud Build in breeze build-image
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12509:
URL: https://github.com/apache/airflow/issues/12509#issuecomment-731619333
And 3.8
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #12509: Add support for using Cloud Build in breeze build-image
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #12509:
URL: https://github.com/apache/airflow/issues/12509
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #12509: Add support for using Cloud Build in breeze build-image
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #12509:
URL: https://github.com/apache/airflow/issues/12509#issuecomment-731230088
WDYT @potiuk ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12509: Add support for using Cloud Build in breeze build-image
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12509:
URL: https://github.com/apache/airflow/issues/12509#issuecomment-731560924
Why not. Might be a good option to add.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12509: Add support for using Cloud Build in breeze build-image
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12509:
URL: https://github.com/apache/airflow/issues/12509#issuecomment-731589794
Fix is in #12534. Once we merge it and I push the images it shoudl be rather fast to rebuild next time
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12509: Add support for using Cloud Build in breeze build-image
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12509:
URL: https://github.com/apache/airflow/issues/12509#issuecomment-731617627
The 3.6 version (default) should be now fast to pull/build please try it out @turbaszek and let's observe it for a while to see if this is really working in a stable way I :crossed_fingers: . Now when rebuild is done it should be pretty much always reasonably fast. Let me know if it is not. I hope I fixed last errors within this part and do not plan any more structural changes that could break it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12509: Add support for using Cloud Build in breeze build-image
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12509:
URL: https://github.com/apache/airflow/issues/12509#issuecomment-731584705
However It might not be needed . I have found one reason why the images after pulling might run a bit longer (docker-context-files were not empty when I built it last time), I am going to prevent that problem in a next PR and then it shoudl be much faster next time. I think when we finally fix it building image via ci build will be much longer than locally all the time (no matter how fast machine you will have) - it will be network-bound. Sending context and pulling image will take far longer than rebuilding it in all circumstances.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12509: Add support for using Cloud Build in breeze build-image
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12509:
URL: https://github.com/apache/airflow/issues/12509#issuecomment-731617778
3.7 out
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org