You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/11/07 09:27:47 UTC
[GitHub] [airflow] potiuk opened a new issue #12150: Production image has only amazon and google providers installed
potiuk opened a new issue #12150:
URL: https://github.com/apache/airflow/issues/12150
When "production" image is prepared for, only amazon and google providers are installed from sources.
**Apache Airflow version**:
master
**What you expected to happen**:
All providers should be installed
**How to reproduce it**:
```
./breeze --production-image --python 3.6
```
Then:
```
./breeze --production-image --python 3.6 shell bash
```
then
```
ls -la ~/.local/lib/python3.6/site-packages/airflow/providers/
amazon
google
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12150: Production image has only amazon and google providers installed
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12150:
URL: https://github.com/apache/airflow/issues/12150#issuecomment-723899987
Then lets not call it production -- if it's not installing modules as close as possible to the same way we will for a release (i.e. with pip from wheels) then it's not production, it's a light(er)-weight CI image.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12150: Production image has only amazon and google providers installed
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12150:
URL: https://github.com/apache/airflow/issues/12150#issuecomment-723868905
The "production" image should be installing the providers packages anyway.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12150: Production image has only amazon and google providers installed
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12150:
URL: https://github.com/apache/airflow/issues/12150#issuecomment-723914964
As discussed - once we release providers via PYPI, I will change the build to use .whl backport packages to make the image from master even closer to the real "production" one we are going to have. Let's do it incrementally :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #12150: Production image has only amazon and google providers installed
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #12150:
URL: https://github.com/apache/airflow/issues/12150#issuecomment-723896440
> Sorry, I wasn't clear. Should be installing the providers packages _from split out providers_, not by having them be part of the apache-airflow dist.
>
> Else it's not "production".
If you install it from released PIP packages it will work (but only for the packages installed by default). but if you want to build master version of the production image which uses the latest sources (and which many developers use to try out the latest and greatest), and which WE use for kubernetes and helm chart testing then yes. We definitely need to do it regardless of #11464.
And we have to do it all the time if we do not want to be surprised at the release time that the image does not work. This is something we do for about 6 months already.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12150: Production image has only amazon and google providers installed
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12150:
URL: https://github.com/apache/airflow/issues/12150#issuecomment-723435780
We need to merge it for Beta1 - otherwise the production images will not be usable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #12150: Production image has only amazon and google providers installed
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #12150:
URL: https://github.com/apache/airflow/issues/12150
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12150: Production image has only amazon and google providers installed
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12150:
URL: https://github.com/apache/airflow/issues/12150#issuecomment-723878714
Sorry, I wasn't clear. Should be installing the providers packages _from split out providers_, not by having them be part of the apache-airflow dist.
Else it's not "production".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12150: Production image has only amazon and google providers installed
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12150:
URL: https://github.com/apache/airflow/issues/12150#issuecomment-723431136
Confirmed that was an aftermath of #10806. adding a fix shortly and protection against missing installation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12150: Production image has only amazon and google providers installed
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12150:
URL: https://github.com/apache/airflow/issues/12150#issuecomment-723896440
> Sorry, I wasn't clear. Should be installing the providers packages _from split out providers_, not by having them be part of the apache-airflow dist.
>
> Else it's not "production".
If you install it from released PIP packages it will work. but if you want to build master version of the production image which uses the latest sources (and which many developers use to try out the latest and greatest), and which WE use for kubernetes and helm chart testing then yes. We definitely need to do it regardless of #11464.
And we have to do it all the time if we do not want to be surprised at the release time that the image does not work. This is something we do for about 6 months already.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12150: Production image has only amazon and google providers installed
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12150:
URL: https://github.com/apache/airflow/issues/12150#issuecomment-723881900
If we do #11464 do we even need to do anything?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12150: Production image has only amazon and google providers installed
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12150:
URL: https://github.com/apache/airflow/issues/12150#issuecomment-723869659
Yep. It was until the implicit packages change
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org