You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/13 09:54:03 UTC
[GitHub] [airflow] mgorsk1 opened a new issue #20849: Provide 'vanilla' Airflow Docker image
mgorsk1 opened a new issue #20849:
URL: https://github.com/apache/airflow/issues/20849
### Description
Currently the 'minimal' version of Airflow Docker image ships with 20+ extra dependencies, which according to my understanding, are the most common used libraries. The idea of this PR to prepare another, pure Airflow image, which wouldn't have those dependencies and would only ship with core Airflow package.
[```
ARG AIRFLOW_EXTRAS="amazon,async,celery,cncf.kubernetes,dask,docker,elasticsearch,ftp,google,google_auth,grpc,hashicorp,http,ldap,microsoft.azure,mysql,odbc,pandas,postgres,redis,sendgrid,sftp,slack,ssh,statsd,virtualenv"
```](https://github.com/apache/airflow/blob/main/Dockerfile#L37)
Alternative solutions would be:
- ship separate Airflow images per each executor type (kubernetes, dask, celery, etc.)
- fix the issue with dask but it's less scalable as any of packages mentioned below could yield similar issues
### Use case/motivation
We are using official Airflow image for Airflow on our k8s cluster, which is constantly scanned for security vulnerabilities. The fact that dask is pinned to very low version is problematic for us cause there are a lot of red flags coming from this package. We do not need dask at all as we rely on KubernetesExecutor.
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20849: Provide 'vanilla' Airflow Docker image
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20849:
URL: https://github.com/apache/airflow/issues/20849#issuecomment-1012021423
Yeah. :slim image is probably a good idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #20849: Provide 'vanilla' Airflow Docker image
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #20849:
URL: https://github.com/apache/airflow/issues/20849#issuecomment-1046247344
Just want to let you know i have not forgotten about it. It is on my list but we are transitioning to python and debian bullseye support and once done i will add slim version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mgorsk1 commented on issue #20849: Provide 'vanilla' Airflow Docker image
Posted by GitBox <gi...@apache.org>.
mgorsk1 commented on issue #20849:
URL: https://github.com/apache/airflow/issues/20849#issuecomment-1012899889
Thanks @potiuk! Let me know if I can be of any help with this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20849: Provide 'vanilla' Airflow Docker image
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20849:
URL: https://github.com/apache/airflow/issues/20849#issuecomment-1081879715
And one more comment - few more things have happened since (ARM support and some optimisations). This one is "still" on my list after those are complete and we migrate to new python based build-image scripts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20849: Provide 'vanilla' Airflow Docker image
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20849:
URL: https://github.com/apache/airflow/issues/20849#issuecomment-1012149673
I will add it next week after I merge #20664 - this will improve vastly the way how we build our images and it will be easy to add then (ARM support and Python 3.10 are following that closely).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #20849: Provide 'vanilla' Airflow Docker image
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #20849:
URL: https://github.com/apache/airflow/issues/20849#issuecomment-1046247344
Just want to let to know i have not forgotten about it. It is on my list but we are transitioning to python and debian bullseye support and once done i will add slim version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org