You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/17 22:50:14 UTC
[GitHub] [airflow] potiuk opened a new pull request #20912: Switch to new MySQL public key
potiuk opened a new pull request #20912:
URL: https://github.com/apache/airflow/pull/20912
MySQL changed key used to sign their apt packages. This caused
docker building failing for prod images as MySQL could not be
installed.
New Public Key is used instead.
Fixes: #20911
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1014995400
If ny committer approves it - please merge it. This is needed to stop PRs/main builds from failing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mhbrown removed a comment on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
mhbrown removed a comment on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1014953406
Will the DockerHub images be updated with this fix? I'm having issues with `dockerhub.docker.artifactory.viasat.com/apache/airflow:1.10.15-python3.6` due to the key change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1014955521
> Will the DockerHub images be updated with this fix? I'm having issues with `dockerhub.docker.artifactory.viasat.com/apache/airflow:1.10.15-python3.6` due to the key change.
Only 2.* images. The 1.10 is already end-of-life and will not receive any fixes. You can manually apply the workaround mentioned in #20911 - add the apt-key line in your extended image if you are going to use it as migration "bridge":
```
USER root
RUN sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 467B942D3A79BD29
RUN apt-get update
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1014946516
Note. I will also need to (doing it now) rebuild past released docker images to allow people to extend those (apt-get update fails without the key) (only 2.* images as 1.10 is end-of-life and there is a workaround available - see below).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj merged pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
mik-laj merged pull request #20912:
URL: https://github.com/apache/airflow/pull/20912
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
potiuk closed pull request #20912:
URL: https://github.com/apache/airflow/pull/20912
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ruben-ortiz-buybay commented on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
ruben-ortiz-buybay commented on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1019197277
One of the side effects of this change, for the previous images, is that some packages versions for MySQL change, from 8.0.27-1debian10 to 8.0.28-1debian10 (mysql-client and libmysqlclient21) In my case, this change on versions ends in an error, connecting to AWS RDS Mysql:
MySQLdb._exceptions.OperationalError: (2026, 'SSL connection error: error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol')
Checking the changelog for version 8.0.28 ( https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-28.html ) includes an important change that can affect legacy systems:
> Support for the TLSv1 and TLSv1.1 connection protocols is removed as of MySQL 8.0.28. The protocols were deprecated from MySQL 8.0.26. For background, refer to the IETF memo Deprecating TLSv1.0 and TLSv1.1. Make connections using the more-secure TLSv1.2 and TLSv1.3 protocols. TLSv1.3 requires that both the MySQL Server software and the client application were compiled with OpenSSL 1.1.1 or higher.
A workaround the issue is using docker images by SHA and not by version (and adding the new key if package updates are needed)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1014995400
If any committer approves it - please merge it. This is needed to stop PRs/main builds from failing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mhbrown commented on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
mhbrown commented on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1014953406
Will the DockerHub images be updated with this fix? I'm having issues with `dockerhub.docker.artifactory.viasat.com/apache/airflow:1.10.15-python3.6` due to the key change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] juroVee commented on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
juroVee commented on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1017284999
Hi @potiuk, can we be assured that these changes will be included in future releases source codes (Dockerfile) here https://github.com/apache/airflow/releases?
Thing is, we are building our custom base images from there and currently it's not possible in 2.2.2/2.2.3 due to this issue as the archives don't seem to be updated. Until then we can use this temporary solution in our extended images (which we are building from custom base images described above):
``
RUN sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 467B942D3A79BD29
``
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1015042665
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1019325710
> A workaround the issue is using docker images by SHA and not by version (and adding the new key if package updates are needed)
Thanks. However I'd heartily recommend to update your MySQL server is updated to support TLS 1.2.
We've seen a lot of vendors following the recommendation to get rid of TLS1.1 because in modern world it is not secure enough. This is actually a regulatory requirement in many countries to disable TLS1.0 and very strong recommendation to disable TLS1.1 as well..
Pretty much all vendors disabled support bu default for TLS1.2 mid 2020.
BTW. One could argue thta another workaround would be to disable SSL at all because, well, TLS1.1 is not secure and gives false sense of security.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1014955521
> Will the DockerHub images be updated with this fix? I'm having issues with `dockerhub.docker.artifactory.viasat.com/apache/airflow:1.10.15-python3.6` due to the key change.
Only 2.* images. The 1.10 is already end-of-life as of June 2021 and will not receive any fixes. You can manually apply the workaround mentioned in #20911 - add the apt-key line in your extended image if you are going to use it as migration "bridge":
```
USER root
RUN sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 467B942D3A79BD29
RUN apt-get update
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1014946516
Note. I will also need to (doing it now) rebuild past released docker images to allow people to extend those (apt-get update fails without the key).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1017305272
> Hi @potiuk, can we be assured that these changes will be included in future releases source codes (Dockerfile) here https://github.com/apache/airflow/releases?
Yes. They will be. Good point. I just marked it as 2.2.4 release - I am going to cherry-pick some more Docker Image changes to 2.2.4 as there were quite a number of refactorings and improvements there. Rest assured it will be added. I am also thinking actually on how we could protect against similar cases in the future. Not sure if this is possible though (embedding the key id used by 3rd-party is the only way can make sure installing a package is "safe" so I am not sure if we can do it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20912: Switch to new MySQL public key
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20912:
URL: https://github.com/apache/airflow/pull/20912#issuecomment-1019325710
> A workaround the issue is using docker images by SHA and not by version (and adding the new key if package updates are needed)
Thanks. However I'd heartily recommend to update your MySQL server is updated TLS 1.2. We've seen a lot of vendors following the recommendation to get rid of TLS1.1 because in modern world it is not secure enough. This is actually a regulatory requirement in many countries to disable TLS1.0 and very strong recommendation to disable TLS1.1 as well..
Pretty much all vendors disabled support bu defaul for TLS1.2 mid 2020.
BTW. One could argue thta another workaround would be to disable SSL at all because, well, TLS1.1 is not secure and gives false sense of security.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org