You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/03/11 21:27:44 UTC
[GitHub] [airflow] talnicolas opened a new issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
talnicolas opened a new issue #22194:
URL: https://github.com/apache/airflow/issues/22194
### Apache Airflow version
2.2.4 (latest released)
### What happened
After upgrading from 2.2.3 to 2.2.4 the Paramiko library upgraded from 2.8.1 to 2.9.2. After that change one of our SSH connections started to fail with the following error:
```
File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1334, in _run_raw_task
self._execute_task_with_callbacks(context)
File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1460, in _execute_task_with_callbacks
result = self._execute_task(context, self.task)
File "/usr/local/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1516, in _execute_task
result = execute_callable(context=context)
File "/usr/local/lib/python3.9/site-packages/airflow/providers/ssh/operators/ssh.py", line 175, in execute
raise AirflowException(f"SSH operator error: {str(e)}")
airflow.exceptions.AirflowException: SSH operator error: encountered RSA key, expected OPENSSH key
```
After looking at [Paramiko changelog](https://github.com/paramiko/paramiko/blob/main/sites/www/changelog.rst#id17) we saw that since 2.9.0 they have made non-backward compatible changes to the way they manage algorithms. In our case we are trying to connect to a server that has the following version of openssh and openssl : OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013.
### What you expected to happen
Our existing SSH connections to keep working regardless of the version of OpenSSH used.
### How to reproduce
Connect through the 2.2.4 SSHOperator to a server that do not support either RSA2 or the ``server-sig-algs`` protocol extension.
### Operating System
Debian GNU/Linux 11 (bullseye)
### Versions of Apache Airflow Providers
apache-airflow-providers-ssh==1!2.4.0
### Deployment
Astronomer
### Deployment details
_No response_
### Anything else
In their [changelog](https://github.com/paramiko/paramiko/blob/main/sites/www/changelog.rst#id17) Paramiko indicates that to workaround that issue it is possible to specify ``disabled_algorithms={'keys': ['rsa-sha2-256', 'rsa-sha2-512']}`` in either SSHClient <paramiko.client.SSHClient.__init__> or Transport <paramiko.transport.Transport.__init__>.
I'm wondering what could be the impact to add this to the [SSHClient](https://github.com/apache/airflow/blob/ee9049c0566b2539a247687de05f9cffa008f871/airflow/providers/ssh/hooks/ssh.py#L260)?
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065732245
You are free to downgrade to earlier paramiko version. Current limitation of SSH Provider is `paramiko>=2.6.0` - the fact that paramiko >=2.9.0 does not accept your key is likely for a good reason (security) and if you choose to continue using version that is older (and might contain some security vulnerabilities) rather than upgrading your key you can still do it.
Follow the examples here:
https://airflow.apache.org/docs/docker-stack/build.html#example-of-upgrading-airflow-provider-packages
You can build your own image using this Dockerfile (feel free to do it)
```
FROM apache/airflow:2.2.4
RUN pip install --no-cache-dir paramiko<2.9.0
```
Expecting that newer version of Airflow is using an older and potentially vulnerable version of a dependency to allow you to use potentially vulnerable key format is a bit selfish.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #22194:
URL: https://github.com/apache/airflow/issues/22194
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065732245
You are free to downgrade to earlier paramiko version. Current limitation of SSH Provider is `paramiko>=2.6.0` - the fact that paramiko >=2.9.0 does not accept your key is likely for a good reason (security) and if you choose to continue using version that is older (and might contain some security vulnerabilities) rather than upgrading your key - you can still do it.
Follow the examples here:
https://airflow.apache.org/docs/docker-stack/build.html#example-of-upgrading-airflow-provider-packages
You can build your own image using this Dockerfile (feel free to do it)
```
FROM apache/airflow:2.2.4
RUN pip install --no-cache-dir paramiko<2.9.0
```
Expecting that newer version of Airflow is using an older and potentially vulnerable version of a dependency to allow you to use potentially vulnerable key format is a bit selfish.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065888527
This is really what I consider as "non selfish" - Identify a problem that small part of others might have, and propose a PR so that not only my problem is fixed but also others. For an open-source software that you get for free this is absolutely the best way to proceed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] talnicolas commented on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
talnicolas commented on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065798803
Thank you for your insightful comment @potiuk.
Just a comment if I may, insulting people when they are just coming for advice from « experts » won’t help building a safe and inclusive community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065887894
> I'm wondering what could be the impact to add this to the [SSHClient]
By all means - please propose a PR to expose this. You are most welcome.
Indeed I missed that part.
I was focusing more on the expectations: "Our existing SSH connections to keep working regardless of the version of OpenSSH used". But If you marked something as a bug, where this is a rather a feature request to expose this feature to the operators - this bug will be closed as invalid. This is not a bug and your `expectation` is more of a wish, than expectation and is not a `bug` by any means. Maybe that's why the part where you proposed a feature (in otherwise a bug request) was simply, well, missed.
No hard feelings - really :). And again - if you felt "insulted" - then that was a total misunderstanding. I simply closed the bug, believing that you "complain on a bug that you expected to work" rather than "proposed a feature to make it work better". The latter is absolutely welcome - actually you don't even need to make an issue for it - just create a PR and proceed as usual. Airflow has almost 2000 contributors - mostly volunteers so you are free to propose this (and I will be happy to review and approve it if it follows all the criteria from CONTRIBUTING.rst - and this is a great way to become one.
Again, apologies if you felt insulted, that was definitely not intention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065855566
> Just a comment if I may, insulting people when they are just coming for advice from « experts » won’t help building a safe and inclusive community.
Unfortunately, I have no control whether you feel insulted or not, that was certainly not my intention, My intention was to raise your awareness that your expectations were a bit selfish as they did not include wider thinking about the community but focused on your own needs. This is what I think is a bit selfish. I just expressed how I perceived it.
But I certainly had no intention to insult you - if you feel insulted - sorrry for that - but again, I have completely no control on your feelings when I express mine. I suggest you to look at it at a wider context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065543512
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065887894
> I'm wondering what could be the impact to add this to the [SSHClient]
By all means - please propose a PR to expose this. You are most welcome.
Indeed I missed that part.
I was focusing more on the expectations: "Our existing SSH connections to keep working regardless of the version of OpenSSH used". But If you marked something as a bug, where this is a rather a feature request to expose this feature to the operators - this bug will be closed as invalid (as it is not a bug and your `expectation` is more of a wish, than expectation and is not a `bug` by any means. Maybe that's why the part where you proposed a feature was simply, well, missed.
No hard feelings - really :). And again - if you felt "insulted" - then that was a total misunderstanding. I simply closed the bug, believing that you "complain on a bug that you expected to work" rather than "proposed a feature to make it work better". The latter is absolutely welcome - actually you don't even need to make an issue for it - just create a PR and proceed as usual. Airflow has almost 2000 contributors - mostly volunteers so you are free to propose this (and I will be happy to review and approve it if it follows all the criteria from CONTRIBUTING.rst
Again, apologies if you felt insulted, that was definitely not intention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065887894
> I'm wondering what could be the impact to add this to the [SSHClient]
By all means - please propose a PR to expose this. You are most welcome.
Indeed I missed that part.
I was focusing more on the expectations: "Our existing SSH connections to keep working regardless of the version of OpenSSH used". But If you marked something as a bug, where this is a rather a feature request to expose this feature to the operators - this bug will be closed as invalid. This is not a bug and your `expectation` is more of a wish, than expectation and is not a `bug` by any means. Maybe that's why the part where you proposed a feature (in otherwise a bug report) was simply, well, missed.
No hard feelings - really :). And again - if you felt "insulted" - then that was a total misunderstanding. I simply closed the bug, believing that you "complain on a bug that you expected to work" rather than "proposed a feature to make it work better". The latter is absolutely welcome - actually you don't even need to make an issue for it - just create a PR and proceed as usual. Airflow has almost 2000 contributors - mostly volunteers so you are free to propose this (and I will be happy to review and approve it if it follows all the criteria from CONTRIBUTING.rst - and this is a great way to become one.
Again, apologies if you felt insulted, that was definitely not intention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065887894
> I'm wondering what could be the impact to add this to the [SSHClient]
By all means - please propose a PR to expose this. You are most welcome.
Indeed I missed that part.
I was focusing more on the expectations: "Our existing SSH connections to keep working regardless of the version of OpenSSH used". But If you marked something as a bug, where this is a rather a feature request to expose this feature to the operators - this bug will be closed as invalid. This is not a bug and your `expectation` is more of a wish, than expectation and is not a `bug` by any means. Maybe that's why the part where you proposed a feature (in otherwise a bug request) was simply, well, missed.
No hard feelings - really :). And again - if you felt "insulted" - then that was a total misunderstanding. I simply closed the bug, believing that you "complain on a bug that you expected to work" rather than "proposed a feature to make it work better". The latter is absolutely welcome - actually you don't even need to make an issue for it - just create a PR and proceed as usual. Airflow has almost 2000 contributors - mostly volunteers so you are free to propose this (and I will be happy to review and approve it if it follows all the criteria from CONTRIBUTING.rst
Again, apologies if you felt insulted, that was definitely not intention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] talnicolas commented on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
talnicolas commented on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065875862
Indeed you do not have that control, just like I don’t have any control on your ability to understand people’s request.
> I'm wondering what could be the impact to add this to the [SSHClient](https://github.com/apache/airflow/blob/ee9049c0566b2539a247687de05f9cffa008f871/airflow/providers/ssh/hooks/ssh.py#L260)?
If that doesn’t feel to you like someone who came here specifically to understand that wider context you are talking about, well I’m sorry too.
That conversation is sterile now, I wish you a good continuation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065578026
What version of the [SSH provider](https://pypi.org/project/apache-airflow-providers-ssh/) do you have installed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065732245
You are free to downgrade to earlier paramiko version. Current limitation of SSH Provider is `paramiko>=2.6.0` - the fact that paramiko >=2.9.0 does not accept your key is likely for a good reason (security) and if you choose to continue using version that is older (and might contain some security vulnerabilities) rather than upgrading your key you can still do it.
Follow the examples here:
https://airflow.apache.org/docs/docker-stack/build.html#example-of-upgrading-airflow-provider-packages
You can build your own image using this Dockerfile (feel free to do it)
```
FROM apache/airflow:2.2.4
RUN pip install --no-cache-dir paramiko<2.9.0
```
Expecting that newer version of Airflow is using an older and potentially vulnerable version of a dependency is a bit selfish.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #22194: Paramiko 2.9.x breaking SSH operator for some OpenSSH configurations
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #22194:
URL: https://github.com/apache/airflow/issues/22194#issuecomment-1065855566
> Just a comment if I may, insulting people when they are just coming for advice from « experts » won’t help building a safe and inclusive community.
Unfortunately, I have no control whether you feel insulted or not, that was certainly not my intention, My intention was to raise your awareness that your expectations were a bit selfish as they did not include wider thinking about the community but focused on your own needs. This is what I think is a bit selfish. I just expressed how I perceived it.
But I certainly had no intention to insult you - if you did - sorrry for that - but again, I have completely no control on your feelings when I express mine. I suggest you to look at it at a wider context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org