You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/26 21:22:44 UTC
[GitHub] [airflow] Gaurang033 opened a new issue #14489: Add a retry with wait interval for SSH operator
Gaurang033 opened a new issue #14489:
URL: https://github.com/apache/airflow/issues/14489
<!--
Welcome to Apache Airflow! For a smooth issue process, try to answer the following questions.
Don't worry if they're not all applicable; just try to include what you can :-)
If you need to include code snippets or logs, please put them in fenced code
blocks. If they're super-long, please use the details tag like
<details><summary>super-long log</summary> lots of stuff </details>
Please delete these comment blocks before submitting the issue.
-->
**Description**
currently SSH operator fails if authentication fails without retying. We can add two more parameters as mentioned below which will retry in case of Authentication failure after waiting for configured time.
- max_retry - maximum time SSH operator should retry in case of exception
- wait - how many seconds it should wait before next retry
<!-- A short description of your feature -->
**Use case / motivation**
We are using SSH operator heavily in our production jobs, And what I have noticed is sometimes SSH operator fails to authenticate, however open re-running jobs it run successfully. And this happens ofthen. We have ended up writing our own custom operator for this. However, if we can implement this, this could help others as well.
<!-- What do you want to happen?
Rather than telling us how you might implement this solution, try to take a
step back and describe what you are trying to achieve.
-->
Implement the suggested feature for ssh operator.
**Are you willing to submit a PR?**
I will submit the PR if feature gets approval.
<!--- We accept contributions! -->
**Related Issues**
N/A
<!-- Is there currently another issue associated with this? -->
No
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #14489: Add a retry with wait interval for SSH operator
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #14489:
URL: https://github.com/apache/airflow/issues/14489#issuecomment-981063837
@nshetty15 are you still working on it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14489: Add a retry with wait interval for SSH operator
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14489:
URL: https://github.com/apache/airflow/issues/14489#issuecomment-922538167
assigned it to you @nshetty15
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Gaurang033 edited a comment on issue #14489: Add a retry with wait interval for SSH operator
Posted by GitBox <gi...@apache.org>.
Gaurang033 edited a comment on issue #14489:
URL: https://github.com/apache/airflow/issues/14489#issuecomment-790204665
@mik-laj can you elaborate on the generic hook thing? I was thinking to modify the existing hook and add an option for this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Gaurang033 commented on issue #14489: Add a retry with wait interval for SSH operator
Posted by GitBox <gi...@apache.org>.
Gaurang033 commented on issue #14489:
URL: https://github.com/apache/airflow/issues/14489#issuecomment-790204665
@Gaurang033 can you elaborate on the generic hook thing? I was thinking to modify the existing hook and add an option for this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nshetty15 commented on issue #14489: Add a retry with wait interval for SSH operator
Posted by GitBox <gi...@apache.org>.
nshetty15 commented on issue #14489:
URL: https://github.com/apache/airflow/issues/14489#issuecomment-984880472
@eladkal could you please review the change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #14489: Add a retry with wait interval for SSH operator
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #14489:
URL: https://github.com/apache/airflow/issues/14489
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Gaurang033 commented on issue #14489: Add a retry with wait interval for SSH operator
Posted by GitBox <gi...@apache.org>.
Gaurang033 commented on issue #14489:
URL: https://github.com/apache/airflow/issues/14489#issuecomment-790746887
@mik-laj perfect. we can similar implementation for this as well, then
https://github.com/apache/airflow/blob/3f7ebfdfe2a1fa90b0854028a5db057adacd46c1/airflow/providers/ssh/hooks/ssh.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nshetty15 commented on issue #14489: Add a retry with wait interval for SSH operator
Posted by GitBox <gi...@apache.org>.
nshetty15 commented on issue #14489:
URL: https://github.com/apache/airflow/issues/14489#issuecomment-981110837
@eladkal I had some issues setting up airflow. Its good now. I will submit a PR soon for this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #14489: Add a retry with wait interval for SSH operator
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #14489:
URL: https://github.com/apache/airflow/issues/14489#issuecomment-790370996
We have 2 hooks that allows us to connect to SSH server. Second hook have retry mechanism. See: https://github.com/apache/airflow/blob/3f7ebfdfe2a1fa90b0854028a5db057adacd46c1/airflow/providers/google/cloud/hooks/compute_ssh.py#L251
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nshetty15 commented on issue #14489: Add a retry with wait interval for SSH operator
Posted by GitBox <gi...@apache.org>.
nshetty15 commented on issue #14489:
URL: https://github.com/apache/airflow/issues/14489#issuecomment-922528336
@eladkal Can I take this up?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #14489: Add a retry with wait interval for SSH operator
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #14489:
URL: https://github.com/apache/airflow/issues/14489#issuecomment-787618674
SGTM. GoogleCompute retry authorizations, so generic
hook can also do it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org