You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/09/28 16:17:14 UTC
[GitHub] [airflow] potiuk opened a new issue #18578: Allow to set execution_timeout default value in airflow.cfg
potiuk opened a new issue #18578:
URL: https://github.com/apache/airflow/issues/18578
### Discussed in https://github.com/apache/airflow/discussions/18411
<div type='discussions-op-text'>
<sup>Originally posted by **alexInhert** September 21, 2021</sup>
### Description
Currently the default value of execution_timeout in base operator is None
https://github.com/apache/airflow/blob/c686241f4ceb62d52e9bfa607822e4b7a3c76222/airflow/models/baseoperator.py#L502
This means that a task will run without limit till finished.
### Use case/motivation
The problem is that there is no way to overwrite this default for all dags. This causes problems where we find that tasks run sometimes for 1 week!!! for no reason. They are just stuck. This mostly happens with tasks that submit work to some 3rd party resource. The 3rd party had some error and terminated but the Airflow task for some reason did not.
This can be handled in cluster policy however it feels that cluster policy is more to enforce something but less about setting defaults values.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
</div>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-929378818
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-974609651
> I'd be glad to take this one over. @eladkal
Assigned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dillonjohnson commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
dillonjohnson commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-974577233
I'd be glad to take this one over.
@eladkal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dillonjohnson commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
dillonjohnson commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-1002610649
Yes. Sorry.
I just need to test a little bit more.
Will try to wrap it up soon. @eladkal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dillonjohnson commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
dillonjohnson commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-1006051741
On second thought, have had some stuff come up and I'm having tons of issues with the various tests failing (not sure why).
I think I'll have to pass it off. Sorry about that. @eladkal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-930172108
> Hello, @potiuk I would like to work on this. Could you guide me more on this?
Just read the discusion in #18411 i think @alexInhert explained it in the last comment -and maybe he can provide more guidance if needed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-973114986
@zevaverbach are you still working on this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-1002608399
@dillonjohnson are you still working on this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] zevaverbach commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
zevaverbach commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-973211876
Oh darn, to be honest I think I over-committed. Can you please unassign me?
On Thu, Nov 18, 2021 at 6:53 PM eladkal ***@***.***> wrote:
> @zevaverbach <https://github.com/zevaverbach> are you still working on
> this issue?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/airflow/issues/18578#issuecomment-973114986>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAVVCNBEMBTLWBXUWAKPPOTUMU4RDANCNFSM5E5Z4S5A>
> .
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-974609651
> I'd be glad to take this one over. @eladkal
Assigned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-929380786
Details in #18411
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-930172108
> Hello, @potiuk I would like to work on this. Could you guide me more on this?
Just read the discusion in #18411 i think @alexInhert explained it in the last comment -and maybe he can provide more guidance if needed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-929378818
cc: @alexinhert
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Ayushsunny commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
Ayushsunny commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-929805404
Hello, @potiuk I would like to work on this.
Could you guide me more on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Ayushsunny commented on issue #18578: Allow to set execution_timeout default value in airflow.cfg
Posted by GitBox <gi...@apache.org>.
Ayushsunny commented on issue #18578:
URL: https://github.com/apache/airflow/issues/18578#issuecomment-929805404
Hello, @potiuk I would like to work on this.
Could you guide me more on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org