You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/04 19:17:53 UTC
[GitHub] [airflow] dstandish opened a new pull request #20658: Rename `to_delete` to `to_cancel` in TriggerRunner
dstandish opened a new pull request #20658:
URL: https://github.com/apache/airflow/pull/20658
The queue's purpose is to track triggers that need to be canceled. I found the language `to_delete` a bit confusing because for one it does not actually delete them but cancel them. The deletion work is actually in `cleanup_finished_triggers`. It seems that the canceling of triggers with this method will usually not do anything and it's only for cancelling triggers that are currently running but for whatever reason no longer should be. E.g. when a task is killed and therefore the trigger is no longer needed, or some multi-triggerer scenarios.
@kaxil i think these methods should be renamed but just not sure if this is considered "breaking" since maybe you could say the triggerer process is "internal"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20658: Rename `to_delete` to `to_cancel` in TriggerRunner
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20658:
URL: https://github.com/apache/airflow/pull/20658#issuecomment-1005930473
> Just chatted with @ashb and his position is that methods like this on the trigger internals, or scheduler internals, should be considered private and can be refactored freely. But added that we should probablly also try to indicate that somewhere in the docs -- that there's no guarantee that they won't change from [minor] version to version.
I think we should - rather than explaining this come up with the list of Classes/APIs (Protocols + Models ?) that would become our official Python API that we will guarantee to not change until major version increase. I don't think there will be awfully lot of those and I think in most cases it wil be enough to just list the classes that we treat this way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on pull request #20658: Rename `to_delete` to `to_cancel` in TriggerRunner
Posted by GitBox <gi...@apache.org>.
dstandish commented on pull request #20658:
URL: https://github.com/apache/airflow/pull/20658#issuecomment-1005849380
> LGTM, Is backward compatibility an issue here?
Just chatted with @ashb and his position is that methods like this on the trigger internals, or scheduler internals, should be considered private and can be refactored freely. But added that we should probablly also try to indicate that somewhere in the docs -- that there's no guarantee that they won't change from version to version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish merged pull request #20658: Rename `to_delete` to `to_cancel` in TriggerRunner
Posted by GitBox <gi...@apache.org>.
dstandish merged pull request #20658:
URL: https://github.com/apache/airflow/pull/20658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish edited a comment on pull request #20658: Rename `to_delete` to `to_cancel` in TriggerRunner
Posted by GitBox <gi...@apache.org>.
dstandish edited a comment on pull request #20658:
URL: https://github.com/apache/airflow/pull/20658#issuecomment-1005849380
> LGTM, Is backward compatibility an issue here?
Just chatted with @ashb and his position is that methods like this on the trigger internals, or scheduler internals, should be considered private and can be refactored freely. But added that we should probablly also try to indicate that somewhere in the docs -- that there's no guarantee that they won't change from [minor] version to version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #20658: Rename `to_delete` to `to_cancel` in TriggerRunner
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #20658:
URL: https://github.com/apache/airflow/pull/20658#issuecomment-1005929146
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #20658: Rename `to_delete` to `to_cancel` in TriggerRunner
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #20658:
URL: https://github.com/apache/airflow/pull/20658#issuecomment-1005943490
Yeah, my view is that the "public" API is, loosely
- anything operator related, or passed in via the Context (so this includes methods and properties on TI and DagRun etc.)
- anything that is used as a base class where the user can configure a custom class to be used.
So Executors are part of the public API, but the SchedulerJob, nor TriggererJob are not.
Clearly this isn't a fully description but the ground work of what I think it should be
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #20658: Rename `to_delete` to `to_cancel` in TriggerRunner
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #20658:
URL: https://github.com/apache/airflow/pull/20658#issuecomment-1005951674
> Clearly this isn't a fully description but the ground work of what I think it should be
Yeah. Some (obvious) models Variables, Connections etc (they will become non-SQLAlchemy Models really when we agree/implement this part of the API in AIP-44) + configuration + some logging part + some base opareators (DBapi for example) . Those on top of what you wrote @ashb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org