You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/05/16 14:59:42 UTC
[GitHub] [airflow] snjypl opened a new pull request, #23720: Fix backfill queued task getting reset to scheduled state.
snjypl opened a new pull request, #23720:
URL: https://github.com/apache/airflow/pull/23720
the backfill queued tasks were getting reset to scheduled state by `kubernetes_executer.clear_not_launched_queued_tasks` of `scheduler` instance.
this fixes the issue by making `clear_not_launched_queued_tasks` only handle the tasks which are created by its own `scheduler/job`.
closes: #23356 , closes: #23693 , closes: #23145
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] snjypl commented on a diff in pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
snjypl commented on code in PR #23720:
URL: https://github.com/apache/airflow/pull/23720#discussion_r879714440
##########
airflow/executors/kubernetes_executor.py:
##########
@@ -530,13 +532,15 @@ def start(self) -> None:
self.kube_config.worker_pods_pending_timeout_check_interval,
self._check_worker_pods_pending_timeout,
)
- self.event_scheduler.call_regular_interval(
- self.kube_config.worker_pods_queued_check_interval,
- self.clear_not_launched_queued_tasks,
- )
- # We also call this at startup as that's the most likely time to see
- # stuck queued tasks
- self.clear_not_launched_queued_tasks()
+
+ if self.job_id != 'manual':
Review Comment:
to me it seemed like, for `manual` the `ti.queued_by_job_id` is not updated, it remains same as the original job_id.
and also, in this case the executor.job_id is str, and ti.queued_by_job_id is expected to the an integer. so the query in `clear_not_launched_queued_tasks` will fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1244183648
> I don't think we should change how Kubernetes executor handles queued task instances that have overstayed in queued state. If a task stayed in queued state for a long time, then something is wrong and the task should be cleared since we have a configuration to detect and clear it. While #26205 can also solve this issue, I think it's not necessary to send a task that's not in `scheduled` state to the executor. This is missing right now in backfill because the possibility to have many tasks set to scheduled was solved in #26161.
After consultations, I'm withdrawing the above statement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on code in PR #23720:
URL: https://github.com/apache/airflow/pull/23720#discussion_r968831618
##########
tests/executors/test_kubernetes_executor.py:
##########
@@ -138,7 +138,7 @@ def test_delete_pod_successfully(self, mock_watcher, mock_client, mock_kube_clie
mock_kube_client.return_value.delete_namespaced_pod = mock_delete_namespace
kube_executor = KubernetesExecutor()
- kube_executor.job_id = "test-job-id"
+ kube_executor.job_id = 1
Review Comment:
What's the reason for changing this to 1?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ei-grad commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ei-grad commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1298394160
Looks like the only meaningful error is:
```
Run black (python formatter) on other..................................................Failed
- hook id: black
- duration: 67.5s
- files were modified by this hook
reformatted tests/executors/test_kubernetes_executor.py
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vchiapaikeo commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
vchiapaikeo commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1204487587
Hi team! New to the scene here, but we are also facing this bug as well in our implementation (2.3.3, KubernetesExecutor). Wondering if this PR could get a second pair of eyes. Similar to what is mentioned in the discussion, we'll have backfills that run (some to completion) but then get stuck because they are later marked `scheduled`.
All tasks getting marked as scheduled after already completing their backfill:
<img width="143" alt="image" src="https://user-images.githubusercontent.com/9200263/182712320-738d768f-9f15-49df-8c10-9656563d5b93.png">
A dag run that completes one task but has its subsequent task marked as scheduled:
<img width="174" alt="image" src="https://user-images.githubusercontent.com/9200263/182712371-36d6a29d-dbee-4710-9261-3e7784c04f5d.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1249241471
Can you add tests to ensure this is not altered in the future?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on code in PR #23720:
URL: https://github.com/apache/airflow/pull/23720#discussion_r1020883376
##########
airflow/executors/kubernetes_executor.py:
##########
@@ -530,13 +532,15 @@ def start(self) -> None:
self.kube_config.worker_pods_pending_timeout_check_interval,
self._check_worker_pods_pending_timeout,
)
- self.event_scheduler.call_regular_interval(
- self.kube_config.worker_pods_queued_check_interval,
- self.clear_not_launched_queued_tasks,
- )
- # We also call this at startup as that's the most likely time to see
- # stuck queued tasks
- self.clear_not_launched_queued_tasks()
+
+ if self.job_id != 'manual':
Review Comment:
@snjypl personally, I'll prefer to have this part in a separate PR because it seems to be addressing something else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] DMilmont commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
DMilmont commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1318956584
I went ahead and manually patched this proposed fix into our airflow environment.
We average around 30 backfills per day, and over the last 4 days since it was manually patched - I have not seen or heard of this issue popping up. So far it seems that the fix has resolved the issue.
If it pops up again before this is merged I will report back. Just wanted to update with some test results.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1291052693
Needs conflict resolving.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1311945735
> Is there anything I can do to help move this one along? We rely on the airflow backfill CLI and this bug really makes it difficult for our users to backfill anything.
You can patch your airflow with those changes and verify that they work in your case. That would be an absolutely awesome input and one that can speed up the merge process if we have some evidences that people are eager to get it, even to the extend that they decide to patch their airflow without waiting for the next release.
The change looks like easy to patch manually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ei-grad commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ei-grad commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1132018441
Having the same issue in airflow==2.2.5. Applying a slightly adapted patch from this pull request fixes the problem with broken backfill.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1240565460
I don't think we should change how Kubernetes executor handles queued task instances that have overstayed in queued state. If a task stayed in queued state for a long time, then something is wrong and the task should be cleared since we have a configuration to detect and clear it.
While https://github.com/apache/airflow/pull/26205 can also solve this issue, I think it's not necessary to send a task that's not in `scheduled` state to the executor. This is missing right now in backfill because the possibility to have many tasks set to scheduled was solved in https://github.com/apache/airflow/pull/26161.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] snjypl commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
snjypl commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1308860838
> Yep. You need to fix it - I recommend installing pre-commit (as per our contribution docs). It will fix it for you.
@potiuk i have fixed the issue with black formatter by installing pre commit hook.
there is still a failing task.
Tests / Sqlite Py3.7: API Always CLI Core Integration Other Providers WWW
does it need to be addressed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1319422275
Thanks @snjypl
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ei-grad commented on a diff in pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ei-grad commented on code in PR #23720:
URL: https://github.com/apache/airflow/pull/23720#discussion_r1006973733
##########
airflow/executors/kubernetes_executor.py:
##########
@@ -464,7 +464,9 @@ def clear_not_launched_queued_tasks(self, session=None) -> None:
if not self.kube_client:
raise AirflowException(NOT_STARTED_MESSAGE)
- query = session.query(TaskInstance).filter(TaskInstance.state == State.QUEUED)
+ query = session.query(TaskInstance).filter(
+ TaskInstance.state == State.QUEUED, TaskInstance.queued_by_job_id == self.job_id
+ )
Review Comment:
Would it be possible to have more than one backfill running then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
potiuk merged PR #23720:
URL: https://github.com/apache/airflow/pull/23720
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #23720:
URL: https://github.com/apache/airflow/pull/23720#discussion_r878998038
##########
airflow/executors/kubernetes_executor.py:
##########
@@ -530,13 +532,15 @@ def start(self) -> None:
self.kube_config.worker_pods_pending_timeout_check_interval,
self._check_worker_pods_pending_timeout,
)
- self.event_scheduler.call_regular_interval(
- self.kube_config.worker_pods_queued_check_interval,
- self.clear_not_launched_queued_tasks,
- )
- # We also call this at startup as that's the most likely time to see
- # stuck queued tasks
- self.clear_not_launched_queued_tasks()
+
+ if self.job_id != 'manual':
Review Comment:
Any reason why this excludes `manual`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ei-grad commented on a diff in pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ei-grad commented on code in PR #23720:
URL: https://github.com/apache/airflow/pull/23720#discussion_r995982091
##########
airflow/executors/kubernetes_executor.py:
##########
@@ -464,7 +464,9 @@ def clear_not_launched_queued_tasks(self, session=None) -> None:
if not self.kube_client:
raise AirflowException(NOT_STARTED_MESSAGE)
- query = session.query(TaskInstance).filter(TaskInstance.state == State.QUEUED)
+ query = session.query(TaskInstance).filter(
+ TaskInstance.state == State.QUEUED, TaskInstance.queued_by_job_id == self.job_id
+ )
Review Comment:
Wouldn't `self.job_id` have value `"backfill"` without the change in [airflow/jobs/backfill_job.py](https://github.com/apache/airflow/pull/23720/files#diff-ce6cb8661a9f4cb5453effb446e6fbccf42a61d597445fe98899051db61436db)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1296540533
Some strange errors to fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] DMilmont commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
DMilmont commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1311932803
Is there anything I can do to help move this one along? We rely on the airflow backfill CLI and this bug really makes it difficult for our users to backfill anything.
Anyone have any workarounds that I can implement now without waiting for this to be merged in?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ei-grad commented on a diff in pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ei-grad commented on code in PR #23720:
URL: https://github.com/apache/airflow/pull/23720#discussion_r995982091
##########
airflow/executors/kubernetes_executor.py:
##########
@@ -464,7 +464,9 @@ def clear_not_launched_queued_tasks(self, session=None) -> None:
if not self.kube_client:
raise AirflowException(NOT_STARTED_MESSAGE)
- query = session.query(TaskInstance).filter(TaskInstance.state == State.QUEUED)
+ query = session.query(TaskInstance).filter(
+ TaskInstance.state == State.QUEUED, TaskInstance.queued_by_job_id == self.job_id
+ )
Review Comment:
> Can we have only this change plus the test on this PR?
Wouldn't `self.job_id` have value `"backfill"` without the change in [airflow/jobs/backfill_job.py](https://github.com/apache/airflow/pull/23720/files#diff-ce6cb8661a9f4cb5453effb446e6fbccf42a61d597445fe98899051db61436db)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1319421907
> If it pops up again before this is merged I will report back. Just wanted to update with some test results.
Cool. Thanks for that !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1298455604
Yep. You need to fix it - I recommend installing pre-commit (as per our contribution docs). It will fix it for you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1298455791
Make sure to rebase BTW.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on code in PR #23720:
URL: https://github.com/apache/airflow/pull/23720#discussion_r993040333
##########
airflow/executors/kubernetes_executor.py:
##########
@@ -464,7 +464,9 @@ def clear_not_launched_queued_tasks(self, session=None) -> None:
if not self.kube_client:
raise AirflowException(NOT_STARTED_MESSAGE)
- query = session.query(TaskInstance).filter(TaskInstance.state == State.QUEUED)
+ query = session.query(TaskInstance).filter(
+ TaskInstance.state == State.QUEUED, TaskInstance.queued_by_job_id == self.job_id
+ )
Review Comment:
Sorry for taking so long to come to this PR, I was on vacation.
If I understand correctly, this is the main change addressing the issue in this PR. Can we have only this change plus the test on this PR? All other changes should be moved to a new PR. This helps us track changes and it's more specific to the issue being resolved too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] agomez-etsy commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
agomez-etsy commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1261100919
Hi all. I can confirm I still see this issue on 2.4.0 KubernetesExecutor. Would this be available in 2.4.2? I see the milestone has being pushed several times. We are extremely interested in this fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on code in PR #23720:
URL: https://github.com/apache/airflow/pull/23720#discussion_r998155482
##########
airflow/executors/kubernetes_executor.py:
##########
@@ -464,7 +464,9 @@ def clear_not_launched_queued_tasks(self, session=None) -> None:
if not self.kube_client:
raise AirflowException(NOT_STARTED_MESSAGE)
- query = session.query(TaskInstance).filter(TaskInstance.state == State.QUEUED)
+ query = session.query(TaskInstance).filter(
+ TaskInstance.state == State.QUEUED, TaskInstance.queued_by_job_id == self.job_id
+ )
Review Comment:
Should have it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] snjypl closed pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
snjypl closed pull request #23720: Fix backfill queued task getting reset to scheduled state.
URL: https://github.com/apache/airflow/pull/23720
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #23720: Fix backfill queued task getting reset to scheduled state.
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #23720:
URL: https://github.com/apache/airflow/pull/23720#issuecomment-1129680570
@ashb @ephraimbuddy @dstandish @jedcunningham -> not sure if that's the right fix, But I guess this addresses the issues for 2.3.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org