You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "dheerajturaga (via GitHub)" <gi...@apache.org> on 2025/10/30 00:41:34 UTC
[PR] Fix executor field not being captured for task instances [airflow]
dheerajturaga opened a new pull request, #57526:
URL: https://github.com/apache/airflow/pull/57526
The executor field was never populated with the actual executor name when
tasks were run. This caused the field to remain NULL in the database and
not display properly in the UI
Before:
<img width="640" height="144" alt="image" src="https://github.com/user-attachments/assets/3c335376-aa2e-4abc-96f1-61df69020f00" />
After:
<img width="1080" height="592" alt="image" src="https://github.com/user-attachments/assets/198dd2a6-3796-4811-82f7-a3e390900882" />
<img width="1723" height="657" alt="image" src="https://github.com/user-attachments/assets/250dc529-48b5-4424-9c3d-98222b88be03" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix executor field not being captured for task instances [airflow]
Posted by "dheerajturaga (via GitHub)" <gi...@apache.org>.
dheerajturaga commented on PR #57526:
URL: https://github.com/apache/airflow/pull/57526#issuecomment-3470771263
Closing with preference to #57526
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix executor field not being captured for task instances [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #57526:
URL: https://github.com/apache/airflow/pull/57526#discussion_r2476115882
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -679,8 +681,20 @@ def _executable_task_instances_to_queued(self, max_tis: int, session: Session) -
.execution_options(synchronize_session=False)
)
+ # Update executor field for each task instance before making them transient
for ti in executable_tis:
+ session.execute(
+ update(TI)
+ .where(
+ TI.dag_id == ti.dag_id,
+ TI.task_id == ti.task_id,
+ TI.run_id == ti.run_id,
+ TI.map_index == ti.map_index,
+ )
+ .values(executor=ti.executor)
+ )
Review Comment:
Let's not do it here, and instead do it at the bulk update on L671-682 above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix executor field not being captured for task instances [airflow]
Posted by "dheerajturaga (via GitHub)" <gi...@apache.org>.
dheerajturaga commented on PR #57526:
URL: https://github.com/apache/airflow/pull/57526#issuecomment-3465629693
@kaxil Another one for 3.1.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix executor field not being captured for task instances [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on PR #57526:
URL: https://github.com/apache/airflow/pull/57526#issuecomment-3470722032
PR: https://github.com/apache/airflow/pull/57590
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix executor field not being captured for task instances [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #57526:
URL: https://github.com/apache/airflow/pull/57526#discussion_r2476142247
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -679,8 +681,20 @@ def _executable_task_instances_to_queued(self, max_tis: int, session: Session) -
.execution_options(synchronize_session=False)
)
+ # Update executor field for each task instance before making them transient
for ti in executable_tis:
+ session.execute(
+ update(TI)
+ .where(
+ TI.dag_id == ti.dag_id,
+ TI.task_id == ti.task_id,
+ TI.run_id == ti.run_id,
+ TI.map_index == ti.map_index,
+ )
+ .values(executor=ti.executor)
+ )
Review Comment:
or does that now work.
The current logic you have will run query for each TI --
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix executor field not being captured for task instances [airflow]
Posted by "dheerajturaga (via GitHub)" <gi...@apache.org>.
dheerajturaga closed pull request #57526: Fix executor field not being captured for task instances
URL: https://github.com/apache/airflow/pull/57526
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix executor field not being captured for task instances [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #57526:
URL: https://github.com/apache/airflow/pull/57526#discussion_r2476160701
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -679,8 +681,20 @@ def _executable_task_instances_to_queued(self, max_tis: int, session: Session) -
.execution_options(synchronize_session=False)
)
+ # Update executor field for each task instance before making them transient
for ti in executable_tis:
+ session.execute(
+ update(TI)
+ .where(
+ TI.dag_id == ti.dag_id,
+ TI.task_id == ti.task_id,
+ TI.run_id == ti.run_id,
+ TI.map_index == ti.map_index,
+ )
+ .values(executor=ti.executor)
+ )
ti.emit_state_change_metric(TaskInstanceState.QUEUED)
+ session.flush()
Review Comment:
We shouldn't need flush
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org