You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2023/05/11 09:19:03 UTC
[GitHub] [airflow] potiuk commented on pull request #31182: Make sure that DAG processor job row has filed value in `job_type` column
potiuk commented on PR #31182:
URL: https://github.com/apache/airflow/pull/31182#issuecomment-1543641376
FYI: This is what I was talking about -> result if we add the typing now.
It's ol how it is now, I won't complai, and I will solve it later.
But just FYI why it has been separated - previously Scheduler and BackfillJobRunner had `Job` only while the others could have either `Job` or `JobPydantic`. By bring it as a common field in Base Runner, we have no easy way to distinguish tose two cases.
```
airflow/jobs/backfill_job_runner.py:258: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
executor = self.job.executor
^
airflow/jobs/backfill_job_runner.py:540: error: Argument 1 to
"import_executor_cls" of "ExecutorLoader" has incompatible type
"Union[Any, str, None]"; expected "str" [arg-type]
self.job.executor_class,
^
airflow/jobs/backfill_job_runner.py:898: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
executor = self.job.executor
^
airflow/jobs/backfill_job_runner.py:964: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
queued_tis = self.job.executor.queued_tasks
^
airflow/jobs/backfill_job_runner.py:966: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
running_tis = self.job.executor.running
^
airflow/jobs/backfill_job_runner.py:1005: error: Argument 4 to
"reduce_in_chunks" has incompatible type "Optional[int]"; expected "int"
[arg-type]
...elpers.reduce_in_chunks(query, tis_to_reset, [], self.job.max_tis_per_...
^
airflow/jobs/scheduler_job_runner.py:264: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.debug_dump()
^
airflow/jobs/scheduler_job_runner.py:280: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "is_alive" [union-attr]
return self.job.is_alive(grace_multiplier=grace_multiplier...
^
airflow/jobs/scheduler_job_runner.py:650: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.queue_command(
^
airflow/jobs/scheduler_job_runner.py:677: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
max_tis = self.job.executor.slots_available
^
airflow/jobs/scheduler_job_runner.py:679: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
... max_tis = min(self.job.max_tis_per_query, self.job.executor.slo...
^
airflow/jobs/scheduler_job_runner.py:690: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
event_buffer = self.job.executor.get_event_buffer()
^
airflow/jobs/scheduler_job_runner.py:772: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
or self.job.executor.has_task(ti) # This scheduler ha...
^
airflow/jobs/scheduler_job_runner.py:802: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.send_callback(request)
^
airflow/jobs/scheduler_job_runner.py:837: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.job_id = self.job.id
^
airflow/jobs/scheduler_job_runner.py:840: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.callback_sink = PipeCallbackSink(
^
airflow/jobs/scheduler_job_runner.py:847: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.callback_sink = DatabaseCallbackSink...
^
airflow/jobs/scheduler_job_runner.py:849: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.start()
^
airflow/jobs/scheduler_job_runner.py:879: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.end()
^
airflow/jobs/scheduler_job_runner.py:994: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.heartbeat()
^
airflow/jobs/scheduler_job_runner.py:1091: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
if self.job.executor.slots_available <= 0:
^
airflow/jobs/scheduler_job_runner.py:1477: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.send_callback(callback)
^
airflow/jobs/scheduler_job_runner.py:1504: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.send_callback(request)
^
airflow/jobs/scheduler_job_runner.py:1529: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
tis_for_warning_message = self.job.executor.cleanup_stuck_...
^
airflow/jobs/scheduler_job_runner.py:1612: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
to_reset = self.job.executor.try_adopt_task_instan...
^
airflow/jobs/scheduler_job_runner.py:1710: error: Item "JobPydantic" of
"Union[Job, JobPydantic]" has no attribute "executor" [union-attr]
self.job.executor.send_callback(request)
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org