[GitHub] [airflow] TomatoEgg commented on issue #7230: [AIRFLOW-6602]: render executor_config before triggering tasks to su…

[GitBox <gi...@apache.org>]

TomatoEgg commented on issue #7230: [AIRFLOW-6602]: render executor_config before triggering tasks to su…
URL: https://github.com/apache/airflow/pull/7230#issuecomment-581299939
 
 
   @nuclearpinguin : basically I found out in my test that the following happens when I triggered a DAG with executor_config is templated (say {{ ds }} as value for a key):
   1. scheduler_job._enqueue_task_instances_with_queued_state is called which invokes
   ```
   self.executor.queue_command(
                   simple_task_instance,
                   command,
                   priority=priority,
                   queue=queue)
   ```
   2. base_executor.queue_command() is called
   ```
   self.queued_tasks[simple_task_instance.key] = (command, priority, queue, simple_task_instance)
   ```
   
   In the above two steps, simple_task_instance is of type SimpleTaskInstance, which means that it doesn't have enough information to render the templates. If it is TaskInstance instead, we could render the executor_config in those places.
   
   What makes this a bit more confusing is that the test case called 'test_enqueue_task_instances_with_queued_state' in test_scheduler_job.py, it is using TaskInstance as test data instead of SimpleTaskInstance.
   
   So my question is
   1) What data type is the correct one?
   2) is there an integration test case that we could run to see the truth?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services