You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/01/04 00:50:55 UTC

[GitHub] [airflow] dazza-codes edited a comment on issue #6750: [WIP] [AIRFLOW-6329] Add AirflowExecutor and native worker

dazza-codes edited a comment on issue #6750: [WIP] [AIRFLOW-6329] Add AirflowExecutor and native worker
URL: https://github.com/apache/airflow/pull/6750#issuecomment-570740844
 
 
   > What I would really love is to change the "type" of task message. Instead of sending `["airflow", "tasks", "run", ...]` it would be nice to send `{'dag_id': dag_id, 'task_id': task_id, ... }`
   
   As an aside, while considering options for an asyncio executor, there could be various ways that the task "type" (data/spec/payload - you name it) needs to be parsed by some kind of asyncio coroutine that can spawn and manage 1000's of coroutines on an event loop, such that each of those task coroutines can `await` any awaitable to yield control back to the event loop.  In a way, this idea might be similar to whatever is managing the messages in this PR.  I'd like to understand any changes to task designs.  I've kinda bumped into this while trying to do a small POC on the dask executor, i.e. #6984 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services