You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/24 05:13:47 UTC

[GitHub] [airflow] houqp commented on pull request #12858: Store per-task TIDeps in serialized blob

houqp commented on pull request #12858:
URL: https://github.com/apache/airflow/pull/12858#issuecomment-750744813


   @ashb I tried to serialize with builtin deps disabled for all of our DAGs, which only seems to only increase the serialized json size by 1-2%, including our biggest DAG that contains 1000+ tasks (total size 3mb). All of our tasks use custom deps to enable special scheduling logic, so they all contributed the size increase. Given that, the size increase seems minor?
   
   In your comment, you mentioned only built-in deps are allowed for now to reduce space, i am curious what edge-case would have caused significant size increase with custom deps. I would like to help lessen this restriction to unblock our production deployment of 2.0.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org