[GitHub] [airflow] potiuk removed a comment on issue #16902: Check for file content hash when caching dag_code

[GitBox <gi...@apache.org>]

potiuk removed a comment on issue #16902:
URL: https://github.com/apache/airflow/issues/16902#issuecomment-877629898


   Why not comparing the content of the code and skip merging in this case? I think you have to read the file content anyway to calculate the hash and at that time we already have read the DB entry with the code (it is all in-memory).
   
   I am not 100% sure if in this case merge() with unchanged "content" will cause additional write to the DB (it would be worth checking), but even if it does then simpler solution will be to compare the content and not perform the merge at all in this case.
   
   We could - of-course -store the hash of the content in the DB and do not read the entry from DB if the hash does not match, but I have a feeling that would increase the complexity, and not really improve the performance (we would have to read the content of all the files and calculate hashes anyway). 
   
   I am not sure if this is worth optimising. WDYT @wolfier ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org