You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2023/05/28 12:40:04 UTC

[GitHub] [arrow-datafusion] alamb commented on issue #6470: Consider providing an option for dynamic linking

alamb commented on issue #6470:
URL: https://github.com/apache/arrow-datafusion/issues/6470#issuecomment-1566124974

   Thanks @sergiimk  -- this is quite interesting
   
   I think one reason for the long cycle time is not just re-link but also that datafusion-core is large and any change to it requires recompiling the entire crate.  
   
   Another approach I have seen taken in the rust community is to break such large crates into smaller ones. Not only does this tend to increase test cycle times it also encourages more encapsulation. 
   
   We have been chipping away at doing this  (e.g. https://github.com/apache/arrow-datafusion/issues/1754) but extracting `datafusion-physical-plan` and `datafusion-datasource` are especially connected.
   
   Maybe it is time to invest some more time trying to split out those crates 🤔 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org