You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/01/06 20:55:05 UTC

[GitHub] [hudi] vinothchandar edited a comment on pull request #2410: [HUDI-1510] Move HoodieEngineContext and its dependencies to hudi-common

vinothchandar edited a comment on pull request #2410:
URL: https://github.com/apache/hudi/pull/2410#issuecomment-755529282


   This points to a deeper discussion. Right now, hudi-common is depended upon by `hudi-hadoop-mr` which has all the Hive/InputFormat code. Should that also be redone as a client-module, if saw we want to support Hive writing ultimately? That is a large change though. 
   
   So, we can do either of the two options here for now.
   
   A) move `HoodieEngineContext` to hudi-common; add a `HoodieMREngineContext`
   B) Move FSUtils#getAllPartitionPaths to hudi-client-common and have another parallelized impl. 
   
   I still prefer A. But want to hear what you both think as well. Let's make a quick call, since we want this in soon :) 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org