[GitHub] [incubator-doris] xy720 commented on issue #4101: [Proposal]Create a repository for SparkLoad‘s dependencies

[GitBox <gi...@apache.org>]

xy720 commented on issue #4101:
URL: https://github.com/apache/incubator-doris/issues/4101#issuecomment-661630805


   1、I am not going to remove the old version yet，because it's not a big problem for now.
   2、At present, I plan to save a DppVersion in FeConstant, fe will try to find or create a subfolders in repository named as DppVersion before load. Different Doris clusters will have its own repository directory.
   3、Fe will soon have a submodule of spark load，for every cluster we maintain a DppVersion in FeConstant, represents a version of spark load submodule(Just like the meta version). Before Fe starts to submit spark load, it will first look for the subfolders under the dppversion command under the repository, and find each library under this folder. Then it will compare each library with MD5, and determine whether it needs to upload again
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org