[GitHub] [incubator-pinot] fx19880617 commented on issue #5218: Fix HDFS copy logic

[GitBox <gi...@apache.org>]

fx19880617 commented on issue #5218: Fix HDFS copy logic
URL: https://github.com/apache/incubator-pinot/pull/5218#issuecomment-614841888
 
 
   > I saw copy method is used in `HadoopSegmentGenerationJobRunner`, where it copies the newly generated segments from staging dir to the final output dir. But why do the new segments store in different sub-dirs that makes you make this change?
   
   For all other pinot fs, the behavior keeps same as directory copying retains sub-dir structure. Otherwise, what if there are files with same name?
   
   If user wants to just copy all the files recursively from a directory and flatten to a new directory, then user should list all the files then do file copy to dest dir.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org