[GitHub] [hive] pvargacl commented on pull request #1779: HIVE-24535: Cleanup AcidUtils.Directory and remove unnecessary filesystem listing

[GitBox <gi...@apache.org>]

pvargacl commented on pull request #1779:
URL: https://github.com/apache/hive/pull/1779#issuecomment-749182942


   The FileSystem calls are saved because on S3 AcidUtils#getAcidState will do a recursive listing (getHdfsDirSnapshots), and when it processes a delta directory, it will store the file list in ParsedDelta, so when AcidDirectory#getBaseAndDeltaFiles is called later the files are returned without a second FS call. If we are on HDFS there is no gain, the getBaseAndDeltaFiles will still do a listing as it did before in OrcInputFormat.FileGenerator#callInternalmoved. 
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org