You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Prasanth Jayachandran (JIRA)" <ji...@apache.org> on 2018/05/17 17:15:00 UTC

[jira] [Created] (HIVE-19588) Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader

Prasanth Jayachandran created HIVE-19588:
--------------------------------------------

             Summary: Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
                 Key: HIVE-19588
                 URL: https://issues.apache.org/jira/browse/HIVE-19588
             Project: Hive
          Issue Type: Bug
          Components: Transactions
    Affects Versions: 3.1.0
            Reporter: Nita Dembla
            Assignee: Prasanth Jayachandran
         Attachments: Screen Shot 2018-05-16 at 2.23.25 PM.png

Looks like we are doing file listing several times when creating one instance of VectorizedOrcAcidRowBatchReader
AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when there are files with bucket_* prefix) just to get a single file out of a path to figure out if it has ACID schema (as part of https://issues.apache.org/jira/browse/HIVE-18190).
There is full file listing where we populate
1) ColumnizedDeleteEventRegistry
2) SortMergedDeleteEventRegistry
3) Twice in computeOffsetAndBucket()

 

Attaching profiles which [~gopalv] took while debugging. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)