You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Prasanth Jayachandran (JIRA)" <ji...@apache.org> on 2018/05/17 21:56:00 UTC
[jira] [Commented] (HIVE-19588) Several invocation of file listing
when creating VectorizedOrcAcidRowBatchReader
[ https://issues.apache.org/jira/browse/HIVE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479793#comment-16479793 ]
Prasanth Jayachandran commented on HIVE-19588:
----------------------------------------------
[~ekoifman] can you please review the patch?
> Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
> --------------------------------------------------------------------------------
>
> Key: HIVE-19588
> URL: https://issues.apache.org/jira/browse/HIVE-19588
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 3.1.0
> Reporter: Nita Dembla
> Assignee: Prasanth Jayachandran
> Priority: Major
> Attachments: HIVE-19588.1.patch, Screen Shot 2018-05-16 at 2.23.25 PM.png
>
>
> Looks like we are doing file listing several times when creating one instance of VectorizedOrcAcidRowBatchReader
> AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when there are files with bucket_* prefix) just to get a single file out of a path to figure out if it has ACID schema (as part of HIVE-18190).
> There is full file listing where we populate
> 1) ColumnizedDeleteEventRegistry
> 2) SortMergedDeleteEventRegistry
> 3) Twice in computeOffsetAndBucket()
>
> Attaching profiles which [~gopalv] took while debugging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)