You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Vihang Karajgaonkar (JIRA)" <ji...@apache.org> on 2017/11/08 20:56:00 UTC
[jira] [Comment Edited] (HIVE-17931) Implement Parquet
vectorization reader for Array type
[ https://issues.apache.org/jira/browse/HIVE-17931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16244711#comment-16244711 ]
Vihang Karajgaonkar edited comment on HIVE-17931 at 11/8/17 8:55 PM:
---------------------------------------------------------------------
Hi [~colinma] I left some comments on the review board. Since the patch is not committed yet, can we refactor the code in this patch itself to remove the duplicate code. Having duplicate code causing problems like HIVE-17961 where the bug was fixed only in one part of the code and the test was really testing duplicate code which didn't catch the regression.
Also, can you add some qtests which for the list types? Specifically, can add q-tests where the number of elements in the List are <1024, =1024 and >1024 as well to make we don't have corner conditions. Its unclear to me from the code how does the vectorized batch handle these cases.
Thanks a lot for the patch!
was (Author: vihangk1):
Hi [~colinma] I left some comments on the review board. Since the patch is not committed yet, can we refactor the code in this patch itself to remove the duplicate code. Having duplicate code causing problems like HIVE-17961 where the bug was fixed only in one part of the code and the test was really testing duplicate code which didn't catch the regression.
Also, can you add some qtests which for the list types? Specifically, can add q-tests where the elements in the List are <1024, =1024 and >1024 as well to make we don't have corner conditions. Its unclear to me from the code how does the vectorized batch handle these cases.
Thanks a lot for the patch!
> Implement Parquet vectorization reader for Array type
> -----------------------------------------------------
>
> Key: HIVE-17931
> URL: https://issues.apache.org/jira/browse/HIVE-17931
> Project: Hive
> Issue Type: Sub-task
> Reporter: Colin Ma
> Assignee: Colin Ma
> Attachments: HIVE-17931.001.patch, HIVE-17931.002.patch
>
>
> Parquet vectorized reader can't support array type, it should be supported to improve the performance when the query with array type.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)