You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Jacques Nadeau (JIRA)" <ji...@apache.org> on 2019/07/02 23:01:00 UTC
[jira] [Commented] (ARROW-5821) [Java] Support compact fixed-width
vectors
[ https://issues.apache.org/jira/browse/ARROW-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16877350#comment-16877350 ]
Jacques Nadeau commented on ARROW-5821:
---------------------------------------
If I understand the ask correctly, it seems like you are proposing to change the format to have a new integer representation that doesn't support constant lookup. I'm not sure now is the right time to do something like this.
> [Java] Support compact fixed-width vectors
> ------------------------------------------
>
> Key: ARROW-5821
> URL: https://issues.apache.org/jira/browse/ARROW-5821
> Project: Apache Arrow
> Issue Type: New Feature
> Reporter: Ji Liu
> Assignee: Ji Liu
> Priority: Minor
>
> In shuffle stage of some applications, FixedWitdhVectors may have very little non-null data.
> In this case, directly serialize vectors is not a good choice, generally we can compact the vector make it only holding non-null value and create a BitVector to trace the indices for non-null values so that it could be deserialized properly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)