[jira] [Resolved] (HIVE-8853) Make vectorization work with Spark [Spark Branch]

["Jimmy Xiang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jimmy Xiang resolved HIVE-8853.
-------------------------------
       Resolution: Fixed
    Fix Version/s: spark-branch

Vectorization works with Spark now. Close this jira.

> Make vectorization work with Spark [Spark Branch]
> -------------------------------------------------
>
>                 Key: HIVE-8853
>                 URL: https://issues.apache.org/jira/browse/HIVE-8853
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Xuefu Zhang
>            Assignee: Jimmy Xiang
>             Fix For: spark-branch
>
>
> In Hive to make vectorization work, the reader needs to be also vectorized, which means that the reader can read a chunk of rows (or a list of column chunks) instead of one row at a time. However, we use Spark RDD for reading, which again utilized the underlying inputformat to read. Subsequent processing also needs to hapen in batches. We need to make sure that vectorizatoin is working as expected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)