You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/07/24 14:12:00 UTC

[jira] [Work logged] (HIVE-26072) Enable vectorization for stats gathering (tablescan op)

     [ https://issues.apache.org/jira/browse/HIVE-26072?focusedWorklogId=794629&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794629 ]

ASF GitHub Bot logged work on HIVE-26072:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 24/Jul/22 14:11
            Start Date: 24/Jul/22 14:11
    Worklog Time Spent: 10m 
      Work Description: ayushtkn opened a new pull request, #3142:
URL: https://github.com/apache/hive/pull/3142

   After https://issues.apache.org/jira/browse/HIVE-23530 , almost all compute stats functions are vectorizable. Only function that is not vectorizable is "compute_bit_vector" for ndv statistics computation. This causes "create table as select" and "insert overwrite select" queries to run in non-vectorized mode. 
   
   Even a very naive implementation of vectorized compute_bit_vector gives about 50% performance improvement on simple "insert overwrite select" queries. That is because entire mapper or reducer can run in vectorized mode.




Issue Time Tracking
-------------------

    Worklog Id:     (was: 794629)
    Time Spent: 1h 20m  (was: 1h 10m)

> Enable vectorization for stats gathering (tablescan op)
> -------------------------------------------------------
>
>                 Key: HIVE-26072
>                 URL: https://issues.apache.org/jira/browse/HIVE-26072
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>            Reporter: Rajesh Balamohan
>            Assignee: Ayush Saxena
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> https://issues.apache.org/jira/browse/HIVE-24510 enabled vectorization for compute_bit_vector. 
> But tablescan operator for stats gathering is disabled by default.
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java#L2577]
> Need to enable vectorization for this. This can significantly reduce runtimes for analyze statements for large tables.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)