[jira] [Commented] (HIVE-19875) increase LLAP IO queue size for perf

["Karen Coppage (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-19875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809604#comment-16809604 ] 

Karen Coppage commented on HIVE-19875:
--------------------------------------

Hi [~prasanth_j], branch 3.0 can't compile because of this error:
 Compilation failure
 [ERROR] ..../hive/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java:[228,54] cannot find symbol
 [ERROR] symbol: variable MAX_DECIMAL64_PRECISION
 [ERROR] location: class org.apache.orc.TypeDescription

It seems in 3.0.x the ORC version is 1.4.3 in which org.apache.orc.TypeDescription.MAX_DECIMAL64_PRECISION was not yet introduced. ORC was upgraded from branch-3.1 (HIVE-19669), to a version where MAX_DECIMAL64_PRECISION exists.

> increase LLAP IO queue size for perf
> ------------------------------------
>
>                 Key: HIVE-19875
>                 URL: https://issues.apache.org/jira/browse/HIVE-19875
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 3.1.0, 4.0.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>            Priority: Major
>             Fix For: 3.1.0, 3.0.1, 4.0.0
>
>         Attachments: HIVE-19875.1.patch, HIVE-19875.2.patch
>
>
> According to [~gopalv] queue limit has perf impact, esp. during hashtable load for mapjoin where in the past IO used to queue up more data for processing to process.
> 1) Overall the default limit could be adjusted higher.
> 2) Depending on Decimal64 availability, the weight for decimal columns could be reduced.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)