You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Peter Toth (Jira)" <ji...@apache.org> on 2020/03/12 06:52:00 UTC

[jira] [Created] (HIVE-23014) ORC reading performance

Peter Toth created HIVE-23014:
---------------------------------

             Summary: ORC reading performance
                 Key: HIVE-23014
                 URL: https://issues.apache.org/jira/browse/HIVE-23014
             Project: Hive
          Issue Type: Bug
          Components: ORC
            Reporter: Peter Toth


Spark 3 adds support for using Hive 2.3.6 besides the old Hive 1.2.1 version. Some of the ORC reading benchmark shows that there is a huge performance difference in ORC reading between the 2 versions. I measured that {{org.apache.hadoop.hive.ql.io.orc.ReaderImpl}} in hive-exec-2.3.6-core.jar is ~3-5 times slower than in hive-exec-1.2.1.spark2.jar.

I'm not sure if more recent Hive versions still suffer from this performance regression.

Please see some details here: SPARK-30565



--
This message was sent by Atlassian Jira
(v8.3.4#803005)