[jira] [Commented] (HIVE-10470) LLAP: NPE in IO when returning 0 rows with no projection

["Sergey Shelukhin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-10470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14575071#comment-14575071 ] 

Sergey Shelukhin commented on HIVE-10470:
-----------------------------------------

[~prasanth_j] may not be an issue, but can you check?
{noformat}
-      if (hasIndexOnlyCols) {
+      if (hasIndexOnlyCols && (includedRgs == null)) {
{noformat}
This only returns the dummy batch if there's no RG filtering at all. Could there be partial RG filtering in this case? I'd assume not, but I'm not sure... after all, someone does filter all the rows. It will probably be hard to cover by a q file test, unless lots of values are added to create partial RG filtering.

> LLAP: NPE in IO when returning 0 rows with no projection
> --------------------------------------------------------
>
>                 Key: HIVE-10470
>                 URL: https://issues.apache.org/jira/browse/HIVE-10470
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Sergey Shelukhin
>            Assignee: Prasanth Jayachandran
>             Fix For: llap
>
>         Attachments: HIVE-10470.1.patch
>
>
> Looks like a trivial fix, unless I'm missing something. I may do it later if you don't ;)
> {noformat}
> aused by: java.lang.NullPointerException
> 	at org.apache.hadoop.hive.ql.io.orc.EncodedTreeReaderFactory.createEncodedTreeReader(EncodedTreeReaderFactory.java:1764)
> 	at org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:92)
> 	at org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:39)
> 	at org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:116)
> 	at org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:36)
> 	at org.apache.hadoop.hive.ql.io.orc.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:329)
> 	at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:299)
> 	at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:55)
> 	at org.apache.hadoop.hive.common.CallableWithNdc.call(CallableWithNdc.java:37)
> 	... 4 more
> {noformat}
> Running q file
> {noformat}
> SET hive.vectorized.execution.enabled=true;
> SET hive.llap.io.enabled=false;
> SET hive.exec.orc.default.row.index.stride=1000;
> SET hive.optimize.index.filter=true;
> DROP TABLE orc_llap;
> CREATE TABLE orc_llap(
>     ctinyint TINYINT,
>     csmallint SMALLINT,
>     cint INT,
>     cbigint BIGINT,
>     cfloat FLOAT,
>     cdouble DOUBLE,
>     cstring1 STRING,
>     cstring2 STRING,
>     ctimestamp1 TIMESTAMP,
>     ctimestamp2 TIMESTAMP,
>     cboolean1 BOOLEAN,
>     cboolean2 BOOLEAN)
>     STORED AS ORC tblproperties ("orc.compress"="ZLIB");
> insert into table orc_llap
> select ctinyint, csmallint, cint, cbigint, cfloat, cdouble, cstring1, cstring2, ctimestamp1, ctimestamp2, cboolean1, cboolean2
> from alltypesorc limit 10;
> SET hive.llap.io.enabled=true;
> select count(*) from orc_llap where cint < 60000000;
> DROP TABLE orc_llap;
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)