[jira] [Resolved] (IMPALA-7929) Impala query on HBASE table failing with InternalException: Required field*

["Yongjun Zhang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yongjun Zhang resolved IMPALA-7929.
-----------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 3.2.0

Many thanks to [~joemcdonnell], [~tgar] and [~paul-rogers] for the review and commit!

> Impala query on HBASE table failing with InternalException: Required field*
> ---------------------------------------------------------------------------
>
>                 Key: IMPALA-7929
>                 URL: https://issues.apache.org/jira/browse/IMPALA-7929
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>    Affects Versions: Impala 3.2.0
>            Reporter: Yongjun Zhang
>            Assignee: Yongjun Zhang
>            Priority: Major
>             Fix For: Impala 3.2.0
>
>
> This looks a corner case bug demonstrated at impala-hbase boundary.
> The way to reproduce:
> Create a table in hive shell,
> {code}
> create database abc;
> CREATE TABLE abc.test_hbase1 (k STRING, c STRING) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('
> hbase.columns.mapping'=':key,cf:c', 'serialization.format'='1') TBLPROPERTIES ('hbase.table.name'='test_hbase1', 'storage_handler'='o
> rg.apache.hadoop.hive.hbase.HBaseStorageHandler');
> {code}
> Then issue query at impala shell:
> {code}
> select * from abc.test_hbase1 where k != "row1"; 
> {code}
> Observe:
> {code}
> Query: select * from abc.test_hbase1 where k != "row1"                                     
> Query submitted at: 2018-12-04 17:02:42 (Coordinator: http://xyz:25000)    
> ERROR: InternalException: Required field 'qualifier' was not present! Struct: THBaseFilter(family::key, qualifier:null, op_ordinal:3, filter_constant:row1)
> {code}
> More observations:
> # Replacing {{k != "row1"}} with {{k <> "row1"}} fails the same way. However, replacing it with other operators, such as ">", "<", "=", all works.
> # Replacing {{k != "row1}} with {{c != "row1"}}, it succeeded without the error reported above.
> The above example uses a two-column table, creating a similar table with three columns fails the same way: adding inequality predicate on the first column fails, adding inequility predicate doesn't fail.
> The code that issues the error message is in HBase, it seems Impala did not pass the needed info to HBase in this special case. Also wonder if it's because the first column of the table is the key in hbase table that could reveal the bug.
> {code}
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/TColumnIncrement.java:      throw new org.apache.thrift.protocol.TProtocolException("Required field 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/TColumnValue.java:      throw new org.apache.thrift.protocol.TProtocolException("Required field 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:        throw new org.apache.thrift.protocol.TProtocolException("Required field 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:        throw new org.apache.thrift.protocol.TProtocolException("Required field 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:        throw new org.apache.thrift.protocol.TProtocolException("Required field 'qualifier' was not present! Struct: " + toString());
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)