[jira] [Created] (HIVE-26298) Selecting complex types on migrated iceberg table does not work

["Gergely Fürnstáhl (Jira)" <ji...@apache.org>]

Gergely Fürnstáhl created HIVE-26298:
----------------------------------------

             Summary: Selecting complex types on migrated iceberg table does not work
                 Key: HIVE-26298
                 URL: https://issues.apache.org/jira/browse/HIVE-26298
             Project: Hive
          Issue Type: Bug
            Reporter: Gergely Fürnstáhl
         Attachments: 00001-a5d522f4-a065-44e6-983b-ba66596b4332.metadata.json

I am working on implementing NameMapping in Impala (mainly replicating Hive's functionality) and ran into the following issue:


{code:java}
CREATE TABLE array_demo
(
  int_primitive INT,
  int_array ARRAY<INT>,
  int_array_array ARRAY<ARRAY<INT>>,
  int_to_array_array_Map MAP<INT,ARRAY<ARRAY<INT>>>
)
STORED AS ORC;

INSERT INTO array_demo values (0, array(1), array(array(2), array(3,4)), map(5,array(array(6),array(7,8))));

select * from array_demo;
+---------------------------+-----------------------+-----------------------------+------------------------------------+
| array_demo.int_primitive  | array_demo.int_array  | array_demo.int_array_array  | array_demo.int_to_array_array_map  |
+---------------------------+-----------------------+-----------------------------+------------------------------------+
| 0                         | [1]                   | [[2],[3,4]]                 | {5:[[6],[7,8]]}                    |
+---------------------------+-----------------------+-----------------------------+------------------------------------+
 {code}
Converting to iceberg

 

 
{code:java}
ALTER TABLE array_demo SET TBLPROPERTIES ('storage_handler'='org.apache.iceberg.mr.hive.HiveIcebergStorageHandler')

select * from array_demo;
INFO  : Compiling command(queryId=gfurnstahl_20220608102746_54bf3e74-e12b-400b-94a9-4e4c9fe460fe): select * from array_demo
INFO  : No Stats for default@array_demo, Columns: int_primitive, int_array, int_to_array_array_map, int_array_array
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:array_demo.int_primitive, type:int, comment:null), FieldSchema(name:array_demo.int_array, type:array<int>, comment:null), FieldSchema(name:array_demo.int_array_array, type:array<array<int>>, comment:null), FieldSchema(name:array_demo.int_to_array_array_map, type:map<int,array<array<int>>>, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=gfurnstahl_20220608102746_54bf3e74-e12b-400b-94a9-4e4c9fe460fe); Time taken: 0.036 seconds
INFO  : Executing command(queryId=gfurnstahl_20220608102746_54bf3e74-e12b-400b-94a9-4e4c9fe460fe): select * from array_demo
INFO  : Completed executing command(queryId=gfurnstahl_20220608102746_54bf3e74-e12b-400b-94a9-4e4c9fe460fe); Time taken: 0.0 seconds
INFO  : OK
Error: java.io.IOException: java.lang.IllegalArgumentException: Can not promote MAP type to INTEGER (state=,code=0)

select int_primitive from array_demo;
+----------------+
| int_primitive  |
+----------------+
| 0              |
+----------------+
1 row selected (0.088 seconds)
 {code}
Removing schema.name-mapping.default solves it
{code:java}
ALTER TABLE array_demo UNSET TBLPROPERTIES ('schema.name-mapping.default');

select * from array_demo;
+---------------------------+-----------------------+-----------------------------+------------------------------------+
| array_demo.int_primitive  | array_demo.int_array  | array_demo.int_array_array  | array_demo.int_to_array_array_map  |
+---------------------------+-----------------------+-----------------------------+------------------------------------+
| 0                         | [1]                   | [[2],[3,4]]                 | {5:[[6],[7,8]]}                    |
+---------------------------+-----------------------+-----------------------------+------------------------------------+
 {code}
Possible cause:

 

The name mapping generated and pushed into schema.name-mapping.default is different from the name mapping in the schema in the metadata.json (attached it)



--
This message was sent by Atlassian Jira
(v8.20.7#820007)