[jira] [Updated] (PHOENIX-3346) Hive PhoenixStorageHandler doesn't work well with column mapping

["Sergey Soldatov (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PHOENIX-3346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sergey Soldatov updated PHOENIX-3346:
-------------------------------------
    Attachment: PHOENIX-3346-1.patch

The patch includes:
1. proper column mapping for both rowkeys and regular columns, fully supported in both MR and Tez jobs
2. Fix for MR/Tez jobs when job fails if no hbase-site.xml in the app classpath. No need for it anymore and all connection parameters comes from table properties
3. Added tez support to IT tests. Now we have a single set of tests that executed for both MR and Tez. 
4. Added integration tests that verify that data written in Hive is accessible not only on Phoenix side, but from Hive as well. 

Notes:
1. Hive always using low case names for columns. Without additional mapping, the phoenix table will have low case names as well. 
2. New tests involve running quite a lot of yarn jobs (for each insert into Hive table, as well as for running Hive queries), so the execution is quite slow and may be unstable due resource limitation. 

> Hive PhoenixStorageHandler doesn't work well with column mapping
> ----------------------------------------------------------------
>
>                 Key: PHOENIX-3346
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3346
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Sergey Soldatov
>            Assignee: Sergey Soldatov
>              Labels: HivePhoenix
>         Attachments: PHOENIX-3346-1.patch
>
>
> If column mapping is used during table creation, the hive table becomes  unusable and throws UnknownColumn exception.
> There are several issues in the current implementation:
> 1. During table creation mapping doesn't applies to primary keys
> 2. During select query building no mapping happen
> 3. PhoenixRow should have backward mapping from phoenix column names to hive names.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)