[jira] [Commented] (HIVE-11233) Include Apache Phoenix support in HBaseStorageHandler

["Hive QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-11233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337960#comment-15337960 ] 

Hive QA commented on HIVE-11233:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12811362/HIVE-11233.5.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 10238 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_repair
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_external_table_ppd
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_binary_storage_queries
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_table_nonprintable
org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker.testPartitionsCheck
org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker.testTableCheck
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.testPigPopulation
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/165/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/165/console
Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-165/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12811362 - PreCommit-HIVE-MASTER-Build

> Include Apache Phoenix support in HBaseStorageHandler
> -----------------------------------------------------
>
>                 Key: HIVE-11233
>                 URL: https://issues.apache.org/jira/browse/HIVE-11233
>             Project: Hive
>          Issue Type: New Feature
>          Components: HBase Handler
>    Affects Versions: 1.2.1, 2.0.0
>            Reporter: Svetozar Ivanov
>            Assignee: Svetozar Ivanov
>              Labels: Binary, Hbase, Numeric, Phoenix, Sortable
>         Attachments: HIVE-11233-branch-1.2.patch, HIVE-11233-branch-2.0.patch, HIVE-11233.1.patch, HIVE-11233.2.patch, HIVE-11233.3.patch, HIVE-11233.4.patch, HIVE-11233.5.patch, HIVE-11233.patch
>
>
> Currently HBaseStorageHandler doesn't provide mechanism for storage of binary sortable key and values. It is necessary when given HBase table is used for persistence by Apache Hive and Apache Phoenix. In that way all byte arrays read or written by Hive will be compatible with binary sortable format used in Phoenix.
> It turns out the major difference is in all numeric data types accordingly officially provided documentation - https://phoenix.apache.org/language/datatypes.html.
> That's how I'm using it in my code:
> {code}
>     private static String buildWithSerDeProperties(TableDescriptor tableDescriptor) {
>         Map<String, String> serdePropertiesMap = new HashMap<>();
>         serdePropertiesMap.put(HBaseSerDe.HBASE_TABLE_NAME, tableDescriptor.getTableName());
>         serdePropertiesMap.put(HBaseSerDe.HBASE_TABLE_DEFAULT_STORAGE_TYPE, BINARY_STORAGE_TYPE);
>         serdePropertiesMap.put(HBaseSerDe.HBASE_COLUMNS_MAPPING, buildHBaseColumnsDefinition(tableDescriptor));
>         serdePropertiesMap.put(HBaseSerDe.HBASE_VALUE_FACTORY_CLASS, PhoenixValueFactory.class.getName());
>         /* Use different key factory for simple and composite primary key */
>         if (tableDescriptor.getPkDescriptors().size() == 1) {
>             serdePropertiesMap.put(HBaseSerDe.HBASE_KEY_FACTORY_CLASS, PhoenixKeyFactory.class.getName());
>         } else {
>             serdePropertiesMap.put(HBaseSerDe.HBASE_COMPOSITE_KEY_FACTORY, PhoenixCompositeKeyFactory.class.getName());
>         }
>         String serDeProperties = serdePropertiesMap.entrySet().stream()
>                 .map(e -> quoteInSingleQuotes(e.getKey()) + " = " + quoteInSingleQuotes(e.getValue()))
>                 .collect(Collectors.joining(COLUMNS_SEPARATOR));
>         logger.debug("SERDEPROPERTIES are [{}]", serDeProperties);
>         return serDeProperties;
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)