You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/12/15 17:01:46 UTC
[jira] [Commented] (DRILL-4169) Upgrade Hive Storage Plugin to work
with latest stable Hive (v1.2.1)
[ https://issues.apache.org/jira/browse/DRILL-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15058242#comment-15058242 ]
ASF GitHub Bot commented on DRILL-4169:
---------------------------------------
GitHub user vkorukanti opened a pull request:
https://github.com/apache/drill/pull/302
DRILL-4169: Upgrade Hive storage plugin to work with Hive 1.2.1
+ HadoopShims.setTokenStr is moved to Utils.setTokenStr. There is no change
in functionality.
+ Disable binary partitions columns in Hive test suites. Binary
partition column feature is regressed in Hive 1.2.1. This should affect
only the Hive execution which is used to generate the test data. If Drill
is talking to Hive v1.0.0 (which has binary partition columns working),
Drill should be able to get the data from Hive without any issues (tested)
+ Update StorageHandler based test as there is an issue with test data
generation in Hive. Need a separate test with custom test StorageHandler.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vkorukanti/drill hive121
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/302.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #302
----
commit 1579d40641a8731b9478233d252349a7bf7166c5
Author: vkorukanti <ve...@gmail.com>
Date: 2015-12-11T19:36:11Z
DRILL-4194: Improve performance of the HiveScan metadata fetch operation
+ Use the stats (numRows) stored in Hive metastore whenever available to
calculate the costs for planning purpose
+ Delay the costly operation of loading of InputSplits until needed. When
InputSplits are loaded, cache them at query level to speedup subsequent
access.
this closes #301
commit ff555e63218038c5dddc5a4eecea7faf8cff058c
Author: vkorukanti <ve...@gmail.com>
Date: 2015-08-26T00:51:19Z
DRILL-4169: Upgrade Hive storage plugin to work with Hive 1.2.1
+ HadoopShims.setTokenStr is moved to Utils.setTokenStr. There is no change
in functionality.
+ Disable binary partitions columns in Hive test suites. Binary
partition column feature is regressed in Hive 1.2.1. This should affect
only the Hive execution which is used to generate the test data. If Drill
is talking to Hive v1.0.0 (which has binary partition columns working),
Drill should be able to get the data from Hive without any issues (tested)
+ Update StorageHandler based test as there is an issue with test data
generation in Hive. Need a separate test with custom test StorageHandler.
----
> Upgrade Hive Storage Plugin to work with latest stable Hive (v1.2.1)
> --------------------------------------------------------------------
>
> Key: DRILL-4169
> URL: https://issues.apache.org/jira/browse/DRILL-4169
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Hive
> Affects Versions: 1.4.0
> Reporter: Venki Korukanti
> Assignee: Venki Korukanti
> Fix For: 1.5.0
>
>
> There have been few bug fixes in Hive SerDes since Hive 1.0.0. Its good to update the Hive storage plugin to work with latest stable Hive version (1.2.1), so that HiveRecordReader can use the latest SerDes.
> Compatibility when working with lower versions (v1.0.0 - currently supported version) of Hive servers: There are no metastore API changes between Hive 1.0.0 and Hive 1.2.1 that affect how Drill's Hive storage plugin is interacting with Hive metastore. Tested to make sure it works fine. So users can use Drill to query Hive 1.0.0 (currently supported) and Hive 1.2.1 (new addition in this JIRA).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)