You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Grant Henke (Code Review)" <ge...@cloudera.org> on 2019/05/07 02:44:12 UTC

[kudu-CR] WIP [java] Upgrade Hive dependency to 3.1.1 

Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13256


Change subject: WIP [java] Upgrade Hive dependency to 3.1.1
......................................................................

WIP [java] Upgrade Hive dependency to 3.1.1

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
3 files changed, 9 insertions(+), 9 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/1
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>

[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#4).

Change subject: WIP: Upgrade Hive dependency to 3.1.1
......................................................................

WIP: Upgrade Hive dependency to 3.1.1

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/CMakeLists.txt
M src/kudu/hms/hive_metastore.thrift
M src/kudu/hms/mini_hms.cc
M thirdparty/package-hadoop.sh
M thirdparty/vars.sh
8 files changed, 823 insertions(+), 103 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/4
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] [hms] Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Adar Lieber-Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#17).

Change subject: [hms] Upgrade Hive dependency to 3.1.1
......................................................................

[hms] Upgrade Hive dependency to 3.1.1

This patch upgrades Hive to 3.1.1 from 2.3.4. Most of the compatibility
work has been done in previous patches. This patch contains
some testing changes that are 3.x specific along with an update
to the thrift interface.

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/hive_metastore.thrift
M thirdparty/vars.sh
5 files changed, 810 insertions(+), 97 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/17
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 17
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Lieber-Dembo <ad...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#9).

Change subject: WIP: Upgrade Hive dependency to 3.1.1
......................................................................

WIP: Upgrade Hive dependency to 3.1.1

This patch
- Upgrades Hive to 3.1.1
- Upgrades Hadoop to 3.2.0
- Uses the Hive `schematool` to initialized the
derby database in the MiniHms. This fixes issues
with autoCreate and is more representative of a
production environment.
- Workaround HIVE-21614 by using `LIKE` instead
of `=` when filtering tables.

TODO:
- Fix Sentry HiveConf compat issues.

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/CMakeLists.txt
M src/kudu/hms/hive_metastore.thrift
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_client-test.cc
M src/kudu/hms/mini_hms.cc
M src/kudu/sentry/mini_sentry.cc
M src/kudu/util/subprocess.cc
M src/kudu/util/subprocess.h
M thirdparty/build-thirdparty.sh
M thirdparty/package-hadoop.sh
M thirdparty/vars.sh
14 files changed, 865 insertions(+), 123 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/9
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 9
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] WIP [java] Upgrade Hive dependency to 3.1.1

Posted by "Hao Hao (Code Review)" <ge...@cloudera.org>.
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: WIP [java] Upgrade Hive dependency to 3.1.1
......................................................................


Patch Set 2:

(1 comment)

> Hao should take a look at this.
 > 
 > In general I want to understand what it means for Kudu's HMS
 > artifacts (the C++ client and the HMS plugin) to be built against
 > Hive 3. Will the artifacts be backwards compatible when deployed
 > alongside HMS from Hive 2? If not, and if we want to maintain both
 > Hive 2 and Hive 3 compatibility, we may need to duplicate the
 > build/test. That's painful for the HMS plugin (we can do it the way
 > we did Spark 1 and Spark 2), but I think it's downright impossible
 > in C++ without a ton of undesirable build hacking. Really crossing
 > my fingers that the C++ client will remain backwards compatible and
 > any interesting changes (see below) can be dealt with at runtime.
 > 
 > Separately, there's also the issue of data model changes between
 > Hive 2 and Hive 3. Has the HMS data model from Hive 2 changed in
 > such a way that we need to change how we use it? In the plugin test
 > that you updated there are now references to a table's "catalog
 > name", which feels like an important data model change.

It looks like the data model changes reflected in the test is mainly introduced in HIVE-18755 for supporting Catalog namespace. Wondering if it is workaround-able by passing a default namespace?

For C++ side upgrade, I briefly looked into the difference in hive_metastore.thrift between Hive 2.3 and Hive3.1, and didn't notice big changes except HIVE-18755. Though there also could be changes in Hive notification log that are important (e.g MessageFactory, which de/serialize the notification message) which I haven't checked.

http://gerrit.cloudera.org:8080/#/c/13256/2/java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
File java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java:

http://gerrit.cloudera.org:8080/#/c/13256/2/java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java@338
PS2, Line 338: client.dropTable
What happens if you use the same dropTable API existed in Hive 2? Will it break the test?



-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 07 May 2019 22:19:59 +0000
Gerrit-HasComments: Yes

[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: WIP: Upgrade Hive dependency to 3.1.1
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13256/2/java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
File java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java:

http://gerrit.cloudera.org:8080/#/c/13256/2/java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java@338
PS2, Line 338: client.dropTable
> What happens if you use the same dropTable API existed in Hive 2? Will it b
They sort of broke the APIs that use envContext. They kept the others though. If a catalog isn't specified, the default catalog is used.


http://gerrit.cloudera.org:8080/#/c/13256/3/thirdparty/vars.sh
File thirdparty/vars.sh:

http://gerrit.cloudera.org:8080/#/c/13256/3/thirdparty/vars.sh@220
PS3, Line 220: HIVE_NAME=hive-$HIVE_VERSION
             : HIVE_SOURCE=$TP_SOURCE_DIR/$HIVE_NAME
> Need to figure out whether this needs to be amended, or removed altogether.
All of these are included in Hive 3. I will remove.



-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 17 Jul 2019 19:27:14 +0000
Gerrit-HasComments: Yes

[kudu-CR] POC: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#16).

Change subject: POC: Upgrade Hive dependency to 3.1.1
......................................................................

POC: Upgrade Hive dependency to 3.1.1

This patch upgrades Hive to 3.1.1

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/hive_metastore.thrift
M thirdparty/vars.sh
5 files changed, 810 insertions(+), 97 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/16
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 16
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] POC: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#10).

Change subject: POC: Upgrade Hive dependency to 3.1.1
......................................................................

POC: Upgrade Hive dependency to 3.1.1

This patch
- Upgrades Hive to 3.1.1
- Upgrades Hadoop to 3.2.0
- Uses the Hive `schematool` to initialized the
derby database in the MiniHms. This fixes issues
with autoCreate and is more representative of a
production environment.
- Workaround HIVE-21614 by using `LIKE` instead
of `=` when filtering tables.

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/CMakeLists.txt
M src/kudu/hms/hive_metastore.thrift
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_client-test.cc
M src/kudu/hms/mini_hms.cc
M src/kudu/hms/mini_hms.h
M src/kudu/util/subprocess.cc
M src/kudu/util/subprocess.h
M thirdparty/package-hadoop.sh
M thirdparty/vars.sh
13 files changed, 862 insertions(+), 119 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/10
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 10
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#7).

Change subject: WIP: Upgrade Hive dependency to 3.1.1
......................................................................

WIP: Upgrade Hive dependency to 3.1.1

This patch
- Upgrades Hive to 3.1.1
- Upgrades Hadoop to 3.2.0
- Uses the Hive `schematool` to initialized the
derby database in the MiniHms. This fixes issues
with autoCreate and is more representative of a
production environment.
- Stripped the Hive jars from Sentry thirdparty
to avoid dependency issues.

TODO:
- Fix Sentry HiveConf compat issues.
- Use Subprocess::call with env vars instead of start/wait on schematool.
- Fix/Workaround HIVE-21614

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/CMakeLists.txt
M src/kudu/hms/hive_metastore.thrift
M src/kudu/hms/hms_catalog-test.cc
M src/kudu/hms/hms_client-test.cc
M src/kudu/hms/mini_hms.cc
M src/kudu/sentry/mini_sentry.cc
M thirdparty/build-thirdparty.sh
M thirdparty/package-hadoop.sh
M thirdparty/vars.sh
12 files changed, 872 insertions(+), 133 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/7
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 7
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] POC: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: POC: Upgrade Hive dependency to 3.1.1
......................................................................


Patch Set 16: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 16
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Thu, 19 Mar 2020 03:10:02 +0000
Gerrit-HasComments: No

[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: WIP: Upgrade Hive dependency to 3.1.1
......................................................................


Patch Set 3:

(1 comment)

Did a pass through the new code. Interesting to see some backwards incompatibilities in hive_metastore.thrift; guess the Hive devs didn't maintain that scrupulously.

http://gerrit.cloudera.org:8080/#/c/13256/3/thirdparty/vars.sh
File thirdparty/vars.sh:

http://gerrit.cloudera.org:8080/#/c/13256/3/thirdparty/vars.sh@220
PS3, Line 220: # TODO(dan): bump to a release version once HIVE-17747 and HIVE-16886/HIVE-18526
             : # are published. The SHA below is the current head of branch-2.
Need to figure out whether this needs to be amended, or removed altogether.



-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 17 Jul 2019 19:16:17 +0000
Gerrit-HasComments: Yes

[kudu-CR] POC: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#12).

Change subject: POC: Upgrade Hive dependency to 3.1.1
......................................................................

POC: Upgrade Hive dependency to 3.1.1

This patch
- Upgrades Hive to 3.1.1
- Upgrades Hadoop to 3.2.0

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/hive_metastore.thrift
M thirdparty/vars.sh
5 files changed, 808 insertions(+), 97 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/12
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 12
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] [hms] Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: [hms] Upgrade Hive dependency to 3.1.1
......................................................................


Patch Set 19:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13256/19//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/13256/19//COMMIT_MSG@7
PS19, Line 7: dependency to 3.1.1
> Mention Hadoop version bump to 3.2.0 as well in this patch?
Done


http://gerrit.cloudera.org:8080/#/c/13256/19//COMMIT_MSG@9
PS19, Line 9: 3.1.1
> Any reason to use 3.1.1 specifically? I saw the latest Apache Hive release 
Primarily because all the packaging, work, and testing was already done long ago. I think a follow on change can bump to the latest.



-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 19
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Thu, 07 May 2020 22:12:53 +0000
Gerrit-HasComments: Yes

[kudu-CR] POC: Upgrade Hive dependency to 3.1.1

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: POC: Upgrade Hive dependency to 3.1.1
......................................................................


Patch Set 13: Code-Review+1

Consider my review a +2, but we don't want to merge this yet, so I'm using a +1 instead.


-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 13
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 13 Aug 2019 05:29:13 +0000
Gerrit-HasComments: No

[kudu-CR] WIP [java] Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#2).

Change subject: WIP [java] Upgrade Hive dependency to 3.1.1
......................................................................

WIP [java] Upgrade Hive dependency to 3.1.1

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-client/build.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
4 files changed, 13 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/2
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#3).

Change subject: WIP: Upgrade Hive dependency to 3.1.1
......................................................................

WIP: Upgrade Hive dependency to 3.1.1

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/CMakeLists.txt
M src/kudu/hms/hive_metastore.thrift
M src/kudu/hms/mini_hms.cc
M thirdparty/package-hadoop.sh
M thirdparty/vars.sh
8 files changed, 823 insertions(+), 101 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/3
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] POC: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#15).

Change subject: POC: Upgrade Hive dependency to 3.1.1
......................................................................

POC: Upgrade Hive dependency to 3.1.1

This patch
- Upgrades Hive to 3.1.1
- Upgrades Hadoop to 3.2.0

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M src/kudu/hms/hive_metastore.thrift
M thirdparty/vars.sh
4 files changed, 801 insertions(+), 90 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/15
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 15
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#8).

Change subject: WIP: Upgrade Hive dependency to 3.1.1
......................................................................

WIP: Upgrade Hive dependency to 3.1.1

This patch
- Upgrades Hive to 3.1.1
- Upgrades Hadoop to 3.2.0
- Uses the Hive `schematool` to initialized the
derby database in the MiniHms. This fixes issues
with autoCreate and is more representative of a
production environment.
- Stripped the Hive jars from Sentry thirdparty
to avoid dependency issues.

TODO:
- Fix Sentry HiveConf compat issues.
- Use Subprocess::call with env vars instead of start/wait on schematool.
- Fix/Workaround HIVE-21614

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/CMakeLists.txt
M src/kudu/hms/hive_metastore.thrift
M src/kudu/hms/hms_catalog-test.cc
M src/kudu/hms/hms_client-test.cc
M src/kudu/hms/mini_hms.cc
M src/kudu/sentry/mini_sentry.cc
M thirdparty/build-thirdparty.sh
M thirdparty/package-hadoop.sh
M thirdparty/vars.sh
12 files changed, 871 insertions(+), 132 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/8
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 8
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#5).

Change subject: WIP: Upgrade Hive dependency to 3.1.1
......................................................................

WIP: Upgrade Hive dependency to 3.1.1

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/CMakeLists.txt
M src/kudu/hms/hive_metastore.thrift
M src/kudu/hms/mini_hms.cc
M thirdparty/package-hadoop.sh
M thirdparty/vars.sh
8 files changed, 835 insertions(+), 117 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/5
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] [hms] Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has removed Adar Lieber-Dembo from this change.  ( http://gerrit.cloudera.org:8080/13256 )

Change subject: [hms] Upgrade Hive dependency to 3.1.1
......................................................................


Removed reviewer Adar Lieber-Dembo.
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteReviewer
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 17
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] WIP [java] Upgrade Hive dependency to 3.1.1

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: WIP [java] Upgrade Hive dependency to 3.1.1
......................................................................


Patch Set 2:

Hao should take a look at this.

In general I want to understand what it means for Kudu's HMS artifacts (the C++ client and the HMS plugin) to be built against Hive 3. Will the artifacts be backwards compatible when deployed alongside HMS from Hive 2? If not, and if we want to maintain both Hive 2 and Hive 3 compatibility, we may need to duplicate the build/test. That's painful for the HMS plugin (we can do it the way we did Spark 1 and Spark 2), but I think it's downright impossible in C++ without a ton of undesirable build hacking. Really crossing my fingers that the C++ client will remain backwards compatible and any interesting changes (see below) can be dealt with at runtime.

Separately, there's also the issue of data model changes between Hive 2 and Hive 3. Has the HMS data model from Hive 2 changed in such a way that we need to change how we use it? In the plugin test that you updated there are now references to a table's "catalog name", which feels like an important data model change.


-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 07 May 2019 17:54:59 +0000
Gerrit-HasComments: No

[kudu-CR] [hms] Upgrade Hive to 3.1.1 and Hadoop to 3.2.0

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Attila Bukor, Kudu Jenkins, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#20).

Change subject: [hms] Upgrade Hive to 3.1.1 and Hadoop to 3.2.0
......................................................................

[hms] Upgrade Hive to 3.1.1 and Hadoop to 3.2.0

This patch upgrades Hive to 3.1.1 from 2.3.4 and Hadoop to 3.2.0
from 2.8.5. Most of the compatibility work has been done in previous
patches. This patch contains some testing changes that are 3.x specific
along with an update to the thrift interface.

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/hive_metastore.thrift
M thirdparty/vars.sh
5 files changed, 810 insertions(+), 97 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/20
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 20
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] POC: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has removed a vote on this change.

Change subject: POC: Upgrade Hive dependency to 3.1.1
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 16
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] WIP [java] Upgrade Hive dependency to 3.1.1

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: WIP [java] Upgrade Hive dependency to 3.1.1
......................................................................


Patch Set 1:

(1 comment)

The Java test failures look interesting to say the least.

http://gerrit.cloudera.org:8080/#/c/13256/1/java/gradle/dependencies.gradle
File java/gradle/dependencies.gradle:

http://gerrit.cloudera.org:8080/#/c/13256/1/java/gradle/dependencies.gradle@42
PS1, Line 42:     hive           : "3.1.1",
Earlier placement was more correct re: alphabetical ordering?



-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 07 May 2019 04:03:13 +0000
Gerrit-HasComments: Yes

[kudu-CR] [hms] Upgrade Hive to 3.1.1 and Hadoop to 3.2.0

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: [hms] Upgrade Hive to 3.1.1 and Hadoop to 3.2.0
......................................................................

[hms] Upgrade Hive to 3.1.1 and Hadoop to 3.2.0

This patch upgrades Hive to 3.1.1 from 2.3.4 and Hadoop to 3.2.0
from 2.8.5. Most of the compatibility work has been done in previous
patches. This patch contains some testing changes that are 3.x specific
along with an update to the thrift interface.

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Reviewed-on: http://gerrit.cloudera.org:8080/13256
Reviewed-by: Hao Hao <ha...@cloudera.com>
Tested-by: Kudu Jenkins
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/hive_metastore.thrift
M thirdparty/vars.sh
5 files changed, 810 insertions(+), 97 deletions(-)

Approvals:
  Hao Hao: Looks good to me, approved
  Kudu Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 21
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] POC: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#11).

Change subject: POC: Upgrade Hive dependency to 3.1.1
......................................................................

POC: Upgrade Hive dependency to 3.1.1

This patch
- Upgrades Hive to 3.1.1
- Upgrades Hadoop to 3.2.0
- Uses the Hive `schematool` to initialized the
derby database in the MiniHms. This fixes issues
with autoCreate and is more representative of a
production environment.
- Workaround HIVE-21614 by using `LIKE` instead
of `=` when filtering tables.

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/CMakeLists.txt
M src/kudu/hms/hive_metastore.thrift
M src/kudu/hms/hms_catalog.cc
M src/kudu/hms/hms_client-test.cc
M src/kudu/hms/mini_hms.cc
M src/kudu/hms/mini_hms.h
M src/kudu/util/subprocess.cc
M src/kudu/util/subprocess.h
M thirdparty/package-hadoop.sh
M thirdparty/vars.sh
13 files changed, 862 insertions(+), 119 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/11
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 11
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)

[kudu-CR] WIP: Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, Hao Hao, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13256

to look at the new patch set (#6).

Change subject: WIP: Upgrade Hive dependency to 3.1.1
......................................................................

WIP: Upgrade Hive dependency to 3.1.1

This patch
- Upgrades Hive to 3.1.1
- Upgrades Hadoop to 3.2.0
- Uses the Hive `schematool` to initialized the
derby database in the MiniHms. This fixes issues
with autoCreate and is more representative of a
production environment.

TODO:
- Use Subprocess::call with env vars instead of start/wait on schematool.
- Fix Sentry depending on Hive 2.2.0 and including hive jars in sentry_home/lib
- Fix/Workaround HIVE-21614

Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
---
M java/gradle/dependencies.gradle
M java/kudu-hive/build.gradle
M java/kudu-hive/src/test/java/org/apache/kudu/hive/metastore/TestKuduMetastorePlugin.java
M src/kudu/hms/CMakeLists.txt
M src/kudu/hms/hive_metastore.thrift
M src/kudu/hms/hms_catalog-test.cc
M src/kudu/hms/hms_client-test.cc
M src/kudu/hms/mini_hms.cc
M thirdparty/package-hadoop.sh
M thirdparty/vars.sh
10 files changed, 869 insertions(+), 132 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/56/13256/6
-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] POC: Upgrade Hive dependency to 3.1.1

Posted by "Hao Hao (Code Review)" <ge...@cloudera.org>.
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: POC: Upgrade Hive dependency to 3.1.1
......................................................................


Patch Set 13: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 13
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Thu, 08 Aug 2019 18:13:09 +0000
Gerrit-HasComments: No

[kudu-CR] WIP [java] Upgrade Hive dependency to 3.1.1

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: WIP [java] Upgrade Hive dependency to 3.1.1
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13256/1/java/gradle/dependencies.gradle
File java/gradle/dependencies.gradle:

http://gerrit.cloudera.org:8080/#/c/13256/1/java/gradle/dependencies.gradle@42
PS1, Line 42:     hive           : "3.1.1",
> Earlier placement was more correct re: alphabetical ordering?
yeah, this was a weird merge issue with another dep update match I was making. Will fix.



-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 07 May 2019 13:14:28 +0000
Gerrit-HasComments: Yes

[kudu-CR] [hms] Upgrade Hive to 3.1.1 and Hadoop to 3.2.0

Posted by "Hao Hao (Code Review)" <ge...@cloudera.org>.
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: [hms] Upgrade Hive to 3.1.1 and Hadoop to 3.2.0
......................................................................


Patch Set 20: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 20
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Thu, 07 May 2020 22:42:10 +0000
Gerrit-HasComments: No

[kudu-CR] [hms] Upgrade Hive dependency to 3.1.1

Posted by "Hao Hao (Code Review)" <ge...@cloudera.org>.
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13256 )

Change subject: [hms] Upgrade Hive dependency to 3.1.1
......................................................................


Patch Set 19:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13256/19//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/13256/19//COMMIT_MSG@7
PS19, Line 7: dependency to 3.1.1
Mention Hadoop version bump to 3.2.0 as well in this patch?


http://gerrit.cloudera.org:8080/#/c/13256/19//COMMIT_MSG@9
PS19, Line 9: 3.1.1
Any reason to use 3.1.1 specifically? I saw the latest Apache Hive release is 3.1.2.



-- 
To view, visit http://gerrit.cloudera.org:8080/13256
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec3ab2cc4e41a26cee8dd7f7098a2f288c56a42c
Gerrit-Change-Number: 13256
Gerrit-PatchSet: 19
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Thu, 07 May 2020 22:01:20 +0000
Gerrit-HasComments: Yes