You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Thomas Marshall (Code Review)" <ge...@cloudera.org> on 2019/06/06 18:28:28 UTC

[Impala-ASF-CR] Improve adding kudu-hive.jar to HADOOP CLASSPATH 

Thomas Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13542


Change subject: Improve adding kudu-hive.jar to HADOOP_CLASSPATH
......................................................................

Improve adding kudu-hive.jar to HADOOP_CLASSPATH

IMPALA-8503 added kudu-hive.jar to HADOOP_CLASSPATH in
run-hive-server.sh to allow the Hive Metastore to start
with Kudu's HMS plugin.

It accomplished this by fully specify the expected jar filename
based on the value of IMPALA_KUDU_JAVA_VERSION, but this is overly
restrictive for users who may wish to override this value in
impala-config-branch.sh to build their own branch with a different
version of the kudu-hive.jar

This patch relaxes this restriction by adding any jar containing the
string kudu-hive in IMPALA_KUDU_HOME to HADOOP_CLASSPATH

Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
---
M testdata/bin/run-hive-server.sh
1 file changed, 7 insertions(+), 3 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/13542/1
-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Thomas Marshall (Code Review)" <ge...@cloudera.org>.
Hello Hao Hao, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13542

to look at the new patch set (#2).

Change subject: Fix integration of kudu-hive.jar
......................................................................

Fix integration of kudu-hive.jar

IMPALA-8503 added downloading kudu-hive.jar and adding it to
HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to
start with Kudu's HMS plugin.

There are two problems with this that are fixed by this patch:
- Previously, we fully specify the expected jar filename based on the
  value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH
  but this is overly restrictive for users who may wish to override
  this value in impala-config-branch.sh to build their own branch with
  a different version of the kudu-hive.jar This patch relaxes this
  restriction by adding any jar containing the string kudu-hive in
  IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH
- In bootstrap_toolchain, we don't download a package if its directory
  already exists. Since the 'kudu' and 'kudu-java' packages download
  to the same directory, this led to a race condition where
  'kudu-java' might not be downloaded if 'kudu' had already been
  unpacked when it started. This patch fixes this by inspecting the
  contents of the Kudu package directory to look for specific files
  expected for each Kudu package.

Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M testdata/bin/run-hive-server.sh
3 files changed, 23 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/13542/2
-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
......................................................................


Patch Set 4: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Fri, 07 Jun 2019 02:18:37 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Thomas Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
......................................................................


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py
File bin/bootstrap_toolchain.py:

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py@433
PS2, Line 433: debug
> does this affect release builds?
Both the 'debug' and the 'release' directories will be present regardless of the build type because we always just unpack and keep everything in the tarball.


http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh
File testdata/bin/run-hive-server.sh:

http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh@105
PS2, Line 105: ${IMPALA_KUDU_JAVA_HOME}/*
> I think this could just be
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:24:54 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Thomas Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
......................................................................


Patch Set 4: Code-Review+2

carrying forward


-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:33:07 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] Improve adding kudu-hive.jar to HADOOP CLASSPATH

Posted by "Hao Hao (Code Review)" <ge...@cloudera.org>.
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Improve adding kudu-hive.jar to HADOOP_CLASSPATH
......................................................................


Patch Set 1: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Jun 2019 18:35:59 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/3526/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Jun 2019 21:11:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Lars Volker (Code Review)" <ge...@cloudera.org>.
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py
File bin/bootstrap_toolchain.py:

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py@433
PS2, Line 433: debug
does this affect release builds?


http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh
File testdata/bin/run-hive-server.sh:

http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh@105
PS2, Line 105: ${IMPALA_KUDU_JAVA_HOME}/*
I think this could just be

  for file in ${IMPALA_KUDU_JAVA_HOME}/*kudu-hive*jar; do

or you could use find | while.



-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:11:13 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Thomas Marshall (Code Review)" <ge...@cloudera.org>.
Hello Lars Volker, Hao Hao, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13542

to look at the new patch set (#3).

Change subject: Fix integration of kudu-hive.jar
......................................................................

Fix integration of kudu-hive.jar

IMPALA-8503 added downloading kudu-hive.jar and adding it to
HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to
start with Kudu's HMS plugin.

There are two problems with this that are fixed by this patch:
- Previously, we fully specify the expected jar filename based on the
  value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH
  but this is overly restrictive for users who may wish to override
  this value in impala-config-branch.sh to build their own branch with
  a different version of the kudu-hive.jar This patch relaxes this
  restriction by adding any jar containing the string kudu-hive in
  IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH
- In bootstrap_toolchain, we don't download a package if its directory
  already exists. Since the 'kudu' and 'kudu-java' packages download
  to the same directory, this led to a race condition where
  'kudu-java' might not be downloaded if 'kudu' had already been
  unpacked when it started. This patch fixes this by inspecting the
  contents of the Kudu package directory to look for specific files
  expected for each Kudu package.

Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M testdata/bin/run-hive-server.sh
3 files changed, 19 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/13542/3
-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/3524/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Jun 2019 21:05:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
......................................................................

Fix integration of kudu-hive.jar

IMPALA-8503 added downloading kudu-hive.jar and adding it to
HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to
start with Kudu's HMS plugin.

There are two problems with this that are fixed by this patch:
- Previously, we fully specify the expected jar filename based on the
  value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH
  but this is overly restrictive for users who may wish to override
  this value in impala-config-branch.sh to build their own branch with
  a different version of the kudu-hive.jar This patch relaxes this
  restriction by adding any jar containing the string kudu-hive in
  IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH
- In bootstrap_toolchain, we don't download a package if its directory
  already exists. Since the 'kudu' and 'kudu-java' packages download
  to the same directory, this led to a race condition where
  'kudu-java' might not be downloaded if 'kudu' had already been
  unpacked when it started. This patch fixes this by inspecting the
  contents of the Kudu package directory to look for specific files
  expected for each Kudu package.

Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Reviewed-on: http://gerrit.cloudera.org:8080/13542
Reviewed-by: Thomas Marshall <tm...@cloudera.com>
Reviewed-by: Hao Hao <ha...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M testdata/bin/run-hive-server.sh
3 files changed, 21 insertions(+), 5 deletions(-)

Approvals:
  Thomas Marshall: Looks good to me, approved
  Hao Hao: Looks good to me, but someone else must approve
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 5
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <tm...@cloudera.com>

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Hao Hao (Code Review)" <ge...@cloudera.org>.
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
......................................................................


Patch Set 4: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Jun 2019 21:20:08 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4424/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:33:20 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Lars Volker (Code Review)" <ge...@cloudera.org>.
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
......................................................................


Patch Set 3: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py
File bin/bootstrap_toolchain.py:

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py@433
PS2, Line 433: debug
> Both the 'debug' and the 'release' directories will be present regardless o
Thx for the explanation, might be good to leave that in a comment here for future reference.



-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:26:52 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/3523/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <tm...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:34:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] Fix integration of kudu-hive.jar

Posted by "Thomas Marshall (Code Review)" <ge...@cloudera.org>.
Hello Lars Volker, Hao Hao, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/13542

to look at the new patch set (#4).

Change subject: Fix integration of kudu-hive.jar
......................................................................

Fix integration of kudu-hive.jar

IMPALA-8503 added downloading kudu-hive.jar and adding it to
HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to
start with Kudu's HMS plugin.

There are two problems with this that are fixed by this patch:
- Previously, we fully specify the expected jar filename based on the
  value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH
  but this is overly restrictive for users who may wish to override
  this value in impala-config-branch.sh to build their own branch with
  a different version of the kudu-hive.jar This patch relaxes this
  restriction by adding any jar containing the string kudu-hive in
  IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH
- In bootstrap_toolchain, we don't download a package if its directory
  already exists. Since the 'kudu' and 'kudu-java' packages download
  to the same directory, this led to a race condition where
  'kudu-java' might not be downloaded if 'kudu' had already been
  unpacked when it started. This patch fixes this by inspecting the
  contents of the Kudu package directory to look for specific files
  expected for each Kudu package.

Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M testdata/bin/run-hive-server.sh
3 files changed, 21 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/13542/4
-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <tm...@cloudera.com>

[Impala-ASF-CR] Improve adding kudu-hive.jar to HADOOP CLASSPATH

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13542 )

Change subject: Improve adding kudu-hive.jar to HADOOP_CLASSPATH
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/3522/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 06 Jun 2019 19:08:36 +0000
Gerrit-HasComments: No