You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Michael Smith (Code Review)" <ge...@cloudera.org> on 2022/10/17 18:45:19 UTC

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Michael Smith has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19146


Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................

IMPALA-11667: Clean up Java dependency exclusions

Use dependencyManagement to simplify Java dependencies by directly
controlling versions of transitive dependencies instead of using
exclusions and direct inclusion.

Dependency management specifies versions authoritatively, so redundant
version declarations are also removed.

Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
---
M fe/pom.xml
M java/TableFlattener/pom.xml
M java/datagenerator/pom.xml
M java/executor-deps/pom.xml
M java/ext-data-source/test/pom.xml
M java/pom.xml
M java/shaded-deps/hive-exec/pom.xml
M java/test-hive-udfs/pom.xml
M java/yarn-extras/pom.xml
9 files changed, 109 insertions(+), 266 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/19146/1
-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19146/5/java/pom.xml
File java/pom.xml:

http://gerrit.cloudera.org:8080/#/c/19146/5/java/pom.xml@255
PS5, Line 255:   <dependencies>
             :     <!-- Provided by reload4j instead. -->
             :     <dependency>
             :       <groupId>log4j</groupId>
             :       <artifactId>log4j</artifactId>
             :       <version>1.2.17</version>
             :       <scope>provided</scope>
             :     </dependency>
             : 
             :     <!-- Provided by our own s3a-aws-sdk shaded jar. -->
             :     <dependency>
             :       <groupId>com.amazonaws</groupId>
             :       <artifactId>aws-java-sdk-bundle</artifactId>
             :       <version>1.12.132</version>
             :       <scope>provided</scope>
             :     </dependency>
             :   </dependencies>
> It does seem error-prone, and with log4j we can't have a check that it's no
Upgrading the CDP build we use might reduce the use of log4j too, since there's an effort to switch to reload4j.



-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 18 Oct 2022 00:12:06 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/19146/3/java/pom.xml
File java/pom.xml:

http://gerrit.cloudera.org:8080/#/c/19146/3/java/pom.xml@255
PS3, Line 255:   <dependencies>
             :     <!-- Provided by reload4j instead. -->
             :     <dependency>
             :       <groupId>log4j</groupId>
             :       <artifactId>log4j</artifactId>
             :       <version>1.2.17</version>
             :       <scope>provided</scope>
             :     </dependency>
             : 
             :     <!-- Provided by our own s3a-aws-sdk shaded jar. -->
             :     <dependency>
             :       <groupId>com.amazonaws</groupId>
             :       <artifactId>aws-java-sdk-bundle</artifactId>
             :       <version>1.12.132</version>
             :       <scope>provided</scope>
             :     </dependency>
             :   </dependencies>
> I think this is a problem for the shaded s3 jar. The minimal shaded s3 jar 
Sorry, somehow commented on an old revision, moving this to the latest


http://gerrit.cloudera.org:8080/#/c/19146/5/java/pom.xml
File java/pom.xml:

http://gerrit.cloudera.org:8080/#/c/19146/5/java/pom.xml@255
PS5, Line 255:   <dependencies>
             :     <!-- Provided by reload4j instead. -->
             :     <dependency>
             :       <groupId>log4j</groupId>
             :       <artifactId>log4j</artifactId>
             :       <version>1.2.17</version>
             :       <scope>provided</scope>
             :     </dependency>
             : 
             :     <!-- Provided by our own s3a-aws-sdk shaded jar. -->
             :     <dependency>
             :       <groupId>com.amazonaws</groupId>
             :       <artifactId>aws-java-sdk-bundle</artifactId>
             :       <version>1.12.132</version>
             :       <scope>provided</scope>
             :     </dependency>
             :   </dependencies>
I think this is a problem for the shaded s3 jar. The minimal shaded s3 jar has a dependency on hadoop-aws and gets aws-java-sdk-bundle as a result of that dependency. That ensures that our version matches hadoop's version.

The provided dependency means that hadoop-aws doesn't pull in aws-java-sdk-bundle and the shaded jar is empty.

Other thought: Using provided as a replacement for exclusions is pretty subtle. It would be ok if these remained as exclusions.



-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 23:59:48 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11647/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 19:10:09 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19146

to look at the new patch set (#4).

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................

IMPALA-11667: Clean up Java dependency exclusions

Use dependencyManagement to simplify Java dependencies by directly
controlling versions of transitive dependencies instead of using
exclusions and direct inclusion.

Dependency management specifies versions authoritatively, so redundant
version declarations are also removed.

Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
---
M fe/pom.xml
M java/TableFlattener/pom.xml
M java/datagenerator/pom.xml
M java/executor-deps/pom.xml
M java/ext-data-source/sample/pom.xml
M java/ext-data-source/test/pom.xml
M java/pom.xml
M java/shaded-deps/hive-exec/pom.xml
M java/test-hive-udfs/pom.xml
M java/yarn-extras/pom.xml
10 files changed, 118 insertions(+), 267 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/19146/4
-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8713/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 18:46:12 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11649/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 21:27:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19146

to look at the new patch set (#3).

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................

IMPALA-11667: Clean up Java dependency exclusions

Use dependencyManagement to simplify Java dependencies by directly
controlling versions of transitive dependencies instead of using
exclusions and direct inclusion.

Dependency management specifies versions authoritatively, so redundant
version declarations are also removed.

Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
---
M fe/pom.xml
M java/TableFlattener/pom.xml
M java/datagenerator/pom.xml
M java/executor-deps/pom.xml
M java/ext-data-source/sample/pom.xml
M java/ext-data-source/test/pom.xml
M java/pom.xml
M java/shaded-deps/hive-exec/pom.xml
M java/test-hive-udfs/pom.xml
M java/yarn-extras/pom.xml
10 files changed, 117 insertions(+), 267 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/19146/3
-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 6: Code-Review+2

I grabbed this locally and compared the generated classpaths. They don't have anything other than minor differences, which is good.

I think this is a good cleanup.


-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 18 Oct 2022 22:35:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 6: Verified+1

Top of stack passed GVO


-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Wed, 19 Oct 2022 15:53:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 6:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11656/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 18 Oct 2022 17:37:58 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 5:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/11650/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 22:24:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 1: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 23:51:50 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11648/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 3
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 19:56:33 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11646/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 19:05:15 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19146

to look at the new patch set (#5).

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................

IMPALA-11667: Clean up Java dependency exclusions

Use dependencyManagement to simplify Java dependencies by directly
controlling versions of transitive dependencies instead of using
exclusions and direct inclusion.

Dependency management specifies versions authoritatively, so redundant
version declarations are also removed.

Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
---
M fe/pom.xml
M java/TableFlattener/pom.xml
M java/datagenerator/pom.xml
M java/executor-deps/pom.xml
M java/ext-data-source/sample/pom.xml
M java/ext-data-source/test/pom.xml
M java/pom.xml
M java/shaded-deps/hive-exec/pom.xml
M java/test-hive-udfs/pom.xml
M java/yarn-extras/pom.xml
10 files changed, 129 insertions(+), 301 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/19146/5
-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19146/5/fe/pom.xml
File fe/pom.xml:

http://gerrit.cloudera.org:8080/#/c/19146/5/fe/pom.xml@a122
PS5, Line 122: 
             : 
             : 
> Can you keep this comment?
Ack



-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 18 Oct 2022 00:07:52 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................

IMPALA-11667: Clean up Java dependency exclusions

Use dependencyManagement to simplify Java dependencies by directly
controlling versions of transitive dependencies instead of using
exclusions and direct inclusion.

Dependency management specifies versions authoritatively, so redundant
version declarations are also removed.

Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Reviewed-on: http://gerrit.cloudera.org:8080/19146
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Joe McDonnell <jo...@cloudera.com>
---
M fe/pom.xml
M java/TableFlattener/pom.xml
M java/datagenerator/pom.xml
M java/executor-deps/pom.xml
M java/ext-data-source/sample/pom.xml
M java/ext-data-source/test/pom.xml
M java/pom.xml
M java/shaded-deps/hive-exec/pom.xml
M java/test-hive-udfs/pom.xml
M java/yarn-extras/pom.xml
10 files changed, 112 insertions(+), 256 deletions(-)

Approvals:
  Joe McDonnell: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 7
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19146

to look at the new patch set (#6).

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................

IMPALA-11667: Clean up Java dependency exclusions

Use dependencyManagement to simplify Java dependencies by directly
controlling versions of transitive dependencies instead of using
exclusions and direct inclusion.

Dependency management specifies versions authoritatively, so redundant
version declarations are also removed.

Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
---
M fe/pom.xml
M java/TableFlattener/pom.xml
M java/datagenerator/pom.xml
M java/executor-deps/pom.xml
M java/ext-data-source/sample/pom.xml
M java/ext-data-source/test/pom.xml
M java/pom.xml
M java/shaded-deps/hive-exec/pom.xml
M java/test-hive-udfs/pom.xml
M java/yarn-extras/pom.xml
10 files changed, 112 insertions(+), 256 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/19146/6
-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 6
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Hello Joe McDonnell, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19146

to look at the new patch set (#2).

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................

IMPALA-11667: Clean up Java dependency exclusions

Use dependencyManagement to simplify Java dependencies by directly
controlling versions of transitive dependencies instead of using
exclusions and direct inclusion.

Dependency management specifies versions authoritatively, so redundant
version declarations are also removed.

Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
---
M fe/pom.xml
M java/TableFlattener/pom.xml
M java/datagenerator/pom.xml
M java/executor-deps/pom.xml
M java/ext-data-source/sample/pom.xml
M java/ext-data-source/test/pom.xml
M java/pom.xml
M java/shaded-deps/hive-exec/pom.xml
M java/test-hive-udfs/pom.xml
M java/yarn-extras/pom.xml
10 files changed, 109 insertions(+), 267 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/19146/2
-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 5:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/19146/5/fe/pom.xml
File fe/pom.xml:

http://gerrit.cloudera.org:8080/#/c/19146/5/fe/pom.xml@a122
PS5, Line 122: 
             : 
             : 
Can you keep this comment?


http://gerrit.cloudera.org:8080/#/c/19146/3/java/pom.xml
File java/pom.xml:

http://gerrit.cloudera.org:8080/#/c/19146/3/java/pom.xml@255
PS3, Line 255:   <dependencies>
             :     <!-- Provided by reload4j instead. -->
             :     <dependency>
             :       <groupId>log4j</groupId>
             :       <artifactId>log4j</artifactId>
             :       <version>1.2.17</version>
             :       <scope>provided</scope>
             :     </dependency>
             : 
             :     <!-- Provided by our own s3a-aws-sdk shaded jar. -->
             :     <dependency>
             :       <groupId>com.amazonaws</groupId>
             :       <artifactId>aws-java-sdk-bundle</artifactId>
             :       <version>1.12.132</version>
             :       <scope>provided</scope>
             :     </dependency>
             :   </dependencies>
I think this is a problem for the shaded s3 jar. The minimal shaded s3 jar has a dependency on hadoop-aws and gets aws-java-sdk-bundle as a result of that dependency. That ensures that our version matches hadoop's version.

The provided dependency means that hadoop-aws doesn't pull in aws-java-sdk-bundle and the shaded jar is empty.

Other thought: Using provided as a replacement for exclusions is pretty subtle. It would be ok if these remained as exclusions.



-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Mon, 17 Oct 2022 23:57:16 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11667: Clean up Java dependency exclusions

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19146 )

Change subject: IMPALA-11667: Clean up Java dependency exclusions
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19146/5/java/pom.xml
File java/pom.xml:

http://gerrit.cloudera.org:8080/#/c/19146/5/java/pom.xml@255
PS5, Line 255:   <dependencies>
             :     <!-- Provided by reload4j instead. -->
             :     <dependency>
             :       <groupId>log4j</groupId>
             :       <artifactId>log4j</artifactId>
             :       <version>1.2.17</version>
             :       <scope>provided</scope>
             :     </dependency>
             : 
             :     <!-- Provided by our own s3a-aws-sdk shaded jar. -->
             :     <dependency>
             :       <groupId>com.amazonaws</groupId>
             :       <artifactId>aws-java-sdk-bundle</artifactId>
             :       <version>1.12.132</version>
             :       <scope>provided</scope>
             :     </dependency>
             :   </dependencies>
> I think this is a problem for the shaded s3 jar. The minimal shaded s3 jar 
It does seem error-prone, and with log4j we can't have a check that it's not present. I'll try removing these and using exclusions, hopefully the other changes have made the need for them less pervasive.



-- 
To view, visit http://gerrit.cloudera.org:8080/19146
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I424a175135855dcbd38ae432ea111cca5f562633
Gerrit-Change-Number: 19146
Gerrit-PatchSet: 5
Gerrit-Owner: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Comment-Date: Tue, 18 Oct 2022 00:06:51 +0000
Gerrit-HasComments: Yes