You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by GitBox <gi...@apache.org> on 2019/10/24 15:45:49 UTC
[GitHub] [metron] nickwallen commented on a change in pull request #1523:
METRON-2232 Upgrade to Hadoop 3.1.1
nickwallen commented on a change in pull request #1523: METRON-2232 Upgrade to Hadoop 3.1.1
URL: https://github.com/apache/metron/pull/1523#discussion_r338651799
##########
File path: metron-platform/metron-hbase-server/pom.xml
##########
@@ -57,16 +63,34 @@
</exclusion>
</exclusions>
</dependency>
+ <dependency>
+ <!-- We depend on this for some Zookeeper upload operations -->
+ <groupId>org.apache.metron</groupId>
+ <artifactId>stellar-common</artifactId>
+ <version>${project.parent.version}</version>
+ <classifier>uber</classifier>
+ </dependency>
<dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-enrichment-common</artifactId>
<version>${project.parent.version}</version>
+ <exclusions>
+ <exclusion>
Review comment:
I've been suspicious of using wildcard exclusions because of differences between what Maven reports it is doing versus what it actually does when using these. I dug into this a little more and here is what I've found.
(1) Wildcard exclusions do indeed work when building the jar. By experimentation I can see that the dependencies from `metron-enrichment-common` are not being included in the uber jar, as we'd expect.
(2) The exclusion is reported correctly when using `mvn dependency:tree`. The output tells me below that no additional transitives are pulled-in.
```
~/tmp/metron-pr1523/metron-platform/metron-hbase-server
$ mvn dependency:tree
...
11:40:35,195 [INFO] +- org.apache.metron:metron-enrichment-common:jar:0.7.2:compile
11:40:35,195 [INFO] +- org.apache.metron:metron-hbase-common:jar:0.7.2:compile
11:40:35,195 [INFO] +- org.apache.hbase:hbase-common:jar:2.0.2:provided
...
```
(3) Oddly, when using `mvn dependency:tree -Dverbose` it makes it seem that the dependencies are NOT excluded. This output makes it seem that it is pulling in `hbase-client` as a transitive when it is not.
```
~/tmp/metron-pr1523/metron-platform/metron-hbase-server
$ mvn dependency:tree -Dverbose
...
11:38:37,776 [INFO] +- org.apache.metron:metron-enrichment-common:jar:0.7.2:compile
11:38:37,776 [INFO] | +- (com.github.ben-manes.caffeine:caffeine:jar:2.6.2:compile - omitted for duplicate)
11:38:37,776 [INFO] | +- org.apache.hbase:hbase-client:jar:2.0.2:compile
11:38:37,776 [INFO] | | +- org.apache.hbase.thirdparty:hbase-shaded-protobuf:jar:2.1.0:compile
11:38:37,776 [INFO] | | +- (org.apache.hbase:hbase-common:jar:2.0.2:compile - omitted for duplicate)
11:38:37,776 [INFO] | | +- org.apache.hbase:hbase-hadoop-compat:jar:2.0.2:compile
11:38:37,776 [INFO] | | | +- (org.apache.hbase.thirdparty:hbase-shaded-miscellaneous:jar:2.1.0:compile - omitted for duplicate)
11:38:37,776 [INFO] | | | +- (org.slf4j:slf4j-api:jar:1.7.25:compile - omitted for duplicate)
11:38:37,776 [INFO] | | | +- org.apache.hbase:hbase-metrics-api:jar:2.0.2:compile
11:38:37,776 [INFO] | | | | +- (org.apache.hbase:hbase-common:jar:2.0.2:compile - omitted for duplicate)
11:38:37,776 [INFO] | | | | +- (org.slf4j:slf4j-api:jar:1.7.25:compile - omitted for duplicate)
11:38:37,777 [INFO] | | | | +- (org.apache.commons:commons-lang3:jar:3.6:compile - omitted for conflict with 3.2)
11:38:37,777 [INFO] | | | | +- (org.apache.hbase.thirdparty:hbase-shaded-miscellaneous:jar:2.1.0:compile - omitted for duplicate)
11:38:37,777 [INFO] | | | | \- (org.apache.yetus:audience-annotations:jar:0.5.0:compile - omitted for duplicate)
11:38:37,777 [INFO] | | | \- (org.apache.yetus:audience-annotations:jar:0.5.0:compile - omitted for duplicate)
11:38:37,777 [INFO] | | +- org.apache.hbase:hbase-hadoop2-compat:jar:2.0.2:compile
```
I am not sure if this is a bug or is intentional when using `-Dverbose`, but it is something we will have to keep in mind when investigating classpath issues when wildcard exclusions are in-play.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services