You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/01/24 19:44:26 UTC
[GitHub] [iceberg] sfc-gh-pbennes opened a new issue #3958: Linkage errors in published jars
sfc-gh-pbennes opened a new issue #3958:
URL: https://github.com/apache/iceberg/issues/3958
Greetings,
I run the [Google Linkage Checker](https://github.com/GoogleCloudPlatform/cloud-opensource-java/wiki/Linkage-Checker-Enforcer-Rule) to look for [linkage errors](https://jlbp.dev/glossary.html#linkage-error) as part of my Java builds and noticed that iceberg has some when adding iceberg-api, iceberg-core, iceberg-hive-metastore to a project:
```
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (enforce-linkage-checker) @ example ---
[ERROR] Linkage Checker rule found 53 reachable errors:
Class org.apache.hadoop.conf.Configurable is not found;
referenced by 6 class files
org.apache.iceberg.CatalogUtil (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.hadoop.HadoopCatalog (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.hadoop.HadoopConfigurable (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.hadoop.HadoopTables (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.jdbc.JdbcCatalog (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.hive.HiveCatalog (org.apache.iceberg:iceberg-hive-metastore:0.12.1)
Cause:
Unknown
Class org.apache.hadoop.conf.Configuration is not found;
referenced by 13 class files
org.apache.iceberg.CatalogUtil (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.SerializableTable (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.hadoop.HadoopCatalog (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.hadoop.HadoopFileIO (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.hadoop.HadoopInputFile (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.hadoop.HadoopTables (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.hadoop.SerializableConfiguration (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.hadoop.Util (org.apache.iceberg:iceberg-core:0.12.1)
org.apache.iceberg.hive.CachedClientPool (org.apache.iceberg:iceberg-hive-metastore:0.12.1)
org.apache.iceberg.hive.HiveCatalog (org.apache.iceberg:iceberg-hive-metastore:0.12.1)
org.apache.iceberg.hive.HiveCatalogs (org.apache.iceberg:iceberg-hive-metastore:0.12.1)
org.apache.iceberg.hive.HiveClientPool (org.apache.iceberg:iceberg-hive-metastore:0.12.1)
org.apache.iceberg.hive.HiveTableOperations (org.apache.iceberg:iceberg-hive-metastore:0.12.1)
Cause:
Unknown
...
```
How to reproduce:
Create this pom.xml and run `mvn verify`:
```
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>dev.null</groupId>
<artifactId>example</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>example</name>
<dependencies>
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-core</artifactId>
<version>0.12.1</version>
</dependency>
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-api</artifactId>
<version>0.12.1</version>
</dependency>
<dependency>
<groupId>org.apache.iceberg</groupId>
<artifactId>iceberg-hive-metastore</artifactId>
<version>0.12.1</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>3.0.0-M3</version>
<dependencies>
<dependency>
<groupId>com.google.cloud.tools</groupId>
<artifactId>linkage-checker-enforcer-rules</artifactId>
<version>1.5.12</version>
</dependency>
</dependencies>
<executions>
<execution>
<id>enforce-linkage-checker</id>
<phase>verify</phase>
<goals>
<goal>enforce</goal>
</goals>
<configuration>
<rules>
<LinkageCheckerRule
implementation="com.google.cloud.tools.dependencies.enforcer.LinkageCheckerRule">
<reportOnlyReachable>true</reportOnlyReachable>
</LinkageCheckerRule>
</rules>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
```
As best I can tell, it's [because hadoop-common is listed as a compileOnly](https://github.com/apache/iceberg/blob/master/build.gradle#L236) dependency which prevents it from ending up in [the public pom](https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-core/0.12.1/iceberg-core-0.12.1.pom). When there are optional dependencies it's normal to have these linkage errors, though it's not obvious this is a case of that. Either hadoop-common etc. should be marked as optional dependencies if they're only needed for functionality related to hadoop, or just included as regular dependencies so that the build system can correctly pull in the needed classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org