You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "rdblue (via GitHub)" <gi...@apache.org> on 2026/04/01 21:58:55 UTC
[PR] Aliyun: Remove leaked transitive dependencies [iceberg]
rdblue opened a new pull request, #15858:
URL: https://github.com/apache/iceberg/pull/15858
While reviewing dependency changes to the Spark runtime bundle in https://github.com/apache/iceberg/pull/15655, we discovered that the Aliyun module was leaking several dependencies as well, including `okio` and the Kotlin stdlibJ:
```diff
137,156c135
< | +--- project :iceberg-common (*)
< | +--- com.aliyun:credentials-java:0.3.12
< | | +--- com.aliyun:tea:[1.1.14, 2.0.0) -> 1.4.1
< | | | +--- com.squareup.okhttp3:okhttp:4.12.0
< | | | | +--- com.squareup.okio:okio:3.6.0
< | | | | | \--- com.squareup.okio:okio-jvm:3.6.0
< | | | | | +--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.9.10
< | | | | | | +--- org.jetbrains.kotlin:kotlin-stdlib:1.9.10
< | | | | | | | \--- org.jetbrains.kotlin:kotlin-stdlib-common:1.9.10
< | | | | | | \--- org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.9.10
< | | | | | | \--- org.jetbrains.kotlin:kotlin-stdlib:1.9.10 (*)
< | | | | | \--- org.jetbrains.kotlin:kotlin-stdlib-common:1.9.10
< | | | | \--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.21 -> 1.9.10 (*)
< | | | \--- com.google.code.gson:gson:2.11.0
< | | | \--- com.google.errorprone:error_prone_annotations:2.27.0
< | | +--- com.google.code.gson:gson:2.11.0 (*)
< | | +--- org.jacoco:org.jacoco.agent:0.8.8
< | | +--- com.sun.xml.bind:jaxb-core:2.3.0
< | | \--- com.sun.xml.bind:jaxb-impl:2.3.0
< | \--- com.aliyun:tea:1.4.1 (*)
---
> | \--- project :iceberg-common (*)
```
This was introduced in https://github.com/apache/iceberg/commit/e93eaab3. This PR changes the dependencies to `compileOnly` to exclude them from inclusion in runtime Jars.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Aliyun: Remove leaked transitive dependencies [iceberg]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on PR #15858:
URL: https://github.com/apache/iceberg/pull/15858#issuecomment-4174463671
@rdblue, I see `libs.aliyun.credentials.java` is used in `AliyunClientFactories`
https://github.com/apache/iceberg/blob/e8b619148b557c5d72522e7f24bdadc9170eabfe/aliyun/src/main/java/org/apache/iceberg/aliyun/AliyunClientFactories.java#L21-L22
could this be a problem/regression at runtime?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Aliyun: Remove leaked transitive dependencies [iceberg]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on PR #15858:
URL: https://github.com/apache/iceberg/pull/15858#issuecomment-4179058152
> libs.aliyun.sdk.oss was already compile only,
good point!
It now throws
```
Caused by: java.lang.NoClassDefFoundError: com/aliyun/credentials/provider/OIDCRoleArnCredentialProvider
at org.apache.iceberg.aliyun.AliyunClientFactories$DefaultAliyunClientFactory.newOSSClient(AliyunClientFactories.java:131)
```
which is a reasonable error message
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Aliyun: Remove leaked transitive dependencies [iceberg]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer merged PR #15858:
URL: https://github.com/apache/iceberg/pull/15858
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Aliyun: Remove leaked transitive dependencies [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on PR #15858:
URL: https://github.com/apache/iceberg/pull/15858#issuecomment-4179054452
It also won't be a regression because this has not been released. We do not include dependencies like this in the runtime libraries and it is not Iceberg's responsibility to ensure everything is on the classpath.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Aliyun: Remove leaked transitive dependencies [iceberg]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on PR #15858:
URL: https://github.com/apache/iceberg/pull/15858#issuecomment-4178707883
@kevinjqliu
libs.aliyun.sdk.oss was already compile only,
So we had this weird condition where where a user could have manually included the SDK but not credentials-java and it would have worked if RRSA was on. Technically this would be a regression but we were wrong to do it before and folks were probably not doing the right thing if they relied on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Aliyun: Remove leaked transitive dependencies [iceberg]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on PR #15858:
URL: https://github.com/apache/iceberg/pull/15858#issuecomment-4174262736
Missed that the checks hadn't finished since I'm on my phone. But I'm sure we will be fine here, I will revert if they fail
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org