You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "deepujain (via GitHub)" <gi...@apache.org> on 2026/03/07 23:35:16 UTC
[PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
deepujain opened a new pull request, #8303:
URL: https://github.com/apache/hadoop/pull/8303
# PR description — copy into GitHub when opening the PR (do not commit this file)
## HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classloader from Configuration in getInstanceFromReflection
### Problem
The configuration `fs.s3a.classloader.isolation=false` (introduced in HADOOP-18993) does not work as intended. The isolation flag is stored correctly in the Hadoop Configuration, but when creating instances via `S3AUtils.getInstanceFromReflection`, the classloader was always taken from `conf.getClassLoader()` without considering the isolation flag. When isolation is disabled, the Configuration may not have had its classloader set (e.g. `maybeIsolateClassloader` does not set it when isolation=false), so we must explicitly respect the flag and, when isolation is false, use the Configuration's classloader when set, otherwise the thread context classloader so that loading is not forced to use an isolated S3A classloader.
### Change
In `S3AUtils.getInstanceFromReflection`:
- When `conf != null` and `fs.s3a.classloader.isolation` is true (default): keep using `conf.getClassLoader()` (set by `maybeIsolateClassloader` to the S3A classloader).
- When `conf != null` and `fs.s3a.classloader.isolation` is false: use the classloader from the Configuration (`conf.getClassLoader()`); if that is null, use `Thread.currentThread().getContextClassLoader()` so that classes are loaded from the application classpath.
- If the resolved classloader is still null, fall back to `S3AUtils.class.getClassLoader()` for backward compatibility.
This ensures that when users set `fs.s3a.classloader.isolation=false`, instance creation uses the Configuration's classloader (or the context classloader) instead of always using the S3A isolated classloader.
### Tests
- Existing test `ITestS3AFileSystemIsolatedClassloader` covers isolation=true and isolation=false; the test expects that when isolation=false, `fs.getConf().getClassLoader()` is not the S3A classloader. This fix ensures the classloader used in `getInstanceFromReflection` is consistent with that expectation.
### JIRA
Fixes HADOOP-19833
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Add regression test for fs.s3a.classloader.isolation=false behavior. [hadoop]
Posted by "ajfabbri (via GitHub)" <gi...@apache.org>.
ajfabbri commented on code in PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#discussion_r3191611494
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFileSystemIsolatedClassloader.java:
##########
@@ -205,4 +205,29 @@ public void notIsolatedClassloader() throws IOException {
.isInstanceOf(CustomCredentialsProvider.class);
});
}
+
+ /**
+ * HADOOP-19833: When isolation=false, getInstanceFromReflection must use
+ * the context classloader (or conf's classloader) so that classes from the
+ * application classpath can be loaded. This test calls getInstanceFromReflection
+ * directly with isolation=false and verifies the custom class is instantiated.
+ */
+ @Test
+ public void testGetInstanceFromReflectionWithIsolationFalse() throws Exception {
Review Comment:
Thanks for the update. Looks like the bug was fixed already but appreciate the additional test coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
Posted by "yunzhaoy (via GitHub)" <gi...@apache.org>.
yunzhaoy commented on code in PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#discussion_r2907154031
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java:
##########
@@ -663,7 +663,17 @@ public static <InstanceT> InstanceT getInstanceFromReflection(String className,
try {
ClassLoader classLoader;
if (conf != null) {
- classLoader = conf.getClassLoader();
+ if (conf.getBoolean(AWS_S3_CLASSLOADER_ISOLATION,
+ DEFAULT_AWS_S3_CLASSLOADER_ISOLATION)) {
+ classLoader = conf.getClassLoader();
+ } else {
+ ClassLoader confLoader = conf.getClassLoader();
+ classLoader = (confLoader != null) ? confLoader
+ : Thread.currentThread().getContextClassLoader();
+ }
+ if (classLoader == null) {
+ classLoader = S3AUtils.class.getClassLoader();
+ }
Review Comment:
Based on the logic [here](https://github.com/apache/hadoop/blob/release-3.5.0-RC0/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L803-L806), the config.getClassLoader should never be null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#issuecomment-4036670953
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 49m 16s | | trunk passed |
| +1 :green_heart: | compile | 1m 9s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | compile | 1m 7s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | checkstyle | 1m 0s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 1s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 35m 27s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 44s | | the patch passed |
| +1 :green_heart: | compile | 0m 38s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javac | 0m 38s | | the patch passed |
| +1 :green_heart: | compile | 0m 38s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | javac | 0m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 30s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 45s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 34m 49s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 3m 44s | | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not generate ASF License warnings. |
| | | 140m 34s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/8303 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux bcea484b57c6 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4a6e322b79f197917caa6fe36094461f9786b91e |
| Default Java | Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/4/testReport/ |
| Max. process+thread count | 583 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/4/console |
| versions | git=2.43.0 maven=3.9.11 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#issuecomment-4017987820
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 18m 46s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 50m 24s | | trunk passed |
| +1 :green_heart: | compile | 1m 8s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | compile | 1m 2s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | checkstyle | 1m 0s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 14s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 2m 44s | | trunk passed |
| +1 :green_heart: | shadedclient | 37m 26s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 45s | | the patch passed |
| +1 :green_heart: | compile | 0m 42s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javac | 0m 42s | | the patch passed |
| +1 :green_heart: | compile | 0m 41s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | javac | 0m 41s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 32s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 49s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 33m 45s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 4m 3s | | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not generate ASF License warnings. |
| | | 161m 19s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/8303 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 82ecd42d922b 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 72bf011fcc339f5e36932081ff6a6803c3e11707 |
| Default Java | Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/1/testReport/ |
| Max. process+thread count | 583 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/1/console |
| versions | git=2.43.0 maven=3.9.11 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
Posted by "deepujain (via GitHub)" <gi...@apache.org>.
deepujain commented on PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#issuecomment-4024160942
My JIRA id is deepujain
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Add regression test for fs.s3a.classloader.isolation=false behavior. [hadoop]
Posted by "ajfabbri (via GitHub)" <gi...@apache.org>.
ajfabbri commented on PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#issuecomment-4792531794
> Seems like @rashworld-max is an AI. We should block this user somehow. @steveloughran @ajfabbri ?
Let's all report abuse via github: https://support.github.com/contact/report-abuse
I will create an INFRA JIRA to block them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Add regression test for fs.s3a.classloader.isolation=false behavior. [hadoop]
Posted by "mukund-thakur (via GitHub)" <gi...@apache.org>.
mukund-thakur commented on PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#issuecomment-4792457518
Seems like @rashworld-max is an AI. We should block this user somehow. @steveloughran @ajfabbri ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
Posted by "jlalwani-amazon (via GitHub)" <gi...@apache.org>.
jlalwani-amazon commented on PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#issuecomment-4315071917
Can a committer look at this please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Add regression test for fs.s3a.classloader.isolation=false behavior. [hadoop]
Posted by "deepujain (via GitHub)" <gi...@apache.org>.
deepujain commented on PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#issuecomment-4376173788
Updated the PR title and description to clarify the current branch scope: this branch is test-only and should be read as regression coverage for the isolation=false behavior, not as a standalone production logic change. I cannot edit the JIRA issue directly from here, but the PR text now matches the actual contents of the branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
Posted by "ajfabbri (via GitHub)" <gi...@apache.org>.
ajfabbri commented on code in PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#discussion_r3184397078
##########
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AFileSystemIsolatedClassloader.java:
##########
@@ -205,4 +205,29 @@ public void notIsolatedClassloader() throws IOException {
.isInstanceOf(CustomCredentialsProvider.class);
});
}
+
+ /**
+ * HADOOP-19833: When isolation=false, getInstanceFromReflection must use
+ * the context classloader (or conf's classloader) so that classes from the
+ * application classpath can be loaded. This test calls getInstanceFromReflection
+ * directly with isolation=false and verifies the custom class is instantiated.
+ */
+ @Test
+ public void testGetInstanceFromReflectionWithIsolationFalse() throws Exception {
Review Comment:
This PR currently only adds a test--it doesn't include any production logic changes. Please clarify if this is supposed to be a bug fix or not, and update the description of the PR and JIRA accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Add regression test for fs.s3a.classloader.isolation=false behavior. [hadoop]
Posted by "ajfabbri (via GitHub)" <gi...@apache.org>.
ajfabbri merged PR #8303:
URL: https://github.com/apache/hadoop/pull/8303
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#issuecomment-4030152571
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 19m 6s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 2s | | trunk passed |
| +1 :green_heart: | compile | 1m 6s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | compile | 1m 1s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | checkstyle | 0m 56s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 10s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 1s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 44s | | trunk passed |
| +1 :green_heart: | shadedclient | 29m 14s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 0m 37s | [/patch-mvninstall-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/3/artifact/out/patch-mvninstall-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
| -1 :x: | compile | 0m 37s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-21.0.10+7-Ubuntu-124.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/3/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-21.0.10+7-Ubuntu-124.04.txt) | hadoop-aws in the patch failed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04. |
| -1 :x: | javac | 0m 37s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-21.0.10+7-Ubuntu-124.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/3/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-21.0.10+7-Ubuntu-124.04.txt) | hadoop-aws in the patch failed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04. |
| -1 :x: | compile | 0m 38s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-17.0.18+8-Ubuntu-124.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/3/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-17.0.18+8-Ubuntu-124.04.1.txt) | hadoop-aws in the patch failed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1. |
| -1 :x: | javac | 0m 38s | [/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-17.0.18+8-Ubuntu-124.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/3/artifact/out/patch-compile-hadoop-tools_hadoop-aws-jdkUbuntu-17.0.18+8-Ubuntu-124.04.1.txt) | hadoop-aws in the patch failed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 26s | | the patch passed |
| -1 :x: | mvnsite | 0m 38s | [/patch-mvnsite-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/3/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| -1 :x: | spotbugs | 0m 35s | [/patch-spotbugs-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/3/artifact/out/patch-spotbugs-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
| +1 :green_heart: | shadedclient | 31m 51s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 0m 39s | [/patch-unit-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/3/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt) | hadoop-aws in the patch failed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not generate ASF License warnings. |
| | | 134m 1s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/8303 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d02193be7b0b 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1f43995dda9df76badc6eb854bf86a2999ca7c64 |
| Default Java | Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/3/testReport/ |
| Max. process+thread count | 634 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/3/console |
| versions | git=2.43.0 maven=3.9.11 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
Posted by "yunzhaoy (via GitHub)" <gi...@apache.org>.
yunzhaoy commented on code in PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#discussion_r2907151206
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java:
##########
@@ -663,7 +663,17 @@ public static <InstanceT> InstanceT getInstanceFromReflection(String className,
try {
ClassLoader classLoader;
if (conf != null) {
- classLoader = conf.getClassLoader();
+ if (conf.getBoolean(AWS_S3_CLASSLOADER_ISOLATION,
+ DEFAULT_AWS_S3_CLASSLOADER_ISOLATION)) {
+ classLoader = conf.getClassLoader();
+ } else {
Review Comment:
I feel this check is not required. Based on this [method](https://github.com/apache/hadoop/blob/release-3.4.0-RC3/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java#L1696-L1714), the config.getClassLoader should be used no matter whether the isolation is true or false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
Posted by "deepujain (via GitHub)" <gi...@apache.org>.
deepujain commented on code in PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#discussion_r2907254825
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java:
##########
@@ -663,7 +663,17 @@ public static <InstanceT> InstanceT getInstanceFromReflection(String className,
try {
ClassLoader classLoader;
if (conf != null) {
- classLoader = conf.getClassLoader();
+ if (conf.getBoolean(AWS_S3_CLASSLOADER_ISOLATION,
+ DEFAULT_AWS_S3_CLASSLOADER_ISOLATION)) {
+ classLoader = conf.getClassLoader();
+ } else {
+ ClassLoader confLoader = conf.getClassLoader();
+ classLoader = (confLoader != null) ? confLoader
+ : Thread.currentThread().getContextClassLoader();
+ }
+ if (classLoader == null) {
+ classLoader = S3AUtils.class.getClassLoader();
+ }
Review Comment:
the implementation was simplified to always use conf.getClassLoader() when conf != null, since the Configuration’s classloader is set by the caller and is never null.
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java:
##########
@@ -663,7 +663,17 @@ public static <InstanceT> InstanceT getInstanceFromReflection(String className,
try {
ClassLoader classLoader;
if (conf != null) {
- classLoader = conf.getClassLoader();
+ if (conf.getBoolean(AWS_S3_CLASSLOADER_ISOLATION,
+ DEFAULT_AWS_S3_CLASSLOADER_ISOLATION)) {
+ classLoader = conf.getClassLoader();
+ } else {
Review Comment:
the implementation was simplified to always use conf.getClassLoader() when conf != null, since the Configuration’s classloader is set by the caller and is never null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
Posted by "deepujain (via GitHub)" <gi...@apache.org>.
deepujain commented on PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#issuecomment-4025937293
The PR now only has unit tests. it documents and verifies that with fs.s3a.classloader.isolation=false, a custom credentials provider can be loaded via getInstanceFromReflection when the Configuration (e.g. from the FS) has the right classloader.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Fix fs.s3a.classloader.isolation=false by using classlo… [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#issuecomment-4018574646
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 55s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 49m 29s | | trunk passed |
| +1 :green_heart: | compile | 1m 12s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | compile | 1m 24s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | checkstyle | 1m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 53s | | trunk passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 35m 45s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 44s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | javac | 0m 43s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | checkstyle | 0m 28s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 47s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK Ubuntu-21.0.10+7-Ubuntu-124.04 |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| +1 :green_heart: | spotbugs | 1m 29s | | the patch passed |
| +1 :green_heart: | shadedclient | 34m 59s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 4m 47s | | hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not generate ASF License warnings. |
| | | 142m 57s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/8303 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 9af70a14d4e2 5.15.0-164-generic #174-Ubuntu SMP Fri Nov 14 20:25:16 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e98c38ccd075da09e08869b60b4e6a956ca92d65 |
| Default Java | Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Multi-JDK versions | /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.10+7-Ubuntu-124.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.18+8-Ubuntu-124.04.1 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/2/testReport/ |
| Max. process+thread count | 570 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8303/2/console |
| versions | git=2.43.0 maven=3.9.11 spotbugs=4.9.7 |
| Powered by | Apache Yetus 0.14.1 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19833. Add regression test for fs.s3a.classloader.isolation=false behavior. [hadoop]
Posted by "ajfabbri (via GitHub)" <gi...@apache.org>.
ajfabbri commented on PR #8303:
URL: https://github.com/apache/hadoop/pull/8303#issuecomment-4792580961
INFRA Jira here: https://issues.apache.org/jira/browse/INFRA-28056. Thanks for the heads-up @mukund-thakur !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org