You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "klaudworks (via GitHub)" <gi...@apache.org> on 2026/02/03 17:21:10 UTC
[PR] [hotfix] Use Thread Context ClassLoader for user class loading [flink-agents]
klaudworks opened a new pull request, #514:
URL: https://github.com/apache/flink-agents/pull/514
## Summary
This PR fixes `ClassNotFoundException` when loading user-defined resource classes (e.g., custom `ChatModel` implementations like `AzureOpenAIChatModelSetup`) from user JARs uploaded via the REST API.
## Problem
When `flink-agents-dist.jar` is deployed in `/opt/flink/lib`, it is loaded by the **System ClassLoader**. User JARs uploaded at runtime are loaded by Flink's **User ClassLoader** (a child of the System ClassLoader).
The existing code uses `Class.forName(className)` which uses the caller's classloader (System ClassLoader). Due to Java's parent-first delegation model, the System ClassLoader cannot see classes in its child classloaders, resulting in `ClassNotFoundException`.
## Solution
Use the **Thread Context ClassLoader (TCCL)** instead:
```java
Class.forName(className, true, Thread.currentThread().getContextClassLoader())
```
Flink sets the TCCL to the User ClassLoader before executing user code, making user-defined classes accessible to framework code.
## Files Changed
- `JavaResourceProvider.java` - Main resource instantiation
- `JavaSerializableResourceProvider.java` - Serializable resource deserialization
- `AgentPlan.java` - PythonResourceWrapper class checks
- `ActionJsonDeserializer.java` - Parameter type and config deserialization
- `FunctionToolJsonDeserializer.java` - Parameter type deserialization
- `EventLogRecordJsonDeserializer.java` - Event class deserialization
## Testing
Validated in a Flink 1.20 cluster with flink-agents, successfully loading custom `AzureOpenAIChatModelSetup` classes from user JARs uploaded via the REST API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [hotfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #514:
URL: https://github.com/apache/flink-agents/pull/514#issuecomment-3842624275
@klaudworks Please add the following content to your PR description and select a checkbox:
```
- [ ] `doc-needed`
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-included`
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [hotfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "klaudworks (via GitHub)" <gi...@apache.org>.
klaudworks commented on PR #514:
URL: https://github.com/apache/flink-agents/pull/514#issuecomment-3842639284
> @klaudworks Please add the following content to your PR description and select a checkbox:
>
> ```
> - [ ] `doc-needed`
> - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
> - [ ] `doc-included`
> ```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Bugfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong merged PR #514:
URL: https://github.com/apache/flink-agents/pull/514
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [hotfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "klaudworks (via GitHub)" <gi...@apache.org>.
klaudworks commented on PR #514:
URL: https://github.com/apache/flink-agents/pull/514#issuecomment-3845826668
Thanks for looking into this!
@xintongsong I'll write a proper issue for this in the next few minutes.
@xintongsong For my personal case the current workaround is quite inconvenient for proper deployment scenarios. I'm providing the platform running flink and can't anticipate what users will try run. Putting flink-agents into the lib folder to load it with the system classloader feels acceptable but rebuilding the flink Docker image whenever someone modifies a custom resource would involve some effort.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [hotfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "klaudworks (via GitHub)" <gi...@apache.org>.
klaudworks commented on PR #514:
URL: https://github.com/apache/flink-agents/pull/514#issuecomment-3842726856
The failed test is a flaky test. It relies on Ollama to do some calculation which is non-deterministic. You can rerun it and it should be fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [hotfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "klaudworks (via GitHub)" <gi...@apache.org>.
klaudworks commented on PR #514:
URL: https://github.com/apache/flink-agents/pull/514#issuecomment-3845830594
@wenjin272 thanks for further validating the fix!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [hotfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong commented on PR #514:
URL: https://github.com/apache/flink-agents/pull/514#issuecomment-3846100040
@klaudworks,
The workaround is admittedly not perfect. There will soon bee a patch release, likely in late Feb or early Mar, which should also carry fixes for other issues discovered during this period. Does it sound good to you?
The reason I don't want to withdraw the RC is that, we are trying to finalize the release before the Chinese New Year vacation. If we can make it, people who don't take the vacation can start trying out the 0.2.0 release and report issues during this time, so that when we are back we can resolve the issues immediately and ship 0.2.1 around the time mentioned above. Otherwise, we'll have to postpone the release to after the vacation, which means everything is paused during this time, and we'll probably have a stable patch release around end of March. And if we withdraw the RC now, it's very likely we won't make it before the vacation. It takes more than one week, building another RC, testing, voting, finalizing the artifacts, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [hotfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong commented on PR #514:
URL: https://github.com/apache/flink-agents/pull/514#issuecomment-3844930564
@klaudworks, thank you for reporting and fixing this bug. This is indeed a valid problem, and the fix sounds good. @wenjin272, please help check the details?
The 0.2.0 version is already in RC vote, and will soon be releases. Given that there's a workaround to this issue (placing user-defined resource in `flink/lib`), I'd suggest not to withdraw the RC and fix this in the next patch release. WDYT?
Additionally, I think we should:
- Create an issue describing the problem, workaround and track the fix version. This is helpful for other people who run into the same problem, because people usually tend to search in issues rather than PRs when they run into problems. @klaudworks, would you like to do that?
- We are planning to add some e2e tests that actually submit jobs into a flink standalone cluster, instead of currently running in a mini-cluster. I think user-defined resource should be included as one of the test cases. @wenjin272
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [hotfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "klaudworks (via GitHub)" <gi...@apache.org>.
klaudworks closed pull request #514: [hotfix] Use Thread Context ClassLoader for user class loading
URL: https://github.com/apache/flink-agents/pull/514
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [hotfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "wenjin272 (via GitHub)" <gi...@apache.org>.
wenjin272 commented on PR #514:
URL: https://github.com/apache/flink-agents/pull/514#issuecomment-3845096948
Hi, @klaudworks, thanks for your detailed bug report and fix. Since you only verified with custom resource, I verify the fix with custom event. It works well. The fix looks good to me.
Additionally, I investigated when we set the TCCL to the UserCodeClassLoader.
1. For agent plan deserialization, the `UserCodeClassLoader` is set by `UserDefinedObjectsHolder` in Flink `StreamGraph`.
2. For resource creation, the `UserCodeClassLoader` is set in `JavaActionTask` before execute action.
3. For event log deserialization, there is no place to set the `UserCodeClassLoader` because deserialization does not actually occur.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Bugfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "klaudworks (via GitHub)" <gi...@apache.org>.
klaudworks commented on PR #514:
URL: https://github.com/apache/flink-agents/pull/514#issuecomment-3846114372
@xintongsong that sounds good. Thank you for considering me but don't worry about it. I am running a custom version of flink-agents with the patch included.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Bugfix] Use Thread Context ClassLoader for user class loading [flink-agents]
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong commented on PR #514:
URL: https://github.com/apache/flink-agents/pull/514#issuecomment-3846140484
Ported to release-0.2 in b8559a3eb8b3439767cbe73877b8a8cb5407a77f
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org