You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by fhueske <gi...@git.apache.org> on 2017/09/20 17:02:03 UTC
[GitHub] flink pull request #4690: [FLINK-7656] [runtime] Switch to user classloader ...
GitHub user fhueske opened a pull request:
https://github.com/apache/flink/pull/4690
[FLINK-7656] [runtime] Switch to user classloader before calling initializeOnMaster and finalizeOnMaster.
## What is the purpose of the change
Fixes a classloading issue for `OutputFormat`s that implement `InitializeOnMaster` or `FinalizeOnMaster`. The initialize and finalize methods are called without setting the context classloader to the usercode classloader.
## Brief change log
* Set the context classloader to the usercode classloader before calling `InitializeOnMaster.initializeGlobal()` and `FinalizeOnMaster.finalizeGlobal()` and restore the original classloader afterwards.
* Extend test to check that the correct classloader is set.
## Verifying this change
* Run `JobTaskVertexTest.testOutputFormatVertex()`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): **no**
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no**
- The serializers: **no**
- The runtime per-record code paths (performance sensitive): **no**
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: **YES**
## Documentation
- Does this pull request introduce a new feature? **no**
- If yes, how is the feature documented? **n/a**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fhueske/flink outputOnMasterFix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4690.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4690
----
commit 2dfc8a4550981b9b8c417585dfb5e02b3282c77a
Author: Fabian Hueske <fh...@apache.org>
Date: 2017-09-20T14:26:47Z
[FLINK-7656] [runtime] Switch to user classloader before calling initializeOnMaster and finalizeOnMaster.
----
---
[GitHub] flink pull request #4690: [FLINK-7656] [runtime] Switch to user classloader ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4690
---
[GitHub] flink issue #4690: [FLINK-7656] [runtime] Switch to user classloader before ...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/4690
Can you take a brief look at this change @aljoscha? Thanks!
---
[GitHub] flink issue #4690: [FLINK-7656] [runtime] Switch to user classloader before ...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/4690
Thanks for the review @aljoscha.
I'll change that and merge the fix.
---
[GitHub] flink pull request #4690: [FLINK-7656] [runtime] Switch to user classloader ...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/4690#discussion_r140515705
--- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/jobgraph/JobTaskVertexTest.java ---
@@ -84,10 +88,10 @@ public void testConnectMultipleTargets() {
@Test
public void testOutputFormatVertex() {
try {
- final TestingOutputFormat outputFormat = new TestingOutputFormat();
+ final OutputFormat outputFormat = new TestingOutputFormat();
final OutputFormatVertex of = new OutputFormatVertex("Name");
new TaskConfig(of.getConfiguration()).setStubWrapper(new UserCodeObjectWrapper<OutputFormat<?>>(outputFormat));
- final ClassLoader cl = getClass().getClassLoader();
+ final ClassLoader cl = new FlinkUserCodeClassLoader(new URL[0], getClass().getClassLoader());
--- End diff --
This doesn't exist anymore but you can use `FlinkUserCodeClassLoaders.childFirst()`/`parentFirst()`.
---