You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by fhueske <gi...@git.apache.org> on 2017/09/20 17:02:03 UTC

[GitHub] flink pull request #4690: [FLINK-7656] [runtime] Switch to user classloader ...

GitHub user fhueske opened a pull request:

    https://github.com/apache/flink/pull/4690

    [FLINK-7656] [runtime] Switch to user classloader before calling initializeOnMaster and finalizeOnMaster.

    ## What is the purpose of the change
    
    Fixes a classloading issue for `OutputFormat`s that implement `InitializeOnMaster` or `FinalizeOnMaster`. The initialize and finalize methods are called without setting the context classloader to the usercode classloader.
    
    ## Brief change log
    
    * Set the context classloader to the usercode classloader before calling `InitializeOnMaster.initializeGlobal()` and `FinalizeOnMaster.finalizeGlobal()` and restore the original classloader afterwards.
    * Extend test to check that the correct classloader is set.
    
    ## Verifying this change
    
    * Run `JobTaskVertexTest.testOutputFormatVertex()`
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): **no**
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no**
      - The serializers: **no**
      - The runtime per-record code paths (performance sensitive): **no**
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: **YES**
    
    ## Documentation
    
      - Does this pull request introduce a new feature? **no**
      - If yes, how is the feature documented? **n/a**
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/fhueske/flink outputOnMasterFix

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/4690.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4690
    
----
commit 2dfc8a4550981b9b8c417585dfb5e02b3282c77a
Author: Fabian Hueske <fh...@apache.org>
Date:   2017-09-20T14:26:47Z

    [FLINK-7656] [runtime] Switch to user classloader before calling initializeOnMaster and finalizeOnMaster.

----


---

[GitHub] flink pull request #4690: [FLINK-7656] [runtime] Switch to user classloader ...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/flink/pull/4690


---

[GitHub] flink issue #4690: [FLINK-7656] [runtime] Switch to user classloader before ...

Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the issue:

    https://github.com/apache/flink/pull/4690
  
    Can you take a brief look at this change @aljoscha? Thanks!


---

[GitHub] flink issue #4690: [FLINK-7656] [runtime] Switch to user classloader before ...

Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the issue:

    https://github.com/apache/flink/pull/4690
  
    Thanks for the review @aljoscha.
    I'll change that and merge the fix.


---

[GitHub] flink pull request #4690: [FLINK-7656] [runtime] Switch to user classloader ...

Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4690#discussion_r140515705
  
    --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/jobgraph/JobTaskVertexTest.java ---
    @@ -84,10 +88,10 @@ public void testConnectMultipleTargets() {
     	@Test
     	public void testOutputFormatVertex() {
     		try {
    -			final TestingOutputFormat outputFormat = new TestingOutputFormat();
    +			final OutputFormat outputFormat = new TestingOutputFormat();
     			final OutputFormatVertex of = new OutputFormatVertex("Name");
     			new TaskConfig(of.getConfiguration()).setStubWrapper(new UserCodeObjectWrapper<OutputFormat<?>>(outputFormat));
    -			final ClassLoader cl = getClass().getClassLoader();
    +			final ClassLoader cl = new FlinkUserCodeClassLoader(new URL[0], getClass().getClassLoader());
    --- End diff --
    
    This doesn't exist anymore but you can use `FlinkUserCodeClassLoaders.childFirst()`/`parentFirst()`.


---