You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zentol <gi...@git.apache.org> on 2018/03/28 11:01:38 UTC
[GitHub] flink pull request #5780: [FLINK-8704][tests] Port ClassLoaderITCase to flip...
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/5780
[FLINK-8704][tests] Port ClassLoaderITCase to flip6
Partially based on #5766.
## What is the purpose of the change
This PR ports the `ClassLoaderITCase` to flip6.
## Verifying this change
Run `ClassLoaderITCase` with flip6 enabled/disabled.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 8704_loader
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5780.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5780
----
----
---
[GitHub] flink issue #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip6
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5780
travis is still failing with mysterious error messages..
```
15:58:07.061 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (integration-tests) on project flink-tests_2.11: ExecutionException: java.lang.RuntimeException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
15:58:07.061 [ERROR] Command was /bin/sh -c cd /home/travis/build/apache/flink/flink-tests/target && /usr/lib/jvm/java-8-oracle/jre/bin/java -Xms256m -Xmx2048m -Dmvn.forkNumber=1 -XX:+UseG1GC -jar /home/travis/build/apache/flink/flink-tests/target/surefire/surefirebooter5397929458643043612.jar /home/travis/build/apache/flink/flink-tests/target/surefire/surefire4818404066082555175tmp /home/travis/build/apache/flink/flink-tests/target/surefire/surefire_1027420331622776884606tmp
```
---
[GitHub] flink issue #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip6
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5780
I may have found the case, we have to set `MANAGED_MEMORY_SIZE` for this test.
---
[GitHub] flink issue #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip6
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/5780
The `ClassLoaderITCase` fails on Travis. There are also checkstyle violations:
```
10:04:01.988 [ERROR] src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java:[62,15] (imports) UnusedImports: Unused import: org.junit.Assert.assertEquals.
10:04:01.988 [ERROR] src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java:[65,15] (imports) UnusedImports: Unused import: org.junit.Assert.fail.
```
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181533175
--- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java ---
@@ -196,21 +188,23 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
// the test also ensures that user specific exceptions are serializable between JobManager <--> JobClient.
PackagedProgram streamingCheckpointedProg = new PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE));
- TestStreamEnvironment.setAsContext(
- testCluster,
+ TestEnvironment.setAsContext(
+ MINI_CLUSTER_RESOURCE,
parallelism,
Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.<URL>emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.<Throwable>hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
- "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")))));
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
+ streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ } catch (Exception e) {
+ // Program should terminate with a 'SuccessException':
+ // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar.
+ Optional<Throwable> expectedCause = ExceptionUtils.findThrowable(e,
+ candidate -> candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"));
--- End diff --
Technically that shouldn't be possible. The exception class is part of the user-jar which is not on the classpath of the test.
---
[GitHub] flink issue #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip6
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5780
checkstyle 😡
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181466728
--- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java ---
@@ -117,6 +138,7 @@ public TestEnvironment getTestEnvironment() {
@Override
public void before() throws Exception {
+ miniClusterResourceConfiguration = Preconditions.checkNotNull(miniClusterResourceConfigurationSupplier.get());
--- End diff --
I think we are mixing concerns by letting the `MiniClusterResource` change its behaviour across multiple tests. I think we should not pass in a configuration supplier which can lazily instantiate a configuration. Instead if you need a different `MiniClusterResource` per test, then the test should instantiate the respective `MiniClusterResource` with the proper configuration.
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181759432
--- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java ---
@@ -202,15 +204,27 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.<URL>emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.<Throwable>hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
- "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")))));
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
--- End diff --
my guess is that the exception stack has changed. The SuccessException is no longer the direct cause of the top-level exception.
```
Expected: exception with cause hasProperty("cause", hasProperty("class", hasProperty("canonicalName", "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")))
but: cause property 'cause' property 'class' null
Stacktrace was: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:420)
at org.apache.flink.test.classloading.ClassLoaderITCase.testCheckpointedStreamingClassloaderJobWithCustomClassLoader(ClassLoaderITCase.java:212)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:239)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:108)
at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:78)
at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:54)
at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:144)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
Caused by: org.apache.flink.runtime.client.JobExecutionException: org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram$SuccessException
at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:621)
at org.apache.flink.streaming.util.TestStreamEnvironment.execute(TestStreamEnvironment.java:79)
at org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.main(CheckpointedStreamingProgram.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
... 42 more
Caused by: org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram$SuccessException
at org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram$StatefulMapper.map(CheckpointedStreamingProgram.java:116)
at org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram$StatefulMapper.map(CheckpointedStreamingProgram.java:84)
at org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:41)
at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:202)
at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:103)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:307)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
at java.lang.Thread.run(Thread.java:745)
```
---
[GitHub] flink issue #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip6
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5780
merging
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181534220
--- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java ---
@@ -49,15 +54,17 @@
* Starts a Flink mini cluster as a resource and registers the respective
* ExecutionEnvironment and StreamExecutionEnvironment.
*/
-public class MiniClusterResource extends ExternalResource {
+public class MiniClusterResource extends ExternalResource implements JobExecutor {
--- End diff --
otherwise we have to revert to using a `MiniCluster` directly, but then we lose the ´@Rule` benefits.
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181532978
--- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java ---
@@ -49,15 +54,17 @@
* Starts a Flink mini cluster as a resource and registers the respective
* ExecutionEnvironment and StreamExecutionEnvironment.
*/
-public class MiniClusterResource extends ExternalResource {
+public class MiniClusterResource extends ExternalResource implements JobExecutor {
--- End diff --
so that we can pass the MiniClusterResource to `TestEnvironment.setAsContext()`
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181467478
--- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java ---
@@ -196,21 +188,23 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
// the test also ensures that user specific exceptions are serializable between JobManager <--> JobClient.
PackagedProgram streamingCheckpointedProg = new PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE));
- TestStreamEnvironment.setAsContext(
- testCluster,
+ TestEnvironment.setAsContext(
+ MINI_CLUSTER_RESOURCE,
parallelism,
Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.<URL>emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.<Throwable>hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
- "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")))));
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
+ streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ } catch (Exception e) {
+ // Program should terminate with a 'SuccessException':
+ // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar.
+ Optional<Throwable> expectedCause = ExceptionUtils.findThrowable(e,
+ candidate -> candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"));
--- End diff --
Can't we use `findThrowable(Throwable throwable, Class<T> searchType)`?
---
[GitHub] flink issue #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip6
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/5780
Failing reproducibly, and only on this branch?
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181467950
--- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java ---
@@ -117,6 +138,7 @@ public TestEnvironment getTestEnvironment() {
@Override
public void before() throws Exception {
+ miniClusterResourceConfiguration = Preconditions.checkNotNull(miniClusterResourceConfigurationSupplier.get());
--- End diff --
The same applies to rule chains. An easy solution would be to create a `MiniClusterWithTemporaryFolderResource` which encapsulates this logic.
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181534097
--- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java ---
@@ -196,21 +188,23 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
// the test also ensures that user specific exceptions are serializable between JobManager <--> JobClient.
PackagedProgram streamingCheckpointedProg = new PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE));
- TestStreamEnvironment.setAsContext(
- testCluster,
+ TestEnvironment.setAsContext(
+ MINI_CLUSTER_RESOURCE,
parallelism,
Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.<URL>emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.<Throwable>hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
- "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")))));
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
+ streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ } catch (Exception e) {
+ // Program should terminate with a 'SuccessException':
+ // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar.
+ Optional<Throwable> expectedCause = ExceptionUtils.findThrowable(e,
+ candidate -> candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"));
--- End diff --
Note that this test is a bit funky at the moment; it should not be possible to access the exception at all since it shouldn't be deserializable. I had one version where the predicate version of `findThrowble` actually failed since it couldn't deserialize it, but this issue for some reason disappeared after a rebase.
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181533944
--- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java ---
@@ -117,6 +138,7 @@ public TestEnvironment getTestEnvironment() {
@Override
public void before() throws Exception {
+ miniClusterResourceConfiguration = Preconditions.checkNotNull(miniClusterResourceConfigurationSupplier.get());
--- End diff --
That's not "easy", it doesn't even solve the issue.
The contained `TemporaryFolder` would only be available after the construction of the `MiniClusterResource`, the folder is however necessary to create the configuration of the `MiniClusterResource`. Naturally we would like both of these to be `Rules`, but that's not possible unless the configuration is generated lazily, as i propose in the PR. Due to the `RuleChain` at the time that `MiniClusterResource#before` is called we can already access the `TemporaryFolder`.
The alternative is for the `TemporaryFolder` to be setup in a `@BeforeClass` method which is then used in the for the configuration. That would totally works, but it is really unfortunate as the setting of temporary paths is rather common and required for virtually all tests that use file-based state backends.
I agree that my proposal is quite hacky, but the question we have to ask is whether we prefer more complexity in the test vs the `MiniClusterResource`, where i would prefer the latter.
---
[GitHub] flink issue #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip6
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5780
I would be fine with changing the PR to work against a `MiniCluster` (to drop the JobExecutor changes) and a `TemporaryFolder` that we manage ourselves, i,e. not as a `@Rule` (to drop the icky configuration changes).
That's the quickest approach for now.
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181582943
--- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java ---
@@ -196,21 +188,23 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
// the test also ensures that user specific exceptions are serializable between JobManager <--> JobClient.
PackagedProgram streamingCheckpointedProg = new PackagedProgram(new File(STREAMING_CHECKPOINTED_PROG_JAR_FILE));
- TestStreamEnvironment.setAsContext(
- testCluster,
+ TestEnvironment.setAsContext(
+ MINI_CLUSTER_RESOURCE,
parallelism,
Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.<URL>emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.<Throwable>hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
- "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")))));
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
+ streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ } catch (Exception e) {
+ // Program should terminate with a 'SuccessException':
+ // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar.
+ Optional<Throwable> expectedCause = ExceptionUtils.findThrowable(e,
+ candidate -> candidate.getClass().getCanonicalName().equals("org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException"));
--- End diff --
The test failed since the `TestEnvironment` was set as the context, and not the `TestStreamEnvironment` as intended.
---
[GitHub] flink issue #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip6
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5780
I've cut down the PR to only not make any changes outside the tests, fixed the checkstyle violations and the failing test.
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/5780
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181746661
--- Diff: flink-tests/src/test/java/org/apache/flink/test/classloading/ClassLoaderITCase.java ---
@@ -202,15 +204,27 @@ public void testCheckpointedStreamingClassloaderJobWithCustomClassLoader() throw
Collections.singleton(new Path(STREAMING_CHECKPOINTED_PROG_JAR_FILE)),
Collections.<URL>emptyList());
- // Program should terminate with a 'SuccessException':
- // we can not access the SuccessException here when executing the tests with maven, because its not available in the jar.
- expectedException.expectCause(
- Matchers.<Throwable>hasProperty("cause",
- hasProperty("class",
- hasProperty("canonicalName", equalTo(
- "org.apache.flink.test.classloading.jar.CheckpointedStreamingProgram.SuccessException")))));
-
- streamingCheckpointedProg.invokeInteractiveModeForExecution();
+ try {
--- End diff --
Why doesn't the old code work here anymore?
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181465966
--- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java ---
@@ -225,6 +247,11 @@ private void startMiniCluster() throws Exception {
}
}
+ @Override
+ public JobExecutionResult executeJobBlocking(JobGraph jobGraph) throws JobExecutionException, InterruptedException {
--- End diff --
I think we should use the `MiniClusterClient` for job submission. The `JobExecutor` interface is only an internal interface to bridge between the two different mini clusters.
---
[GitHub] flink pull request #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5780#discussion_r181468061
--- Diff: flink-test-utils-parent/flink-test-utils/src/main/java/org/apache/flink/test/util/MiniClusterResource.java ---
@@ -49,15 +54,17 @@
* Starts a Flink mini cluster as a resource and registers the respective
* ExecutionEnvironment and StreamExecutionEnvironment.
*/
-public class MiniClusterResource extends ExternalResource {
+public class MiniClusterResource extends ExternalResource implements JobExecutor {
--- End diff --
Why do we need to implement `JobExecutor`?
---
[GitHub] flink issue #5780: [FLINK-8955][tests] Port ClassLoaderITCase to flip6
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:
https://github.com/apache/flink/pull/5780
still failures because of this?
---