You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by ch...@apache.org on 2018/02/21 17:20:02 UTC

[1/5] flink git commit: [hotfix] Fix various typos

Repository: flink
Updated Branches:
  refs/heads/master 2fa8a1586 -> b17610e1c


[hotfix] Fix various typos

This closes #5497.


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/b17610e1
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/b17610e1
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/b17610e1

Branch: refs/heads/master
Commit: b17610e1c4d7a299f73b6f4cc21856bac61c1fa4
Parents: 3d22f7c
Author: John Eismeier <jo...@gmail.com>
Authored: Thu Feb 15 13:54:03 2018 -0500
Committer: zentol <ch...@apache.org>
Committed: Wed Feb 21 18:19:38 2018 +0100

----------------------------------------------------------------------
 README.md                                       |  5 ++--
 docs/dev/migration.md                           | 26 ++++++++++----------
 docs/quickstart/run_example_quickstart.md       |  4 +--
 docs/redirects/storm_compat.md                  |  2 +-
 flink-contrib/flink-storm-examples/README.md    |  6 ++---
 flink-filesystems/flink-s3-fs-hadoop/README.md  |  2 +-
 flink-filesystems/flink-s3-fs-presto/README.md  |  2 +-
 .../flink-swift-fs-hadoop/README.md             |  2 +-
 .../calcite/FlinkCalciteSqlValidator.scala      |  2 +-
 .../jobs/job.plan.node.checkpoints.details.jade |  2 +-
 .../jobs/job.plan.node.checkpoints.details.html |  5 ++--
 .../flink/runtime/taskmanager/TaskManager.scala |  2 +-
 tools/merge_flink_pr.py                         |  2 +-
 13 files changed, 30 insertions(+), 32 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index f6bf2ba..1e00e65 100644
--- a/README.md
+++ b/README.md
@@ -15,7 +15,7 @@ Learn more about Flink at [http://flink.apache.org/](http://flink.apache.org/)
 
 * Support for *event time* and *out-of-order* processing in the DataStream API, based on the *Dataflow Model*
 
-* Flexible windowing (time, count, sessions, custom triggers) accross different time semantics (event time, processing time)
+* Flexible windowing (time, count, sessions, custom triggers) across different time semantics (event time, processing time)
 
 * Fault-tolerance with *exactly-once* processing guarantees
 
@@ -127,7 +127,7 @@ or in the `docs/` directory of the source code.
 
 ## Fork and Contribute
 
-This is an active open-source project. We are always open to people who want to use the system or contribute to it. 
+This is an active open-source project. We are always open to people who want to use the system or contribute to it.
 Contact us if you are looking for implementation tasks that fit your skills.
 This article describes [how to contribute to Apache Flink](http://flink.apache.org/how-to-contribute.html).
 
@@ -136,4 +136,3 @@ This article describes [how to contribute to Apache Flink](http://flink.apache.o
 
 Apache Flink is an open source project of The Apache Software Foundation (ASF).
 The Apache Flink project originated from the [Stratosphere](http://stratosphere.eu) research project.
-

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/docs/dev/migration.md
----------------------------------------------------------------------
diff --git a/docs/dev/migration.md b/docs/dev/migration.md
index dea8b7a..b9ed8fb 100644
--- a/docs/dev/migration.md
+++ b/docs/dev/migration.md
@@ -55,8 +55,8 @@ Please visit the [CEP Migration docs]({{ site.baseurl }}/dev/libs/cep.html#migra
 
 In Flink 1.3, to make sure that users can use their own custom logging framework, core Flink artifacts are
 now clean of specific logger dependencies.
- 
-Example and quickstart archtypes already have loggers specified and should not be affected.
+
+Example and quickstart archetypes already have loggers specified and should not be affected.
 For other custom projects, make sure to add logger dependencies. For example, in Maven's `pom.xml`, you can add:
 
 ~~~xml
@@ -145,16 +145,16 @@ public class BufferingSink implements SinkFunction<Tuple2<String, Integer>>,
         bufferedElements.add(value);
         if (bufferedElements.size() == threshold) {
             for (Tuple2<String, Integer> element: bufferedElements) {
-	        // send it to the sink
-	    }
-	    bufferedElements.clear();
-	}
+                // send it to the sink
+            }
+            bufferedElements.clear();
+        }
     }
 
     @Override
     public ArrayList<Tuple2<String, Integer>> snapshotState(
         long checkpointId, long checkpointTimestamp) throws Exception {
-	    return bufferedElements;
+        return bufferedElements;
     }
 
     @Override
@@ -445,15 +445,15 @@ The code to use the aligned window operators in Flink 1.2 is presented below:
 
 // for tumbling windows
 DataStream<Tuple2<String, Integer>> window1 = source
-	.keyBy(0)
-	.window(TumblingAlignedProcessingTimeWindows.of(Time.of(1000, TimeUnit.MILLISECONDS)))
-	.apply(your-function)
+    .keyBy(0)
+    .window(TumblingAlignedProcessingTimeWindows.of(Time.of(1000, TimeUnit.MILLISECONDS)))
+    .apply(your-function)
 
 // for sliding windows
 DataStream<Tuple2<String, Integer>> window1 = source
-	.keyBy(0)
-	.window(SlidingAlignedProcessingTimeWindows.of(Time.seconds(1), Time.milliseconds(100)))
-	.apply(your-function)
+    .keyBy(0)
+    .window(SlidingAlignedProcessingTimeWindows.of(Time.seconds(1), Time.milliseconds(100)))
+    .apply(your-function)
 
 {% endhighlight %}
 </div>

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/docs/quickstart/run_example_quickstart.md
----------------------------------------------------------------------
diff --git a/docs/quickstart/run_example_quickstart.md b/docs/quickstart/run_example_quickstart.md
index d5c48c9..4ad8994 100644
--- a/docs/quickstart/run_example_quickstart.md
+++ b/docs/quickstart/run_example_quickstart.md
@@ -36,7 +36,7 @@ give you a good foundation from which to start building more complex analysis pr
 
 ## Setting up a Maven Project
 
-We are going to use a Flink Maven Archetype for creating our project stucture. Please
+We are going to use a Flink Maven Archetype for creating our project structure. Please
 see [Java API Quickstart]({{ site.baseurl }}/quickstart/java_api_quickstart.html) for more details
 about this. For our purposes, the command to run is this:
 
@@ -284,7 +284,7 @@ similar to this:
 The number in front of each line tells you on which parallel instance of the print sink the output
 was produced.
 
-This should get you started with writing your own Flink programs. To learn more 
+This should get you started with writing your own Flink programs. To learn more
 you can check out our guides
 about [basic concepts]({{ site.baseurl }}/dev/api_concepts.html) and the
 [DataStream API]({{ site.baseurl }}/dev/datastream_api.html). Stick

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/docs/redirects/storm_compat.md
----------------------------------------------------------------------
diff --git a/docs/redirects/storm_compat.md b/docs/redirects/storm_compat.md
index adad140..f2df117 100644
--- a/docs/redirects/storm_compat.md
+++ b/docs/redirects/storm_compat.md
@@ -1,5 +1,5 @@
 ---
-title: "Storm Compatability"
+title: "Storm Compatibility"
 layout: redirect
 redirect: /dev/libs/storm_compatibility.html
 permalink: /apis/streaming/storm_compatibility.html

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/flink-contrib/flink-storm-examples/README.md
----------------------------------------------------------------------
diff --git a/flink-contrib/flink-storm-examples/README.md b/flink-contrib/flink-storm-examples/README.md
index c3247f6..00d7906 100644
--- a/flink-contrib/flink-storm-examples/README.md
+++ b/flink-contrib/flink-storm-examples/README.md
@@ -9,10 +9,10 @@ This module contains multiple versions of a simple Word-Count example to illustr
 
 * how to submit a whole Storm topology to Flink
    3. `WordCountTopology` plugs a Storm topology together
-      * `StormWordCountLocal` submits the topology to a local Flink cluster (similiar to a `LocalCluster` in Storm)
+      * `StormWordCountLocal` submits the topology to a local Flink cluster (similar to a `LocalCluster` in Storm)
         (`WordCountLocalByName` accesses attributes by field names rather than index)
-      * `WordCountRemoteByClient` submits the topology to a remote Flink cluster (simliar to the usage of `NimbusClient` in Storm)
-      * `WordCountRemoteBySubmitter` submits the topology to a remote Flink cluster (simliar to the usage of `StormSubmitter` in Storm)
+      * `WordCountRemoteByClient` submits the topology to a remote Flink cluster (similar to the usage of `NimbusClient` in Storm)
+      * `WordCountRemoteBySubmitter` submits the topology to a remote Flink cluster (similar to the usage of `StormSubmitter` in Storm)
 
 Additionally, this module package the three example Word-Count programs as jar files to be submitted to a Flink cluster via `bin/flink run example.jar`.
 (Valid jars are `WordCount-SpoutSource.jar`, `WordCount-BoltTokenizer.jar`, and `WordCount-StormTopology.jar`)

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/flink-filesystems/flink-s3-fs-hadoop/README.md
----------------------------------------------------------------------
diff --git a/flink-filesystems/flink-s3-fs-hadoop/README.md b/flink-filesystems/flink-s3-fs-hadoop/README.md
index ca8fdab..0d61ef3 100644
--- a/flink-filesystems/flink-s3-fs-hadoop/README.md
+++ b/flink-filesystems/flink-s3-fs-hadoop/README.md
@@ -25,7 +25,7 @@ steps are required to keep the shading correct:
   }
 ```
   - copy `core-default.xml` to `src/main/resources/core-default-shaded.xml` and
-    - change every occurence of `org.apache.hadoop` into `org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop`
+    - change every occurrence of `org.apache.hadoop` into `org.apache.flink.fs.s3hadoop.shaded.org.apache.hadoop`
   - copy `core-site.xml` to `src/test/resources/core-site.xml` (as is)
 2. verify the shaded jar:
   - does not contain any unshaded classes except for `org.apache.flink.fs.s3hadoop.S3FileSystemFactory`

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/flink-filesystems/flink-s3-fs-presto/README.md
----------------------------------------------------------------------
diff --git a/flink-filesystems/flink-s3-fs-presto/README.md b/flink-filesystems/flink-s3-fs-presto/README.md
index cae39c9..9a71abf 100644
--- a/flink-filesystems/flink-s3-fs-presto/README.md
+++ b/flink-filesystems/flink-s3-fs-presto/README.md
@@ -26,7 +26,7 @@ steps are required to keep the shading correct:
   }
 ```
   - copy `core-default.xml` to `src/main/resources/core-default-shaded.xml` and
-    - change every occurence of `org.apache.hadoop` into `org.apache.flink.fs.s3presto.shaded.org.apache.hadoop`
+    - change every occurrence of `org.apache.hadoop` into `org.apache.flink.fs.s3presto.shaded.org.apache.hadoop`
   - copy `core-site.xml` to `src/test/resources/core-site.xml` (as is)
 2. verify the shaded jar:
   - does not contain any unshaded classes except for `org.apache.flink.fs.s3presto.S3FileSystemFactory`

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/flink-filesystems/flink-swift-fs-hadoop/README.md
----------------------------------------------------------------------
diff --git a/flink-filesystems/flink-swift-fs-hadoop/README.md b/flink-filesystems/flink-swift-fs-hadoop/README.md
index 0b6d091..e78f630 100644
--- a/flink-filesystems/flink-swift-fs-hadoop/README.md
+++ b/flink-filesystems/flink-swift-fs-hadoop/README.md
@@ -25,7 +25,7 @@ steps are required to keep the shading correct:
     }
     ```
   - copy `core-default.xml` to `src/main/resources/core-default-shaded.xml` and
-    - change every occurence of `org.apache.hadoop` into `org.apache.flink.fs.openstackhadoop.shaded.org.apache.hadoop`
+    - change every occurrence of `org.apache.hadoop` into `org.apache.flink.fs.openstackhadoop.shaded.org.apache.hadoop`
   - copy `core-site.xml` to `src/test/resources/core-site.xml` (as is)
 2. verify the shaded jar:
   - does not contain any unshaded classes except for `org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem`

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/calcite/FlinkCalciteSqlValidator.scala
----------------------------------------------------------------------
diff --git a/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/calcite/FlinkCalciteSqlValidator.scala b/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/calcite/FlinkCalciteSqlValidator.scala
index 137a199..b15733d 100644
--- a/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/calcite/FlinkCalciteSqlValidator.scala
+++ b/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/calcite/FlinkCalciteSqlValidator.scala
@@ -60,7 +60,7 @@ class FlinkCalciteSqlValidator(
           // We accept only literal true
         case c if null != c =>
           throw new ValidationException(
-            s"Left outer joins with a table function do not accept a predicte such as $c. " +
+            s"Left outer joins with a table function do not accept a predicate such as $c. " +
               s"Only literal TRUE is accepted.")
       }
     }

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/flink-runtime-web/web-dashboard/app/partials/jobs/job.plan.node.checkpoints.details.jade
----------------------------------------------------------------------
diff --git a/flink-runtime-web/web-dashboard/app/partials/jobs/job.plan.node.checkpoints.details.jade b/flink-runtime-web/web-dashboard/app/partials/jobs/job.plan.node.checkpoints.details.jade
index 401a426..d0055de 100644
--- a/flink-runtime-web/web-dashboard/app/partials/jobs/job.plan.node.checkpoints.details.jade
+++ b/flink-runtime-web/web-dashboard/app/partials/jobs/job.plan.node.checkpoints.details.jade
@@ -56,7 +56,7 @@ div(ng-if="checkpoint")
     thead
       tr
         td #[strong Name]
-        td #[strong Acknowleged]
+        td #[strong Acknowledged]
         td #[strong Latest Acknowledgment]
         td #[strong End to End Duration]
         td #[strong State Size]

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/flink-runtime-web/web-dashboard/web/partials/jobs/job.plan.node.checkpoints.details.html
----------------------------------------------------------------------
diff --git a/flink-runtime-web/web-dashboard/web/partials/jobs/job.plan.node.checkpoints.details.html b/flink-runtime-web/web-dashboard/web/partials/jobs/job.plan.node.checkpoints.details.html
index d3b1613..3b7768a 100644
--- a/flink-runtime-web/web-dashboard/web/partials/jobs/job.plan.node.checkpoints.details.html
+++ b/flink-runtime-web/web-dashboard/web/partials/jobs/job.plan.node.checkpoints.details.html
@@ -1,4 +1,3 @@
-
 <!--
 Licensed to the Apache Software Foundation (ASF) under one
 or more contributor license agreements.  See the NOTICE file
@@ -62,7 +61,7 @@ limitations under the License.
     <thead>
       <tr>
         <td><strong>Name</strong></td>
-        <td><strong>Acknowleged</strong></td>
+        <td><strong>Acknowledged</strong></td>
         <td><strong>Latest Acknowledgment</strong></td>
         <td><strong>End to End Duration</strong></td>
         <td><strong>State Size</strong></td>
@@ -168,4 +167,4 @@ limitations under the License.
   <p ng-if="unknown_checkpoint" role="alert" class="alert alert-danger"><strong>Unknown or expired checkpoint ID.</strong></p>
   <p ng-if="!unknown_checkpoint" role="alert" class="alert alert-info"><strong>Waiting for response from JobManager with checkpoint details...</strong> <i aria-hidden="true" class="fa fa-circle-o-notch fa-spin fa-fw"></i>
   </p>
-</div>
\ No newline at end of file
+</div>

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala
----------------------------------------------------------------------
diff --git a/flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala b/flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala
index 54549ef..106bea1 100644
--- a/flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala
+++ b/flink-runtime/src/main/scala/org/apache/flink/runtime/taskmanager/TaskManager.scala
@@ -556,7 +556,7 @@ class TaskManager(
               "TaskManager was triggered to register at JobManager, but is already registered")
           } else if (deadline.exists(_.isOverdue())) {
             // we failed to register in time. that means we should quit
-            log.error("Failed to register at the JobManager withing the defined maximum " +
+            log.error("Failed to register at the JobManager within the defined maximum " +
                         "connect time. Shutting down ...")
 
             // terminate ourselves (hasta la vista)

http://git-wip-us.apache.org/repos/asf/flink/blob/b17610e1/tools/merge_flink_pr.py
----------------------------------------------------------------------
diff --git a/tools/merge_flink_pr.py b/tools/merge_flink_pr.py
index 1799a55..4a6f416 100755
--- a/tools/merge_flink_pr.py
+++ b/tools/merge_flink_pr.py
@@ -274,7 +274,7 @@ def resolve_jira(title, merge_branches, comment):
     asf_jira.transition_issue(
         jira_id, resolve["id"], fixVersions=jira_fix_versions, comment=comment)
 
-    print "Succesfully resolved %s with fixVersions=%s!" % (jira_id, fix_versions)
+    print "Successfully resolved %s with fixVersions=%s!" % (jira_id, fix_versions)
 
 
 #branches = get_json("%s/branches" % GITHUB_API_BASE)


[2/5] flink git commit: [hotfix][table][tests] Set @Ignore description for RowCsvInputFormatTest#testParserCorrectness

Posted by ch...@apache.org.
[hotfix][table][tests] Set @Ignore description for RowCsvInputFormatTest#testParserCorrectness

This closes #5413.


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/99c25566
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/99c25566
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/99c25566

Branch: refs/heads/master
Commit: 99c255663c6536f8122964a8eba760af47d76678
Parents: 7410491
Author: zentol <ch...@apache.org>
Authored: Mon Feb 5 16:01:48 2018 +0100
Committer: zentol <ch...@apache.org>
Committed: Wed Feb 21 18:19:38 2018 +0100

----------------------------------------------------------------------
 .../java/org/apache/flink/api/java/io/RowCsvInputFormatTest.java  | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/flink/blob/99c25566/flink-java/src/test/java/org/apache/flink/api/java/io/RowCsvInputFormatTest.java
----------------------------------------------------------------------
diff --git a/flink-java/src/test/java/org/apache/flink/api/java/io/RowCsvInputFormatTest.java b/flink-java/src/test/java/org/apache/flink/api/java/io/RowCsvInputFormatTest.java
index f44d5bf..666b1df 100644
--- a/flink-java/src/test/java/org/apache/flink/api/java/io/RowCsvInputFormatTest.java
+++ b/flink-java/src/test/java/org/apache/flink/api/java/io/RowCsvInputFormatTest.java
@@ -715,9 +715,8 @@ public class RowCsvInputFormatTest {
 		}
 	}
 
-	// Test disabled because we do not support double-quote escaped quotes right now.
 	@Test
-	@Ignore
+	@Ignore("Test disabled because we do not support double-quote escaped quotes right now.")
 	public void testParserCorrectness() throws Exception {
 		// RFC 4180 Compliance Test content
 		// Taken from http://en.wikipedia.org/wiki/Comma-separated_values#Example


[5/5] flink git commit: [FLINK-8713][tests] Ensure outdated mod time is smaller than minimum

Posted by ch...@apache.org.
[FLINK-8713][tests] Ensure outdated mod time is smaller than minimum

This closes #5534.


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/7410491b
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/7410491b
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/7410491b

Branch: refs/heads/master
Commit: 7410491bd68eed2deec0e334bd811471621ff1ae
Parents: cb08aca
Author: zentol <ch...@apache.org>
Authored: Tue Feb 20 13:16:30 2018 +0100
Committer: zentol <ch...@apache.org>
Committed: Wed Feb 21 18:19:38 2018 +0100

----------------------------------------------------------------------
 .../api/common/io/FileInputFormatTest.java      | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/flink/blob/7410491b/flink-core/src/test/java/org/apache/flink/api/common/io/FileInputFormatTest.java
----------------------------------------------------------------------
diff --git a/flink-core/src/test/java/org/apache/flink/api/common/io/FileInputFormatTest.java b/flink-core/src/test/java/org/apache/flink/api/common/io/FileInputFormatTest.java
index 9cbe215..b464a43 100644
--- a/flink-core/src/test/java/org/apache/flink/api/common/io/FileInputFormatTest.java
+++ b/flink-core/src/test/java/org/apache/flink/api/common/io/FileInputFormatTest.java
@@ -24,6 +24,7 @@ import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
 import org.apache.flink.core.fs.FSDataInputStream;
 import org.apache.flink.core.fs.FileInputSplit;
+import org.apache.flink.core.fs.FileSystem;
 import org.apache.flink.core.fs.Path;
 import org.apache.flink.testutils.TestFileUtils;
 import org.apache.flink.types.IntValue;
@@ -362,13 +363,22 @@ public class FileInputFormatTest {
 	@Test
 	public void testGetStatisticsMultipleFilesWithCachedVersion() {
 		try {
+			FileSystem fs = FileSystem.getLocalFileSystem();
+
 			final long SIZE1 = 2077;
 			final long SIZE2 = 31909;
 			final long SIZE3 = 10;
 			final long TOTAL = SIZE1 + SIZE2 + SIZE3;
 			final long FAKE_SIZE = 10065;
 			
-			String tempDir = TestFileUtils.createTempFileDir(SIZE1, SIZE2, SIZE3);
+			File tempDirFile = temporaryFolder.newFolder();
+			String tempDir = tempDirFile.getAbsolutePath();
+			String f1 = TestFileUtils.createTempFileInDirectory(tempDir, SIZE1);
+			long modTime1 = fs.getFileStatus(new Path(f1)).getModificationTime();
+			String f2 = TestFileUtils.createTempFileInDirectory(tempDir, SIZE2);
+			long modTime2 = fs.getFileStatus(new Path(f2)).getModificationTime();
+			String f3 = TestFileUtils.createTempFileInDirectory(tempDir, SIZE3);
+			long modTime3 = fs.getFileStatus(new Path(f3)).getModificationTime();
 			
 			DummyFileInputFormat format = new DummyFileInputFormat();
 			format.setFilePath(tempDir);
@@ -398,7 +408,7 @@ public class FileInputFormatTest {
 			format.setFilePath(tempDir);
 			format.configure(new Configuration());
 			
-			FileBaseStatistics outDatedFakeStats = new FileBaseStatistics(stats.getLastModificationTime() - 1, FAKE_SIZE, BaseStatistics.AVG_RECORD_BYTES_UNKNOWN);
+			FileBaseStatistics outDatedFakeStats = new FileBaseStatistics(Math.min(Math.min(modTime1, modTime2), modTime3) - 1, FAKE_SIZE, BaseStatistics.AVG_RECORD_BYTES_UNKNOWN);
 			BaseStatistics reGathered = format.getStatistics(outDatedFakeStats);
 			Assert.assertEquals("The file size from the statistics is wrong.", TOTAL, reGathered.getTotalInputSize());
 			
@@ -463,12 +473,16 @@ public class FileInputFormatTest {
 	
 	@Test
 	public void testGetStatisticsMultipleOneFileWithCachedVersion() throws IOException {
+		FileSystem fs = FileSystem.getLocalFileSystem();
+
 		final long size1 = 50873;
 		final long fakeSize = 10065;
 		String tempFile1 = TestFileUtils.createTempFile(size1);
+		final long lastModTime1 = fs.getFileStatus(new Path(tempFile1)).getModificationTime();
 
 		final long size2 = 52573;
 		String tempFile2 = TestFileUtils.createTempFile(size2);
+		final long lastModTime2 = fs.getFileStatus(new Path(tempFile2)).getModificationTime();
 
 		final long sizeTotal = size1 + size2;
 		
@@ -500,7 +514,7 @@ public class FileInputFormatTest {
 		format.setFilePaths(tempFile1, tempFile2);
 		format.configure(new Configuration());
 		
-		FileBaseStatistics outDatedFakeStats = new FileBaseStatistics(stats.getLastModificationTime() - 1, fakeSize, BaseStatistics.AVG_RECORD_BYTES_UNKNOWN);
+		FileBaseStatistics outDatedFakeStats = new FileBaseStatistics(Math.min(lastModTime1, lastModTime2) - 1, fakeSize, BaseStatistics.AVG_RECORD_BYTES_UNKNOWN);
 		BaseStatistics reGathered = format.getStatistics(outDatedFakeStats);
 		Assert.assertEquals("The file size from the statistics is wrong.", sizeTotal, reGathered.getTotalInputSize());
 	}


[3/5] flink git commit: [FLINK-8703][tests] Migrate tests to MiniClusterResource (batch #1)

Posted by ch...@apache.org.
[FLINK-8703][tests] Migrate tests to MiniClusterResource (batch #1)

This closes #5535.


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/cb08aca3
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/cb08aca3
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/cb08aca3

Branch: refs/heads/master
Commit: cb08aca3585bde903bc879b2009c7abdcd225019
Parents: 2fa8a15
Author: zentol <ch...@apache.org>
Authored: Tue Feb 20 13:41:45 2018 +0100
Committer: zentol <ch...@apache.org>
Committed: Wed Feb 21 18:19:38 2018 +0100

----------------------------------------------------------------------
 .../operations/DegreesWithExceptionITCase.java  | 29 +-----------
 .../ReduceOnEdgesWithExceptionITCase.java       | 29 +-----------
 .../ReduceOnNeighborsWithExceptionITCase.java   | 29 +-----------
 .../EventTimeAllWindowCheckpointingITCase.java  | 39 +++++++---------
 .../KeyedStateCheckpointingITCase.java          | 32 +++++--------
 .../StreamFaultToleranceTestBase.java           | 49 ++++----------------
 .../WindowCheckpointingITCase.java              | 39 ++++++----------
 .../flink/test/misc/AutoParallelismITCase.java  | 42 ++++-------------
 .../test/misc/CustomSerializationITCase.java    | 36 ++++++--------
 .../test/misc/MiscellaneousIssuesITCase.java    | 42 +++++------------
 ...SuccessAfterNetworkBuffersFailureITCase.java | 36 +++++++-------
 .../flink/test/recovery/FastFailuresITCase.java | 19 ++------
 ...SimpleRecoveryFailureRateStrategyITBase.java | 21 +++++----
 ...RecoveryFixedDelayRestartStrategyITBase.java | 21 +++++----
 .../test/recovery/SimpleRecoveryITCaseBase.java | 33 +++----------
 .../streaming/experimental/CollectITCase.java   | 39 ++++++----------
 16 files changed, 159 insertions(+), 376 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/DegreesWithExceptionITCase.java
----------------------------------------------------------------------
diff --git a/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/DegreesWithExceptionITCase.java b/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/DegreesWithExceptionITCase.java
index f01daec..717233a 100644
--- a/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/DegreesWithExceptionITCase.java
+++ b/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/DegreesWithExceptionITCase.java
@@ -20,16 +20,10 @@ package org.apache.flink.graph.test.operations;
 
 import org.apache.flink.api.java.ExecutionEnvironment;
 import org.apache.flink.api.java.io.DiscardingOutputFormat;
-import org.apache.flink.configuration.ConfigConstants;
-import org.apache.flink.configuration.Configuration;
 import org.apache.flink.graph.Graph;
 import org.apache.flink.graph.test.TestGraphUtils;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
-import org.apache.flink.test.util.TestEnvironment;
-import org.apache.flink.util.TestLogger;
+import org.apache.flink.test.util.AbstractTestBase;
 
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
 import org.junit.Test;
 
 import static org.junit.Assert.fail;
@@ -38,29 +32,10 @@ import static org.junit.Assert.fail;
  * Test expected errors for {@link Graph#inDegrees()},
  * {@link Graph#outDegrees()}, and {@link Graph#getDegrees()}.
  */
-public class DegreesWithExceptionITCase extends TestLogger {
+public class DegreesWithExceptionITCase extends AbstractTestBase {
 
 	private static final int PARALLELISM = 4;
 
-	private static LocalFlinkMiniCluster cluster;
-
-	@BeforeClass
-	public static void setupCluster() {
-		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, PARALLELISM);
-		cluster = new LocalFlinkMiniCluster(config, false);
-		cluster.start();
-
-		TestEnvironment.setAsContext(cluster, PARALLELISM);
-	}
-
-	@AfterClass
-	public static void tearDownCluster() {
-		cluster.stop();
-
-		TestEnvironment.unsetAsContext();
-	}
-
 	/**
 	 * Test outDegrees() with an edge having a srcId that does not exist in the vertex DataSet.
 	 */

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/ReduceOnEdgesWithExceptionITCase.java
----------------------------------------------------------------------
diff --git a/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/ReduceOnEdgesWithExceptionITCase.java b/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/ReduceOnEdgesWithExceptionITCase.java
index 6733ba1..f383f02 100644
--- a/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/ReduceOnEdgesWithExceptionITCase.java
+++ b/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/ReduceOnEdgesWithExceptionITCase.java
@@ -22,21 +22,15 @@ import org.apache.flink.api.java.DataSet;
 import org.apache.flink.api.java.ExecutionEnvironment;
 import org.apache.flink.api.java.io.DiscardingOutputFormat;
 import org.apache.flink.api.java.tuple.Tuple2;
-import org.apache.flink.configuration.ConfigConstants;
-import org.apache.flink.configuration.Configuration;
 import org.apache.flink.graph.Edge;
 import org.apache.flink.graph.EdgeDirection;
 import org.apache.flink.graph.EdgesFunctionWithVertexValue;
 import org.apache.flink.graph.Graph;
 import org.apache.flink.graph.Vertex;
 import org.apache.flink.graph.test.TestGraphUtils;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
-import org.apache.flink.test.util.TestEnvironment;
+import org.apache.flink.test.util.AbstractTestBase;
 import org.apache.flink.util.Collector;
-import org.apache.flink.util.TestLogger;
 
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
 import org.junit.Test;
 
 import static org.junit.Assert.fail;
@@ -44,29 +38,10 @@ import static org.junit.Assert.fail;
 /**
  * Test expected exceptions for {@link Graph#groupReduceOnEdges}.
  */
-public class ReduceOnEdgesWithExceptionITCase extends TestLogger {
+public class ReduceOnEdgesWithExceptionITCase extends AbstractTestBase {
 
 	private static final int PARALLELISM = 4;
 
-	private static LocalFlinkMiniCluster cluster;
-
-	@BeforeClass
-	public static void setupCluster() {
-		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, PARALLELISM);
-		cluster = new LocalFlinkMiniCluster(config, false);
-		cluster.start();
-
-		TestEnvironment.setAsContext(cluster, PARALLELISM);
-	}
-
-	@AfterClass
-	public static void tearDownCluster() {
-		cluster.stop();
-
-		TestEnvironment.unsetAsContext();
-	}
-
 	/**
 	 * Test groupReduceOnEdges() with an edge having a srcId that does not exist in the vertex DataSet.
 	 */

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/ReduceOnNeighborsWithExceptionITCase.java
----------------------------------------------------------------------
diff --git a/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/ReduceOnNeighborsWithExceptionITCase.java b/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/ReduceOnNeighborsWithExceptionITCase.java
index 1a60874..b84831b 100644
--- a/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/ReduceOnNeighborsWithExceptionITCase.java
+++ b/flink-libraries/flink-gelly/src/test/java/org/apache/flink/graph/test/operations/ReduceOnNeighborsWithExceptionITCase.java
@@ -22,8 +22,6 @@ import org.apache.flink.api.java.DataSet;
 import org.apache.flink.api.java.ExecutionEnvironment;
 import org.apache.flink.api.java.io.DiscardingOutputFormat;
 import org.apache.flink.api.java.tuple.Tuple2;
-import org.apache.flink.configuration.ConfigConstants;
-import org.apache.flink.configuration.Configuration;
 import org.apache.flink.graph.Edge;
 import org.apache.flink.graph.EdgeDirection;
 import org.apache.flink.graph.Graph;
@@ -31,13 +29,9 @@ import org.apache.flink.graph.NeighborsFunctionWithVertexValue;
 import org.apache.flink.graph.ReduceNeighborsFunction;
 import org.apache.flink.graph.Vertex;
 import org.apache.flink.graph.test.TestGraphUtils;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
-import org.apache.flink.test.util.TestEnvironment;
+import org.apache.flink.test.util.AbstractTestBase;
 import org.apache.flink.util.Collector;
-import org.apache.flink.util.TestLogger;
 
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
 import org.junit.Test;
 
 import static org.junit.Assert.fail;
@@ -45,29 +39,10 @@ import static org.junit.Assert.fail;
 /**
  * Test expected exceptions for {@link Graph#groupReduceOnNeighbors} and {@link Graph#reduceOnNeighbors}.
  */
-public class ReduceOnNeighborsWithExceptionITCase extends TestLogger {
+public class ReduceOnNeighborsWithExceptionITCase extends AbstractTestBase {
 
 	private static final int PARALLELISM = 4;
 
-	private static LocalFlinkMiniCluster cluster;
-
-	@BeforeClass
-	public static void setupCluster() {
-		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, PARALLELISM);
-		cluster = new LocalFlinkMiniCluster(config, false);
-		cluster.start();
-
-		TestEnvironment.setAsContext(cluster, PARALLELISM);
-	}
-
-	@AfterClass
-	public static void tearDownCluster() {
-		cluster.stop();
-
-		TestEnvironment.unsetAsContext();
-	}
-
 	/**
 	 * Test groupReduceOnNeighbors() -NeighborsFunctionWithVertexValue-
 	 * with an edge having a srcId that does not exist in the vertex DataSet.

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/checkpointing/EventTimeAllWindowCheckpointingITCase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/checkpointing/EventTimeAllWindowCheckpointingITCase.java b/flink-tests/src/test/java/org/apache/flink/test/checkpointing/EventTimeAllWindowCheckpointingITCase.java
index fd4ecd4..07167a9 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/checkpointing/EventTimeAllWindowCheckpointingITCase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/checkpointing/EventTimeAllWindowCheckpointingITCase.java
@@ -24,26 +24,24 @@ import org.apache.flink.api.common.restartstrategy.RestartStrategies;
 import org.apache.flink.api.java.tuple.Tuple2;
 import org.apache.flink.api.java.tuple.Tuple4;
 import org.apache.flink.configuration.AkkaOptions;
-import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
 import org.apache.flink.configuration.TaskManagerOptions;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
 import org.apache.flink.runtime.state.CheckpointListener;
 import org.apache.flink.streaming.api.TimeCharacteristic;
 import org.apache.flink.streaming.api.checkpoint.ListCheckpointed;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
 import org.apache.flink.streaming.api.functions.sink.RichSinkFunction;
 import org.apache.flink.streaming.api.functions.source.RichSourceFunction;
 import org.apache.flink.streaming.api.functions.windowing.RichAllWindowFunction;
 import org.apache.flink.streaming.api.watermark.Watermark;
 import org.apache.flink.streaming.api.windowing.time.Time;
 import org.apache.flink.streaming.api.windowing.windows.TimeWindow;
-import org.apache.flink.streaming.util.TestStreamEnvironment;
+import org.apache.flink.test.util.MiniClusterResource;
 import org.apache.flink.test.util.SuccessException;
 import org.apache.flink.util.Collector;
 import org.apache.flink.util.TestLogger;
 
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
+import org.junit.ClassRule;
 import org.junit.Test;
 
 import java.util.Collections;
@@ -66,29 +64,19 @@ public class EventTimeAllWindowCheckpointingITCase extends TestLogger {
 
 	private static final int PARALLELISM = 4;
 
-	private static LocalFlinkMiniCluster cluster;
+	@ClassRule
+	public static final MiniClusterResource MINI_CLUSTER_RESOURCE = new MiniClusterResource(
+		new MiniClusterResource.MiniClusterResourceConfiguration(
+			getConfiguration(),
+			2,
+			PARALLELISM / 2));
 
-	private static TestStreamEnvironment env;
-
-	@BeforeClass
-	public static void startTestCluster() {
+	private static Configuration getConfiguration() {
 		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, 2);
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, PARALLELISM / 2);
 		config.setLong(TaskManagerOptions.MANAGED_MEMORY_SIZE, 48L);
 		config.setString(AkkaOptions.LOOKUP_TIMEOUT, "60 s");
 		config.setString(AkkaOptions.ASK_TIMEOUT, "60 s");
-		cluster = new LocalFlinkMiniCluster(config, false);
-		cluster.start();
-
-		env = new TestStreamEnvironment(cluster, PARALLELISM);
-	}
-
-	@AfterClass
-	public static void stopTestCluster() {
-		if (cluster != null) {
-			cluster.stop();
-		}
+		return config;
 	}
 
 	// ------------------------------------------------------------------------
@@ -101,6 +89,7 @@ public class EventTimeAllWindowCheckpointingITCase extends TestLogger {
 		FailingSource.reset();
 
 		try {
+			StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARALLELISM);
 			env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
 			env.enableCheckpointing(100);
@@ -161,6 +150,7 @@ public class EventTimeAllWindowCheckpointingITCase extends TestLogger {
 		FailingSource.reset();
 
 		try {
+			StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARALLELISM);
 			env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
 			env.enableCheckpointing(100);
@@ -218,6 +208,7 @@ public class EventTimeAllWindowCheckpointingITCase extends TestLogger {
 		FailingSource.reset();
 
 		try {
+			StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARALLELISM);
 			env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
 			env.enableCheckpointing(100);
@@ -286,6 +277,7 @@ public class EventTimeAllWindowCheckpointingITCase extends TestLogger {
 		FailingSource.reset();
 
 		try {
+			StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARALLELISM);
 			env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
 			env.enableCheckpointing(100);
@@ -354,6 +346,7 @@ public class EventTimeAllWindowCheckpointingITCase extends TestLogger {
 		FailingSource.reset();
 
 		try {
+			StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARALLELISM);
 			env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
 			env.enableCheckpointing(100);

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/checkpointing/KeyedStateCheckpointingITCase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/checkpointing/KeyedStateCheckpointingITCase.java b/flink-tests/src/test/java/org/apache/flink/test/checkpointing/KeyedStateCheckpointingITCase.java
index 76a18c3..e5868c3 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/checkpointing/KeyedStateCheckpointingITCase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/checkpointing/KeyedStateCheckpointingITCase.java
@@ -24,11 +24,9 @@ import org.apache.flink.api.common.state.ValueState;
 import org.apache.flink.api.common.state.ValueStateDescriptor;
 import org.apache.flink.api.java.functions.KeySelector;
 import org.apache.flink.api.java.tuple.Tuple2;
-import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
 import org.apache.flink.configuration.TaskManagerOptions;
 import org.apache.flink.contrib.streaming.state.RocksDBStateBackend;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
 import org.apache.flink.runtime.state.AbstractStateBackend;
 import org.apache.flink.runtime.state.CheckpointListener;
 import org.apache.flink.runtime.state.filesystem.FsStateBackend;
@@ -38,11 +36,10 @@ import org.apache.flink.streaming.api.datastream.DataStream;
 import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
 import org.apache.flink.streaming.api.functions.sink.RichSinkFunction;
 import org.apache.flink.streaming.api.functions.source.RichParallelSourceFunction;
-import org.apache.flink.streaming.util.TestStreamEnvironment;
+import org.apache.flink.test.util.MiniClusterResource;
 import org.apache.flink.util.TestLogger;
 
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
+import org.junit.ClassRule;
 import org.junit.Rule;
 import org.junit.Test;
 import org.junit.rules.TemporaryFolder;
@@ -80,24 +77,17 @@ public class KeyedStateCheckpointingITCase extends TestLogger {
 
 	// ------------------------------------------------------------------------
 
-	private static LocalFlinkMiniCluster cluster;
+	@ClassRule
+	public static final MiniClusterResource MINI_CLUSTER_RESOURCE = new MiniClusterResource(
+		new MiniClusterResource.MiniClusterResourceConfiguration(
+			getConfiguration(),
+			NUM_TASK_MANAGERS,
+			NUM_TASK_SLOTS));
 
-	@BeforeClass
-	public static void startCluster() throws Exception {
+	private static Configuration getConfiguration() {
 		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, NUM_TASK_MANAGERS);
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, NUM_TASK_SLOTS);
 		config.setLong(TaskManagerOptions.MANAGED_MEMORY_SIZE, 12L);
-
-		cluster = new LocalFlinkMiniCluster(config, false);
-		cluster.start();
-	}
-
-	@AfterClass
-	public static void stopCluster() throws Exception{
-		if (cluster != null) {
-			cluster.stop();
-		}
+		return config;
 	}
 
 	// ------------------------------------------------------------------------
@@ -150,7 +140,7 @@ public class KeyedStateCheckpointingITCase extends TestLogger {
 	protected void testProgramWithBackend(AbstractStateBackend stateBackend) throws Exception {
 		assertEquals("Broken test setup", 0, (NUM_STRINGS / 2) % NUM_KEYS);
 
-		final StreamExecutionEnvironment env = new TestStreamEnvironment(cluster, PARALLELISM);
+		final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 		env.setParallelism(PARALLELISM);
 		env.enableCheckpointing(500);
 		env.getConfig().disableSysoutLogging();

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/checkpointing/StreamFaultToleranceTestBase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/checkpointing/StreamFaultToleranceTestBase.java b/flink-tests/src/test/java/org/apache/flink/test/checkpointing/StreamFaultToleranceTestBase.java
index 5d902ff..5ac1214 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/checkpointing/StreamFaultToleranceTestBase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/checkpointing/StreamFaultToleranceTestBase.java
@@ -19,24 +19,18 @@
 package org.apache.flink.test.checkpointing;
 
 import org.apache.flink.api.common.restartstrategy.RestartStrategies;
-import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
-import org.apache.flink.configuration.TaskManagerOptions;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
 import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
-import org.apache.flink.streaming.util.TestStreamEnvironment;
+import org.apache.flink.test.util.MiniClusterResource;
 import org.apache.flink.test.util.TestUtils;
 import org.apache.flink.util.TestLogger;
 
-import org.junit.AfterClass;
 import org.junit.Assert;
-import org.junit.BeforeClass;
+import org.junit.ClassRule;
 import org.junit.Test;
 
 import java.io.Serializable;
 
-import static org.junit.Assert.fail;
-
 /**
  * Test base for fault tolerant streaming programs.
  */
@@ -46,37 +40,12 @@ public abstract class StreamFaultToleranceTestBase extends TestLogger {
 	protected static final int NUM_TASK_SLOTS = 4;
 	protected static final int PARALLELISM = NUM_TASK_MANAGERS * NUM_TASK_SLOTS;
 
-	private static LocalFlinkMiniCluster cluster;
-
-	@BeforeClass
-	public static void startCluster() {
-		try {
-			Configuration config = new Configuration();
-			config.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, NUM_TASK_MANAGERS);
-			config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, NUM_TASK_SLOTS);
-			config.setLong(TaskManagerOptions.MANAGED_MEMORY_SIZE, 12L);
-
-			cluster = new LocalFlinkMiniCluster(config, false);
-
-			cluster.start();
-		}
-		catch (Exception e) {
-			e.printStackTrace();
-			fail("Failed to start test cluster: " + e.getMessage());
-		}
-	}
-
-	@AfterClass
-	public static void stopCluster() {
-		try {
-			cluster.stop();
-			cluster = null;
-		}
-		catch (Exception e) {
-			e.printStackTrace();
-			fail("Failed to stop test cluster: " + e.getMessage());
-		}
-	}
+	@ClassRule
+	public static final MiniClusterResource MINI_CLUSTER_RESOURCE = new MiniClusterResource(
+		new MiniClusterResource.MiniClusterResourceConfiguration(
+			new Configuration(),
+			NUM_TASK_MANAGERS,
+			NUM_TASK_SLOTS));
 
 	/**
 	 * Implementations are expected to assemble the test topology in this function
@@ -96,7 +65,7 @@ public abstract class StreamFaultToleranceTestBase extends TestLogger {
 	@Test
 	public void runCheckpointedProgram() throws Exception {
 		try {
-			TestStreamEnvironment env = new TestStreamEnvironment(cluster, PARALLELISM);
+			StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARALLELISM);
 			env.enableCheckpointing(500);
 			env.getConfig().disableSysoutLogging();

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/checkpointing/WindowCheckpointingITCase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/checkpointing/WindowCheckpointingITCase.java b/flink-tests/src/test/java/org/apache/flink/test/checkpointing/WindowCheckpointingITCase.java
index 7ec4f86..bc73717 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/checkpointing/WindowCheckpointingITCase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/checkpointing/WindowCheckpointingITCase.java
@@ -23,25 +23,23 @@ import org.apache.flink.api.common.functions.ReduceFunction;
 import org.apache.flink.api.common.restartstrategy.RestartStrategies;
 import org.apache.flink.api.java.tuple.Tuple;
 import org.apache.flink.api.java.tuple.Tuple2;
-import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
 import org.apache.flink.configuration.TaskManagerOptions;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
 import org.apache.flink.runtime.state.CheckpointListener;
 import org.apache.flink.streaming.api.TimeCharacteristic;
 import org.apache.flink.streaming.api.checkpoint.ListCheckpointed;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
 import org.apache.flink.streaming.api.functions.sink.RichSinkFunction;
 import org.apache.flink.streaming.api.functions.source.RichSourceFunction;
 import org.apache.flink.streaming.api.functions.windowing.RichWindowFunction;
 import org.apache.flink.streaming.api.windowing.time.Time;
 import org.apache.flink.streaming.api.windowing.windows.TimeWindow;
-import org.apache.flink.streaming.util.TestStreamEnvironment;
+import org.apache.flink.test.util.MiniClusterResource;
 import org.apache.flink.test.util.SuccessException;
 import org.apache.flink.util.Collector;
 import org.apache.flink.util.TestLogger;
 
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
+import org.junit.ClassRule;
 import org.junit.Test;
 import org.junit.runner.RunWith;
 import org.junit.runners.Parameterized;
@@ -74,28 +72,17 @@ public class WindowCheckpointingITCase extends TestLogger {
 
 	private static final int PARALLELISM = 4;
 
-	private static LocalFlinkMiniCluster cluster;
+	@ClassRule
+	public static final MiniClusterResource MINI_CLUSTER_RESOURCE = new MiniClusterResource(
+		new MiniClusterResource.MiniClusterResourceConfiguration(
+			getConfiguration(),
+			2,
+			PARALLELISM / 2));
 
-	private static TestStreamEnvironment env;
-
-	@BeforeClass
-	public static void startTestCluster() {
+	private static Configuration getConfiguration() {
 		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, 2);
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, PARALLELISM / 2);
 		config.setLong(TaskManagerOptions.MANAGED_MEMORY_SIZE, 48L);
-
-		cluster = new LocalFlinkMiniCluster(config, false);
-		cluster.start();
-
-		env = new TestStreamEnvironment(cluster, PARALLELISM);
-	}
-
-	@AfterClass
-	public static void stopTestCluster() {
-		if (cluster != null) {
-			cluster.stop();
-		}
+		return config;
 	}
 
 	// ------------------------------------------------------------------------
@@ -106,6 +93,7 @@ public class WindowCheckpointingITCase extends TestLogger {
 		FailingSource.reset();
 
 		try {
+			StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARALLELISM);
 			env.setStreamTimeCharacteristic(timeCharacteristic);
 			env.getConfig().setAutoWatermarkInterval(10);
@@ -160,6 +148,7 @@ public class WindowCheckpointingITCase extends TestLogger {
 		FailingSource.reset();
 
 		try {
+			StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARALLELISM);
 			env.setStreamTimeCharacteristic(timeCharacteristic);
 			env.getConfig().setAutoWatermarkInterval(10);
@@ -214,6 +203,7 @@ public class WindowCheckpointingITCase extends TestLogger {
 		FailingSource.reset();
 
 		try {
+			StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARALLELISM);
 			env.setStreamTimeCharacteristic(timeCharacteristic);
 			env.getConfig().setAutoWatermarkInterval(10);
@@ -258,6 +248,7 @@ public class WindowCheckpointingITCase extends TestLogger {
 		FailingSource.reset();
 
 		try {
+			StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARALLELISM);
 			env.setStreamTimeCharacteristic(timeCharacteristic);
 			env.getConfig().setAutoWatermarkInterval(10);

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/misc/AutoParallelismITCase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/misc/AutoParallelismITCase.java b/flink-tests/src/test/java/org/apache/flink/test/misc/AutoParallelismITCase.java
index 9cafee6..c25dbf0 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/misc/AutoParallelismITCase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/misc/AutoParallelismITCase.java
@@ -22,17 +22,15 @@ import org.apache.flink.api.common.ExecutionConfig;
 import org.apache.flink.api.common.functions.RichMapPartitionFunction;
 import org.apache.flink.api.common.io.GenericInputFormat;
 import org.apache.flink.api.java.DataSet;
+import org.apache.flink.api.java.ExecutionEnvironment;
 import org.apache.flink.api.java.io.LocalCollectionOutputFormat;
-import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
 import org.apache.flink.core.io.GenericInputSplit;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
-import org.apache.flink.test.util.TestEnvironment;
+import org.apache.flink.test.util.MiniClusterResource;
 import org.apache.flink.util.Collector;
 import org.apache.flink.util.TestLogger;
 
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
+import org.junit.ClassRule;
 import org.junit.Test;
 
 import java.io.IOException;
@@ -52,37 +50,17 @@ public class AutoParallelismITCase extends TestLogger {
 	private static final int SLOTS_PER_TM = 7;
 	private static final int PARALLELISM = NUM_TM * SLOTS_PER_TM;
 
-	private static LocalFlinkMiniCluster cluster;
-
-	private static TestEnvironment env;
-
-	@BeforeClass
-	public static void setupCluster() {
-		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, NUM_TM);
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, SLOTS_PER_TM);
-		cluster = new LocalFlinkMiniCluster(config, false);
-
-		cluster.start();
-
-		env = new TestEnvironment(cluster, NUM_TM * SLOTS_PER_TM, false);
-	}
-
-	@AfterClass
-	public static void teardownCluster() {
-		try {
-			cluster.stop();
-		}
-		catch (Throwable t) {
-			System.err.println("Error stopping cluster on shutdown");
-			t.printStackTrace();
-			fail("ClusterClient shutdown caused an exception: " + t.getMessage());
-		}
-	}
+	@ClassRule
+	public static final MiniClusterResource MINI_CLUSTER_RESOURCE = new MiniClusterResource(
+		new MiniClusterResource.MiniClusterResourceConfiguration(
+			new Configuration(),
+			2,
+			7));
 
 	@Test
 	public void testProgramWithAutoParallelism() {
 		try {
+			ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(ExecutionConfig.PARALLELISM_AUTO_MAX);
 			env.getConfig().disableSysoutLogging();
 

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/misc/CustomSerializationITCase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/misc/CustomSerializationITCase.java b/flink-tests/src/test/java/org/apache/flink/test/misc/CustomSerializationITCase.java
index ad359f6..ab937c9 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/misc/CustomSerializationITCase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/misc/CustomSerializationITCase.java
@@ -19,21 +19,19 @@
 package org.apache.flink.test.misc;
 
 import org.apache.flink.api.common.functions.MapFunction;
+import org.apache.flink.api.java.ExecutionEnvironment;
 import org.apache.flink.api.java.io.DiscardingOutputFormat;
 import org.apache.flink.client.program.ProgramInvocationException;
-import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
 import org.apache.flink.configuration.TaskManagerOptions;
 import org.apache.flink.core.memory.DataInputView;
 import org.apache.flink.core.memory.DataOutputView;
 import org.apache.flink.runtime.client.JobExecutionException;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
-import org.apache.flink.test.util.TestEnvironment;
+import org.apache.flink.test.util.MiniClusterResource;
 import org.apache.flink.types.Value;
 import org.apache.flink.util.TestLogger;
 
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
+import org.junit.ClassRule;
 import org.junit.Test;
 
 import java.io.IOException;
@@ -50,30 +48,23 @@ public class CustomSerializationITCase extends TestLogger {
 
 	private static final int PARLLELISM = 5;
 
-	private static LocalFlinkMiniCluster cluster;
+	@ClassRule
+	public static final MiniClusterResource MINI_CLUSTER_RESOURCE = new MiniClusterResource(
+		new MiniClusterResource.MiniClusterResourceConfiguration(
+			getConfiguration(),
+			1,
+			PARLLELISM));
 
-	private static TestEnvironment env;
-
-	@BeforeClass
-	public static void startCluster() {
+	public static Configuration getConfiguration() {
 		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, PARLLELISM);
 		config.setLong(TaskManagerOptions.MANAGED_MEMORY_SIZE, 30L);
-		cluster = new LocalFlinkMiniCluster(config, false);
-		cluster.start();
-
-		env = new TestEnvironment(cluster, PARLLELISM, false);
-	}
-
-	@AfterClass
-	public static void shutdownCluster() {
-		cluster.stop();
-		cluster = null;
+		return config;
 	}
 
 	@Test
 	public void testIncorrectSerializer1() {
 		try {
+			ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARLLELISM);
 			env.getConfig().disableSysoutLogging();
 
@@ -104,6 +95,7 @@ public class CustomSerializationITCase extends TestLogger {
 	@Test
 	public void testIncorrectSerializer2() {
 		try {
+			ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARLLELISM);
 			env.getConfig().disableSysoutLogging();
 
@@ -134,6 +126,7 @@ public class CustomSerializationITCase extends TestLogger {
 	@Test
 	public void testIncorrectSerializer3() {
 		try {
+			ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARLLELISM);
 			env.getConfig().disableSysoutLogging();
 
@@ -164,6 +157,7 @@ public class CustomSerializationITCase extends TestLogger {
 	@Test
 	public void testIncorrectSerializer4() {
 		try {
+			ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(PARLLELISM);
 			env.getConfig().disableSysoutLogging();
 

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/misc/MiscellaneousIssuesITCase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/misc/MiscellaneousIssuesITCase.java b/flink-tests/src/test/java/org/apache/flink/test/misc/MiscellaneousIssuesITCase.java
index d79b339..6099934 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/misc/MiscellaneousIssuesITCase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/misc/MiscellaneousIssuesITCase.java
@@ -23,19 +23,16 @@ import org.apache.flink.api.common.accumulators.LongCounter;
 import org.apache.flink.api.common.functions.MapFunction;
 import org.apache.flink.api.common.functions.RichFlatMapFunction;
 import org.apache.flink.api.java.DataSet;
+import org.apache.flink.api.java.ExecutionEnvironment;
 import org.apache.flink.api.java.io.DiscardingOutputFormat;
-import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
-import org.apache.flink.configuration.TaskManagerOptions;
 import org.apache.flink.core.fs.FileSystem;
 import org.apache.flink.runtime.client.JobExecutionException;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
-import org.apache.flink.test.util.TestEnvironment;
+import org.apache.flink.test.util.MiniClusterResource;
 import org.apache.flink.util.Collector;
 import org.apache.flink.util.TestLogger;
 
-import org.junit.AfterClass;
-import org.junit.BeforeClass;
+import org.junit.ClassRule;
 import org.junit.Test;
 
 import static org.junit.Assert.assertEquals;
@@ -56,34 +53,17 @@ import static org.junit.Assert.fail;
 @SuppressWarnings("serial")
 public class MiscellaneousIssuesITCase extends TestLogger {
 
-	private static final int PARALLELISM = 6;
-
-	private static LocalFlinkMiniCluster cluster;
-
-	private static TestEnvironment env;
-
-	@BeforeClass
-	public static void startCluster() {
-		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, 2);
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, 3);
-		config.setLong(TaskManagerOptions.MANAGED_MEMORY_SIZE, 12L);
-		cluster = new LocalFlinkMiniCluster(config, false);
-
-		cluster.start();
-
-		env = new TestEnvironment(cluster, PARALLELISM, false);
-	}
-
-	@AfterClass
-	public static void shutdownCluster() {
-		cluster.stop();
-		cluster = null;
-	}
+	@ClassRule
+	public static final MiniClusterResource MINI_CLUSTER_RESOURCE = new MiniClusterResource(
+		new MiniClusterResource.MiniClusterResourceConfiguration(
+			new Configuration(),
+			2,
+			3));
 
 	@Test
 	public void testNullValues() {
 		try {
+			ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(1);
 			env.getConfig().disableSysoutLogging();
 
@@ -114,6 +94,7 @@ public class MiscellaneousIssuesITCase extends TestLogger {
 	@Test
 	public void testDisjointDataflows() {
 		try {
+			ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(5);
 			env.getConfig().disableSysoutLogging();
 
@@ -134,6 +115,7 @@ public class MiscellaneousIssuesITCase extends TestLogger {
 		final String accName = "test_accumulator";
 
 		try {
+			ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 			env.setParallelism(6);
 			env.getConfig().disableSysoutLogging();
 

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/misc/SuccessAfterNetworkBuffersFailureITCase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/misc/SuccessAfterNetworkBuffersFailureITCase.java b/flink-tests/src/test/java/org/apache/flink/test/misc/SuccessAfterNetworkBuffersFailureITCase.java
index 1210e85..02fdaf9 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/misc/SuccessAfterNetworkBuffersFailureITCase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/misc/SuccessAfterNetworkBuffersFailureITCase.java
@@ -25,7 +25,6 @@ import org.apache.flink.api.java.io.DiscardingOutputFormat;
 import org.apache.flink.api.java.operators.DeltaIteration;
 import org.apache.flink.api.java.operators.IterativeDataSet;
 import org.apache.flink.api.java.tuple.Tuple2;
-import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
 import org.apache.flink.configuration.TaskManagerOptions;
 import org.apache.flink.examples.java.clustering.KMeans;
@@ -34,9 +33,10 @@ import org.apache.flink.examples.java.graph.ConnectedComponents;
 import org.apache.flink.examples.java.graph.util.ConnectedComponentsData;
 import org.apache.flink.runtime.client.JobExecutionException;
 import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
-import org.apache.flink.test.util.TestEnvironment;
+import org.apache.flink.test.util.MiniClusterResource;
 import org.apache.flink.util.TestLogger;
 
+import org.junit.ClassRule;
 import org.junit.Test;
 
 import static org.junit.Assert.assertTrue;
@@ -50,22 +50,27 @@ import static org.junit.Assert.fail;
 public class SuccessAfterNetworkBuffersFailureITCase extends TestLogger {
 
 	private static final int PARALLELISM = 16;
+
+	@ClassRule
+	public static final MiniClusterResource MINI_CLUSTER_RESOURCE = new MiniClusterResource(
+		new MiniClusterResource.MiniClusterResourceConfiguration(
+			getConfiguration(),
+			2,
+			8));
+
+	private static Configuration getConfiguration() {
+		Configuration config = new Configuration();
+		config.setLong(TaskManagerOptions.MANAGED_MEMORY_SIZE, 80L);
+		config.setInteger(TaskManagerOptions.NETWORK_NUM_BUFFERS, 1024);
+		return config;
+	}
+
 	@Test
 	public void testSuccessfulProgramAfterFailure() {
 		LocalFlinkMiniCluster cluster = null;
 
 		try {
-			Configuration config = new Configuration();
-			config.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, 2);
-			config.setLong(TaskManagerOptions.MANAGED_MEMORY_SIZE, 80L);
-			config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, 8);
-			config.setInteger(TaskManagerOptions.NETWORK_NUM_BUFFERS, 1024);
-
-			cluster = new LocalFlinkMiniCluster(config, false);
-
-			cluster.start();
-
-			TestEnvironment env = new TestEnvironment(cluster, PARALLELISM, false);
+			ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 
 			try {
 				runConnectedComponents(env);
@@ -95,11 +100,6 @@ public class SuccessAfterNetworkBuffersFailureITCase extends TestLogger {
 			e.printStackTrace();
 			fail(e.getMessage());
 		}
-		finally {
-			if (cluster != null) {
-				cluster.stop();
-			}
-		}
 	}
 
 	private static void runConnectedComponents(ExecutionEnvironment env) throws Exception {

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/recovery/FastFailuresITCase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/recovery/FastFailuresITCase.java b/flink-tests/src/test/java/org/apache/flink/test/recovery/FastFailuresITCase.java
index 9a59e88..be13c1a 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/recovery/FastFailuresITCase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/recovery/FastFailuresITCase.java
@@ -21,14 +21,12 @@ package org.apache.flink.test.recovery;
 import org.apache.flink.api.common.functions.MapFunction;
 import org.apache.flink.api.common.restartstrategy.RestartStrategies;
 import org.apache.flink.api.java.tuple.Tuple2;
-import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
 import org.apache.flink.streaming.api.datastream.DataStream;
+import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
 import org.apache.flink.streaming.api.functions.sink.SinkFunction;
 import org.apache.flink.streaming.api.functions.source.RichSourceFunction;
-import org.apache.flink.streaming.util.TestStreamEnvironment;
-import org.apache.flink.util.TestLogger;
+import org.apache.flink.test.util.AbstractTestBase;
 
 import org.junit.Test;
 
@@ -40,7 +38,7 @@ import static org.junit.Assert.fail;
  * Test program with very fast failure rate.
  */
 @SuppressWarnings("serial")
-public class FastFailuresITCase extends TestLogger {
+public class FastFailuresITCase extends AbstractTestBase {
 
 	static final AtomicInteger FAILURES_SO_FAR = new AtomicInteger();
 	static final int NUM_FAILURES = 200;
@@ -49,14 +47,7 @@ public class FastFailuresITCase extends TestLogger {
 	public void testThis() {
 		final int parallelism = 4;
 
-		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, 2);
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, 2);
-
-		LocalFlinkMiniCluster cluster = new LocalFlinkMiniCluster(config, false);
-		cluster.start();
-
-		TestStreamEnvironment env = new TestStreamEnvironment(cluster, parallelism);
+		StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
 
 		env.getConfig().disableSysoutLogging();
 		env.setParallelism(parallelism);
@@ -99,8 +90,6 @@ public class FastFailuresITCase extends TestLogger {
 		catch (Exception e) {
 			e.printStackTrace();
 			fail(e.getMessage());
-		} finally {
-			cluster.stop();
 		}
 	}
 }

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryFailureRateStrategyITBase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryFailureRateStrategyITBase.java b/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryFailureRateStrategyITBase.java
index 6fc876f..424f540 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryFailureRateStrategyITBase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryFailureRateStrategyITBase.java
@@ -20,26 +20,29 @@ package org.apache.flink.test.recovery;
 
 import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
+import org.apache.flink.test.util.MiniClusterResource;
 
-import org.junit.BeforeClass;
+import org.junit.ClassRule;
 
 /**
  * Test cluster configuration with failure-rate recovery.
  */
 public class SimpleRecoveryFailureRateStrategyITBase extends SimpleRecoveryITCaseBase {
-	@BeforeClass
-	public static void setupCluster() {
+
+	@ClassRule
+	public static final MiniClusterResource MINI_CLUSTER_RESOURCE = new MiniClusterResource(
+		new MiniClusterResource.MiniClusterResourceConfiguration(
+			getConfiguration(),
+			2,
+			2));
+
+	private static Configuration getConfiguration() {
 		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, 2);
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, 2);
 		config.setString(ConfigConstants.RESTART_STRATEGY, "failure-rate");
 		config.setInteger(ConfigConstants.RESTART_STRATEGY_FAILURE_RATE_MAX_FAILURES_PER_INTERVAL, 1);
 		config.setString(ConfigConstants.RESTART_STRATEGY_FAILURE_RATE_FAILURE_RATE_INTERVAL, "1 second");
 		config.setString(ConfigConstants.RESTART_STRATEGY_FAILURE_RATE_DELAY, "100 ms");
 
-		cluster = new LocalFlinkMiniCluster(config, false);
-
-		cluster.start();
+		return config;
 	}
 }

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryFixedDelayRestartStrategyITBase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryFixedDelayRestartStrategyITBase.java b/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryFixedDelayRestartStrategyITBase.java
index 405a1c7..1f52d27 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryFixedDelayRestartStrategyITBase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryFixedDelayRestartStrategyITBase.java
@@ -20,25 +20,28 @@ package org.apache.flink.test.recovery;
 
 import org.apache.flink.configuration.ConfigConstants;
 import org.apache.flink.configuration.Configuration;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
+import org.apache.flink.test.util.MiniClusterResource;
 
-import org.junit.BeforeClass;
+import org.junit.ClassRule;
 
 /**
  * Test cluster configuration with fixed-delay recovery.
  */
 public class SimpleRecoveryFixedDelayRestartStrategyITBase extends SimpleRecoveryITCaseBase {
-	@BeforeClass
-	public static void setupCluster() {
+
+	@ClassRule
+	public static final MiniClusterResource MINI_CLUSTER_RESOURCE = new MiniClusterResource(
+		new MiniClusterResource.MiniClusterResourceConfiguration(
+			getConfiguration(),
+			2,
+			2));
+
+	private static Configuration getConfiguration() {
 		Configuration config = new Configuration();
-		config.setInteger(ConfigConstants.LOCAL_NUMBER_TASK_MANAGER, 2);
-		config.setInteger(ConfigConstants.TASK_MANAGER_NUM_TASK_SLOTS, 2);
 		config.setString(ConfigConstants.RESTART_STRATEGY, "fixed-delay");
 		config.setInteger(ConfigConstants.RESTART_STRATEGY_FIXED_DELAY_ATTEMPTS, 1);
 		config.setString(ConfigConstants.RESTART_STRATEGY_FIXED_DELAY_DELAY, "100 ms");
 
-		cluster = new LocalFlinkMiniCluster(config, false);
-
-		cluster.start();
+		return config;
 	}
 }

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryITCaseBase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryITCaseBase.java b/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryITCaseBase.java
index ceef78b..d81274d 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryITCaseBase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/recovery/SimpleRecoveryITCaseBase.java
@@ -24,11 +24,8 @@ import org.apache.flink.api.common.functions.RichMapFunction;
 import org.apache.flink.api.common.restartstrategy.RestartStrategies;
 import org.apache.flink.api.java.ExecutionEnvironment;
 import org.apache.flink.api.java.io.LocalCollectionOutputFormat;
-import org.apache.flink.client.program.ProgramInvocationException;
 import org.apache.flink.runtime.client.JobExecutionException;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
 
-import org.junit.AfterClass;
 import org.junit.Test;
 
 import java.util.ArrayList;
@@ -46,30 +43,15 @@ import static org.junit.Assert.fail;
 @SuppressWarnings("serial")
 public abstract class SimpleRecoveryITCaseBase {
 
-	protected static LocalFlinkMiniCluster cluster;
-
-	@AfterClass
-	public static void teardownCluster() {
-		try {
-			cluster.stop();
-		}
-		catch (Throwable t) {
-			System.err.println("Error stopping cluster on shutdown");
-			t.printStackTrace();
-			fail("ClusterClient shutdown caused an exception: " + t.getMessage());
-		}
-	}
-
 	@Test
-	public void testFailedRunThenSuccessfulRun() {
+	public void testFailedRunThenSuccessfulRun() throws Exception {
 
 		try {
 			List<Long> resultCollection = new ArrayList<Long>();
 
 			// attempt 1
 			{
-				ExecutionEnvironment env = ExecutionEnvironment.createRemoteEnvironment(
-						"localhost", cluster.getLeaderRPCPort());
+				ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 
 				env.setParallelism(4);
 				env.setRestartStrategy(RestartStrategies.noRestart());
@@ -91,15 +73,14 @@ public abstract class SimpleRecoveryITCaseBase {
 					String msg = res == null ? "null result" : "result in " + res.getNetRuntime() + " ms";
 					fail("The program should have failed, but returned " + msg);
 				}
-				catch (ProgramInvocationException e) {
+				catch (JobExecutionException e) {
 					// expected
 				}
 			}
 
 			// attempt 2
 			{
-				ExecutionEnvironment env = ExecutionEnvironment.createRemoteEnvironment(
-						"localhost", cluster.getLeaderRPCPort());
+				ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 
 				env.setParallelism(4);
 				env.setRestartStrategy(RestartStrategies.noRestart());
@@ -149,8 +130,7 @@ public abstract class SimpleRecoveryITCaseBase {
 		try {
 			List<Long> resultCollection = new ArrayList<Long>();
 
-			ExecutionEnvironment env = ExecutionEnvironment.createRemoteEnvironment(
-					"localhost", cluster.getLeaderRPCPort());
+			ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 
 			env.setParallelism(4);
 			// the default restart strategy should be taken
@@ -186,8 +166,7 @@ public abstract class SimpleRecoveryITCaseBase {
 		try {
 			List<Long> resultCollection = new ArrayList<Long>();
 
-			ExecutionEnvironment env = ExecutionEnvironment.createRemoteEnvironment(
-					"localhost", cluster.getLeaderRPCPort());
+			ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
 
 			env.setParallelism(4);
 			env.setRestartStrategy(RestartStrategies.fixedDelayRestart(5, 100));

http://git-wip-us.apache.org/repos/asf/flink/blob/cb08aca3/flink-tests/src/test/java/org/apache/flink/test/streaming/experimental/CollectITCase.java
----------------------------------------------------------------------
diff --git a/flink-tests/src/test/java/org/apache/flink/test/streaming/experimental/CollectITCase.java b/flink-tests/src/test/java/org/apache/flink/test/streaming/experimental/CollectITCase.java
index 0535bf7..5591c81 100644
--- a/flink-tests/src/test/java/org/apache/flink/test/streaming/experimental/CollectITCase.java
+++ b/flink-tests/src/test/java/org/apache/flink/test/streaming/experimental/CollectITCase.java
@@ -18,13 +18,10 @@
 
 package org.apache.flink.test.streaming.experimental;
 
-import org.apache.flink.configuration.Configuration;
-import org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster;
 import org.apache.flink.streaming.api.datastream.DataStream;
 import org.apache.flink.streaming.api.datastream.DataStreamUtils;
 import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
-import org.apache.flink.streaming.util.TestStreamEnvironment;
-import org.apache.flink.util.TestLogger;
+import org.apache.flink.test.util.AbstractTestBase;
 
 import org.junit.Test;
 
@@ -38,33 +35,23 @@ import static org.junit.Assert.assertEquals;
  * <p>This experimental class is relocated from flink-streaming-contrib. Please see package-info.java
  * for more information.
  */
-public class CollectITCase extends TestLogger {
+public class CollectITCase extends AbstractTestBase {
 
 	@Test
 	public void testCollect() throws Exception {
-		final LocalFlinkMiniCluster cluster = new LocalFlinkMiniCluster(new Configuration(), false);
-		try {
-			cluster.start();
+		final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
+		env.setParallelism(1);
 
-			TestStreamEnvironment.setAsContext(cluster, 1);
+		final long n = 10;
+		DataStream<Long> stream = env.generateSequence(1, n);
 
-			final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
-
-			final long n = 10;
-			DataStream<Long> stream = env.generateSequence(1, n);
-
-			long i = 1;
-			for (Iterator<Long> it = DataStreamUtils.collect(stream); it.hasNext(); ) {
-				long x = it.next();
-				assertEquals("received wrong element", i, x);
-				i++;
-			}
-
-			assertEquals("received wrong number of elements", n + 1, i);
-		}
-		finally {
-			TestStreamEnvironment.unsetAsContext();
-			cluster.stop();
+		long i = 1;
+		for (Iterator<Long> it = DataStreamUtils.collect(stream); it.hasNext(); ) {
+			long x = it.next();
+			assertEquals("received wrong element", i, x);
+			i++;
 		}
+
+		assertEquals("received wrong number of elements", n + 1, i);
 	}
 }


[4/5] flink git commit: [hotfix][build] Set flink-shaded-jackson to provided

Posted by ch...@apache.org.
[hotfix][build] Set flink-shaded-jackson to provided

This closes #5539.


Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/3d22f7c0
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/3d22f7c0
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/3d22f7c0

Branch: refs/heads/master
Commit: 3d22f7c0b4b71324bf77b7456145ba37052b33ce
Parents: 99c2556
Author: zentol <ch...@apache.org>
Authored: Wed Feb 21 10:43:09 2018 +0100
Committer: zentol <ch...@apache.org>
Committed: Wed Feb 21 18:19:38 2018 +0100

----------------------------------------------------------------------
 flink-libraries/flink-table/pom.xml | 1 +
 flink-mesos/pom.xml                 | 1 +
 2 files changed, 2 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/flink/blob/3d22f7c0/flink-libraries/flink-table/pom.xml
----------------------------------------------------------------------
diff --git a/flink-libraries/flink-table/pom.xml b/flink-libraries/flink-table/pom.xml
index f336037..bf6585a 100644
--- a/flink-libraries/flink-table/pom.xml
+++ b/flink-libraries/flink-table/pom.xml
@@ -46,6 +46,7 @@ under the License.
 		<dependency>
 			<groupId>org.apache.flink</groupId>
 			<artifactId>flink-shaded-jackson</artifactId>
+			<scope>provided</scope>
 		</dependency>
 
 		<dependency>

http://git-wip-us.apache.org/repos/asf/flink/blob/3d22f7c0/flink-mesos/pom.xml
----------------------------------------------------------------------
diff --git a/flink-mesos/pom.xml b/flink-mesos/pom.xml
index 4abf50d..4e1ec76 100644
--- a/flink-mesos/pom.xml
+++ b/flink-mesos/pom.xml
@@ -57,6 +57,7 @@ under the License.
 		<dependency>
 			<groupId>org.apache.flink</groupId>
 			<artifactId>flink-shaded-jackson</artifactId>
+			<scope>provided</scope>
 		</dependency>
 
 		<dependency>