You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by aljoscha <gi...@git.apache.org> on 2017/10/11 16:36:31 UTC
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
GitHub user aljoscha opened a pull request:
https://github.com/apache/flink/pull/4807
[FLINK-7810] Switch from custom Flakka to Akka 2.4.x
## What is the purpose of the change
Drop support for Scala 2.10 and then update to a newer Akka version. Before, we were forced to stay on our custom Flakka 2.3.x version with back ports because newer Akka does not support Scala 2.10.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aljoscha/flink jira-7729-drop-210-akka-update
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4807.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4807
----
commit 24e70936eac780792dd00d63f833b286f4aa2aaf
Author: Aljoscha Krettek <al...@gmail.com>
Date: 2017-10-06T09:42:00Z
[FLINK-7809] Remove support for Scala 2.10
commit e64135f507256130b4d352e93152b736265de9c2
Author: Aljoscha Krettek <al...@gmail.com>
Date: 2017-10-06T12:33:43Z
[FLINK-7810] Switch from custom Flakka to Akka 2.4.x
We can do this now that we dropped support for Scala 2.10.
----
---
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4807
---
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/4807#discussion_r144529895
--- Diff: .travis.yml ---
@@ -61,31 +61,6 @@ matrix:
- TEST="misc"
- PROFILE="-Dhadoop.version=2.8.0"
- CACHE_NAME=JDK8_H280_M
- - jdk: "openjdk8"
--- End diff --
You're right, of course. Undropping ...
---
[GitHub] flink issue #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4.x
Posted by bowenli86 <gi...@git.apache.org>.
Github user bowenli86 commented on the issue:
https://github.com/apache/flink/pull/4807
wonderful! +1
---
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
Posted by EronWright <gi...@git.apache.org>.
Github user EronWright commented on a diff in the pull request:
https://github.com/apache/flink/pull/4807#discussion_r144320740
--- Diff: docs/start/building.md ---
@@ -115,19 +115,7 @@ The `-Pvendor-repos` activates a Maven [build profile](http://maven.apache.org/g
Flink has APIs, libraries, and runtime modules written in [Scala](http://scala-lang.org). Users of the Scala API and libraries may have to match the Scala version of Flink with the Scala version of their projects (because Scala is not strictly backwards compatible).
-**By default, Flink is built with the Scala 2.11**. To build Flink with Scala *2.10*, you can change the default Scala *binary version* by using *scala-2.10* build profile:
-
-~~~bash
-# Build with Scala version 2.10
-mvn clean install -DskipTests -Pscala-2.10
-~~~
-
-To build against custom Scala versions, you need to define new custom build profile that will override *scala.version* and *scala.binary.version* values.
-
-Flink is developed against Scala *2.11* and tested additionally against Scala *2.10*. These two versions are known to be compatible. Earlier versions (like Scala *2.9*) are *not* compatible.
--- End diff --
I read this paragraph to imply that Flink is still compatible with 2.10, but is that accurate given the use of Akka 2.4?
---
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/4807#discussion_r144518199
--- Diff: .travis.yml ---
@@ -61,31 +61,6 @@ matrix:
- TEST="misc"
- PROFILE="-Dhadoop.version=2.8.0"
- CACHE_NAME=JDK8_H280_M
- - jdk: "openjdk8"
--- End diff --
How about retaining these Travis profiles for now for the Hadoop and JDK versions?
Just bump the Scala dependency.
---
[GitHub] flink issue #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4.x
Posted by EronWright <gi...@git.apache.org>.
Github user EronWright commented on the issue:
https://github.com/apache/flink/pull/4807
Thanks for doing this, we need Akka 2.4 badly for improved SSL support.
---
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/4807#discussion_r144528124
--- Diff: flink-queryable-state/flink-queryable-state-java/pom.xml ---
@@ -110,8 +110,8 @@ under the License.
</dependency>
<dependency>
- <groupId>com.data-artisans</groupId>
- <artifactId>flakka-testkit_${scala.binary.version}</artifactId>
+ <groupId>com.typesafe.akka</groupId>
--- End diff --
There was a test in there that was using Akka but I think that's not needed anymore. Will remove.
On a side node, currently, the queryable-state-client module contains both the client and server implementations (that's why there was the akka test) while flunk-runtime only has interfaces. Fully tweezing the client and server apart is still in progress.
---
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/4807#discussion_r144518821
--- Diff: flink-queryable-state/flink-queryable-state-java/pom.xml ---
@@ -110,8 +110,8 @@ under the License.
</dependency>
<dependency>
- <groupId>com.data-artisans</groupId>
- <artifactId>flakka-testkit_${scala.binary.version}</artifactId>
+ <groupId>com.typesafe.akka</groupId>
--- End diff --
I could not find any use of akka testkit in the queryable state client (and there should not be, design wise).
Was this just copy/paste forwarded and can be dropped?
---
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
Posted by GJL <gi...@git.apache.org>.
Github user GJL commented on a diff in the pull request:
https://github.com/apache/flink/pull/4807#discussion_r144245061
--- Diff: docs/start/building.md ---
@@ -115,19 +115,7 @@ The `-Pvendor-repos` activates a Maven [build profile](http://maven.apache.org/g
Flink has APIs, libraries, and runtime modules written in [Scala](http://scala-lang.org). Users of the Scala API and libraries may have to match the Scala version of Flink with the Scala version of their projects (because Scala is not strictly backwards compatible).
-**By default, Flink is built with the Scala 2.11**. To build Flink with Scala *2.10*, you can change the default Scala *binary version* by using *scala-2.10* build profile:
-
-~~~bash
-# Build with Scala version 2.10
-mvn clean install -DskipTests -Pscala-2.10
-~~~
-
-To build against custom Scala versions, you need to define new custom build profile that will override *scala.version* and *scala.binary.version* values.
-
-Flink is developed against Scala *2.11* and tested additionally against Scala *2.10*. These two versions are known to be compatible. Earlier versions (like Scala *2.9*) are *not* compatible.
-
-Newer versions may be compatible, depending on breaking changes in the language features used by Flink, and the availability of Flink's dependencies in those Scala versions. The dependencies written in Scala include for example *Kafka*, *Akka*, *Scalatest*, and *scopt*.
-
+**By default, Flink is built with the Scala 2.11**.
--- End diff --
I would drop *the*, i.e.,
> **By default, Flink is built with Scala 2.11**.
---
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/4807#discussion_r144530220
--- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/taskmanager/TaskManagerComponentsStartupShutdownTest.java ---
@@ -202,8 +202,6 @@ protected void run() {
assertTrue(ioManager.isProperlyShutDown());
assertTrue(memManager.isShutdown());
} finally {
- TestingUtils.stopActorsGracefully(Arrays.asList(jobManager, taskManager));
--- End diff --
If the test succeeds, they will have been shutdown (the previous asserts also assert (hehe) this.
It seems the previous Akka version didn't mind another shutdown attempt but 2.4 fails with an exception when you try to shutdown after already being shut down.
---
[GitHub] flink issue #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4.x
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/4807
I think this is good now, +1
---
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/4807#discussion_r144326685
--- Diff: docs/start/building.md ---
@@ -115,19 +115,7 @@ The `-Pvendor-repos` activates a Maven [build profile](http://maven.apache.org/g
Flink has APIs, libraries, and runtime modules written in [Scala](http://scala-lang.org). Users of the Scala API and libraries may have to match the Scala version of Flink with the Scala version of their projects (because Scala is not strictly backwards compatible).
-**By default, Flink is built with the Scala 2.11**. To build Flink with Scala *2.10*, you can change the default Scala *binary version* by using *scala-2.10* build profile:
-
-~~~bash
-# Build with Scala version 2.10
-mvn clean install -DskipTests -Pscala-2.10
-~~~
-
-To build against custom Scala versions, you need to define new custom build profile that will override *scala.version* and *scala.binary.version* values.
-
-Flink is developed against Scala *2.11* and tested additionally against Scala *2.10*. These two versions are known to be compatible. Earlier versions (like Scala *2.9*) are *not* compatible.
--- End diff --
Yes, I don't explicitly say what we're compatible with now. In fact we only support 2.11 now, supporting 2.12 is a bit harder than I though.
---
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/4807#discussion_r144325824
--- Diff: docs/start/building.md ---
@@ -115,19 +115,7 @@ The `-Pvendor-repos` activates a Maven [build profile](http://maven.apache.org/g
Flink has APIs, libraries, and runtime modules written in [Scala](http://scala-lang.org). Users of the Scala API and libraries may have to match the Scala version of Flink with the Scala version of their projects (because Scala is not strictly backwards compatible).
-**By default, Flink is built with the Scala 2.11**. To build Flink with Scala *2.10*, you can change the default Scala *binary version* by using *scala-2.10* build profile:
-
-~~~bash
-# Build with Scala version 2.10
-mvn clean install -DskipTests -Pscala-2.10
-~~~
-
-To build against custom Scala versions, you need to define new custom build profile that will override *scala.version* and *scala.binary.version* values.
-
-Flink is developed against Scala *2.11* and tested additionally against Scala *2.10*. These two versions are known to be compatible. Earlier versions (like Scala *2.9*) are *not* compatible.
-
-Newer versions may be compatible, depending on breaking changes in the language features used by Flink, and the availability of Flink's dependencies in those Scala versions. The dependencies written in Scala include for example *Kafka*, *Akka*, *Scalatest*, and *scopt*.
-
+**By default, Flink is built with the Scala 2.11**.
--- End diff --
fixing
---
[GitHub] flink pull request #4807: [FLINK-7810] Switch from custom Flakka to Akka 2.4...
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/4807#discussion_r144519154
--- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/taskmanager/TaskManagerComponentsStartupShutdownTest.java ---
@@ -202,8 +202,6 @@ protected void run() {
assertTrue(ioManager.isProperlyShutDown());
assertTrue(memManager.isShutdown());
} finally {
- TestingUtils.stopActorsGracefully(Arrays.asList(jobManager, taskManager));
--- End diff --
Is this now happening automatically?
---