You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by StephanEwen <gi...@git.apache.org> on 2015/12/31 18:56:17 UTC
[GitHub] flink pull request: [FLINK-3195] [examples] Consolidate Java/Scala...
GitHub user StephanEwen opened a pull request:
https://github.com/apache/flink/pull/1482
[FLINK-3195] [examples] Consolidate Java/Scala and batch/streaming examples
This pull request consolidates the examples:
- One root project `flink-examples`
- Subprojects `flink-examples-batch` and `flink-examples-streaming`
- Batch examples are put into the distribution under `examples/batch`
- Streaming examples are now also put into the distribution under `examples/streaming`
- Java / Scala examples are always in one project. Since Java code never references Scala code, this should be okay, even for Eclipse
- Removed redundant jars, such as `WordCoundPOJO`, and `EnumTrianglesOpt`. Since the JARs do not add any visible difference, they are not necessary and only confusing.
- Rename `PageRankBasic` to `PageRank`
Open question: Moving the final example jars to `examples/batch` breaks a few web tutorials. Since these tutorials usually mention a Flink version (and this is simple to figure out for users), I think it is okay.
An alternative is to have all examples in the `/examples` folder and changing the names to make them unique, for example `WordCount.jar` and `StreamingWordCount.jar`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink examples
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1482.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1482
----
commit 336494a5cdac48d8f905082864a2d183d9f1dc22
Author: Stephan Ewen <se...@apache.org>
Date: 2015-12-23T19:17:56Z
[FLINK-3195] [examples] Consolidate batch examples into one project, unify batch and streaming examples under on parent project
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-3195] [examples] Consolidate Java/Scala...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1482#discussion_r48831257
--- Diff: flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/ml/util/LinearRegressionData.java ---
@@ -20,8 +20,7 @@
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
-import org.apache.flink.examples.java.ml.LinearRegression.Data;
--- End diff --
Re-add these imports?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-3195] [examples] Consolidate Java/Scala...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1482#discussion_r48831214
--- Diff: flink-dist/pom.xml ---
@@ -85,13 +85,7 @@ under the License.
<dependency>
<groupId>org.apache.flink</groupId>
- <artifactId>flink-java-examples</artifactId>
- <version>${project.version}</version>
- </dependency>
-
- <dependency>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-scala-examples</artifactId>
+ <artifactId>flink-examples-batch</artifactId>
<version>${project.version}</version>
</dependency>
--- End diff --
I think we need to add `flink-examples-streaming` as well, no?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-3195] [examples] Consolidate Java/Scala...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/1482#issuecomment-168966850
I like the restructuring. I am also fine with remove redundant examples. However, I think we should either not rename the triangle enumeration jar to `EnumerateGraphTriangles.jar` or also rename the source file. Otherwise it is not clear which code is contained in the jar. I would also be OK with removing the EnumTriangleOpt example completely.
I think it is OK to move the batch example to a subfolder. They should be rather easy to find.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-3195] [examples] Consolidate Java/Scala...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1482#discussion_r48831157
--- Diff: flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/graph/EnumTrianglesOpt.java ---
@@ -18,25 +18,23 @@
package org.apache.flink.examples.java.graph;
-import java.util.ArrayList;
-import java.util.Iterator;
-import java.util.List;
-
import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.common.functions.GroupReduceFunction;
import org.apache.flink.api.common.functions.JoinFunction;
import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.api.common.functions.ReduceFunction;
import org.apache.flink.api.common.operators.Order;
-import org.apache.flink.api.java.functions.FunctionAnnotation.ForwardedFields;
-import org.apache.flink.api.java.tuple.Tuple2;
-import org.apache.flink.util.Collector;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
+import org.apache.flink.api.java.functions.FunctionAnnotation.ForwardedFields;
+import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.examples.java.graph.util.EnumTrianglesData;
-import org.apache.flink.examples.java.graph.util.EnumTrianglesDataTypes.Edge;
--- End diff --
`Edge`, `EdgeWithDegrees`, and `Triad` should be added back to the imports to make the code more readable.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-3195] [examples] Consolidate Java/Scala...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/1482
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-3195] [examples] Consolidate Java/Scala...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1482#issuecomment-169362242
I'd like to finish the project restructure and I'll use this PR as a base. I'll also address Fabian's comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-3195] [examples] Consolidate Java/Scala...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1482#discussion_r48831304
--- Diff: flink-examples/flink-examples-batch/src/main/java/org/apache/flink/examples/java/graph/util/EnumTrianglesData.java ---
@@ -24,7 +24,6 @@
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
-import org.apache.flink.examples.java.graph.util.EnumTrianglesDataTypes.Edge;
--- End diff --
Re-add this import?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---