You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by rmetzger <gi...@git.apache.org> on 2016/01/07 10:32:10 UTC
[GitHub] flink pull request: [FLINK-1712] Remove "flink-staging" module
GitHub user rmetzger opened a pull request:
https://github.com/apache/flink/pull/1492
[FLINK-1712] Remove "flink-staging" module
This PR is based on (and contains) https://github.com/apache/flink/pull/1482. I addressed all comments from #1482 and reworked the quickstart example (with new screenshots).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rmetzger/flink flink1712
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1492.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1492
----
commit 4793d63edfb15445b8333a9057c396a916ef4f17
Author: Stephan Ewen <se...@apache.org>
Date: 2015-12-23T19:17:56Z
[FLINK-3195] [examples] Consolidate batch examples into one project, unify batch and streaming examples under on parent project
commit c72ae164f4d9904582a6ad4b3966175ed6cbd5ef
Author: Robert Metzger <rm...@apache.org>
Date: 2016-01-06T16:37:26Z
[FLINK-3195] Rework quickstart example & some cleanups on the new examples
commit 924aa5b429774872e62fd981f2c036797aadd6f3
Author: Robert Metzger <rm...@apache.org>
Date: 2016-01-06T17:02:06Z
[FLINK-1712] Remove "flink-stagin" module
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1712] Remove "flink-staging" module
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/1492#issuecomment-170033507
Can you summarize what the changes from this pull request are? The set of changes is too large for github to display.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1712] Remove "flink-staging" module
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1492#discussion_r49589801
--- Diff: flink-examples/flink-examples-batch/pom.xml ---
@@ -280,208 +246,155 @@ under the License.
<!-- EnumTriangles Basic -->
<execution>
- <id>EnumTrianglesBasic</id>
+ <id>EnumerateGraphTriangles</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
- <classifier>EnumTrianglesBasic</classifier>
+ <classifier>EnumerateGraphTriangles</classifier>
--- End diff --
Rename to `EnumTriangles` to be in sync with class name.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1712] Remove "flink-staging" module
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/1492
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1712] Remove "flink-staging" module
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/1492#issuecomment-170869915
That seems like a good design.
+1 to merge from my side
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1712] Remove "flink-staging" module
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1492#issuecomment-170840541
I moved the following modules out of "flink-staging":
flink-tez (to flink-contrib)
flink-ml (to flink-libraries)
flink-table (to flink-libraries)
flink-fs-tests (to root)
flink-scala-shell (to root)
I renamed "flink-staging" to "flink-batch-connectors", which contains the following modules
-> flink-avro
-> flink-jdbc
-> flink-hadoop-compatibility
-> flink-hbase
-> flink-hcatalog
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1712] Remove "flink-staging" module
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1492#discussion_r49589855
--- Diff: flink-examples/flink-examples-batch/pom.xml ---
@@ -280,208 +246,155 @@ under the License.
<!-- EnumTriangles Basic -->
<execution>
- <id>EnumTrianglesBasic</id>
+ <id>EnumerateGraphTriangles</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
- <classifier>EnumTrianglesBasic</classifier>
+ <classifier>EnumerateGraphTriangles</classifier>
<archive>
<manifestEntries>
- <program-class>org.apache.flink.examples.scala.graph.EnumTrianglesBasic</program-class>
+ <program-class>org.apache.flink.examples.java.graph.EnumTriangles</program-class>
</manifestEntries>
</archive>
<includes>
- <include>**/scala/graph/EnumTrianglesBasic.class</include>
- <include>**/scala/graph/EnumTrianglesBasic$*.class</include>
+ <include>**/java/graph/EnumTriangles.class</include>
+ <include>**/java/graph/EnumTriangles$*.class</include>
+ <include>**/java/graph/util/EnumTrianglesDataTypes.class</include>
+ <include>**/java/graph/util/EnumTrianglesDataTypes$*.class</include>
<include>**/java/graph/util/EnumTrianglesData.class</include>
</includes>
</configuration>
</execution>
- <!-- EnumTriangles Opt -->
+ <!-- PageRank -->
<execution>
- <id>EnumTrianglesOpt</id>
+ <id>PageRank</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
- <classifier>EnumTrianglesOpt</classifier>
+ <classifier>PageRank</classifier>
<archive>
<manifestEntries>
- <program-class>org.apache.flink.examples.scala.graph.EnumTrianglesOpt</program-class>
+ <program-class>org.apache.flink.examples.java.graph.PageRank</program-class>
</manifestEntries>
</archive>
<includes>
- <include>**/scala/graph/EnumTrianglesOpt.class</include>
- <include>**/scala/graph/EnumTrianglesOpt$*.class</include>
- <include>**/java/graph/util/EnumTrianglesData.class</include>
+ <include>**/java/graph/PageRank.class</include>
+ <include>**/java/graph/PageRank$*.class</include>
+ <include>**/java/graph/util/PageRankData.class</include>
</includes>
</configuration>
</execution>
-
- <!-- PageRank Basic-->
+
+ <!-- WebLogAnalysis -->
<execution>
- <id>PageRankBasic</id>
+ <id>WebLogAnalysis</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
- <classifier>PageRankBasic</classifier>
+ <classifier>WebLogAnalysis</classifier>
<archive>
<manifestEntries>
- <program-class>org.apache.flink.examples.scala.graph.PageRankBasic</program-class>
+ <program-class>org.apache.flink.examples.java.relational.WebLogAnalysis</program-class>
</manifestEntries>
</archive>
<includes>
- <include>**/scala/graph/PageRankBasic.class</include>
- <include>**/scala/graph/PageRankBasic$*.class</include>
- <include>**/java/graph/util/PageRankData.class</include>
+ <include>**/java/relational/WebLogAnalysis.class</include>
+ <include>**/java/relational/WebLogAnalysis$*.class</include>
+ <include>**/java/relational/util/WebLogData.class</include>
+ <include>**/java/relational/util/WebLogDataGenerator.class</include>
</includes>
</configuration>
</execution>
-
- <!-- These queries are currently not self-contained -->
- <!-- TPC-H Query 10 -->
-
- <!--
+ <!-- WordCount -->
<execution>
- <id>TPCHQuery10</id>
+ <id>WordCount</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
- <classifier>TPCHQuery10</classifier>
+ <classifier>WordCount</classifier>
<archive>
<manifestEntries>
- <program-class>org.apache.flink.examples.scala.relational.TPCHQuery10</program-class>
+ <program-class>org.apache.flink.examples.java.wordcount.WordCount</program-class>
</manifestEntries>
</archive>
- <includes>
- <include>**/scala/relational/TPCHQuery10.class</include>
- <include>**/scala/relational/TPCHQuery10$*.class</include>
- </includes>
- </configuration>
- </execution> -->
-
- <!-- TPC-H Query 3 -->
- <!--
- <execution>
- <id>TPCHQuery3</id>
- <phase>package</phase>
- <goals>
- <goal>jar</goal>
- </goals>
- <configuration>
- <classifier>TPCHQuery3</classifier>
- <archive>
- <manifestEntries>
- <program-class>org.apache.flink.examples.scala.relational.TPCHQuery3</program-class>
- </manifestEntries>
- </archive>
<includes>
- <include>**/scala/relational/TPCHQuery3.class</include>
- <include>**/scala/relational/TPCHQuery3$*.class</include>
+ <include>**/java/wordcount/WordCount.class</include>
+ <include>**/java/wordcount/WordCount$*.class</include>
+ <include>**/java/wordcount/util/WordCountData.class</include>
</includes>
</configuration>
- </execution> -->
-
- <!-- WebLogAnalysis -->
+ </execution>
+
+ <!-- Distributed Copy -->
<execution>
- <id>WebLogAnalysis</id>
+ <id>DistCp</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
- <classifier>WebLogAnalysis</classifier>
+ <classifier>DistCp</classifier>
<archive>
<manifestEntries>
- <program-class>org.apache.flink.examples.scala.relational.WebLogAnalysis</program-class>
+ <program-class>org.apache.flink.examples.java.distcp.DistCp</program-class>
</manifestEntries>
</archive>
<includes>
- <include>**/scala/relational/WebLogAnalysis.class</include>
- <include>**/scala/relational/WebLogAnalysis$*.class</include>
- <include>**/java/relational/util/WebLogData.class</include>
- <include>**/java/relational/util/WebLogDataGenerator.class</include>
+ <include>**/java/distcp/*</include>
</includes>
</configuration>
</execution>
+ </executions>
+ </plugin>
- <!-- WordCount -->
+ <!--simplify the name of example JARs for build-target/examples -->
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-antrun-plugin</artifactId>
+ <version>1.7</version>
+ <executions>
<execution>
- <id>WordCount</id>
+ <id>rename</id>
<phase>package</phase>
<goals>
- <goal>jar</goal>
+ <goal>run</goal>
</goals>
- <configuration>
- <classifier>WordCount</classifier>
-
- <archive>
- <manifestEntries>
- <program-class>org.apache.flink.examples.scala.wordcount.WordCount</program-class>
- </manifestEntries>
- </archive>
-
- <includes>
- <include>**/scala/wordcount/WordCount.class</include>
- <include>**/scala/wordcount/WordCount$*.class</include>
- <include>**/java/wordcount/util/WordCountData.class</include>
- </includes>
+ <configuration>
+ <target>
+ <copy file="${project.basedir}/target/flink-examples-batch-${project.version}-KMeans.jar" tofile="${project.basedir}/target/KMeans.jar" />
+ <copy file="${project.basedir}/target/flink-examples-batch-${project.version}-ConnectedComponents.jar" tofile="${project.basedir}/target/ConnectedComponents.jar" />
+ <copy file="${project.basedir}/target/flink-examples-batch-${project.version}-EnumerateGraphTriangles.jar" tofile="${project.basedir}/target/EnumerateGraphTriangles.jar" />
--- End diff --
Rename to `EnumTriangles.jar` to be in sync with class name.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1712] Remove "flink-staging" module
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/1492#issuecomment-171596184
Thank you for the reviews ... I'll merge the PR
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1712] Remove "flink-staging" module
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/1492#issuecomment-171298711
Looks good to me, except for the name of the Triangle Enumeration example jar file.
Would be nice to have this merged soon.
I'd like to start working on putting the Table API on top of Apache Calcite.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-1712] Remove "flink-staging" module
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1492#discussion_r49589787
--- Diff: flink-examples/flink-examples-batch/pom.xml ---
@@ -280,208 +246,155 @@ under the License.
<!-- EnumTriangles Basic -->
<execution>
- <id>EnumTrianglesBasic</id>
+ <id>EnumerateGraphTriangles</id>
--- End diff --
Rename to `EnumTriangles` to be in sync with class name.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---