You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by nk...@apache.org on 2019/01/16 09:29:29 UTC
[flink] branch master updated (ea0f283 -> bfbf986)
This is an automated email from the ASF dual-hosted git repository.
nkruber pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git.
from ea0f283 [FLINK-11331][table][docs] Fix errors in tableApi.md and functions.md
new ecb57a9 [hotfix][docs] fix some typos
new bfd1302 [FLINK-9222][docs] add documentation for setting up Gradle projects
new bfbf986 [hotfix][quickstart] make the note about IDE memory requirements non-specific to MacOS X users
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/_config.yml | 2 +
docs/dev/projectsetup/java_api_quickstart.md | 234 ++++++++++++++++++++++++--
docs/dev/projectsetup/scala_api_quickstart.md | 2 +-
3 files changed, 226 insertions(+), 12 deletions(-)
[flink] 03/03: [hotfix][quickstart] make the note about IDE memory
requirements non-specific to MacOS X users
Posted by nk...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nkruber pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
commit bfbf98697bc4dacab5b689fcbdbfcb080b331801
Author: Nico Kruber <ni...@data-artisans.com>
AuthorDate: Tue Jul 10 13:44:41 2018 +0200
[hotfix][quickstart] make the note about IDE memory requirements non-specific to MacOS X users
---
docs/dev/projectsetup/java_api_quickstart.md | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/docs/dev/projectsetup/java_api_quickstart.md b/docs/dev/projectsetup/java_api_quickstart.md
index 91ef70e..fd2eef5 100644
--- a/docs/dev/projectsetup/java_api_quickstart.md
+++ b/docs/dev/projectsetup/java_api_quickstart.md
@@ -111,11 +111,9 @@ allows to [import Maven projects](http://books.sonatype.com/m2eclipse-book/refer
Some Eclipse bundles include that plugin by default, others require you
to install it manually.
-*A note to Mac OS X users*: The default JVM heapsize for Java may be too
+*Please note*: The default JVM heapsize for Java may be too
small for Flink. You have to manually increase it.
-In Eclipse, choose
-`Run Configurations -> Arguments` and write into the `VM Arguments`
-box: `-Xmx800m`.
+In Eclipse, choose `Run Configurations -> Arguments` and write into the `VM Arguments` box: `-Xmx800m`.
In IntelliJ IDEA recommended way to change JVM options is from the `Help | Edit Custom VM Options` menu. See [this article](https://intellij-support.jetbrains.com/hc/en-us/articles/206544869-Configuring-JVM-options-and-platform-properties) for details.
### Build Project
@@ -317,10 +315,10 @@ You may also use [Gradle's IDE integration](https://docs.gradle.org/current/user
to create project files from Gradle.
-*A note to Mac OS X users*: The default JVM heapsize for Java may be too
-small for Flink. You have to manually increase it. In Eclipse, choose
-`Run Configurations -> Arguments` and write into the `VM Arguments`
-box: `-Xmx800m`.
+*Please note*: The default JVM heapsize for Java may be too
+small for Flink. You have to manually increase it.
+In Eclipse, choose `Run Configurations -> Arguments` and write into the `VM Arguments` box: `-Xmx800m`.
+In IntelliJ IDEA recommended way to change JVM options is from the `Help | Edit Custom VM Options` menu. See [this article](https://intellij-support.jetbrains.com/hc/en-us/articles/206544869-Configuring-JVM-options-and-platform-properties) for details.
### Build Project
[flink] 02/03: [FLINK-9222][docs] add documentation for setting up
Gradle projects
Posted by nk...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nkruber pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
commit bfd13024cab0a561fe3246845e6e0901d7a04cd4
Author: Nico Kruber <ni...@data-artisans.com>
AuthorDate: Mon Apr 23 18:58:28 2018 +0200
[FLINK-9222][docs] add documentation for setting up Gradle projects
---
docs/_config.yml | 2 +
docs/dev/projectsetup/java_api_quickstart.md | 228 ++++++++++++++++++++++++++-
2 files changed, 223 insertions(+), 7 deletions(-)
diff --git a/docs/_config.yml b/docs/_config.yml
index aea41bd..5a83c9a 100644
--- a/docs/_config.yml
+++ b/docs/_config.yml
@@ -34,6 +34,8 @@ version_title: "1.8-SNAPSHOT"
# Branch on Github for this version
github_branch: "master"
+# Plain Scala version is needed for e.g. the Gradle quickstart.
+scala_version: "2.11"
# This suffix is appended to the Scala-dependent Maven artifact names
scala_version_suffix: "_2.11"
diff --git a/docs/dev/projectsetup/java_api_quickstart.md b/docs/dev/projectsetup/java_api_quickstart.md
index 03d75c1..91ef70e 100644
--- a/docs/dev/projectsetup/java_api_quickstart.md
+++ b/docs/dev/projectsetup/java_api_quickstart.md
@@ -26,14 +26,24 @@ under the License.
* This will be replaced by the TOC
{:toc}
-Start working on your Flink Java program in a few simple steps.
+## Build Tools
-## Requirements
+Flink projects can be built with different build tools.
+In order to get started quickly, Flink provides project templates for the following build tools:
+
+- [Maven](#maven)
+- [Gradle](#gradle)
+
+These templates help you to set up the project structure and to create the initial build files.
+
+## Maven
+
+### Requirements
The only requirements are working __Maven 3.0.4__ (or higher) and __Java 8.x__ installations.
-## Create Project
+### Create Project
Use one of the following commands to __create a project__:
@@ -69,7 +79,7 @@ Use one of the following commands to __create a project__:
{% endunless %}
</div>
-## Inspect Project
+### Inspect Project
There will be a new directory in your working directory. If you've used
the _curl_ approach, the directory is called `quickstart`. Otherwise,
@@ -108,7 +118,7 @@ In Eclipse, choose
box: `-Xmx800m`.
In IntelliJ IDEA recommended way to change JVM options is from the `Help | Edit Custom VM Options` menu. See [this article](https://intellij-support.jetbrains.com/hc/en-us/articles/206544869-Configuring-JVM-options-and-platform-properties) for details.
-## Build Project
+### Build Project
If you want to __build/package your project__, go to your project directory and
run the '`mvn clean package`' command.
@@ -116,8 +126,212 @@ You will __find a JAR file__ that contains your application, plus connectors and
that you may have added as dependencies to the application: `target/<artifact-id>-<version>.jar`.
__Note:__ If you use a different class than *StreamingJob* as the application's main class / entry point,
-we recommend you change the `mainClass` setting in the `pom.xml` file accordingly. That way, the Flink
-can run time application from the JAR file without additionally specifying the main class.
+we recommend you change the `mainClass` setting in the `pom.xml` file accordingly. That way, Flink
+can run the application from the JAR file without additionally specifying the main class.
+
+## Gradle
+
+### Requirements
+
+The only requirements are working __Gradle 3.x__ (or higher) and __Java 8.x__ installations.
+
+### Create Project
+
+Use one of the following commands to __create a project__:
+
+<ul class="nav nav-tabs" style="border-bottom: none;">
+ <li class="active"><a href="#gradle-example" data-toggle="tab"><strong>Gradle example</strong></a></li>
+ <li><a href="#gradle-script" data-toggle="tab">Run the <strong>quickstart script</strong></a></li>
+</ul>
+<div class="tab-content">
+ <div class="tab-pane active" id="gradle-example">
+
+ <ul class="nav nav-tabs" style="border-bottom: none;">
+ <li class="active"><a href="#gradle-build" data-toggle="tab"><tt>build.gradle</tt></a></li>
+ <li><a href="#gradle-settings" data-toggle="tab"><tt>settings.gradle</tt></a></li>
+ </ul>
+ <div class="tab-content">
+<!-- NOTE: Any change to the build scripts here should also be reflected in flink-web/q/gradle-quickstart.sh !! -->
+ <div class="tab-pane active" id="gradle-build">
+ {% highlight gradle %}
+buildscript {
+ repositories {
+ jcenter() // this applies only to the Gradle 'Shadow' plugin
+ }
+ dependencies {
+ classpath 'com.github.jengelman.gradle.plugins:shadow:2.0.4'
+ }
+}
+
+plugins {
+ id 'java'
+ id 'application'
+ // shadow plugin to produce fat JARs
+ id 'com.github.johnrengelman.shadow' version '2.0.4'
+}
+
+
+// artifact properties
+group = 'org.myorg.quickstart'
+version = '0.1-SNAPSHOT'
+mainClassName = 'org.myorg.quickstart.StreamingJob'
+description = """Flink Quickstart Job"""
+
+ext {
+ javaVersion = '1.8'
+ flinkVersion = '{{ site.version }}'
+ scalaBinaryVersion = '{{ site.scala_version }}'
+ slf4jVersion = '1.7.7'
+ log4jVersion = '1.2.17'
+}
+
+
+sourceCompatibility = javaVersion
+targetCompatibility = javaVersion
+tasks.withType(JavaCompile) {
+ options.encoding = 'UTF-8'
+}
+
+applicationDefaultJvmArgs = ["-Dlog4j.configuration=log4j.properties"]
+
+task wrapper(type: Wrapper) {
+ gradleVersion = '3.1'
+}
+
+// declare where to find the dependencies of your project
+repositories {
+ mavenCentral()
+ maven { url "https://repository.apache.org/content/repositories/snapshots/" }
+}
+
+// NOTE: We cannot use "compileOnly" or "shadow" configurations since then we could not run code
+// in the IDE or with "gradle run". We also cannot exclude transitive dependencies from the
+// shadowJar yet (see https://github.com/johnrengelman/shadow/issues/159).
+// -> Explicitly define the // libraries we want to be included in the "flinkShadowJar" configuration!
+configurations {
+ flinkShadowJar // dependencies which go into the shadowJar
+
+ // always exclude these (also from transitive dependencies) since they are provided by Flink
+ flinkShadowJar.exclude group: 'org.apache.flink', module: 'force-shading'
+ flinkShadowJar.exclude group: 'com.google.code.findbugs', module: 'jsr305'
+ flinkShadowJar.exclude group: 'org.slf4j'
+ flinkShadowJar.exclude group: 'log4j'
+}
+
+// declare the dependencies for your production and test code
+dependencies {
+ // --------------------------------------------------------------
+ // Compile-time dependencies that should NOT be part of the
+ // shadow jar and are provided in the lib folder of Flink
+ // --------------------------------------------------------------
+ compile "org.apache.flink:flink-java:${flinkVersion}"
+ compile "org.apache.flink:flink-streaming-java_${scalaBinaryVersion}:${flinkVersion}"
+
+ // --------------------------------------------------------------
+ // Dependencies that should be part of the shadow jar, e.g.
+ // connectors. These must be in the flinkShadowJar configuration!
+ // --------------------------------------------------------------
+ //flinkShadowJar "org.apache.flink:flink-connector-kafka-0.11_${scalaBinaryVersion}:${flinkVersion}"
+
+ compile "log4j:log4j:${log4jVersion}"
+ compile "org.slf4j:slf4j-log4j12:${slf4jVersion}"
+
+ // Add test dependencies here.
+ // testCompile "junit:junit:4.12"
+}
+
+// make compileOnly dependencies available for tests:
+sourceSets {
+ main.compileClasspath += configurations.flinkShadowJar
+ main.runtimeClasspath += configurations.flinkShadowJar
+
+ test.compileClasspath += configurations.flinkShadowJar
+ test.runtimeClasspath += configurations.flinkShadowJar
+
+ javadoc.classpath += configurations.flinkShadowJar
+}
+
+run.classpath = sourceSets.main.runtimeClasspath
+
+jar {
+ manifest {
+ attributes 'Built-By': System.getProperty('user.name'),
+ 'Build-Jdk': System.getProperty('java.version')
+ }
+}
+
+shadowJar {
+ configurations = [project.configurations.flinkShadowJar]
+}
+ {% endhighlight %}
+ </div>
+ <div class="tab-pane" id="gradle-settings">
+ {% highlight gradle %}
+rootProject.name = 'quickstart'
+ {% endhighlight %}
+ </div>
+ </div>
+ </div>
+
+ <div class="tab-pane" id="gradle-script">
+ {% highlight bash %}
+ bash -c "$(curl https://flink.apache.org/q/gradle-quickstart.sh)" -- {{site.version}} {{site.scala_version}}
+ {% endhighlight %}
+ This allows you to <strong>name your newly created project</strong>. It will interactively ask
+ you for the project name, organization (also used for the package name), project version,
+ Scala and Flink version.
+ </div>
+</div>
+
+### Inspect Project
+
+There will be a new directory in your working directory based on the
+project name you provided, e.g. for `quickstart`:
+
+{% highlight bash %}
+$ tree quickstart/
+quickstart/
+├── README
+├── build.gradle
+├── settings.gradle
+└── src
+ └── main
+ ├── java
+ │ └── org
+ │ └── myorg
+ │ └── quickstart
+ │ ├── BatchJob.java
+ │ └── StreamingJob.java
+ └── resources
+ └── log4j.properties
+{% endhighlight %}
+
+The sample project is a __Gradle project__, which contains two classes: _StreamingJob_ and _BatchJob_ are the basic skeleton programs for a *DataStream* and *DataSet* program.
+The _main_ method is the entry point of the program, both for in-IDE testing/execution and for proper deployments.
+
+We recommend you __import this project into your IDE__ to develop and
+test it. IntelliJ IDEA supports Gradle projects after installing the `Gradle` plugin.
+Eclipse does so via the [Eclipse Buildship](https://projects.eclipse.org/projects/tools.buildship) plugin
+(make sure to specify a Gradle version >= 3.0 in the last step of the import wizard; the `shadow` plugin requires it).
+You may also use [Gradle's IDE integration](https://docs.gradle.org/current/userguide/userguide.html#ide-integration)
+to create project files from Gradle.
+
+
+*A note to Mac OS X users*: The default JVM heapsize for Java may be too
+small for Flink. You have to manually increase it. In Eclipse, choose
+`Run Configurations -> Arguments` and write into the `VM Arguments`
+box: `-Xmx800m`.
+
+### Build Project
+
+If you want to __build/package your project__, go to your project directory and
+run the '`gradle clean shadowJar`' command.
+You will __find a JAR file__ that contains your application, plus connectors and libraries
+that you may have added as dependencies to the application: `build/libs/<project-name>-<version>-all.jar`.
+
+__Note:__ If you use a different class than *StreamingJob* as the application's main class / entry point,
+we recommend you change the `mainClassName` setting in the `build.gradle` file accordingly. That way, Flink
+can run the application from the JAR file without additionally specifying the main class.
## Next Steps
[flink] 01/03: [hotfix][docs] fix some typos
Posted by nk...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nkruber pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
commit ecb57a913e3284e96ddec29811d5836187294c5a
Author: Nico Kruber <ni...@data-artisans.com>
AuthorDate: Mon Apr 23 18:58:15 2018 +0200
[hotfix][docs] fix some typos
---
docs/dev/projectsetup/scala_api_quickstart.md | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docs/dev/projectsetup/scala_api_quickstart.md b/docs/dev/projectsetup/scala_api_quickstart.md
index bcb4045..da38eb7 100644
--- a/docs/dev/projectsetup/scala_api_quickstart.md
+++ b/docs/dev/projectsetup/scala_api_quickstart.md
@@ -30,7 +30,7 @@ under the License.
## Build Tools
Flink projects can be built with different build tools.
-In order to get quickly started, Flink provides project templates for the following build tools:
+In order to get started quickly, Flink provides project templates for the following build tools:
- [SBT](#sbt)
- [Maven](#maven)