You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by mbalassi <gi...@git.apache.org> on 2015/06/14 22:29:58 UTC

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

GitHub user mbalassi opened a pull request:

    https://github.com/apache/flink/pull/835

    [FLINK-2209] Document linking with jars not in the binary dist

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mbalassi/flink flink-2209

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/835.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #835
    
----
commit 5abcb8b90da9e20f1ca0b4da7fd7f139435c3e10
Author: mbalassi <mb...@apache.org>
Date:   2015-06-14T20:21:43Z

    [FLINK-2209] [docs] Document linking with jars not in the binary dist

commit c8d526da18ca0a882d9eb37ad9282c1152effb06
Author: mbalassi <mb...@apache.org>
Date:   2015-06-14T20:24:52Z

    [docs] Update obsolate cluster execution guide

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/835#discussion_r32404022
  
    --- Diff: docs/apis/cluster_execution.md ---
    @@ -80,67 +80,73 @@ Note that the program contains custom user code and hence requires a JAR file wi
     the classes of the code attached. The constructor of the remote environment
     takes the path(s) to the JAR file(s).
     
    -## Remote Executor
    +## Linking with modules not contained in the binary distribution
     
    -Similar to the RemoteEnvironment, the RemoteExecutor lets you execute
    -Flink programs on a cluster directly. The remote executor accepts a
    -*Plan* object, which describes the program as a single executable unit.
    +The binary distribution contains jar packages in the `lib` folder that are automatically
    +provided to the classpath of your distrbuted programs. Almost all of Flink classes are
    +located there with a few exceptions, for example the streaming connectors and some freshly
    +added modules. To run code depending on these modules you need to make them accessible
    +during runtime, for which we suggest two options:
     
    -### Maven Dependency
    -
    -If you are developing your program in a Maven project, you have to add the
    -`flink-clients` module using this dependency:
    -
    -~~~xml
    -<dependency>
    -  <groupId>org.apache.flink</groupId>
    -  <artifactId>flink-clients</artifactId>
    -  <version>{{ site.version }}</version>
    -</dependency>
    -~~~
    -
    -### Example
    -
    -The following illustrates the use of the `RemoteExecutor` with the Scala API:
    -
    -~~~scala
    -def main(args: Array[String]) {
    -    val input = TextFile("hdfs://path/to/file")
    +1. Either copy the required jar files to the `lib` folder onto all of your TaskManagers.
    +2. Or package them with your usercode.
     
    -    val words = input flatMap { _.toLowerCase().split("""\W+""") filter { _ != "" } }
    -    val counts = words groupBy { x => x } count()
    +The latter version is recommended as it respects the classloader management in Flink.
     
    -    val output = counts.write(wordsOutput, CsvOutputFormat())
    -  
    -    val plan = new ScalaPlan(Seq(output), "Word Count")
    -    val executor = new RemoteExecutor("strato-master", 7881, "/path/to/jarfile.jar")
    -    executor.executePlan(p);
    -}
    -~~~
    +### Packaging dependencies with your usercode with Maven
     
    -The following illustrates the use of the `RemoteExecutor` with the Java API (as
    -an alternative to the RemoteEnvironment):
    +To provide these dependencies not included by Flink we suggest two options with Maven.
     
    -~~~java
    -public static void main(String[] args) throws Exception {
    -    ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    +1. The maven assembly plugin builds a so called fat jar cointaining all your dependencies.
    +Easy to configure, but is an overkill in many cases. See 
    +[usage](http://maven.apache.org/plugins/maven-assembly-plugin/usage.html).
    +2. The maven unpack plugin, for unpacking the relevant parts of the dependencies and
    +then package it with your code.
     
    -    DataSet<String> data = env.readTextFile("hdfs://path/to/file");
    +To the the latter for example for the streaming Kafka connector, `flink-connector-kafka`
    --- End diff --
    
    Wording of the first sentence. Maybe something like: "Using the latter approach in order to bundle the Kafka connecter..."


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/flink/pull/835


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/835#discussion_r32403242
  
    --- Diff: docs/apis/cluster_execution.md ---
    @@ -80,67 +80,73 @@ Note that the program contains custom user code and hence requires a JAR file wi
     the classes of the code attached. The constructor of the remote environment
     takes the path(s) to the JAR file(s).
     
    -## Remote Executor
    +## Linking with modules not contained in the binary distribution
     
    -Similar to the RemoteEnvironment, the RemoteExecutor lets you execute
    -Flink programs on a cluster directly. The remote executor accepts a
    -*Plan* object, which describes the program as a single executable unit.
    +The binary distribution contains jar packages in the `lib` folder that are automatically
    +provided to the classpath of your distrbuted programs. Almost all of Flink classes are
    +located there with a few exceptions, for example the streaming connectors and some freshly
    +added modules. To run code depending on these modules you need to make them accessible
    +during runtime, for which we suggest two options:
     
    -### Maven Dependency
    -
    -If you are developing your program in a Maven project, you have to add the
    -`flink-clients` module using this dependency:
    -
    -~~~xml
    -<dependency>
    -  <groupId>org.apache.flink</groupId>
    -  <artifactId>flink-clients</artifactId>
    -  <version>{{ site.version }}</version>
    -</dependency>
    -~~~
    -
    -### Example
    -
    -The following illustrates the use of the `RemoteExecutor` with the Scala API:
    -
    -~~~scala
    -def main(args: Array[String]) {
    -    val input = TextFile("hdfs://path/to/file")
    +1. Either copy the required jar files to the `lib` folder onto all of your TaskManagers.
    --- End diff --
    
    Maybe add that the TMs have to be restarted.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on a diff in the pull request:

    https://github.com/apache/flink/pull/835#discussion_r32392995
  
    --- Diff: docs/apis/cluster_execution.md ---
    @@ -80,67 +80,73 @@ Note that the program contains custom user code and hence requires a JAR file wi
     the classes of the code attached. The constructor of the remote environment
     takes the path(s) to the JAR file(s).
     
    -## Remote Executor
    +## Linking with modules not contained in the binary distribution
     
    -Similar to the RemoteEnvironment, the RemoteExecutor lets you execute
    -Flink programs on a cluster directly. The remote executor accepts a
    -*Plan* object, which describes the program as a single executable unit.
    +The binary distribution contains jar packages in the `lib` folder that are automatically
    +provided to the classpath of your distrbuted programs. Almost all of Flink classes are
    +located there with a few exceptions, for example the streaming connectors and some freshly
    +added modules. To run code depending on these modules you need to make them accessible
    +during runtime, for which we suggest two options:
     
    -### Maven Dependency
    -
    -If you are developing your program in a Maven project, you have to add the
    -`flink-clients` module using this dependency:
    -
    -~~~xml
    -<dependency>
    -  <groupId>org.apache.flink</groupId>
    -  <artifactId>flink-clients</artifactId>
    -  <version>{{ site.version }}</version>
    -</dependency>
    -~~~
    -
    -### Example
    -
    -The following illustrates the use of the `RemoteExecutor` with the Scala API:
    -
    -~~~scala
    -def main(args: Array[String]) {
    -    val input = TextFile("hdfs://path/to/file")
    +1. Either copy the required jar files to the `lib` folder onto all of your TaskManagers.
    +2. Or package them with your usercode.
     
    -    val words = input flatMap { _.toLowerCase().split("""\W+""") filter { _ != "" } }
    -    val counts = words groupBy { x => x } count()
    +The latter version is recommended as it respects the classloader management in Flink.
     
    -    val output = counts.write(wordsOutput, CsvOutputFormat())
    -  
    -    val plan = new ScalaPlan(Seq(output), "Word Count")
    -    val executor = new RemoteExecutor("strato-master", 7881, "/path/to/jarfile.jar")
    -    executor.executePlan(p);
    -}
    -~~~
    +### Packaging dependencies with your usercode with Maven
     
    -The following illustrates the use of the `RemoteExecutor` with the Java API (as
    -an alternative to the RemoteEnvironment):
    +To provide these dependencies not included by Flink we suggest two options with Maven.
     
    -~~~java
    -public static void main(String[] args) throws Exception {
    -    ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    +1. The maven assembly plugin builds a so called fat jar cointaining all your dependencies.
    +Easy to configure, but is an overkill in many cases. See 
    +[usage](http://maven.apache.org/plugins/maven-assembly-plugin/usage.html).
    +2. The maven unpack plugin, for unpacking the relevant parts of the dependencies and
    +then package it with your code.
     
    -    DataSet<String> data = env.readTextFile("hdfs://path/to/file");
    +To the the latter for example for the streaming Kafka connector, `flink-connector-kafka`
    --- End diff --
    
    the the 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by mbalassi <gi...@git.apache.org>.
Github user mbalassi commented on a diff in the pull request:

    https://github.com/apache/flink/pull/835#discussion_r32403849
  
    --- Diff: docs/apis/cluster_execution.md ---
    @@ -80,67 +80,73 @@ Note that the program contains custom user code and hence requires a JAR file wi
     the classes of the code attached. The constructor of the remote environment
     takes the path(s) to the JAR file(s).
     
    -## Remote Executor
    +## Linking with modules not contained in the binary distribution
     
    -Similar to the RemoteEnvironment, the RemoteExecutor lets you execute
    -Flink programs on a cluster directly. The remote executor accepts a
    -*Plan* object, which describes the program as a single executable unit.
    +The binary distribution contains jar packages in the `lib` folder that are automatically
    +provided to the classpath of your distrbuted programs. Almost all of Flink classes are
    +located there with a few exceptions, for example the streaming connectors and some freshly
    +added modules. To run code depending on these modules you need to make them accessible
    +during runtime, for which we suggest two options:
     
    -### Maven Dependency
    -
    -If you are developing your program in a Maven project, you have to add the
    -`flink-clients` module using this dependency:
    -
    -~~~xml
    -<dependency>
    -  <groupId>org.apache.flink</groupId>
    -  <artifactId>flink-clients</artifactId>
    -  <version>{{ site.version }}</version>
    -</dependency>
    -~~~
    -
    -### Example
    -
    -The following illustrates the use of the `RemoteExecutor` with the Scala API:
    -
    -~~~scala
    -def main(args: Array[String]) {
    -    val input = TextFile("hdfs://path/to/file")
    +1. Either copy the required jar files to the `lib` folder onto all of your TaskManagers.
    --- End diff --
    
    Sure.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/835#discussion_r32403415
  
    --- Diff: docs/libs/gelly_guide.md ---
    @@ -43,6 +43,8 @@ Add the following dependency to your `pom.xml` to use Gelly.
     </dependency>
     ~~~
     
    +Note that the Gelly is currently not part of the binary distribution. See linking with it for cluster execution [here](../apis/cluster_execution.html#linking-with-modules-not-contained-in-the-binary-distribution).
    --- End diff --
    
    no the


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by mbalassi <gi...@git.apache.org>.
Github user mbalassi commented on the pull request:

    https://github.com/apache/flink/pull/835#issuecomment-114149626
  
    I do not think that we can expect that every user starts from the quickstart poms, but it is true that a reference to that would have been nicer. 
    
    I have to admit that I had the old version of quickstart poms in mind when writing this commit. That  excluded anything in the `org.apache.flink` namespace and was obviously not a good solution. Thanks for pointing this out, we should modify this guide then.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by mbalassi <gi...@git.apache.org>.
Github user mbalassi commented on a diff in the pull request:

    https://github.com/apache/flink/pull/835#discussion_r32403843
  
    --- Diff: docs/apis/cluster_execution.md ---
    @@ -80,67 +80,73 @@ Note that the program contains custom user code and hence requires a JAR file wi
     the classes of the code attached. The constructor of the remote environment
     takes the path(s) to the JAR file(s).
     
    -## Remote Executor
    +## Linking with modules not contained in the binary distribution
     
    -Similar to the RemoteEnvironment, the RemoteExecutor lets you execute
    -Flink programs on a cluster directly. The remote executor accepts a
    -*Plan* object, which describes the program as a single executable unit.
    +The binary distribution contains jar packages in the `lib` folder that are automatically
    +provided to the classpath of your distrbuted programs. Almost all of Flink classes are
    +located there with a few exceptions, for example the streaming connectors and some freshly
    +added modules. To run code depending on these modules you need to make them accessible
    --- End diff --
    
    The intention here was to write a sentence that does not have to be updated. We might want to add FlinkML to the binary later, and are introducing new modules frequently, e.g. Storm compatibility will be in the code shortly. I do not think that it is realistic to maintain a sentence here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the pull request:

    https://github.com/apache/flink/pull/835#issuecomment-114790171
  
    Can you take care of this @mbalassi ?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/835#discussion_r32403522
  
    --- Diff: docs/apis/cluster_execution.md ---
    @@ -80,67 +80,73 @@ Note that the program contains custom user code and hence requires a JAR file wi
     the classes of the code attached. The constructor of the remote environment
     takes the path(s) to the JAR file(s).
     
    -## Remote Executor
    +## Linking with modules not contained in the binary distribution
     
    -Similar to the RemoteEnvironment, the RemoteExecutor lets you execute
    -Flink programs on a cluster directly. The remote executor accepts a
    -*Plan* object, which describes the program as a single executable unit.
    +The binary distribution contains jar packages in the `lib` folder that are automatically
    +provided to the classpath of your distrbuted programs. Almost all of Flink classes are
    +located there with a few exceptions, for example the streaming connectors and some freshly
    +added modules. To run code depending on these modules you need to make them accessible
    --- End diff --
    
    Why listing the streaming connectors and not the other libraries? Maybe we can list/link all of them here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by mbalassi <gi...@git.apache.org>.
Github user mbalassi commented on the pull request:

    https://github.com/apache/flink/pull/835#issuecomment-112072013
  
    Thanks for the comments, I have addressed them. If no other objections I would like to merge this in the evening.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the pull request:

    https://github.com/apache/flink/pull/835#issuecomment-114143834
  
    I am a bit confused here. With the quickstart maven projects, fat jar building works without configuring, as it defines proper assembly and share configurations for fat jars.
    
    This description reads as if one needs to always use the maven unpack plugin when using any dependency not in `lib`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/835#discussion_r32403433
  
    --- Diff: docs/libs/ml/index.md ---
    @@ -69,6 +69,8 @@ Next, you have to add the FlinkML dependency to the `pom.xml` of your project.
     </dependency>
     {% endhighlight %}
     
    +Note that the FlinkML is currently not part of the binary distribution. See linking with it for cluster execution [here](../apis/cluster_execution.html#linking-with-modules-not-contained-in-the-binary-distribution).
    --- End diff --
    
    no the


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/835#discussion_r32403929
  
    --- Diff: docs/apis/cluster_execution.md ---
    @@ -80,67 +80,73 @@ Note that the program contains custom user code and hence requires a JAR file wi
     the classes of the code attached. The constructor of the remote environment
     takes the path(s) to the JAR file(s).
     
    -## Remote Executor
    +## Linking with modules not contained in the binary distribution
     
    -Similar to the RemoteEnvironment, the RemoteExecutor lets you execute
    -Flink programs on a cluster directly. The remote executor accepts a
    -*Plan* object, which describes the program as a single executable unit.
    +The binary distribution contains jar packages in the `lib` folder that are automatically
    +provided to the classpath of your distrbuted programs. Almost all of Flink classes are
    +located there with a few exceptions, for example the streaming connectors and some freshly
    +added modules. To run code depending on these modules you need to make them accessible
    +during runtime, for which we suggest two options:
     
    -### Maven Dependency
    -
    -If you are developing your program in a Maven project, you have to add the
    -`flink-clients` module using this dependency:
    -
    -~~~xml
    -<dependency>
    -  <groupId>org.apache.flink</groupId>
    -  <artifactId>flink-clients</artifactId>
    -  <version>{{ site.version }}</version>
    -</dependency>
    -~~~
    -
    -### Example
    -
    -The following illustrates the use of the `RemoteExecutor` with the Scala API:
    -
    -~~~scala
    -def main(args: Array[String]) {
    -    val input = TextFile("hdfs://path/to/file")
    +1. Either copy the required jar files to the `lib` folder onto all of your TaskManagers.
    +2. Or package them with your usercode.
     
    -    val words = input flatMap { _.toLowerCase().split("""\W+""") filter { _ != "" } }
    -    val counts = words groupBy { x => x } count()
    +The latter version is recommended as it respects the classloader management in Flink.
     
    -    val output = counts.write(wordsOutput, CsvOutputFormat())
    -  
    -    val plan = new ScalaPlan(Seq(output), "Word Count")
    -    val executor = new RemoteExecutor("strato-master", 7881, "/path/to/jarfile.jar")
    -    executor.executePlan(p);
    -}
    -~~~
    +### Packaging dependencies with your usercode with Maven
     
    -The following illustrates the use of the `RemoteExecutor` with the Java API (as
    -an alternative to the RemoteEnvironment):
    +To provide these dependencies not included by Flink we suggest two options with Maven.
     
    -~~~java
    -public static void main(String[] args) throws Exception {
    -    ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    +1. The maven assembly plugin builds a so called fat jar cointaining all your dependencies.
    +Easy to configure, but is an overkill in many cases. See 
    --- End diff --
    
    overkill is maybe a little bit too colloquial here


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request: [FLINK-2209] Document linking with jars not in...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/835#discussion_r32403908
  
    --- Diff: docs/apis/cluster_execution.md ---
    @@ -80,67 +80,73 @@ Note that the program contains custom user code and hence requires a JAR file wi
     the classes of the code attached. The constructor of the remote environment
     takes the path(s) to the JAR file(s).
     
    -## Remote Executor
    +## Linking with modules not contained in the binary distribution
     
    -Similar to the RemoteEnvironment, the RemoteExecutor lets you execute
    -Flink programs on a cluster directly. The remote executor accepts a
    -*Plan* object, which describes the program as a single executable unit.
    +The binary distribution contains jar packages in the `lib` folder that are automatically
    +provided to the classpath of your distrbuted programs. Almost all of Flink classes are
    +located there with a few exceptions, for example the streaming connectors and some freshly
    +added modules. To run code depending on these modules you need to make them accessible
    +during runtime, for which we suggest two options:
     
    -### Maven Dependency
    -
    -If you are developing your program in a Maven project, you have to add the
    -`flink-clients` module using this dependency:
    -
    -~~~xml
    -<dependency>
    -  <groupId>org.apache.flink</groupId>
    -  <artifactId>flink-clients</artifactId>
    -  <version>{{ site.version }}</version>
    -</dependency>
    -~~~
    -
    -### Example
    -
    -The following illustrates the use of the `RemoteExecutor` with the Scala API:
    -
    -~~~scala
    -def main(args: Array[String]) {
    -    val input = TextFile("hdfs://path/to/file")
    +1. Either copy the required jar files to the `lib` folder onto all of your TaskManagers.
    +2. Or package them with your usercode.
     
    -    val words = input flatMap { _.toLowerCase().split("""\W+""") filter { _ != "" } }
    -    val counts = words groupBy { x => x } count()
    +The latter version is recommended as it respects the classloader management in Flink.
     
    -    val output = counts.write(wordsOutput, CsvOutputFormat())
    -  
    -    val plan = new ScalaPlan(Seq(output), "Word Count")
    -    val executor = new RemoteExecutor("strato-master", 7881, "/path/to/jarfile.jar")
    -    executor.executePlan(p);
    -}
    -~~~
    +### Packaging dependencies with your usercode with Maven
     
    -The following illustrates the use of the `RemoteExecutor` with the Java API (as
    -an alternative to the RemoteEnvironment):
    +To provide these dependencies not included by Flink we suggest two options with Maven.
     
    -~~~java
    -public static void main(String[] args) throws Exception {
    -    ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    +1. The maven assembly plugin builds a so called fat jar cointaining all your dependencies.
    +Easy to configure, but is an overkill in many cases. See 
    --- End diff --
    
    There is no subject in the sentence.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---