You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by rmetzger <gi...@git.apache.org> on 2017/02/13 19:51:06 UTC
[GitHub] flink pull request #3300: [FLINK-5690][docs] Add note on shading to best pra...
GitHub user rmetzger opened a pull request:
https://github.com/apache/flink/pull/3300
[FLINK-5690][docs] Add note on shading to best practices guide
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rmetzger/flink flink5690
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3300.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3300
----
commit 504cf17d28932921da9b27ea924d777107b248ed
Author: Robert Metzger <rm...@apache.org>
Date: 2017-02-13T19:50:23Z
[FLINK-5690][docs] Add note on shading to best practices guide
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3300: [FLINK-5690][docs] Add note on shading to best pra...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/3300
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3300: [FLINK-5690][docs] Add note on shading to best pra...
Posted by alpinegizmo <gi...@git.apache.org>.
Github user alpinegizmo commented on a diff in the pull request:
https://github.com/apache/flink/pull/3300#discussion_r101038332
--- Diff: docs/dev/best_practices.md ---
@@ -310,3 +310,17 @@ Next, you need to put the following jar files into the `lib/` folder:
Note that you need to explicitly set the `lib/` directory when using a per job YARN cluster.
The command to submit Flink on YARN with a custom logger is: `./bin/flink run -yt $FLINK_HOME/lib <... remaining arguments ...>`
+
+
+## Resolving Dependency Conflicts with Flink using the maven-shade-plugin.
+
+Apache Flink loads many classes by default into its classpath (the [classloading is described here in detail]({{site.baseurl}}/monitoring/debugging_classloading.html)). If a user uses a different version of a library that Flink is using, often `IllegalAccessExceptions` or `NoSuchMethodError` are the result.
+
+Through Hadoop, Flink for example depends on the `aws-sdk` library or on `protobuf-java`. If your user code is using these libraries and you run into issues we recommend relocating the dependency in your user code jar.
+
+Apache Maven offers the [maven-shade-plugin](https://maven.apache.org/plugins/maven-shade-plugin/), that allows to change the package of a class *after* compiling it (so the code you are writing is not affected by the shading). For example if you have the `com.amazonaws` packages from the aws sdk in your user code jar, the shade plugin would relocate them into the `org.myorg.shaded.com.amazonaws` package, so that your code is calling your aws sdk version.
--- End diff --
I suggest a minor rewording:
Apache Maven offers the [maven-shade-plugin](https://maven.apache.org/plugins/maven-shade-plugin/), which allows one to change the package of a class *after* compiling it ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3300: [FLINK-5690][docs] Add note on shading to best pra...
Posted by uce <gi...@git.apache.org>.
Github user uce commented on a diff in the pull request:
https://github.com/apache/flink/pull/3300#discussion_r100977226
--- Diff: docs/dev/best_practices.md ---
@@ -310,3 +310,17 @@ Next, you need to put the following jar files into the `lib/` folder:
Note that you need to explicitly set the `lib/` directory when using a per job YARN cluster.
The command to submit Flink on YARN with a custom logger is: `./bin/flink run -yt $FLINK_HOME/lib <... remaining arguments ...>`
+
+
+## Resolving Dependency Conflicts with Flink using the maven-shade-plugin.
--- End diff --
I think this is better for the debugging section
https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/debugging_classloading.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3300: [FLINK-5690][docs] Add note on shading to best practices ...
Posted by rmetzger <gi...@git.apache.org>.
Github user rmetzger commented on the issue:
https://github.com/apache/flink/pull/3300
Thank you for your feedback. I've addressed both concerns and will merge the change now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---