You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by jlist9 <jl...@gmail.com> on 2018/07/24 18:49:59 UTC

Recommended fat jar excludes?

We started out with a sample project from an earlier version of flink-java.
The sample project's pom.xml contained a long list of <exclude/> elements
for building the fat jar. The fat jar size is slightly over 100MB in our
case.

We are looking to upgrade to Flink 1.5 so we updated the pom.xml using one
generated with the maven command on Flink 1.5 quick start page:
https://ci.apache.org/projects/flink/flink-docs-release-1.5/quickstart/java_api_quickstart.html


The fat jar from the same code increases to over 200MB. I notice that the
new pom.xml file only has a very short <exclude/> list (4 lines). I wonder
if there is a recommended exclude list for Flink 1.5? I could use the
exclude list from the earlier version and do a bit of trial and error to
clean it up. If there is an updated list for 1.5, it'll probably save us a
lot time. File size isn't so much of an issue for storage. In our process
we copy the pipeline jar files so it'll take twice of the time before we
can start the job.

Thanks!
Jack

Re: Recommended fat jar excludes?

Posted by Chesnay Schepler <ch...@apache.org>.
The previous list exclude a number of dependencies to prevent clashes 
with Flink (for example netty)
which is no longer required.

If you could provide the output of "mvn dependency:tree" we might be 
able to figure out why the jar is larger.

On 24.07.2018 20:49, jlist9 wrote:
> We started out with a sample project from an earlier version of 
> flink-java. The sample project's pom.xml contained a long list of 
> <exclude/> elements for building the fat jar. The fat jar size is 
> slightly over 100MB in our case.
>
> We are looking to upgrade to Flink 1.5 so we updated the pom.xml using 
> one generated with the maven command on Flink 1.5 quick start page:
> https://ci.apache.org/projects/flink/flink-docs-release-1.5/quickstart/java_api_quickstart.html 
>
>
> The fat jar from the same code increases to over 200MB. I notice that 
> the new pom.xml file only has a very short <exclude/> list (4 lines). 
> I wonder if there is a recommended exclude list for Flink 1.5? I could 
> use the exclude list from the earlier version and do a bit of trial 
> and error to clean it up. If there is an updated list for 1.5, it'll 
> probably save us a lot time. File size isn't so much of an issue for 
> storage. In our process we copy the pipeline jar files so it'll take 
> twice of the time before we can start the job.
>
> Thanks!
> Jack