You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by Siyuan Hua <si...@datatorrent.com> on 2015/11/26 02:25:10 UTC
fastutil takes 17mb, can we exclude this jar?
I use archetype to build an empty apex app, it's about 20mb, but one jar
(fastutil) alone takes 17mb. I use mvn dependency:tree and find out
malhar-library directly depends on that, what is it used for? can we
exclude that or is there any thin replacement?
Thanks
Siyuan
Re: fastutil takes 17mb, can we exclude this jar?
Posted by Timothy Farkas <ti...@datatorrent.com>.
Thanks Chinmay,
There are also some maven plugin alternatives which can do something
similar except they don't auto discover the classes that need to be pulled
out of the jar.
http://stackoverflow.com/questions/13142511/unpack-dependency-and-repack-classes-using-maven
We are stuck with fast util until the next major version of malhar since it
has creeped into the signatures of some methods. For now I can manually use
auto jar to discover the classes to repack, and then list those classes to
be repacked for the fastutil dependency in the maven shade plugin. An
example of this can be included in the demos, so if someone runs into an
issue with large app packages they will have cut and paste modification to
their pom which will resolve it. The long term solution will be to replace
fast util with another library, I think the best contender for that now is
mahout.
On Fri, Nov 27, 2015 at 10:39 PM, Chinmay Kolhatkar <chinmay@datatorrent.com
> wrote:
> How about using AutoJar?
> http:// <http://autojar.sourceforge.net/en_d/index.html>
> autojar.sourceforge.net <http://autojar.sourceforge.net/en_d/index.html>/
> <http://autojar.sourceforge.net/en_d/index.html>en
> <http://autojar.sourceforge.net/en_d/index.html>_d/
> <http://autojar.sourceforge.net/en_d/index.html>index.html
> <http://autojar.sourceforge.net/en_d/index.html>
>
> From the looks of it, this seems go pick only those classes which are
> required.
>
> Fastutils also mentions about it to deal with the problem of large jar file
> size:
> http://fastutil.di.unimi.it/#install
>
> -Chinmay.
> In spite of the qualities of Mahout's collections, my guess is that
> fastutil will be more developed.
>
>
>
> On Fri, Nov 27, 2015 at 6:09 AM, Timothy Farkas <ti...@datatorrent.com>
> wrote:
>
> > Thanks Ted for providing an alternative! I will look into replacing
> > fastutil with it.
> >
> > On Thu, Nov 26, 2015 at 3:09 AM, Ted Dunning <te...@gmail.com>
> > wrote:
> >
> > > On Thu, Nov 26, 2015 at 5:46 PM, Timothy Farkas <ti...@datatorrent.com>
> > > wrote:
> > >
> > > > There are not very many
> > > > replacement options the only Apache license alternative is hppc which
> I
> > > > think is underdeveloped.
> > > >
> > >
> > > Mahout Collections as well.
> > >
> >
>
Re: fastutil takes 17mb, can we exclude this jar?
Posted by Chinmay Kolhatkar <ch...@datatorrent.com>.
How about using AutoJar?
http:// <http://autojar.sourceforge.net/en_d/index.html>
autojar.sourceforge.net <http://autojar.sourceforge.net/en_d/index.html>/
<http://autojar.sourceforge.net/en_d/index.html>en
<http://autojar.sourceforge.net/en_d/index.html>_d/
<http://autojar.sourceforge.net/en_d/index.html>index.html
<http://autojar.sourceforge.net/en_d/index.html>
>From the looks of it, this seems go pick only those classes which are
required.
Fastutils also mentions about it to deal with the problem of large jar file
size:
http://fastutil.di.unimi.it/#install
-Chinmay.
In spite of the qualities of Mahout's collections, my guess is that
fastutil will be more developed.
On Fri, Nov 27, 2015 at 6:09 AM, Timothy Farkas <ti...@datatorrent.com> wrote:
> Thanks Ted for providing an alternative! I will look into replacing
> fastutil with it.
>
> On Thu, Nov 26, 2015 at 3:09 AM, Ted Dunning <te...@gmail.com>
> wrote:
>
> > On Thu, Nov 26, 2015 at 5:46 PM, Timothy Farkas <ti...@datatorrent.com>
> > wrote:
> >
> > > There are not very many
> > > replacement options the only Apache license alternative is hppc which
I
> > > think is underdeveloped.
> > >
> >
> > Mahout Collections as well.
> >
>
Re: fastutil takes 17mb, can we exclude this jar?
Posted by Ted Dunning <te...@gmail.com>.
In spite of the qualities of Mahout's collections, my guess is that
fastutil will be more developed.
On Fri, Nov 27, 2015 at 6:09 AM, Timothy Farkas <ti...@datatorrent.com> wrote:
> Thanks Ted for providing an alternative! I will look into replacing
> fastutil with it.
>
> On Thu, Nov 26, 2015 at 3:09 AM, Ted Dunning <te...@gmail.com>
> wrote:
>
> > On Thu, Nov 26, 2015 at 5:46 PM, Timothy Farkas <ti...@datatorrent.com>
> > wrote:
> >
> > > There are not very many
> > > replacement options the only Apache license alternative is hppc which I
> > > think is underdeveloped.
> > >
> >
> > Mahout Collections as well.
> >
>
Re: fastutil takes 17mb, can we exclude this jar?
Posted by Timothy Farkas <ti...@datatorrent.com>.
Thanks Ted for providing an alternative! I will look into replacing
fastutil with it.
On Thu, Nov 26, 2015 at 3:09 AM, Ted Dunning <te...@gmail.com> wrote:
> On Thu, Nov 26, 2015 at 5:46 PM, Timothy Farkas <ti...@datatorrent.com>
> wrote:
>
> > There are not very many
> > replacement options the only Apache license alternative is hppc which I
> > think is underdeveloped.
> >
>
> Mahout Collections as well.
>
Re: fastutil takes 17mb, can we exclude this jar?
Posted by Ted Dunning <te...@gmail.com>.
On Thu, Nov 26, 2015 at 5:46 PM, Timothy Farkas <ti...@datatorrent.com> wrote:
> There are not very many
> replacement options the only Apache license alternative is hppc which I
> think is underdeveloped.
>
Mahout Collections as well.
Re: fastutil takes 17mb, can we exclude this jar?
Posted by Timothy Farkas <ti...@datatorrent.com>.
That is my doing. It is used by app data for primitive collections. If you
are not using app data you could exclude it. There are not very many
replacement options the only Apache license alternative is hppc which I
think is underdeveloped.
On Wed, Nov 25, 2015 at 5:25 PM, Siyuan Hua <si...@datatorrent.com> wrote:
> I use archetype to build an empty apex app, it's about 20mb, but one jar
> (fastutil) alone takes 17mb. I use mvn dependency:tree and find out
> malhar-library directly depends on that, what is it used for? can we
> exclude that or is there any thin replacement?
>
> Thanks
>
> Siyuan
>