You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Gustavo Beneitez <gu...@gmail.com> on 2018/09/17 10:23:26 UTC
High performance issues with Elastic Connector
Hi there,
I was indexing in production several intranet websites and I realized the
performance was becoming poor (no tomcat restart for two weeks).
After a few thoughts about what is going on I decided to create a dump
file and review it with eclipse MAT and I observed that the class
Leak Suspects
*org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnection*
had a lot of instances and was growing up and never released. Do you have
any issue related to Elastic? Maybe there is something related to our
version (2.4.6) of Elastic.
I also have attached the MAT report. We have all components "by default",
i.e. neither no customization of any component nor any new one created.
Thanks in advance.
Re: High performance issues with Elastic Connector
Posted by Gustavo Beneitez <gu...@gmail.com>.
also, here are the top instances for each object, after a few navigation
they seem somehow retained by the connector
Class Name
Objects Shallow
Heap Retained Heap
-----------------------------------------------------------------------------------------------------------
char[] 75.233
197.316.528 >= 197.316.528
byte[] 7.842
9.589.952 >= 9.589.952
java.lang.String 73.108
1.754.592 >= 65.437.048
java.util.HashMap$Node 40.294
1.289.408 >= 3.626.760
java.util.concurrent.ConcurrentHashMap$Node 23.259
744.288 >= 2.563.680
java.util.HashMap$Node[] 4.140
598.560 >= 4.222.672
int[] 4.993
573.072 >= 573.072
java.lang.Object[] 4.299
335.232 >= 3.012.440
java.lang.reflect.Method 3.628
319.264 >= 445.016
java.util.HashMap 6.074
291.552 >= 4.481.752
java.util.Hashtable$Entry 6.423
205.536 >= 738.760
java.util.concurrent.ConcurrentHashMap$Node[] 461
202.608 >= 2.760.224
java.util.regex.Pattern 2.546
183.312 >= 642.104
java.lang.Object 10.149
162.384 >= 162.384
java.lang.String[] 1.768
120.888 >= 556.048
java.lang.Integer 7.269
116.304 >= 116.928
com.mysql.jdbc.ConnectionPropertiesImpl$BooleanConnectionProperty 1.666
106.624 >= 106.632
java.util.ArrayList 3.479
83.496 >= 2.185.496
java.util.TreeMap$Entry 1.931
77.240 >= 294.136
org.apache.tika.mime.MimeType 1.540
73.920 >= 192.576
java.lang.Long 3.061
73.464 >= 73.504
java.util.LinkedList$Node 3.052
73.248 >= 82.152
java.util.Hashtable$Entry[] 317
70.144
>= 829.168
java.util.LinkedHashMap$Entry 1.726
69.040 >= 132.560
org.apache.catalina.loader.ResourceEntry 2.819
67.656 >= 67.656
java.lang.Class 5.735
62.712 >= 6.118.936
java.util.regex.Pattern$Slice 2.434
58.416 >= 141.216
java.util.regex.Pattern$Start 2.431
58.344 >= 58.344
org.apache.tika.mime.MediaType 1.818
58.176 >= 171.440
java.util.concurrent.ConcurrentHashMap 899 57.536
>= 2.814.184
org.apache.manifoldcf.crawler.interfaces.DocumentDescription 1.303
52.120 >= 190.520
sun.security.util.DerValue 1.541
49.312 >= 212.248
sun.security.util.DerInputBuffer 1.541
49.312 >= 125.928
java.net.URL 736
47.104
>= 138.832
java.util.LinkedList 1.452
46.464 >= 128.624
org.apache.tomcat.util.modeler.AttributeInfo 845
40.560
>= 109.608
-----------------------------------------------------------------------------------------------------------
El lun., 17 sept. 2018 a las 15:32, Gustavo Beneitez (<
gustavo.beneitez@gmail.com>) escribió:
> you are right, in fact there is only one instance that manages more than
> 200 MB, with Top Consumers
> Biggest Top-Level Dominator Class Loaders
> LabelNumber of ObjectsUsed Heap SizeRetained Heap SizeRetained Heap, %
> java.net.FactoryURLClassLoader @ 0xed503040 2.437 89.384 212.569.400
> 89,81%
> org.apache.catalina.loader.ParallelWebappClassLoader @ 0xecc5f4a0 3.212
> 121.744 11.547.920 4,88%
> <system class loader> 21.740 938.712 9.264.456 3,91%
> java.net.URLClassLoader @ 0xecc27b48
>
>
> 519 18.784 2.762.008 1,17%
>
> - Total: 4 entries
>
> 27.908 1.168.624 236.143.784
>
>
> I am trying to gather further info but eclipse MAT is sometimes a bit
> confuse
>
>
> El lun., 17 sept. 2018 a las 14:18, Karl Wright (<da...@gmail.com>)
> escribió:
>
>> How many instances does MAT show for ElasticSearchConnection?
>>
>> A "leak suspect" is the class whose number of instances is the largest.
>> It
>> doesn't mean there's an actual memory leak.
>>
>> Karl
>>
>>
>> On Mon, Sep 17, 2018 at 7:49 AM Gustavo Beneitez <
>> gustavo.beneitez@gmail.com>
>> wrote:
>>
>> > Hi Karl,
>> >
>> > I reviewed the UI and there are a total of
>> > ElasticSearch 100
>> > Seems not so many connections...
>> >
>> >
>> >
>> > El lun., 17 sept. 2018 a las 12:56, Karl Wright (<da...@gmail.com>)
>> > escribió:
>> >
>> > > Hi Gustavo,
>> > >
>> > > Connector instances are pooled in general, so once a connector
>> instance
>> > is
>> > > created it is kept around. The number of instances will max out at
>> the
>> > > number of connections you allow in the UI when defining the
>> connection.
>> > > How many did you allocate?
>> > >
>> > > Karl
>> > >
>> > >
>> > > On Mon, Sep 17, 2018 at 6:23 AM Gustavo Beneitez <
>> > > gustavo.beneitez@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi there,
>> > > >
>> > > > I was indexing in production several intranet websites and I
>> realized
>> > the
>> > > > performance was becoming poor (no tomcat restart for two weeks).
>> > > >
>> > > > After a few thoughts about what is going on I decided to create a
>> dump
>> > > > file and review it with eclipse MAT and I observed that the class
>> > > >
>> > > > Leak Suspects
>> > > >
>> > >
>> >
>> *org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnection*
>> > > >
>> > > > had a lot of instances and was growing up and never released. Do you
>> > have
>> > > > any issue related to Elastic? Maybe there is something related to
>> our
>> > > > version (2.4.6) of Elastic.
>> > > >
>> > > > I also have attached the MAT report. We have all components "by
>> > default",
>> > > > i.e. neither no customization of any component nor any new one
>> created.
>> > > >
>> > > > Thanks in advance.
>> > > >
>> > >
>> >
>>
>
Re: High performance issues with Elastic Connector
Posted by Gustavo Beneitez <gu...@gmail.com>.
you are right, in fact there is only one instance that manages more than
200 MB, with Top Consumers
Biggest Top-Level Dominator Class Loaders
LabelNumber of ObjectsUsed Heap SizeRetained Heap SizeRetained Heap, %
java.net.FactoryURLClassLoader @ 0xed503040 2.437 89.384 212.569.400 89,81%
org.apache.catalina.loader.ParallelWebappClassLoader @ 0xecc5f4a0 3.212
121.744 11.547.920 4,88%
<system class loader> 21.740 938.712 9.264.456 3,91%
java.net.URLClassLoader @ 0xecc27b48
519 18.784 2.762.008 1,17%
- Total: 4 entries
27.908 1.168.624 236.143.784
I am trying to gather further info but eclipse MAT is sometimes a bit
confuse
El lun., 17 sept. 2018 a las 14:18, Karl Wright (<da...@gmail.com>)
escribió:
> How many instances does MAT show for ElasticSearchConnection?
>
> A "leak suspect" is the class whose number of instances is the largest. It
> doesn't mean there's an actual memory leak.
>
> Karl
>
>
> On Mon, Sep 17, 2018 at 7:49 AM Gustavo Beneitez <
> gustavo.beneitez@gmail.com>
> wrote:
>
> > Hi Karl,
> >
> > I reviewed the UI and there are a total of
> > ElasticSearch 100
> > Seems not so many connections...
> >
> >
> >
> > El lun., 17 sept. 2018 a las 12:56, Karl Wright (<da...@gmail.com>)
> > escribió:
> >
> > > Hi Gustavo,
> > >
> > > Connector instances are pooled in general, so once a connector instance
> > is
> > > created it is kept around. The number of instances will max out at the
> > > number of connections you allow in the UI when defining the connection.
> > > How many did you allocate?
> > >
> > > Karl
> > >
> > >
> > > On Mon, Sep 17, 2018 at 6:23 AM Gustavo Beneitez <
> > > gustavo.beneitez@gmail.com>
> > > wrote:
> > >
> > > > Hi there,
> > > >
> > > > I was indexing in production several intranet websites and I realized
> > the
> > > > performance was becoming poor (no tomcat restart for two weeks).
> > > >
> > > > After a few thoughts about what is going on I decided to create a
> dump
> > > > file and review it with eclipse MAT and I observed that the class
> > > >
> > > > Leak Suspects
> > > >
> > >
> >
> *org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnection*
> > > >
> > > > had a lot of instances and was growing up and never released. Do you
> > have
> > > > any issue related to Elastic? Maybe there is something related to our
> > > > version (2.4.6) of Elastic.
> > > >
> > > > I also have attached the MAT report. We have all components "by
> > default",
> > > > i.e. neither no customization of any component nor any new one
> created.
> > > >
> > > > Thanks in advance.
> > > >
> > >
> >
>
Re: High performance issues with Elastic Connector
Posted by Karl Wright <da...@gmail.com>.
How many instances does MAT show for ElasticSearchConnection?
A "leak suspect" is the class whose number of instances is the largest. It
doesn't mean there's an actual memory leak.
Karl
On Mon, Sep 17, 2018 at 7:49 AM Gustavo Beneitez <gu...@gmail.com>
wrote:
> Hi Karl,
>
> I reviewed the UI and there are a total of
> ElasticSearch 100
> Seems not so many connections...
>
>
>
> El lun., 17 sept. 2018 a las 12:56, Karl Wright (<da...@gmail.com>)
> escribió:
>
> > Hi Gustavo,
> >
> > Connector instances are pooled in general, so once a connector instance
> is
> > created it is kept around. The number of instances will max out at the
> > number of connections you allow in the UI when defining the connection.
> > How many did you allocate?
> >
> > Karl
> >
> >
> > On Mon, Sep 17, 2018 at 6:23 AM Gustavo Beneitez <
> > gustavo.beneitez@gmail.com>
> > wrote:
> >
> > > Hi there,
> > >
> > > I was indexing in production several intranet websites and I realized
> the
> > > performance was becoming poor (no tomcat restart for two weeks).
> > >
> > > After a few thoughts about what is going on I decided to create a dump
> > > file and review it with eclipse MAT and I observed that the class
> > >
> > > Leak Suspects
> > >
> >
> *org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnection*
> > >
> > > had a lot of instances and was growing up and never released. Do you
> have
> > > any issue related to Elastic? Maybe there is something related to our
> > > version (2.4.6) of Elastic.
> > >
> > > I also have attached the MAT report. We have all components "by
> default",
> > > i.e. neither no customization of any component nor any new one created.
> > >
> > > Thanks in advance.
> > >
> >
>
Re: High performance issues with Elastic Connector
Posted by Gustavo Beneitez <gu...@gmail.com>.
Hi Karl,
I reviewed the UI and there are a total of
ElasticSearch 100
Seems not so many connections...
El lun., 17 sept. 2018 a las 12:56, Karl Wright (<da...@gmail.com>)
escribió:
> Hi Gustavo,
>
> Connector instances are pooled in general, so once a connector instance is
> created it is kept around. The number of instances will max out at the
> number of connections you allow in the UI when defining the connection.
> How many did you allocate?
>
> Karl
>
>
> On Mon, Sep 17, 2018 at 6:23 AM Gustavo Beneitez <
> gustavo.beneitez@gmail.com>
> wrote:
>
> > Hi there,
> >
> > I was indexing in production several intranet websites and I realized the
> > performance was becoming poor (no tomcat restart for two weeks).
> >
> > After a few thoughts about what is going on I decided to create a dump
> > file and review it with eclipse MAT and I observed that the class
> >
> > Leak Suspects
> >
> *org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnection*
> >
> > had a lot of instances and was growing up and never released. Do you have
> > any issue related to Elastic? Maybe there is something related to our
> > version (2.4.6) of Elastic.
> >
> > I also have attached the MAT report. We have all components "by default",
> > i.e. neither no customization of any component nor any new one created.
> >
> > Thanks in advance.
> >
>
Re: High performance issues with Elastic Connector
Posted by Karl Wright <da...@gmail.com>.
Hi Gustavo,
Connector instances are pooled in general, so once a connector instance is
created it is kept around. The number of instances will max out at the
number of connections you allow in the UI when defining the connection.
How many did you allocate?
Karl
On Mon, Sep 17, 2018 at 6:23 AM Gustavo Beneitez <gu...@gmail.com>
wrote:
> Hi there,
>
> I was indexing in production several intranet websites and I realized the
> performance was becoming poor (no tomcat restart for two weeks).
>
> After a few thoughts about what is going on I decided to create a dump
> file and review it with eclipse MAT and I observed that the class
>
> Leak Suspects
> *org.apache.manifoldcf.agents.output.elasticsearch.ElasticSearchConnection*
>
> had a lot of instances and was growing up and never released. Do you have
> any issue related to Elastic? Maybe there is something related to our
> version (2.4.6) of Elastic.
>
> I also have attached the MAT report. We have all components "by default",
> i.e. neither no customization of any component nor any new one created.
>
> Thanks in advance.
>