You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by yangjiajun <13...@qq.com> on 2018/11/08 13:07:25 UTC
Why Ignite use so many heap space?
HI.
I have an ignite node which is version 2.6 and has fixed 12GB heap memory
and 30GB data region.I use it as a database with persistence.It uses 90%-95%
of heap memory when my application is busy(My application uses jdbc thin
connection).And it still uses 80%-85% of heap memory after I stop my
application.Ignite is in bad performance when its heap usage is high.
How to reduce ignite heap usage?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Why Ignite use so many heap space?
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
https://issues.apache.org/jira/browse/IGNITE-10224 might be related. Try
closing connections every now and then. If it does not help, collect heap
dump & analyze it.
Regards,
--
Ilya Kasnacheev
пн, 12 нояб. 2018 г. в 12:44, yangjiajun <13...@qq.com>:
> My test scenario:
> 1.One ignite node uses 12GB heap memory and 30GB off-heap memory with
> persistence。Here is my cmd to start the node:
> nohup ./ignite.sh ../examples/config/example-ignite.xml -J-server -J-Xms12g
> -J-Xmx12g -J-XX:+AlwaysPreTouch -J-XX:+UseG1GC -J-XX:+ScavengeBeforeFullGC
> -J-XX:+DisableExplicitGC -J-XX:+HeapDumpOnOutOfMemoryError
> -J-XX:HeapDumpPath=./logs -J-XX:+ExitOnOutOfMemoryError
> -J-XX:+PrintGCDetails -J-XX:+PrintGCTimeStamps -J-XX:+PrintGCDateStamps
> -J-XX:+UseGCLogFileRotation -J-XX:NumberOfGCLogFiles=10
> -J-XX:GCLogFileSize=10M -J-Xloggc:./gclog.txt -v&
> 2.My jdbc thin connection is:
>
> jdbc:ignite:thin://ip:port;lazy=true;skipReducerOnUpdate=true;replicatedOnly=true
> 2.Excute merge into statements.Each statement has hundreds of data
> rows.The
> test runs about one statement per second.
> 3.The ignite node run out heap memory when there is no idle limit(gc is not
> work as I describe before).The ignite node uses 30%-70% heap memory when I
> set Idle connections limit to 50.The connection pool size is 200.
> I also tried to limit idle connections in ignite with idleTimeout
> setting.But it makes my connection pool not work.
> 4.I think gc frequency is natural.I am still doing more test and sorry for
> no more details about it.
>
>
> Mikael wrote
> > Hi!
> >
> > You said: "And it still uses 80%-85% of heap memory after I stop my
> > application", I assume you mean your client application ?
> >
> > So after a GC it will still stay at 80% java heap usage ?
> >
> > I assume you are using off-heap memory ?
> >
> > I am not sure what the problem is, I am running an application that
> > updates around 10.000 cache entries every 10 seconds (persistence
> > enabled) and run a lot of other code that generates java heap garbage,
> > and the application generates around 40MB garbage per second, it fills
> > up the heap in a minute or so but goes down to around 20% after every gc.
> >
> > How often does you application GC ? if you have 80% heap filled it
> > should GC pretty often I would think ?
> >
> > There is garbage generated, there is not much you can do about that if
> > it's not your own code that generates the garbage, Ignite is not bad and
> > with the cache keys and values and even index of heap it should work
> > fine, but i guess it all depends on what your application do.
> >
> > Mikael
> >
> > Den 2018-11-08 kl. 14:07, skrev yangjiajun:
> >> HI.
> >>
> >> I have an ignite node which is version 2.6 and has fixed 12GB heap
> >> memory
> >> and 30GB data region.I use it as a database with persistence.It uses
> >> 90%-95%
> >> of heap memory when my application is busy(My application uses jdbc thin
> >> connection).And it still uses 80%-85% of heap memory after I stop my
> >> application.Ignite is in bad performance when its heap usage is high.
> >>
> >> How to reduce ignite heap usage?
> >>
> >>
> >>
> >> --
> >> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
> >>
> >>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Why Ignite use so many heap space?
Posted by yangjiajun <13...@qq.com>.
My test scenario:
1.One ignite node uses 12GB heap memory and 30GB off-heap memory with
persistence。Here is my cmd to start the node:
nohup ./ignite.sh ../examples/config/example-ignite.xml -J-server -J-Xms12g
-J-Xmx12g -J-XX:+AlwaysPreTouch -J-XX:+UseG1GC -J-XX:+ScavengeBeforeFullGC
-J-XX:+DisableExplicitGC -J-XX:+HeapDumpOnOutOfMemoryError
-J-XX:HeapDumpPath=./logs -J-XX:+ExitOnOutOfMemoryError
-J-XX:+PrintGCDetails -J-XX:+PrintGCTimeStamps -J-XX:+PrintGCDateStamps
-J-XX:+UseGCLogFileRotation -J-XX:NumberOfGCLogFiles=10
-J-XX:GCLogFileSize=10M -J-Xloggc:./gclog.txt -v&
2.My jdbc thin connection is:
jdbc:ignite:thin://ip:port;lazy=true;skipReducerOnUpdate=true;replicatedOnly=true
2.Excute merge into statements.Each statement has hundreds of data rows.The
test runs about one statement per second.
3.The ignite node run out heap memory when there is no idle limit(gc is not
work as I describe before).The ignite node uses 30%-70% heap memory when I
set Idle connections limit to 50.The connection pool size is 200.
I also tried to limit idle connections in ignite with idleTimeout
setting.But it makes my connection pool not work.
4.I think gc frequency is natural.I am still doing more test and sorry for
no more details about it.
Mikael wrote
> Hi!
>
> You said: "And it still uses 80%-85% of heap memory after I stop my
> application", I assume you mean your client application ?
>
> So after a GC it will still stay at 80% java heap usage ?
>
> I assume you are using off-heap memory ?
>
> I am not sure what the problem is, I am running an application that
> updates around 10.000 cache entries every 10 seconds (persistence
> enabled) and run a lot of other code that generates java heap garbage,
> and the application generates around 40MB garbage per second, it fills
> up the heap in a minute or so but goes down to around 20% after every gc.
>
> How often does you application GC ? if you have 80% heap filled it
> should GC pretty often I would think ?
>
> There is garbage generated, there is not much you can do about that if
> it's not your own code that generates the garbage, Ignite is not bad and
> with the cache keys and values and even index of heap it should work
> fine, but i guess it all depends on what your application do.
>
> Mikael
>
> Den 2018-11-08 kl. 14:07, skrev yangjiajun:
>> HI.
>>
>> I have an ignite node which is version 2.6 and has fixed 12GB heap
>> memory
>> and 30GB data region.I use it as a database with persistence.It uses
>> 90%-95%
>> of heap memory when my application is busy(My application uses jdbc thin
>> connection).And it still uses 80%-85% of heap memory after I stop my
>> application.Ignite is in bad performance when its heap usage is high.
>>
>> How to reduce ignite heap usage?
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>>
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Why Ignite use so many heap space?
Posted by yangjiajun <13...@qq.com>.
Hi!
Thanks for your reply.
My client application uses jdbc thin connection to test performance of merge
into statement. We also use DPCP connection pooling providers to improve
performance.According to your suggestions and my experiment,I think idle
connections in the connection pool cause memory leak in the ignite node.But
I am still not sure and doing more test.
Mikael wrote
> Hi!
>
> You said: "And it still uses 80%-85% of heap memory after I stop my
> application", I assume you mean your client application ?
>
> So after a GC it will still stay at 80% java heap usage ?
>
> I assume you are using off-heap memory ?
>
> I am not sure what the problem is, I am running an application that
> updates around 10.000 cache entries every 10 seconds (persistence
> enabled) and run a lot of other code that generates java heap garbage,
> and the application generates around 40MB garbage per second, it fills
> up the heap in a minute or so but goes down to around 20% after every gc.
>
> How often does you application GC ? if you have 80% heap filled it
> should GC pretty often I would think ?
>
> There is garbage generated, there is not much you can do about that if
> it's not your own code that generates the garbage, Ignite is not bad and
> with the cache keys and values and even index of heap it should work
> fine, but i guess it all depends on what your application do.
>
> Mikael
>
> Den 2018-11-08 kl. 14:07, skrev yangjiajun:
>> HI.
>>
>> I have an ignite node which is version 2.6 and has fixed 12GB heap
>> memory
>> and 30GB data region.I use it as a database with persistence.It uses
>> 90%-95%
>> of heap memory when my application is busy(My application uses jdbc thin
>> connection).And it still uses 80%-85% of heap memory after I stop my
>> application.Ignite is in bad performance when its heap usage is high.
>>
>> How to reduce ignite heap usage?
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>>
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Why Ignite use so many heap space?
Posted by Mikael <mi...@telia.com>.
Hi!
You said: "And it still uses 80%-85% of heap memory after I stop my
application", I assume you mean your client application ?
So after a GC it will still stay at 80% java heap usage ?
I assume you are using off-heap memory ?
I am not sure what the problem is, I am running an application that
updates around 10.000 cache entries every 10 seconds (persistence
enabled) and run a lot of other code that generates java heap garbage,
and the application generates around 40MB garbage per second, it fills
up the heap in a minute or so but goes down to around 20% after every gc.
How often does you application GC ? if you have 80% heap filled it
should GC pretty often I would think ?
There is garbage generated, there is not much you can do about that if
it's not your own code that generates the garbage, Ignite is not bad and
with the cache keys and values and even index of heap it should work
fine, but i guess it all depends on what your application do.
Mikael
Den 2018-11-08 kl. 14:07, skrev yangjiajun:
> HI.
>
> I have an ignite node which is version 2.6 and has fixed 12GB heap memory
> and 30GB data region.I use it as a database with persistence.It uses 90%-95%
> of heap memory when my application is busy(My application uses jdbc thin
> connection).And it still uses 80%-85% of heap memory after I stop my
> application.Ignite is in bad performance when its heap usage is high.
>
> How to reduce ignite heap usage?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>