Working set memory usage

[Clive Borrageiro <cl...@cibecs.com>]

Hi,

When I use derby as an embedded database that contains a large amount of
records (in excess of 100,000), the Working set memory (as seen under
Windows Task Manager > Mem Usage) of the application's process grows with
time & usage.

I created a simple test application that inserts 100,000 records into a
table, then retrieves these and updates each record individually. At the end
of this process the Working set memory displays a value of around 160MB.
I have profiled the application but the maximum java heap memory used is
17MB.

I also tried running the same process using the database through a network
server; the test application's Working set memory displayed a maximum value
of 17.5MB (java heap around 1MB) but the process running the network server
climbed to 60MB (Working set).

What causes this Working set memory to rise?

Thank you,
Clive

Re: Working set memory usage

["Bernt M. Johnsen" <Be...@Sun.COM>]

Clive Borrageiro wrote:
> Hi,
> 
> When I use derby as an embedded database that contains a large amount of
> records (in excess of 100,000), the Working set memory (as seen under
> Windows Task Manager > Mem Usage) of the application's process grows with
> time & usage.
> 
> I created a simple test application that inserts 100,000 records into a
> table, then retrieves these and updates each record individually. At the end
> of this process the Working set memory displays a value of around 160MB.
> I have profiled the application but the maximum java heap memory used is
> 17MB.
> 
> I also tried running the same process using the database through a network
> server; the test application's Working set memory displayed a maximum value
> of 17.5MB (java heap around 1MB) but the process running the network server
> climbed to 60MB (Working set).
> 
> What causes this Working set memory to rise?

It might be generated code (SQL is compiled into Java byte code and
loaded). Could you post your test-app. It might be easier to pinpoint
your problem if I saw the code.

> 
> Thank you,
> Clive
> 


-- 
Bernt Marius Johnsen, Database Technology Group,
Staff Engineer, Technical Lead Derby/Java DB
Sun Microsystems, Trondheim, Norway

Re: Working set memory usage

[David Van Couvering <Da...@Sun.COM>]

One standard answer is, make sure you are closing all of your statement 
and result set objects.   Also, what version are you running?  A number 
of memory leaks have been fixed in the latest release (currently in beta).

A good way to find out what is out there in terms of memory leaks is to 
go to http://issues.apache.org/jira/browse/DERBY and search for items 
with the work "leak" in them...

David

Clive Borrageiro wrote:
> Hi,
> 
> When I use derby as an embedded database that contains a large amount of
> records (in excess of 100,000), the Working set memory (as seen under
> Windows Task Manager > Mem Usage) of the application's process grows with
> time & usage.
> 
> I created a simple test application that inserts 100,000 records into a
> table, then retrieves these and updates each record individually. At the end
> of this process the Working set memory displays a value of around 160MB.
> I have profiled the application but the maximum java heap memory used is
> 17MB.
> 
> I also tried running the same process using the database through a network
> server; the test application's Working set memory displayed a maximum value
> of 17.5MB (java heap around 1MB) but the process running the network server
> climbed to 60MB (Working set).
> 
> What causes this Working set memory to rise?
> 
> Thank you,
> Clive
>