TDB memory usage

[Laurent Rucquoy <la...@telemis.com>]

Hello,

We have questions about the TDB memory usage.

1. On some Microsoft Windows 64 bit servers, we noted that there is no
mapped file used by the TDB while on other servers mapped file use is
substantial. What is the trigger for mapped file use ?
2. In some cases, RAM is mostly used by the TDB. When some other
applications require more RAM, the space used by the TDB is not released
and we encounter memory lack issues. Despite the fact that the problem is
maybe caused by a wrong memory handling done by the other applications, is
there a way to limit the TDB RAM usage in order to anticipate such a
blocking situation (a bit like defining the 'innodb_buffer_pool_size' in
MariaDB) ?

Environment:
- Windows Server 2008R2 (physical machine)
- Apache Jena 3.1.1 (still with this release because of the long
development cycle of our product)

Thank you in advance for your help.

Laurent

Re: TDB memory usage

[Andy Seaborne <an...@apache.org>]

On 03/04/18 11:49, Laurent Rucquoy wrote:
> Hello,
> 
> We have questions about the TDB memory usage.
> 
> 1. On some Microsoft Windows 64 bit servers, we noted that there is no
> mapped file used by the TDB while on other servers mapped file use is
> substantial. What is the trigger for mapped file use ?

SystemTDB.determineFileMode
https://github.com/apache/jena/blob/master/jena-tdb/src/main/java/org/apache/jena/tdb/sys/SystemTDB.java

It pokes around in property "java.vm.info" but also there is system 
property tdb:fileMode.  A bit fragile (anyone know if there is a better 
way these days?)

If you want to delete a database while the JVM is running, then on 
windows, you have to run "direct" on MS Windows.  It's a Java issue, 
very old; marked as "won't fix" in the java tracker.

> 2. In some cases, RAM is mostly used by the TDB. When some other
> applications require more RAM, the space used by the TDB is not released
> and we encounter memory lack issues. Despite the fact that the problem is
> maybe caused by a wrong memory handling done by the other applications, is
> there a way to limit the TDB RAM usage in order to anticipate such a
> blocking situation (a bit like defining the 'innodb_buffer_pool_size' in
> MariaDB) ?

See
https://jena.apache.org/documentation/tdb/store-parameters.html

and that can be used to set the index cache size in "direct" mode.

Use with care!

     Andy

> 
> Environment:
> - Windows Server 2008R2 (physical machine)
> - Apache Jena 3.1.1 (still with this release because of the long
> development cycle of our product)
> 
> Thank you in advance for your help.
> 
> Laurent
>