You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Elton Soares <el...@ibm.com.INVALID> on 2023/03/10 23:50:36 UTC
How one can obtain a reliable estimate of the maximum memory allocation of a container running Jena Fuseki?
Dear Jena Community,
I'm running Jena Fuseki Version 4.7.0 as a container on an OpenShift Cluster.
OS Version Info (cat /etc/os-release):
NAME="Red Hat Enterprise Linux"
VERSION="8.7 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.7"
...
Hardware Info (from Jena Fuseki initialization log):
[2023-03-10 18:34:36] Server INFO Memory: 32.0 GiB
[2023-03-10 18:34:36] Server INFO Java: 11.0.18
[2023-03-10 18:34:36] Server INFO OS: Linux 3.10.0-1160.76.1.el7.x86_64 amd64
[2023-03-10 18:34:36] Server INFO PID: 1
Disk Info (df -h):
Filesystem Size Used Avail Use% Mounted on
overlay 99G 77G 17G 83% /
tmpfs 64M 0 64M 0% /dev
tmpfs 63G 0 63G 0% /sys/fs/cgroup
shm 64M 0 64M 0% /dev/shm
/docker_data 99G 77G 17G 83% /config
/data 1.0T 364G 661G 36% /usr/app/run
…
For some unknown reason, the memory consumption of the container keeps exceeding the threshold specified at the server initialization command:
java -Xmx32G -jar fuseki-server.jar --port 3030
At our latest experiment, we’ve set the maximum amount of memory available to the container to 48GB, which leaves 16GB (about 50% of Jena’s maximum expected memory allocation) free for any other processes that might be triggered in the background. This was based on a recommendation found at https://www.ibm.com/support/pages/understanding-memory-requirements-32-and-64-bit-systems which suggested that:
“When allocating memory to a JVM, 10% to 25% of the memory is for the JVM process itself and the remainder is for the code running in the JVM. This means that a JVM that has been allocated 4 GB may only have 3.5 GB available to the code running in the JVM.”
Although we expected that this would be sufficient to avoid any issues with regards to memory consumption, the container end’s up trying to allocate more than 48GB which results in an OOMKilled error.
Provided that, would there be any specific recommendations or known strategies to deal with this issue besides increasing the memory available to the container? If not, would there be a better way to determine the ideal amount of memory that should be made available to the container based on the value of -Xmx parameter (currently 32G)?
Thanks for your help!
Best Regards,
Elton Figueiredo de Souza Soares
Research Software Engineer
E-mail: eltons@ibm.com<ma...@ibm.com>
Re: How one can obtain a reliable estimate of the maximum memory allocation of a container running Jena Fuseki?
Posted by Andy Seaborne <an...@apache.org>.
Hi Elton,
Sounds like
https://lists.apache.org/thread/dmmkndmy2ds8pf95zvqbcxpv84bj7cz6
where heap size is set too big.
But maybe host is configured to limit container size?
On 10/03/2023 23:50, Elton Soares wrote:
> Dear Jena Community,
>
> I'm running Jena Fuseki Version 4.7.0 as a container on an OpenShift Cluster.
> OS Version Info (cat /etc/os-release):
> NAME="Red Hat Enterprise Linux"
> VERSION="8.7 (Ootpa)"
> ID="rhel"
> ID_LIKE="fedora"
> VERSION_ID="8.7"
> ...
>
> Hardware Info (from Jena Fuseki initialization log):
> [2023-03-10 18:34:36] Server INFO Memory: 32.0 GiB
> [2023-03-10 18:34:36] Server INFO Java: 11.0.18
> [2023-03-10 18:34:36] Server INFO OS: Linux 3.10.0-1160.76.1.el7.x86_64 amd64
> [2023-03-10 18:34:36] Server INFO PID: 1
>
> Disk Info (df -h):
> Filesystem Size Used Avail Use% Mounted on
> overlay 99G 77G 17G 83% /
> tmpfs 64M 0 64M 0% /dev
> tmpfs 63G 0 63G 0% /sys/fs/cgroup
> shm 64M 0 64M 0% /dev/shm
> /docker_data 99G 77G 17G 83% /config
> /data 1.0T 364G 661G 36% /usr/app/run
> …
>
>
> For some unknown reason, the memory consumption of the container keeps exceeding the threshold specified at the server initialization command:
> java -Xmx32G -jar fuseki-server.jar --port 3030
> At our latest experiment, we’ve set the maximum amount of memory available to the container to 48GB, which leaves 16GB (about 50% of Jena’s maximum expected memory allocation)
What's the estimate based on?
What sort of databases are configured?
> free for any other processes that might be triggered in the background. This was based on a recommendation found at https://www.ibm.com/support/pages/understanding-memory-requirements-32-and-64-bit-systems
(Not related to your report)
If you are running Fuseki in a container with nothing else, that's not a
good balance. TDB2, per database needs about 2G heap, and the rest is
non-heap memory (memory map files) which are managed by the OS and doe
not need to be set in anyway.
which suggested that:
> “When allocating memory to a JVM, 10% to 25% of the memory is for the JVM process itself and the remainder is for the code running in the JVM. This means that a JVM that has been allocated 4 GB may only have 3.5 GB available to the code running in the JVM.”
> Although we expected that this would be sufficient to avoid any issues with regards to memory consumption, the container end’s up trying to allocate more than 48GB which results in an OOMKilled error.
That is a container being killed by the host, not a heap issue.
Hwo long does it take before it happens? (And what's the workload?)
> Provided that, would there be any specific recommendations or known strategies to deal with this issue besides increasing the memory available to the container? If not, would there be a better way to determine the ideal amount of memory that should be made available to the container based on the value of -Xmx parameter (currently 32G)?
Depends on what's configured.
TDB2, per database needs about 2G heap - add an bit for the JVM. And
make sure the host configuration allows the container to grow.
Andy
> Thanks for your help!
> Best Regards,
> Elton Figueiredo de Souza Soares
> Research Software Engineer
> E-mail: eltons@ibm.com<ma...@ibm.com>
>
>