RE: [External]Re: OOME on startup with 2.11.1 and 2.12

[Sa...@ril.com]

Thank you Zhenya. Reducing the JDBC connection did the trick.

But I feel the error is misleading. Error talks about “java.lang.OutOfMemoryError: Direct buffer memory”. So initially we were increasing
-XX:MaxDirectMemorySize=<size>[g|G|m|M|k|K]

Thanks again.



From: Zhenya Stanilovsky <ar...@mail.ru>
Sent: Friday, February 4, 2022 11:56 AM
To: user@ignite.apache.org
Subject: [External]Re: OOME on startup with 2.11.1 and 2.12

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.



hello, seems you really use all available memory, of course without heap dump i have no clue why it`s happen, plz increase -Xmx and check once more.





We are planning to upgrade from 2.9 to 2.11.1. But as soon as we start the 2.11.1 or 2.12 ignite instance crashes with OOME error.



[20:15:05,474][SEVERE][grid-nio-worker-client-listener-2-#34][] Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=java.lang.OutOfMemoryError: Direct buffer memory]]

java.lang.OutOfMemoryError: Direct buffer memory



Some of the pointers



1)      2.9 has no problem.

2)      2.11 and 2.12 crashes even with default supplied config of ignite.sh. OOM is within a min of startup.

3)      No other application is running on the server

4)      Details about the environment

[oracle@jmngd1s05ppv014 log]$ free -g

              total        used        free      shared  buff/cache   available

Mem:             46          18          21           0           5          23

Swap:            47           0          47



[oracle@jmngd1s05ppv014 log]$ java -version

java version "1.8.0_301"

Java(TM) SE Runtime Environment (build 1.8.0_301-b25)

Java HotSpot(TM) 64-Bit Server VM (build 25.301-b25, mixed mode)



[oracle@jmngd1s05ppv014 log]$ uname -a

Linux jmngd1s05ppv014 3.10.0-1160.49.1.el7.x86_64 #1 SMP Tue Nov 9 16:09:48 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux



[oracle@jmngd1s05ppv014 log]$ cat /etc/os-release

NAME="Red Hat Enterprise Linux Server"

VERSION="7.9 (Maipo)"

ID="rhel"

ID_LIKE="fedora"

VARIANT="Server"

VARIANT_ID="server"

VERSION_ID="7.9"

PRETTY_NAME="Red Hat Enterprise Linux Server 7.9 (Maipo)"

ANSI_COLOR="0;31"

CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server"

HOME_URL="https://www.redhat.com/"

BUG_REPORT_URL="https://bugzilla.redhat.com/"



REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"

REDHAT_BUGZILLA_PRODUCT_VERSION=7.9

REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"

REDHAT_SUPPORT_PRODUCT_VERSION="7.9"

5)      I have also attached the ignite log.

6)      O/S administrator confirmed there are no hardware failure.



If anyone can help how to further diagnosis the issue. We will be grateful.





Thanks,

Sachin.

"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s), are confidential and may be privileged. If you are not the intended recipient, you are hereby notified that any review, re-transmission, conversion to hard copy, copying, circulation or other use of this message and any attachments is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."




"Confidentiality Warning: This message and any attachments are intended only for the use of the intended recipient(s). 
are confidential and may be privileged. If you are not the intended recipient. you are hereby notified that any 
review. re-transmission. conversion to hard copy. copying. circulation or other use of this message and any attachments is 
strictly prohibited. If you are not the intended recipient. please notify the sender immediately by return email. 
and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure no viruses are present in this email. 
The company cannot accept responsibility for any loss or damage arising from the use of this email or attachment."