OOME causing caches to be removed

[djax <dj...@sharklasers.com>]

Ignite version: 2.5
This is on a 2 node cluster in AWS with 32GB RAM and 8 cores each.

When executing the following SQL query, the query fills the JVM heap of each
node (specified at 10 GB each node) and exits afterwards. On restart, the
USER cache is removed from persistence (the cache on which the SELECT
statement is being performed)

Any tips? Is this expected behavior?

SQL query:
INSERT INTO user_email
SELECT
email,max(sex),year(max(birthdate)),max(country),max(province),substr(email,position('@',email)+1)
FROM USER WHERE email IS NOT NULL GROUP BY email;

JVM options (excluding ignite libs and ignite home): -Xms18g -Xmx18g -server
-XX:+AggressiveOpts -XX:MaxMetaspaceSize=2048m
-Djava.net.preferIPv4Stack=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m -XX:MaxTenuringThreshold=0
-XX:SurvivorRatio=1024 -XX:+UseCMSInitiatingOccupancyOnly
-XX:CMSInitiatingOccupancyFraction=40 -XX:MaxGCPauseMillis=1000
-XX:InitiatingHeapOccupancyPercent=50 -XX:+UseCompressedOops
-XX:ParallelGCThreads=8 -XX:ConcGCThreads=8 -XX:+DisableExplicitGC
-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/share/apache-ignite
-XX:+ExitOnOutOfMemoryError -DIGNITE_QUIET=false
-DIGNITE_SUCCESS_FILE=/usr/share/apache-ignite/work/ignite_success_ccedb5ad-5e37-4fee-aeae-19456d7da609
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=49124
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

Config (AWS credentials removed):


Logs:


Restart log:




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: OOME causing caches to be removed

["ilya.kasnacheev" <il...@gmail.com>]

Hello!

My recommendation here is breaking this query down into ranges (by using
LIMIT or better yet, range WHERE condition), processing every range
separately. Otherwise I can see how Ignite will try to keep all data on heap
at some point.

Regards,



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: OOME causing caches to be removed

[djax <dj...@sharklasers.com>]

I use the following SQL sentence:

CREATE TABLE user (
                user_id BIGINT NOT NULL,
                db_id INTEGER NOT NULL,
                list_id INTEGER NOT NULL,
                con_id INTEGER NOT NULL,
                status VARCHAR NOT NULL,
                procedence VARCHAR,
                sex CHAR(1),
                birthdate DATE,
                country VARCHAR,
                province INTEGER,
                email VARCHAR,
                CONSTRAINT user_id PRIMARY KEY (user_id)
) WITH "template=parallel";

The error has also occurred with the default template.

The query did finish executing when I set the JVM heap of each node to 18
GB, but that seems excessive, given that the total amount of data on disk of
the USER table is less than 7 GB. During the execution of the query Ignite
reached a peak of around 25 GB of JVM heap usage.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: OOME causing caches to be removed

[akurbanov <an...@gmail.com>]

Hi,

How do you create USER cache?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/