Result set processing in off heap mode

[Sri Ganesh V <vs...@yahoo.co.in>]

Hi,

We use cache in *off-heap *mode.

Recently we faced GC issue when there were multiple client connections
executing SQL queries
 
[via JDBC connection - DriverManager
		
.getConnection("jdbc:ignite:cfg://cache=TEST_CACHE:distributedJoins=true@file:///C:\\ignite-client-config.xml")
]

The returned query result set is often huge and GC kicks in very often and
there is a huge impact on the query response time. 

Also we see "Query execution is too long" warning messages when GC kicks in.
There were scenarios when ignite server node went into non-responsive state
and never recovered back.

We would like to know if "Result Set" processing can happen out of GC
collectable heap like the way we can use cache in off-heap mode ??

Thanks.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Result-set-processing-in-off-heap-mode-tp7988.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Result set processing in off heap mode

[Sri Ganesh V <vs...@yahoo.co.in>]

Hi Denis,

Thank you for the information provided.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Result-set-processing-in-off-heap-mode-tp7988p8003.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Result set processing in off heap mode

[Denis Magda <dm...@gridgain.com>]

Hi,

Right now result set processing can be done in Java heap only. There is a ticket to improve a situation around this by moving result set processing to off-heap
https://issues.apache.org/jira/browse/IGNITE-3870 <https://issues.apache.org/jira/browse/IGNITE-3870>

Until this functionality is not available I can suggest you the following options:
- tune Java garbage collector [1], queries rate and result set size.
- use a commercial JVM like Azul Zing that doesn’t have stop-the-world pauses at all. 

[1] https://apacheignite.readme.io/docs/jvm-and-system-tuning 

—
Denis

> On Sep 28, 2016, at 6:01 AM, Sri Ganesh V <vs...@yahoo.co.in> wrote:
> 
> Hi,
> 
> We use cache in *off-heap *mode.
> 
> Recently we faced GC issue when there were multiple client connections
> executing SQL queries
> 
> [via JDBC connection - DriverManager
> 		
> .getConnection("jdbc:ignite:cfg://cache=TEST_CACHE:distributedJoins=true@file:///C:\\ignite-client-config.xml")
> ]
> 
> The returned query result set is often huge and GC kicks in very often and
> there is a huge impact on the query response time. 
> 
> Also we see "Query execution is too long" warning messages when GC kicks in.
> There were scenarios when ignite server node went into non-responsive state
> and never recovered back.
> 
> We would like to know if "Result Set" processing can happen out of GC
> collectable heap like the way we can use cache in off-heap mode ??
> 
> Thanks.
> 
> 
> 
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Result-set-processing-in-off-heap-mode-tp7988.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.