You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@ignite.apache.org by "igor.tanackovic" <ig...@gmail.com> on 2018/10/17 16:13:58 UTC

SQL Engine

Moving to dev forum...


Hello,

Seems that SQL engine always deserialize whole objects instead of using just
SQL enabled fields (annotated with @QuerySqlField). This may have a huge
impact on Ignite heap usage and GC overhead as well. 

For example, we have a cache holding big objects but with only two sql query
fields which for each query execution (SELECT COUNT(*) FROM 'cache')
consumes large amount on heap memory (~300MB). As a proof of concept, we
divided the same cache to *index* cache with only sql query field and a
*data* holding whole object for materialization. The same query (SELECT
COUNT(*) FROM 'index-cache') consumes ~25 time less memory! The same is true
for all other queries.

The obvious workaround would be to always have separated regions for indexes
(sql query enabled region) and a data/value region for materialization, but
it might be a good idea to fix this in a systematic way during off heap
deserialization.

Regards,
Igor



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/