You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Denis Mekhanikov (JIRA)" <ji...@apache.org> on 2018/08/15 08:40:00 UTC
[jira] [Issue Comment Deleted] (IGNITE-9196) SQL: Memory leak in
MapNodeResults
[ https://issues.apache.org/jira/browse/IGNITE-9196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Denis Mekhanikov updated IGNITE-9196:
-------------------------------------
Comment: was deleted
(was: [~vozerov] we can do that. We could look at contents of the following map: {{MapNodeResults#res}}
To get a handle of it, you need to follow a pretty long chain of links: {{GridQueryProcessor->idx->mapQryExec->qryRess->res}}. Each one would be accessed via reflection, since all the fields on the way are private and don't have accessors.
It would be too implementation-dependent. I don't really like the idea of digging through an incapsulated code via reflection.
Testing a memory leak by comparing sizes of heap dumps seems less ugly to me.
There is already a test, that checks for memory leaks this way: [MemoryLeaksOnRestartNodeTest|https://github.com/apache/ignite/blob/9a4a145be514e650258715a7e682d427d5812d16/modules/core/src/test/java/org/apache/ignite/internal/MemoryLeaksOnRestartNodeTest.java]. This test is not added to any test suite. I guess, it's because it takes too much time to finish. How about creating a test suite, that is run only once in a while? We could add such long-running tests there and run them e.g. before each release.)
> SQL: Memory leak in MapNodeResults
> ----------------------------------
>
> Key: IGNITE-9196
> URL: https://issues.apache.org/jira/browse/IGNITE-9196
> Project: Ignite
> Issue Type: Bug
> Components: sql
> Affects Versions: 2.6
> Reporter: Denis Mekhanikov
> Assignee: Denis Mekhanikov
> Priority: Blocker
> Labels: sql-stability
> Fix For: 2.7
>
>
> When size of a SQL query result set is a multiple of {{Query#pageSize}}, then {{MapQueryResult}} is never closed and removed from {{MapNodeResults#res}} collection.
> The following code leads to OOME when run with 1Gb heap:
> {code:java}
> public class MemLeakRepro {
> public static void main(String[] args) {
> Ignition.start(getConfiguration("server"));
> try (Ignite client = Ignition.start(getConfiguration("client").setClientMode(true))) {
> IgniteCache<Integer, Person> cache = startPeopleCache(client);
> int pages = 10;
> int pageSize = 1024;
> for (int i = 0; i < pages * pageSize; i++) {
> Person p = new Person("Person #" + i, 25);
> cache.put(i, p);
> }
> for (int i = 0; i < 1_000_000; i++) {
> if (i % 1000 == 0)
> System.out.println("Select iteration #" + i);
> Query<List<?>> qry = new SqlFieldsQuery("select * from people");
> qry.setPageSize(pageSize);
> QueryCursor<List<?>> cursor = cache.query(qry);
> cursor.getAll();
> cursor.close();
> }
> }
> }
> private static IgniteConfiguration getConfiguration(String instanceName) {
> IgniteConfiguration igniteCfg = new IgniteConfiguration();
> igniteCfg.setIgniteInstanceName(instanceName);
> TcpDiscoverySpi discoSpi = new TcpDiscoverySpi();
> discoSpi.setIpFinder(new TcpDiscoveryVmIpFinder(true));
> return igniteCfg;
> }
> private static IgniteCache<Integer, Person> startPeopleCache(Ignite node) {
> CacheConfiguration<Integer, Person> cacheCfg = new CacheConfiguration<>("cache");
> QueryEntity qe = new QueryEntity(Integer.class, Person.class);
> qe.setTableName("people");
> cacheCfg.setQueryEntities(Collections.singleton(qe));
> cacheCfg.setSqlSchema("PUBLIC");
> return node.getOrCreateCache(cacheCfg);
> }
> public static class Person {
> @QuerySqlField
> private String name;
> @QuerySqlField
> private int age;
> public Person(String name, int age) {
> this.name = name;
> this.age = age;
> }
> }
> }
> {code}
>
> At the same time it works perfectly fine, when there are, for example, {{pages * pageSize - 1}} records in cache instead.
> The reason for it is that {{MapQueryResult#fetchNextPage(...)}} method doesn't return true, when the result set size is a multiple of the page size.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)