[jira] [Commented] (JOHNZON-396) JsonStreamParserImpl has a memory leak problem

["Romain Manni-Bucau (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/JOHNZON-396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17709949#comment-17709949 ] 

Romain Manni-Bucau commented on JOHNZON-396:
--------------------------------------------

Hi

 

This observatio is right and intended, only case it cant occur at runtime is when you have some huge initial string and light at runtime or a single temporary huge string request. In such case you want a dedicated instance tuned for that but for main case you want buffers to be more likely to be adjusted.

 

Indeed I am all for adjusting the logic but it would be more on the side to reverse the release order than allocating less which is an user setup issue IMHO.

> JsonStreamParserImpl has a memory leak problem
> ----------------------------------------------
>
>                 Key: JOHNZON-396
>                 URL: https://issues.apache.org/jira/browse/JOHNZON-396
>             Project: Johnzon
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.13, 1.2.19
>            Reporter: liucongcong
>            Priority: Major
>         Attachments: image-2023-04-09-12-44-02-385.png, image-2023-04-09-12-44-39-048.png, image-2023-04-09-12-45-10-166.png
>
>
> When some large strings are parsed, the JsonStreamParserImpl's buffer will doAutoAdjust(..).
> In doAutoAdjust method, the first time fallBackCopyBuffer will release to bufferProvider, i found the fallBackCopyBuffer is from valueBuffer but release to bufferProvider.
> When using the default QUEUE BufferStrategy and always trigger the doAutoAdjust will cause a OOM fault.
>  
> this is my unit test case:
>  
> {code:java}
> import java.io.StringReader;
> import java.util.HashMap;
> import java.util.Map;
> import javax.json.spi.JsonProvider;
> import javax.json.stream.JsonParser;
> import javax.json.stream.JsonParserFactory;
> import org.apache.johnzon.core.JsonParserFactoryImpl;
> import org.junit.Test;
> public class JsonStreamBufferMemoryLeakTestCase {
>   @Test
>   public void normalCase() {
>     Map<String, Object> config = new HashMap<>();
>     config.put(JsonParserFactoryImpl.MAX_STRING_LENGTH, 4);
>     JsonParserFactory parserFactory = JsonProvider.provider()
>         .createParserFactory(config);
>     JsonParser parser = parserFactory
>         .createParser(new StringReader("{ \"a\": \"\\\"a2\\\"\" }"));
>     parser.getValue();
>     parser.close();
>     // parserFactory.bufferProvider.queue.size() = 1
>     // parserFactory.valueBufferProvider.queue.size() = 1
>     System.out.println(parserFactory); // debugger breakpoint for view queue size.
>   }
>   @Test
>   public void leakCase() {
>     Map<String, Object> config = new HashMap<>();
>     config.put(JsonParserFactoryImpl.MAX_STRING_LENGTH, 2);
>     JsonParserFactory parserFactory = JsonProvider.provider()
>         .createParserFactory(config);
>     JsonParser parser = parserFactory
>         .createParser(new StringReader("{ \"a\": \"\\\"a2\\\"\" }"));
>     parser.getValue();
>     parser.close();
>     // expect: parserFactory.bufferProvider.queue.size() = 1 and parserFactory.valueBufferProvider.queue.size() = 1
>     // but: parserFactory.bufferProvider.queue.size() = 2 and parserFactory.valueBufferProvider.queue.size() = 0
>     parser = parserFactory
>         .createParser(new StringReader("{ \"a\": \"\\\"a2\\\"\" }"));
>     parser.getValue();
>     parser.close();
>     // expect: parserFactory.bufferProvider.queue.size() = 1 and parserFactory.valueBufferProvider.queue.size() = 1
>     // but: parserFactory.bufferProvider.queue.size() = 3 and parserFactory.valueBufferProvider.queue.size() = 0
>     parser = parserFactory
>         .createParser(new StringReader("{ \"a\": \"\\\"a2\\\"\" }"));
>     parser.getValue();
>     parser.close();
>     // expect: parserFactory.bufferProvider.queue.size() = 1 and parserFactory.valueBufferProvider.queue.size() = 1
>     // but: parserFactory.bufferProvider.queue.size() = 4 and parserFactory.valueBufferProvider.queue.size() = 0
>     System.out.println(parserFactory); // debugger breakpoint for view queue size.
>   }
> }{code}
> source code analysis:
>   !image-2023-04-09-12-44-02-385.png|width=734,height=185!
>  
> !image-2023-04-09-12-44-39-048.png|width=716,height=203!
> !image-2023-04-09-12-45-10-166.png|width=702,height=122!
>  
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)