JSONOutput.toJSON() memory performance while serializing large models

[joben Sinclair <jo...@gmail.com>]

I have a project using Groovy 2.4.9 that I upgraded from Java 1.7 to 1.8
earlier this year.  The analysis performed here was also performed on
Groovy 2.4.9 but I checked the code and verified it also will affect Groovy
3.X+. I have not upgraded Groovy because I wanted to first identify why the
Java 1.8 upgrade introduced crippling OutOfMemoryErrors during JSON
serialization of large data models on my servers with no other changes.

The below commit shows Java version dependent selection of LazyMap, a
component of groovy-json-x.x.x.jar
Commit # c612fe94df7ba871e0618cc0806f1cb23834e394

To evaluate the issue, I made a test framework to serialize a large json
text file and then deserialize it to another file.   The test file was 42Mb
in size and consisted of lots of long arrays, with Maps, and Properties
about 3 - 5 layers deep. Although the same issues were detected on many
different files of varying large sizes.

//Test code sample
def model = new JsonSlurper(type:
JsonParserType.CHARACTER_SOURCE).parseText(text)
def newText = JsonOutput.toJson(model)

When I ran this code with Java 1.7, it only required the VM to have
600-700MB of memory.
When I ran this code with Java 1.8, it would throw an out of memory
exception until I increased the VM to above 3000MB of memory.   Please take
a look at the attached Memory Profile screenshots for both situations.
Larger files resulted in more extreme results.

The memory performance was much better in these conditions when a TreeMap
was used instead of a LinkedHashMap as the underlying data structure for
groovy.json.internal.LazyMap.  I have recompiled
groovy.json.internal.LazyMap to force the usage of TreeMap for all Java
versions and the memory performance at these levels returns to normal
levels.

Could someone look into making JsonOutput.toJson() have a switch for
handling large data models similar to the switch for JsonSlurper to handle
large data models in future versions of Groovy?

Thanks,
John
[image: Memory Profile - Java 1.7.png]
[image: Memory Profile - Java 1.8.png]