You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Andrew Khoury (Jira)" <ji...@apache.org> on 2023/07/08 21:28:00 UTC
[jira] [Comment Edited] (IMAGING-355) Large animated GIF takes too much heap memory in getMetadata
[ https://issues.apache.org/jira/browse/IMAGING-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17741211#comment-17741211 ]
Andrew Khoury edited comment on IMAGING-355 at 7/8/23 9:27 PM:
---------------------------------------------------------------
Hi [~ggregory], yes I tested against master as well and the same issue occurs.
Also, I was able to fix the issue using this change [here|https://github.com/andrewmkhoury/commons-imaging/commit/d4a5f6a039e8ccf232750d5427f88638162db3f1]
{code:java}
readFile(byteSource, true); // instead of false to skip storing the imageData which isn't required for getMetadata and getImageInfo{code}
Here is my updated test code I used to test the issue against git master:
[https://github.com/andrewmkhoury/commons-imaging-gif-test/tree/IMAGING-355]
was (Author: akhoury):
Hi [~ggregory], yes I tested against master as well and the same issue occurs.
Also, I was able to fix the issue using this change [here|https://github.com/andrewmkhoury/commons-imaging/commit/d4a5f6a039e8ccf232750d5427f88638162db3f1]
{code:java}
readFile(byteSource, true); // instead of false to skip storing the imageData which isn't required for getMetadata and getImageInfo{code}
> Large animated GIF takes too much heap memory in getMetadata
> ------------------------------------------------------------
>
> Key: IMAGING-355
> URL: https://issues.apache.org/jira/browse/IMAGING-355
> Project: Commons Imaging
> Issue Type: Bug
> Components: Format: GIF
> Affects Versions: 1.0-alpha3
> Reporter: Andrew Khoury
> Priority: Major
> Attachments: commons-imaging-test.gif, image-2023-06-29-15-18-17-076.png, screenshot-1.png
>
>
> When calling ImageParser.getMetadata on large animated gif files, the java heap consumption is extremely high.
> For example, see the test project I created:
> [https://github.com/andrewmkhoury/commons-imaging-gif-test]
> When calling ImageParser.getMetadata on the attached 5MB gif [^commons-imaging-test.gif], it uses ~1.5GB of heap space. When the max heap is set to -Xmx1488M or lower it fails with this exception. When the heap is set to -Xmx1489M it works.
> {code:java}
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
> at java.base/java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:211)
> at org.apache.commons.imaging.mylzw.MyLzwDecompressor.decompress(MyLzwDecompressor.java:143)
> at org.apache.commons.imaging.formats.gif.GifImageParser.readImageDescriptor(GifImageParser.java:881)
> at org.apache.commons.imaging.formats.gif.GifImageParser.readBlocks(GifImageParser.java:596)
> at org.apache.commons.imaging.formats.gif.GifImageParser.readFile(GifImageParser.java:696)
> at org.apache.commons.imaging.formats.gif.GifImageParser.readFile(GifImageParser.java:680)
> at org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata(GifImageParser.java:485)
> at org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata(GifImageParser.java:58)
> at org.apache.commons.imaging.ImageParser.getMetadata(ImageParser.java:832)
> at Test.main(Test.java:28)
> {code}
> To generate the large gif file I did the following:
> 1. Install ffmpeg and gifsicle
> 2. Use quicktime to create a screen recording
> 3. Generate a gif out of the screen recording
> {code:bash}
> ffmpeg -i ~/Desktop/Screen\ Recording\ 2023-06-29\ at\ 12.09.21\ PM.mov -pix_fmt rgb8 -r 10 commons-imaging-test.gif && gifsicle -O3 commons-imaging-test.gif -o commons-imaging-test.gif
> {code}
> To run the test program:
> {code:bash}
> git clone git@github.com:andrewmkhoury/commons-imaging-gif-test.git
> cd commons-imaging-gif-test
> mvn assembly:assembly
> java -Xmx1g -jar target/IMAGING-test-0.0.1-SNAPSHOT-jar-with-dependencies.jar commons-imaging-test.gif{code}
> Heap analysis via Eclipse MAT shows that the ImageDescriptor.imageData storing the bytes of each frame is the cause of the problem:
> !image-2023-06-29-15-18-17-076.png|width=169,height=93!!screenshot-1.png|width=226,height=86!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)