You are viewing a plain text version of this content. The canonical link for it is here.
Posted to batik-dev@xmlgraphics.apache.org by bu...@apache.org on 2009/03/16 20:11:18 UTC
DO NOT REPLY [Bug 46863] New: DeflaterOutputStream not closed,
causes OutOfMemoryError
https://issues.apache.org/bugzilla/show_bug.cgi?id=46863
Summary: DeflaterOutputStream not closed, causes
OutOfMemoryError
Product: Batik
Version: 1.7
Platform: PC
OS/Version: Windows XP
Status: NEW
Severity: major
Priority: P2
Component: SVGGraphics2D
AssignedTo: batik-dev@xmlgraphics.apache.org
ReportedBy: liamsb@ca.ibm.com
The close() method should be called on java.util.zip.DeflaterOutputStream
instances when they are no longer needed. In particular, the writeIDAT() and
writeZTXT() methods in org.apache.batik.ext.awt.image.codec.png.PNGImageEncoder
do not call close() on DeflaterOutputStream.
Since close() is not called, end() is not called on the java.util.zip.Deflater
instance until it is garbage collected. Due to the implementation of Deflater,
this causes an OutOfMemoryError to occur in some cases when the PNGImageEncoder
is invoked repeatedly. This issue with Deflater is described in detail at
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4797189, where the
recommended solution is to ensure that end() is called on the Deflater in a
timely fashion.
Tested in Batik versions 1.5 and 1.7.
Stack trace:
[...]
Caused by: java.lang.OutOfMemoryError
at java.util.zip.Deflater.init(Native Method)
at java.util.zip.Deflater.<init>(Deflater.java:134)
at java.util.zip.Deflater.<init>(Deflater.java:143)
at org.apache.batik.ext.awt.image.codec.PNGImageEncoder.writeIDAT(Unknown
Source)
at org.apache.batik.ext.awt.image.codec.PNGImageEncoder.encode(Unknown
Source)
at org.apache.batik.svggen.ImageHandlerBase64Encoder.encodeImage(Unknown
Source)
at org.apache.batik.svggen.ImageHandlerBase64Encoder.handleHREF(Unknown
Source)
at org.apache.batik.svggen.ImageHandlerBase64Encoder.handleHREF(Unknown
Source)
at org.apache.batik.svggen.DefaultImageHandler.handleImage(Unknown Source)
at org.apache.batik.svggen.SimpleImageHandler.handleImage(Unknown Source)
at org.apache.batik.svggen.SVGGraphics2D.drawImage(Unknown Source)
at org.apache.batik.svggen.SVGGraphics2D.drawImage(Unknown Source)
[...]
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: batik-dev-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-dev-help@xmlgraphics.apache.org
DO NOT REPLY [Bug 46863] DeflaterOutputStream not closed, causes
OutOfMemoryError
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46863
Cameron McCormack <ca...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #2 from Cameron McCormack <ca...@apache.org> 2009-03-18 20:30:47 PST ---
(In reply to comment #1)
> Created an attachment (id=23388)
--> (https://issues.apache.org/bugzilla/attachment.cgi?id=23388) [details]
> Untested patch adapted from proposal
Thanks.
> First of all, I'm not sure if we should fix this, as it's a Java bug -- should
> this be marked as invalid instead?
If we can easily work around a known problem, I think we should.
> Nevertheless, by making a short investigation and taking a look at the class
> documentation [1], it seems that simply replacing finish() by close() would fix
> this and also shouldn't hurt anything so here goes an untested patch.
Interestingly, it seems Sun's DeflaterOutputStream won't call end() on its
Deflater from inside DeflaterOutputStream.close() if a Deflater was supplied to
the DeflaterOutputStream constructor. We might need to call end() on the
Deflater directly. But I'd like to test this first, so...
> If anyone has the opportunity to review/test this patch, please follow up with
> results. Other proposals are (naturally) welcome. :-)
...liamsb, can you provide a short test case that demonstrates the OOM problem?
I tried with this small test of PNGImageEncoder, but memory usage didn't grow:
import java.awt.image.BufferedImage;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import org.apache.batik.ext.awt.image.codec.png.PNGImageEncoder;
public class A {
public static void main(String[] args) {
BufferedImage bi = new BufferedImage(100, 100,
BufferedImage.TYPE_INT_ARGB);
for (;;) {
PNGImageEncoder e = new PNGImageEncoder(new
ByteArrayOutputStream(), null);
try {
e.encode(bi);
} catch (IOException ex) {
}
}
}
}
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: batik-dev-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-dev-help@xmlgraphics.apache.org
DO NOT REPLY [Bug 46863] DeflaterOutputStream not closed, causes
OutOfMemoryError
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46863
liamsb@ca.ibm.com changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #3 from liamsb@ca.ibm.com 2009-03-26 08:48:01 PST ---
Hi, thanks for looking into this issue.
Here is a test case that results in the OOM exception after running for about
10 minutes on my computer:
import java.awt.RenderingHints;
import java.awt.RenderingHints.Key;
import java.awt.geom.AffineTransform;
import java.awt.image.AffineTransformOp;
import java.awt.image.BufferedImage;
import java.awt.image.BufferedImageOp;
import java.util.HashMap;
import org.apache.batik.dom.GenericDOMImplementation;
import org.apache.batik.svggen.SVGGraphics2D;
import org.w3c.dom.Document;
public class A {
public static void main(String[] args) {
BufferedImage bi = new BufferedImage(100, 100,
BufferedImage.TYPE_INT_ARGB);
Document document =
GenericDOMImplementation.getDOMImplementation().createDocument(null, "svg",
null);
SVGGraphics2D svgGraphics = new SVGGraphics2D(document);
BufferedImageOp bio = new AffineTransformOp(new AffineTransform(), new
RenderingHints(new HashMap<Key, Object>()));
for (;;) {
//for (int i = 0; i < 200; i++) //20
svgGraphics.drawImage(bi, bio, 0, 0);
//System.gc(); //22
}
}
}
If lines 20 and 22 are uncommented, then the program seems to run forever. I
did not experience an OOM exception after running the program for more than an
hour.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: batik-dev-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-dev-help@xmlgraphics.apache.org
DO NOT REPLY [Bug 46863] DeflaterOutputStream not closed, causes
OutOfMemoryError
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46863
--- Comment #4 from Helder Magalhães <he...@gmail.com> 2009-04-20 06:40:17 PST ---
(In reply to comment #3)
> Here is a test case that results in the OOM exception after running for about
> 10 minutes on my computer:
>
[...]
> for (;;) {
> //for (int i = 0; i < 200; i++) //20
> svgGraphics.drawImage(bi, bio, 0, 0);
> //System.gc(); //22
> }
[...]
>
> If lines 20 and 22 are uncommented, then the program seems to run forever. I
> did not experience an OOM exception after running the program for more than an
> hour.
As far as I know, automatic GC is a low priority task which runs in the
background. As you are running this within an infinite loop ("for (;;)"), the
GC will probably never be allowed a chance to run and therefore the OOM error.
Also, the fact that explicitely invoking GC seems to work around the issue
makes me suggests that this may be an invalid report: it hints that the thing
is being garbage collected, maybe just not fast enough. On the other hand, you
may not have tested this enough (more than one hour may not have been enough).
I'd suggest making a few experiments (leaving the test application run
overnight, using the verbose GC switches [1], etc.) and report results back.
There's an interesting article [1] on GC tuning you may be interested in. ;-)
[1] http://www.petefreitag.com/articles/gctuning/
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: batik-dev-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-dev-help@xmlgraphics.apache.org
DO NOT REPLY [Bug 46863] DeflaterOutputStream not closed, causes
OutOfMemoryError
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46863
--- Comment #1 from Helder Magalhães <he...@gmail.com> 2009-03-17 04:28:51 PST ---
Created an attachment (id=23388)
--> (https://issues.apache.org/bugzilla/attachment.cgi?id=23388)
Untested patch adapted from proposal
First of all, I'm not sure if we should fix this, as it's a Java bug -- should
this be marked as invalid instead?
Nevertheless, by making a short investigation and taking a look at the class
documentation [1], it seems that simply replacing finish() by close() would fix
this and also shouldn't hurt anything so here goes an untested patch.
If anyone has the opportunity to review/test this patch, please follow up with
results. Other proposals are (naturally) welcome. :-)
Hope this helps,
Helder Magalhães
[1]
http://java.sun.com/j2se/1.4.2/docs/api/java/util/zip/DeflaterOutputStream.html
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: batik-dev-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-dev-help@xmlgraphics.apache.org
DO NOT REPLY [Bug 46863] DeflaterOutputStream not closed, causes
OutOfMemoryError
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46863
--- Comment #5 from liamsb@ca.ibm.com 2009-04-21 08:09:25 PST ---
Hi,
I have run some more tests. Here are the results:
With the gc() calls, the test case ran out of java heap space after running for
almost four hours:
Exception in thread "main" java.lang.OutOfMemoryError
at java.awt.image.DataBufferInt.<init>(DataBufferInt.java:52)
at
java.awt.image.SinglePixelPackedSampleModel.createDataBuffer(SinglePixelPackedSampleModel.java:231)
at
sun.awt.image.IntegerInterleavedRaster.<init>(IntegerInterleavedRaster.java)
at
sun.awt.image.IntegerInterleavedRaster.createCompatibleWritableRaster(IntegerInterleavedRaster.java:527)
at
java.awt.image.AffineTransformOp.createCompatibleDestImage(AffineTransformOp.java:459)
at java.awt.image.AffineTransformOp.filter(AffineTransformOp.java:220)
at org.apache.batik.svggen.SVGGraphics2D.drawImage(SVGGraphics2D.java:995)
at A.main(A.java:21)
Since we are drawing images repeatedly to the SVGGraphics2D, it's not
surprising that it eventually runs out of java heap space.
Running the test case (without the gc() calls) in a profiler shows frequent
garbage collection and still crashes within 10 minutes. (I saw between 2-50
garbage collections per 10 second period, and 350+ total collections.) Thus,
it seems unlikely that the OOME is caused by the garbage collector not having a
chance to run.
My understanding is that the jvm will not throw an OutOfMemoryError on an
allocation without first running a garbage collection to attempt to free enough
space for the allocation.
The Sun defect (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4797189)
explains that the garbage collector is not aware of the native memory
associated with the Deflater instances, and therefore does not anticipate that
freeing these objects will provide the needed native heap space.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: batik-dev-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-dev-help@xmlgraphics.apache.org
DO NOT REPLY [Bug 46863] DeflaterOutputStream not closed, causes
OutOfMemoryError
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46863
--- Comment #6 from akatti@ca.ibm.com 2009-05-21 09:22:23 PST ---
Hi,
Any update on the suggested fix?
Thank you.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: batik-dev-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-dev-help@xmlgraphics.apache.org
DO NOT REPLY [Bug 46863] DeflaterOutputStream not closed, causes
OutOfMemoryError
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=46863
--- Comment #7 from Helder Magalhães <he...@gmail.com> 2010-02-21 15:58:10 UTC ---
(In reply to comment #6)
> Any update on the suggested fix?
Thomas seems to have addressed this in revision 796596. :-)
I'm not sure if this has caused bug 48693, though, so I'm leaving this issue
open until further clarifications are made. :-|
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: batik-dev-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-dev-help@xmlgraphics.apache.org