You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@xmlgraphics.apache.org by "Matthias Reischenbacher (JIRA)" <ji...@apache.org> on 2015/10/29 20:35:27 UTC
[jira] [Commented] (XGC-93) Calling
DeflaterOutputStream.write(byte) byte by byte causes blocked threads in
multi session-environments
[ https://issues.apache.org/jira/browse/XGC-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981116#comment-14981116 ]
Matthias Reischenbacher commented on XGC-93:
--------------------------------------------
Added a patch based on Andre's suggestion. This indeed can have a big performance impact. In a local test case with around 30 PNG images the fix reduced the processing time during PDF output from approx. 6 seconds to 1 second (!).
> Calling DeflaterOutputStream.write(byte) byte by byte causes blocked threads in multi session-environments
> ----------------------------------------------------------------------------------------------------------
>
> Key: XGC-93
> URL: https://issues.apache.org/jira/browse/XGC-93
> Project: XMLGraphicsCommons
> Issue Type: Improvement
> Components: postscript
> Affects Versions: trunk
> Environment: WebSphere Application Server 7.0 with AIX 7.1 and JDK 1.6.0
> fop.jar, Implementation-Version: 1.1, Build-Id: 20121016-224756-ICT (glenn [Mac OS X 10.8.1 x86_64, Java 1.6.0_35-b10-428-11M3811, Target Java 1.5])
> xmlgraphics-commons-1.5.jar, Implementation-Version: 1.5, Build-Id: 20121016-215324-ICT (glenn [Mac OS X 10.8.1 x86_64, Java 1.6.0_35-b10-428-11M3811])
> Reporter: Andre Klemann
> Labels: easyfix, performance
> Fix For: trunk
>
> Attachments: xgc-93.patch
>
>
> The method org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(OutputStream) from xmlgraphics-commons-1.5.jar calls the method DeflaterOutputStream.write(byte) byte by byte (see lines 242 - 244):
> private boolean optimizedWriteTo(OutputStream out)
> throws IOException {
> if (this.firstTileDump) {
> Raster raster = image.getTile(0, 0);
> DataBuffer buffer = raster.getDataBuffer();
> if (buffer instanceof DataBufferByte) {
> byte[] bytes = ((DataBufferByte) buffer).getData();
> // see determineEncodingColorModel() to see why we permute B and R here
> if (isBGR) {
> for (int i = 0; i < bytes.length; i += 3) {
> out.write(bytes[i + 2]); // 242
> out.write(bytes[i + 1]); // 243
> out.write(bytes[i]); // 244
> }
> } else {
> out.write(bytes);
> }
> return true;
> }
> }
> return false;
> }
> Under WebSphere, in a multi-session environment, this leads to warnings about possibly hanging threads:
> [07.03.13 08:34:22:673 MEZ] 0000000d ThreadMonitor W WSVR0605W: Thread "WebContainer : 1303" (00000547) has been active for 165445 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung.
> at java.util.zip.Deflater.deflateBytes(Native Method)
> at java.util.zip.Deflater.deflate(Deflater.java:306)
> at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:153)
> at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:112)
> at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:89)
> at org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(ImageEncodingHelper.java:242)
> ...
> [07.03.13 08:38:18:886 MEZ] 00000547 ThreadMonitor W WSVR0606W: Thread "WebContainer : 1303" (00000547) was previously reported to be hung but has completed. It was active for approximately 401662 milliseconds. There is/are 0 thread(s) in total in the server that still may be hung.
> [07.03.13 08:40:22:723 MEZ] 0000002d ThreadMonitor W WSVR0605W: Thread "WebContainer : 1323" (0000055b) has been active for 221920 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung.
> at java.util.zip.Deflater.deflateBytes(Native Method)
> at java.util.zip.Deflater.deflate(Deflater.java:306)
> at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:153)
> at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:112)
> at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:89)
> at org.apache.xmlgraphics.ps.ImageEncodingHelper.optimizedWriteTo(ImageEncodingHelper.java:244)
> ...
> The threads will finish their work, but this takes a long time.
> The reason for this is, that the number n of calls to java.util.zip.DeflaterOutputStream.write(byte) might be in a hight range. I have seen cases, where n=4094496. This leads to n calls of DeflaterOutputSteam.write(byte[] b, int off, int len), where off=0 and len=1:
> public void write(byte[] b, int off, int len) throws IOException {
> if (def.finished()) {
> throw new IOException("write beyond end of stream");
> }
> if ((off | len | (off + len) | (b.length - (off + len))) < 0) {
> throw new IndexOutOfBoundsException();
> } else if (len == 0) {
> return;
> }
> if (!def.finished()) {
> // Deflate no more than stride bytes at a time. This avoids
> // excess copying in deflateBytes (see Deflater.c)
> int stride = buf.length;
> for (int i = 0; i < len; i+= stride) {
> def.setInput(b, off + i, Math.min(stride, len - i)); // line 116
> while (!def.needsInput()) {
> deflate(); // line 118
> }
> }
> }
> }
> Since java.util.zip.Deflater internally uses synchronized blocks, esspecially in setInput(byte[], int, int) and deflate(byte[], int, int), one should reduce the number of calls to these methods as much as possible.
> The number of calls of Deflater.setInput(byte[] b, int off, int len) in line 116
> above could, e.g., be reduced by the factor 512 (=buffer.length), if the method ImageEncodingHelper.optimizedWriteTo() would be written in this way:
> private boolean optimizedWriteTo(OutputStream out)
> throws IOException {
> if (this.firstTileDump) {
> Raster raster = image.getTile(0, 0);
> DataBuffer buffer = raster.getDataBuffer();
> if (buffer instanceof DataBufferByte) {
> byte[] bytes = ((DataBufferByte) buffer).getData();
> // see determineEncodingColorModel() to see why we permute B and R here
> if (isBGR) {
> byte[] bytesPermutated = new byte[bytes.length];
> for (int i = 0; i < bytes.length; i += 3) {
> bytesPermutated[i] = bytes[i+2];
> bytesPermutated[i+1] = bytes[i+1];
> bytesPermutated[i+2] = bytes[i];
> }
> out.write(bytesPermutated);
> } else {
> out.write(bytes);
> }
> return true;
> }
> }
> return false;
> }
> In this version, n calls of DeflaterOutputStream.write(byte) are replaced by one call of DeflaterOutputStream(byte[]), which is much faster and offers less blocking potential.
> Without this optimization and with activated zlib compression in FOP 1.1, we always got warnings from WeSphere's thread monitor, that some threads may hang.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: general-help@xmlgraphics.apache.org