You are viewing a plain text version of this content. The canonical link for it is here.
Posted to cvs@cocoon.apache.org by jo...@apache.org on 2008/06/10 04:40:16 UTC

svn commit: r665954 - in /cocoon/branches/BRANCH_2_1_X: ./ src/java/org/apache/cocoon/environment/ src/java/org/apache/cocoon/environment/wrapper/ src/java/org/apache/cocoon/util/ src/test/org/apache/cocoon/util/

Author: joerg
Date: Mon Jun  9 19:40:16 2008
New Revision: 665954

URL: http://svn.apache.org/viewvc?rev=665954&view=rev
Log:
as a follow-up of COCOON-2168 (http://marc.info/?t=120899868500003&r=1&w=4):
Cocoon's pipeline buffer increases from an initial buffer size of 8192 bytes to the configurable flush buffer size rather than allocating the complete buffer beforehand.

Added:
    cocoon/branches/BRANCH_2_1_X/src/test/org/apache/cocoon/util/BufferedOutputStreamTestCase.java
Modified:
    cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/environment/AbstractEnvironment.java
    cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/environment/wrapper/EnvironmentWrapper.java
    cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/util/BufferedOutputStream.java
    cocoon/branches/BRANCH_2_1_X/status.xml

Modified: cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/environment/AbstractEnvironment.java
URL: http://svn.apache.org/viewvc/cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/environment/AbstractEnvironment.java?rev=665954&r1=665953&r2=665954&view=diff
==============================================================================
--- cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/environment/AbstractEnvironment.java (original)
+++ cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/environment/AbstractEnvironment.java Mon Jun  9 19:40:16 2008
@@ -479,28 +479,27 @@
      *
      * <br>This method replaces {@link #getOutputStream()}.
      */
-    public OutputStream getOutputStream(int bufferSize)
-    throws IOException {
-
+    public OutputStream getOutputStream(int bufferSize) throws IOException {
         // This method could be called several times during request processing
         // with differing values of bufferSize and should handle this situation
         // correctly.
-
-        if (bufferSize == -1) {
-            if (this.secureOutputStream == null) {
-                this.secureOutputStream = new BufferedOutputStream(this.outputStream);
-            }
-            return this.secureOutputStream;
-        } else if (bufferSize == 0) {
+        // FIXME (JH): Question is what "correctly" means. The current behavior
+        // seems to be inconsistent: On a second call with bufferSize == 0 we
+        // discard whatever the first called set up. With a bufferSize != 0 the
+        // first call's setup is preserved. Why not always creating new
+        // BufferedOutputStream in the else block replacing a potentially
+        // existing one?
+        if (bufferSize == 0) {
             // Discard secure output stream if it was created before.
             if (this.secureOutputStream != null) {
                 this.secureOutputStream = null;
             }
             return this.outputStream;
         } else {
-            // FIXME Triple buffering, anyone?
-            this.outputStream = new java.io.BufferedOutputStream(this.outputStream, bufferSize);
-            return this.outputStream;
+            if (this.secureOutputStream == null) {
+                this.secureOutputStream = new BufferedOutputStream(this.outputStream, bufferSize);
+            }
+            return this.secureOutputStream;
         }
     }
 
@@ -513,8 +512,8 @@
     */
     public boolean tryResetResponse()
     throws IOException {
-        if (this.secureOutputStream != null) {
-            this.secureOutputStream.clearBuffer();
+        if (this.secureOutputStream != null && this.secureOutputStream.isResettable()) {
+            this.secureOutputStream.reset();
             return true;
         }
         return false;
@@ -526,8 +525,10 @@
     public void commitResponse()
     throws IOException {
         if (this.secureOutputStream != null) {
-            this.setContentLength(this.secureOutputStream.getCount());
-            this.secureOutputStream.realFlush();
+            if (this.secureOutputStream.isResettable()) {
+                this.setContentLength(this.secureOutputStream.getCount());
+            }
+            this.secureOutputStream.flush();
         } else if (this.outputStream != null) {
             this.outputStream.flush();
         }

Modified: cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/environment/wrapper/EnvironmentWrapper.java
URL: http://svn.apache.org/viewvc/cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/environment/wrapper/EnvironmentWrapper.java?rev=665954&r1=665953&r2=665954&view=diff
==============================================================================
--- cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/environment/wrapper/EnvironmentWrapper.java (original)
+++ cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/environment/wrapper/EnvironmentWrapper.java Mon Jun  9 19:40:16 2008
@@ -317,7 +317,7 @@
     public boolean tryResetResponse() throws IOException {
         final OutputStream outputStream = getOutputStream(-1);
         if (outputStream != null && outputStream instanceof BufferedOutputStream) {
-            ((BufferedOutputStream)outputStream).clearBuffer();
+            ((BufferedOutputStream)outputStream).reset();
             return true;
         } else {
           return super.tryResetResponse();
@@ -330,7 +330,7 @@
     public void commitResponse() throws IOException {
         final OutputStream outputStream = getOutputStream(-1);
         if (outputStream != null && outputStream instanceof BufferedOutputStream) {
-            ((BufferedOutputStream)outputStream).realFlush();
+            outputStream.flush();
         } else {
           super.commitResponse();
         }

Modified: cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/util/BufferedOutputStream.java
URL: http://svn.apache.org/viewvc/cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/util/BufferedOutputStream.java?rev=665954&r1=665953&r2=665954&view=diff
==============================================================================
--- cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/util/BufferedOutputStream.java (original)
+++ cocoon/branches/BRANCH_2_1_X/src/java/org/apache/cocoon/util/BufferedOutputStream.java Mon Jun  9 19:40:16 2008
@@ -5,9 +5,9 @@
  * The ASF licenses this file to You under the Apache License, Version 2.0
  * (the "License"); you may not use this file except in compliance with
  * the License.  You may obtain a copy of the License at
- * 
+ *
  *      http://www.apache.org/licenses/LICENSE-2.0
- * 
+ *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,
  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -21,63 +21,93 @@
 import java.io.OutputStream;
 
 /**
- * This class is like the {@link java.io.BufferedOutputStream} but it
- * extends it with a logic to count the number of bytes written to
- * the output stream.
- * 
+ * This class is similar to the {@link java.io.BufferedOutputStream}. In
+ * addition it provides an increasing buffer, the possibility to reset the
+ * buffer and it counts the number of bytes written to the output stream.
+ *
  * @author <a href="mailto:cziegeler@apache.org">Carsten Ziegeler</a>
  * @version CVS $Id$
  * @since   2.1
  */
-public final class BufferedOutputStream extends FilterOutputStream {
-    
-    protected byte buf[];
-
-    protected int count;
-    
-    /**
-     * Creates a new buffered output stream to write data to the 
-     * specified underlying output stream with a default 8192-byte
-     * buffer size.
+public class BufferedOutputStream extends FilterOutputStream {
+
+    private byte buffer[];
+
+    private int count;
+    private int totalCount;
+
+    private final int flushBufferSize;
+
+    /**
+     * Creates a new buffered output stream to write data to the specified
+     * underlying output stream with a default flush buffer size of 32768 bytes
+     * and a default initial buffer size of 8192 bytes.
+     *
+     * @param   out   the underlying output stream.
+     */
+    public BufferedOutputStream(final OutputStream out) {
+        this(out, 32768);
+    }
+
+    /**
+     * Creates a new buffered output stream to write data to the specified
+     * underlying output stream with the specified flush buffer size and a
+     * default initial buffer size of 8192 bytes.
      *
      * @param   out   the underlying output stream.
      */
-    public BufferedOutputStream(OutputStream out) {
-        this(out, 8192);
+    public BufferedOutputStream(final OutputStream out, final int flushBufferSize) {
+        this(out, flushBufferSize, 8192);
     }
 
     /**
-     * Creates a new buffered output stream to write data to the 
-     * specified underlying output stream with the specified buffer 
-     * size. 
+     * Creates a new buffered output stream to write data to the specified
+     * underlying output stream with the specified buffer sizes.
      *
-     * @param   out    the underlying output stream.
-     * @param   size   the buffer size.
-     * @exception IllegalArgumentException if size <= 0.
+     * @param out    the underlying output stream.
+     * @param flushBufferSize  the buffer size when the stream is flushed. Must
+     *                         be greater than 0 or -1 meaning the stream never
+     *                         flushes itself.
+     * @param initialBufferSize  the initial buffer size. Must be greater than 0.
+     *                           Will be limited to the flush buffer size.
      */
-    public BufferedOutputStream(OutputStream out, int size) {
+    public BufferedOutputStream(final OutputStream out,
+                                final int flushBufferSize,
+                                final int initialBufferSize) {
         super(out);
-        if (size <= 0) {
-            throw new IllegalArgumentException("Buffer size <= 0");
+        if (flushBufferSize <= 0 && flushBufferSize != -1) {
+            throw new IllegalArgumentException("Flush buffer size <= 0 && != -1");
         }
-        this.buf = new byte[size];
+        if (initialBufferSize <= 0) {
+            throw new IllegalArgumentException("Initial buffer size <= 0");
+        }
+        int actualInitialBufferSize =
+            flushBufferSize > 0 && initialBufferSize > flushBufferSize ? flushBufferSize
+                                                                       : initialBufferSize;
+        this.buffer = new byte[actualInitialBufferSize];
+        this.flushBufferSize = flushBufferSize;
     }
 
     /**
-     * Writes the specified byte to this buffered output stream. 
+     * Writes the specified byte to this buffered output stream.
      *
      * @param      b   the byte to be written.
      * @exception  IOException  if an I/O error occurs.
      */
-    public void write(int b) throws IOException {
-        if (this.count >= this.buf.length) {
-            this.incBuffer();
+    public void write(final int b) throws IOException {
+        if (this.count == this.buffer.length) {
+            // No need to check return value, can NEVER be 0.
+            this.increaseBuffer(1);
         }
-        this.buf[count++] = (byte)b;
+
+        this.buffer[this.count++] = (byte)b;
+        this.totalCount++;
+
+        checkForFlush();
     }
 
     /**
-     * Writes <code>len</code> bytes from the specified byte array 
+     * Writes <code>len</code> bytes from the specified byte array
      * starting at offset <code>off</code> to this buffered output stream.
      *
      * <p> Ordinarily this method stores bytes from the given array into this
@@ -92,22 +122,54 @@
      * @param      len   the number of bytes to write.
      * @exception  IOException  if an I/O error occurs.
      */
-    public void write(byte b[], int off, int len) throws IOException {
-        while (len > buf.length - count) {
-            this.incBuffer();
+    public void write(final byte[] b, final int off, final int len) throws IOException {
+        int free = this.buffer.length - this.count;
+        int necessaryIncrease = len - free;
+        if (necessaryIncrease > 0) {
+            int actualIncrease = this.increaseBuffer(necessaryIncrease);
+            if (actualIncrease < necessaryIncrease) {
+                free += actualIncrease;
+                // Needs to be written in chunks by recursive calls to this method.
+                writeToBuffer(b, off, free);
+                int newOff = off + free;
+                int newLen = len - free;
+                while (newLen > 0) {
+                    writeToBuffer(b, newOff, Math.min(newLen, this.flushBufferSize));
+                    newOff += this.flushBufferSize;
+                    newLen -= this.flushBufferSize;
+                }
+                return;
+            }
+        }
+
+        writeToBuffer(b, off, len);
+    }
+
+    private void writeToBuffer(final byte[] b, final int off, final int len) throws IOException {
+        System.arraycopy(b, off, this.buffer, this.count, len);
+        this.count += len;
+        this.totalCount += len;
+
+        checkForFlush();
+    }
+
+    private void checkForFlush() throws IOException {
+        if (this.count == this.flushBufferSize) {
+            flush();
         }
-        System.arraycopy(b, off, buf, count, len);
-        count += len;
     }
 
     /**
-     * Flushes this buffered output stream. 
-     * We don't flush here, flushing is done during closing.
+     * Flushes this buffered output stream.
      *
      * @exception  IOException  if an I/O error occurs.
      */
     public void flush() throws IOException {
-        // nothing
+        if (this.count > 0) {
+            this.out.write(this.buffer, 0, this.count);
+            this.count = 0;
+        }
+        this.out.flush();
     }
 
     /**
@@ -117,52 +179,73 @@
      * @exception  IOException  if an I/O error occurs.
      */
     public void close() throws IOException {
-        realFlush();
-        super.close ();
+        flush();
+        super.close();
     }
 
     /**
-     * Flushes this buffered output stream. 
+     * Increase the buffer by at least as many bytes as specified via the
+     * parameter, but not exceeding the flushBufferSize. The actual increase is
+     * returned.
+     *
+     * @return  increase in buffer size.
      */
-    public void realFlush() throws IOException {
-        this.writeBuffer();
-        this.out.flush();
+    private int increaseBuffer(final int increase) {
+        int oldLength = this.buffer.length;
+        if (oldLength == this.flushBufferSize) {
+            // fast way out
+            return 0;
+        }
+
+        int newLength = oldLength;
+        int actualIncrease;
+        do {
+            newLength = newLength * 2;
+            if (this.flushBufferSize > 0 && newLength >= this.flushBufferSize) {
+                newLength = this.flushBufferSize;
+                actualIncrease = newLength - oldLength;
+                break;
+            }
+            actualIncrease = newLength - oldLength;
+        } while (actualIncrease < increase);
+
+        // Because of the "fast way out" above at this point there should always be an increase.
+        byte[] newBuffer = new byte[newLength];
+        if (this.count > 0) {
+            System.arraycopy(this.buffer, 0, newBuffer, 0, this.count);
+        }
+        this.buffer = newBuffer;
+        return actualIncrease;
     }
-    
+
     /**
-     * Write the buffer
+     * Clear the buffer.
+     * @deprecated Public access is deprecated. Use {@link #reset()} instead.
      */
-    private void writeBuffer() 
-    throws IOException {
-        if (this.count > 0) {
-            this.out.write(this.buf, 0, this.count);
-            this.clearBuffer();
-        }
+    public void clearBuffer() {
+        this.totalCount -= this.count;
+        this.count = 0;
     }
 
     /**
-     * Increment the buffer
+     * Reset the BufferedOutputStream to the last {@link #flush()}.
      */
-    private void incBuffer() {
-        // currently we double the buffer size
-        // this is not so fast but is a very simple logic
-        byte[] newBuf = new byte[this.buf.length * 2];
-        System.arraycopy(this.buf, 0, newBuf, 0, this.buf.length);
-        this.buf = newBuf;
+    public void reset() {
+        clearBuffer();
     }
-    
+
     /**
-     * Clear/reset the buffer
+     * @return if it is possible to reset the buffer completely, i.e. nothing has been flushed yet.
      */
-    public void clearBuffer() {
-        this.count = 0;
+    public boolean isResettable() {
+        return this.count == this.totalCount;
     }
 
     /**
      * Return the size of the current buffer
      */
     public int getCount() {
-        return this.count;
+        return this.totalCount;
     }
-}
 
+}

Added: cocoon/branches/BRANCH_2_1_X/src/test/org/apache/cocoon/util/BufferedOutputStreamTestCase.java
URL: http://svn.apache.org/viewvc/cocoon/branches/BRANCH_2_1_X/src/test/org/apache/cocoon/util/BufferedOutputStreamTestCase.java?rev=665954&view=auto
==============================================================================
--- cocoon/branches/BRANCH_2_1_X/src/test/org/apache/cocoon/util/BufferedOutputStreamTestCase.java (added)
+++ cocoon/branches/BRANCH_2_1_X/src/test/org/apache/cocoon/util/BufferedOutputStreamTestCase.java Mon Jun  9 19:40:16 2008
@@ -0,0 +1,210 @@
+package org.apache.cocoon.util;
+
+import java.io.ByteArrayOutputStream;
+import java.util.Arrays;
+
+import junit.framework.TestCase;
+
+public class BufferedOutputStreamTestCase extends TestCase {
+
+    public void testUnlimitedBuffering() throws Exception
+    {
+        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+        BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(byteArrayOutputStream, -1, 32);
+
+        byte[] bytes = new byte[1024];
+        for (int i = 0; i < bytes.length; i++) {
+            bytes[i] = (byte) i;
+        }
+
+        assertEquals("Count of buffered and flushed bytes is wrong", 0, bufferedOutputStream.getCount());
+        assertEquals("Streamed bytes is not empty", 0, byteArrayOutputStream.size());
+
+        // write bytes into buffer
+        bufferedOutputStream.write(bytes);
+
+        assertEquals("Count of buffered and flushed bytes is wrong", bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Streamed bytes is not empty", 0, byteArrayOutputStream.size());
+
+        // reset, all back to beginning
+        bufferedOutputStream.reset();
+
+        assertEquals("Count of buffered and flushed bytes is wrong", 0, bufferedOutputStream.getCount());
+        assertEquals("Streamed bytes is not empty", 0, byteArrayOutputStream.size());
+
+        // write bytes into buffer
+        bufferedOutputStream.write(bytes);
+
+        assertEquals("Count of buffered and flushed bytes is wrong", bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Streamed bytes is not empty", 0, byteArrayOutputStream.size());
+
+        // eventually flush
+        bufferedOutputStream.flush();
+        byte[] streamedBytes = byteArrayOutputStream.toByteArray();
+
+        assertEquals("Count of buffered and flushed bytes is wrong", bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", bytes.length, byteArrayOutputStream.size());
+        assertTrue("Streamed bytes are wrong", Arrays.equals(bytes, streamedBytes));
+    }
+
+    public void testCompleteBuffering() throws Exception
+    {
+        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+        BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(byteArrayOutputStream, 1024, 32);
+
+        byte[] bytes = new byte[1000];
+        for (int i = 0; i < bytes.length; i++) {
+            bytes[i] = (byte) i;
+        }
+
+        assertEquals("Count of buffered and flushed bytes is wrong", 0, bufferedOutputStream.getCount());
+        assertEquals("Streamed bytes is not empty", 0, byteArrayOutputStream.size());
+
+        // write bytes into buffer
+        bufferedOutputStream.write(bytes);
+
+        assertEquals("Count of buffered and flushed bytes is wrong", bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Streamed bytes is not empty", 0, byteArrayOutputStream.size());
+
+        // reset, all back to beginning
+        bufferedOutputStream.reset();
+
+        assertEquals("Count of buffered and flushed bytes is wrong", 0, bufferedOutputStream.getCount());
+        assertEquals("Streamed bytes is not empty", 0, byteArrayOutputStream.size());
+
+        // write bytes into buffer
+        bufferedOutputStream.write(bytes);
+
+        assertEquals("Count of buffered and flushed bytes is wrong", bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Streamed bytes is not empty", 0, byteArrayOutputStream.size());
+
+        // eventually flush
+        bufferedOutputStream.flush();
+        byte[] streamedBytes = byteArrayOutputStream.toByteArray();
+
+        assertEquals("Count of buffered and flushed bytes is wrong", bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", bytes.length, byteArrayOutputStream.size());
+        assertTrue("Streamed bytes are wrong", Arrays.equals(bytes, streamedBytes));
+    }
+
+    public void testCompleteBufferingHittingFlushSize() throws Exception
+    {
+        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+        BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(byteArrayOutputStream, 1024, 32);
+
+        byte[] bytes = new byte[1024];
+        for (int i = 0; i < bytes.length; i++) {
+            bytes[i] = (byte) i;
+        }
+
+        assertEquals("Count of buffered and flushed bytes is wrong", 0, bufferedOutputStream.getCount());
+        assertEquals("Streamed bytes is not empty", 0, byteArrayOutputStream.size());
+
+        // write bytes into buffer, buffer will be flushed automatically
+        bufferedOutputStream.write(bytes);
+        byte[] streamedBytes = byteArrayOutputStream.toByteArray();
+
+        assertEquals("Count of buffered and flushed bytes is wrong", bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", bytes.length, byteArrayOutputStream.size());
+        assertTrue("Streamed bytes are wrong", Arrays.equals(bytes, streamedBytes));
+
+        // reset should not change anything
+        bufferedOutputStream.reset();
+        streamedBytes = byteArrayOutputStream.toByteArray();
+
+        assertEquals("Count of buffered and flushed bytes is wrong", bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", bytes.length, byteArrayOutputStream.size());
+        assertTrue("Streamed bytes are wrong", Arrays.equals(bytes, streamedBytes));
+    }
+
+    public void testBufferingExceedingFlushSize() throws Exception
+    {
+        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+        int flushSize = 512;
+        BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(byteArrayOutputStream, flushSize, 32);
+
+        byte[] bytes = new byte[2000];
+        for (int i = 0; i < bytes.length; i++) {
+            bytes[i] = (byte) i;
+        }
+
+        assertEquals("Count of buffered and flushed bytes is wrong", 0, bufferedOutputStream.getCount());
+        assertEquals("Streamed bytes is not empty", 0, byteArrayOutputStream.size());
+
+        // write bytes into buffer
+        bufferedOutputStream.write(bytes);
+        int flushed = ((int)bytes.length / flushSize) * flushSize;
+
+        assertEquals("Count of buffered and flushed bytes is wrong", bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", flushed, byteArrayOutputStream.size());
+
+        // reset, back to last flush
+        bufferedOutputStream.reset();
+
+        assertEquals("Count of buffered and flushed bytes is wrong", flushed, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", flushed, byteArrayOutputStream.size());
+
+        // write bytes into buffer
+        bufferedOutputStream.write(bytes, flushed, bytes.length - flushed);
+
+        assertEquals("Count of buffered and flushed bytes is wrong", bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", flushed, byteArrayOutputStream.size());
+
+        // eventually flush
+        bufferedOutputStream.flush();
+        byte[] streamedBytes = byteArrayOutputStream.toByteArray();
+
+        assertEquals("Count of buffered and flushed bytes is wrong", bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", bytes.length, byteArrayOutputStream.size());
+        assertTrue("Streamed bytes are wrong", Arrays.equals(bytes, streamedBytes));
+    }
+
+    public void testBufferingExceedingFlushSizeWithInitialContent() throws Exception
+    {
+        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+        int flushSize = 512;
+        BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(byteArrayOutputStream, flushSize, 32);
+
+        byte[] bytes = new byte[2000];
+        for (int i = 0; i < bytes.length; i++) {
+            bytes[i] = (byte) i;
+        }
+
+        int initialContentSize = 50;
+        bufferedOutputStream.write(bytes, 0, initialContentSize);
+
+        assertEquals("Count of buffered and flushed bytes is wrong", initialContentSize, bufferedOutputStream.getCount());
+        assertEquals("Streamed bytes is not empty", 0, byteArrayOutputStream.size());
+
+        // write bytes into buffer
+        bufferedOutputStream.write(bytes);
+        int flushed = ((int)((initialContentSize + bytes.length) / flushSize)) * flushSize;
+
+        assertEquals("Count of buffered and flushed bytes is wrong", initialContentSize + bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", flushed, byteArrayOutputStream.size());
+
+        // reset, back to last flush
+        bufferedOutputStream.reset();
+
+        assertEquals("Count of buffered and flushed bytes is wrong", flushed, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", flushed, byteArrayOutputStream.size());
+
+        // write bytes into buffer
+        bufferedOutputStream.write(bytes, flushed - initialContentSize, bytes.length - flushed + initialContentSize);
+
+        assertEquals("Count of buffered and flushed bytes is wrong", initialContentSize + bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", flushed, byteArrayOutputStream.size());
+
+        // eventually flush
+        bufferedOutputStream.flush();
+        byte[] streamedBytes = byteArrayOutputStream.toByteArray();
+
+        assertEquals("Count of buffered and flushed bytes is wrong", initialContentSize + bytes.length, bufferedOutputStream.getCount());
+        assertEquals("Count of streamed bytes is wrong", initialContentSize + bytes.length, byteArrayOutputStream.size());
+        byte[] expectedBytes = new byte[2050];
+        System.arraycopy(bytes, 0, expectedBytes, 0, initialContentSize);
+        System.arraycopy(bytes, 0, expectedBytes, initialContentSize, bytes.length);
+        assertTrue("Streamed bytes are wrong", Arrays.equals(expectedBytes, streamedBytes));
+    }
+
+}

Modified: cocoon/branches/BRANCH_2_1_X/status.xml
URL: http://svn.apache.org/viewvc/cocoon/branches/BRANCH_2_1_X/status.xml?rev=665954&r1=665953&r2=665954&view=diff
==============================================================================
--- cocoon/branches/BRANCH_2_1_X/status.xml (original)
+++ cocoon/branches/BRANCH_2_1_X/status.xml Mon Jun  9 19:40:16 2008
@@ -182,6 +182,10 @@
 
   <changes>
   <release version="2.1.12" date="TBD">
+    <action dev="JH" type="update">
+      Core: Cocoon's pipeline buffer increases from an initial buffer size of 8192 bytes to the configurable flush
+      buffer size rather than allocating the complete buffer beforehand.
+    </action>
     <action dev="AG" type="fix" fixes-bug="COCOON-2209" due-to="Reynaldo Porras" due-to-email="rporras@agssa.net">
       POI Block: formatted style regions stop creating style at 2000 rows.
     </action>
@@ -246,7 +250,7 @@
     </action>
     <action dev="JH" type="fix">
       Core: Close streams properly after copying Parts (MultipartParser). Allow to access InputStream of PartInMemory
-      multiple times. 
+      multiple times.
     </action>
     <action dev="CZ" type="fix">
       Portal: Remove user management tools.
@@ -435,10 +439,10 @@
     <action dev="JQ" type="update">
       Updating Dojo to 0.4.1.
       <ul>
-      	<li>use namespaces for loading widgets</li>
-      	<li>cleanup of client-side libraries into cocoon.forms.common</li>
-      	<li>deprecation and replacement of functions in forms_lib.js cocoon.js CFormsForm.js</li>
-      	<li>all cforms forms now use a dojo widget (forms:SimpleForm or forms:AjaxForm)</li>
+        <li>use namespaces for loading widgets</li>
+        <li>cleanup of client-side libraries into cocoon.forms.common</li>
+        <li>deprecation and replacement of functions in forms_lib.js cocoon.js CFormsForm.js</li>
+        <li>all cforms forms now use a dojo widget (forms:SimpleForm or forms:AjaxForm)</li>
       </ul>
     </action>
     <action dev="CZ" type="fix" fixes-bug="COCOON-1776" due-to="Vincent Demay">



Re: svn commit: r665954 - in /cocoon/branches/BRANCH_2_1_X: ./ src/java/org/apache/cocoon/environment/ src/java/org/apache/cocoon/environment/wrapper/ src/java/org/apache/cocoon/util/ src/test/org/apache/cocoon/util/

Posted by Joerg Heinicke <jo...@gmx.de>.
On 09.06.2008 22:40, joerg@apache.org wrote:

> Author: joerg
> Date: Mon Jun  9 19:40:16 2008
> New Revision: 665954
> 
> URL: http://svn.apache.org/viewvc?rev=665954&view=rev
> Log:
> as a follow-up of COCOON-2168 (http://marc.info/?t=120899868500003&r=1&w=4):
> Cocoon's pipeline buffer increases from an initial buffer size of 8192 bytes to the configurable flush buffer size rather than allocating the complete buffer beforehand.

I'm still working on making initial buffer size configurable in sitemap.

> +    public OutputStream getOutputStream(int bufferSize) throws IOException {
>          // This method could be called several times during request processing
>          // with differing values of bufferSize and should handle this situation
>          // correctly.

> +        // FIXME (JH): Question is what "correctly" means. The current behavior
> +        // seems to be inconsistent: On a second call with bufferSize == 0 we
> +        // discard whatever the first called set up. With a bufferSize != 0 the
> +        // first call's setup is preserved. Why not always creating new
> +        // BufferedOutputStream in the else block replacing a potentially
> +        // existing one?

Any idea/comment on this? If the current behavior is correct or expected 
it should probably commented appropriately.

Joerg