You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/09/27 20:44:20 UTC
[jira] [Commented] (IO-515) Allow Specifying Initial Buffer Size of
DeferredFileOutputStream
[ https://issues.apache.org/jira/browse/IO-515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527366#comment-15527366 ]
ASF GitHub Bot commented on IO-515:
-----------------------------------
GitHub user brettlounsbury opened a pull request:
https://github.com/apache/commons-io/pull/22
[IO-515] Add constructors that allow the initial size of the in memory buffer
Add constructors that allow the initial size of the in memory buffer to be set to a value other than the default value in ByteArrayOutputStream. The current default is 32B which can have large performance impact if the threshold is large enough due to multiple array doubling iterations. As an example, to buffer 1MB the array would double 15 times and would create approximately 1MB of garbage arrays in addition to the 1MB of buffer at the end.
https://issues.apache.org/jira/browse/IO-515
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/brettlounsbury/commons-io master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/commons-io/pull/22.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22
----
commit 112d6c02a4c0d3fe9caa658505f31da3908b1467
Author: Brett Lounsbury <br...@amazon.com>
Date: 2016-09-27T19:58:21Z
Add constructors that allow the initial size of the in memory buffer to be set to a value other than the default value in ByteArrayOutputStream. The current default is 32B which can have large performance impact if the threshold is large enough due to multiple array doubling iterations. As an example, to buffer 1MB the array would double 15 times and would create approximately 1MB of garbage arrays in addition to the 1MB of buffer at the end.
----
> Allow Specifying Initial Buffer Size of DeferredFileOutputStream
> ----------------------------------------------------------------
>
> Key: IO-515
> URL: https://issues.apache.org/jira/browse/IO-515
> Project: Commons IO
> Issue Type: Improvement
> Components: Streams/Writers
> Reporter: Brett Lounsbury
>
> Right now the DeferredFileOutputStream uses a ByteArrayOputStream for its in memory buffer. It simply does new ByteArrayOutputStream() which initializes the ByteArrayOutputStream's internal buffer to 32B. In the case of large threshold values for in-memory buffering this can become very inefficient (Requiring 15 array doubling operations to grow to 1MB and producing an extra 1MB of garbage along the way).
> By allowing the user to specify the initial buffer size we allow them to tune how many times the array will need to double to something that meets their application's needs.
> In our testing we had an application that used a DeferredFileOutputStream to capture incoming data. Once that data was fully captured it did some fairly CPU intensive processing of the data. The ByteArrayOutputStream used by DeferredFileOutputStream was using 60% of our host's CPU simply on array copying.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)