You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Xiaoming Shi (JIRA)" <ji...@apache.org> on 2011/02/28 18:22:36 UTC
[jira] Created: (CHUKWA-583) Copying data from inputStream to
OuputStream needs appropriate buffer size
Copying data from inputStream to OuputStream needs appropriate buffer size
--------------------------------------------------------------------------
Key: CHUKWA-583
URL: https://issues.apache.org/jira/browse/CHUKWA-583
Project: Chukwa
Issue Type: Bug
Components: Data Processors
Affects Versions: 0.4.0
Reporter: Xiaoming Shi
In the file
./chukwa-0.4.0/src/test/org/apache/hadoop/chukwa/validationframework/util/DataOperations.java line: 54-58
In the function copyFile, the buffer size is fixed as 4096 bytes. With the size of the data varies, the performance can be damaged a lot.
We need an appropriate buffer size which depends on the size of the data to be copied.
This is the same as the Appache Bug (https://issues.apache.org/bugzilla/show_bug.cgi?id=32546)
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CHUKWA-583) Copying data from inputStream to
OuputStream needs appropriate buffer size
Posted by "Xiaoming Shi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13006140#comment-13006140 ]
Xiaoming Shi commented on CHUKWA-583:
-------------------------------------
To copy 10M data with 1024-byte buffer it takes 338,591,180 nano-second, while it takes 105,384,689 nano-second with a 10240-byte buffer.
Maybe you can check the parent bug report, if you have time.
> Copying data from inputStream to OuputStream needs appropriate buffer size
> --------------------------------------------------------------------------
>
> Key: CHUKWA-583
> URL: https://issues.apache.org/jira/browse/CHUKWA-583
> Project: Chukwa
> Issue Type: Bug
> Components: Data Processors
> Affects Versions: 0.4.0
> Reporter: Xiaoming Shi
> Priority: Trivial
>
> In the file
> ./chukwa-0.4.0/src/test/org/apache/hadoop/chukwa/validationframework/util/DataOperations.java line: 54-58
> In the function copyFile, the buffer size is fixed as 4096 bytes. With the size of the data varies, the performance can be damaged a lot.
> We need an appropriate buffer size which depends on the size of the data to be copied.
> This is the same as the Appache Bug (https://issues.apache.org/bugzilla/show_bug.cgi?id=32546)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CHUKWA-583) Copying data from inputStream to
OuputStream needs appropriate buffer size
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-583:
------------------------------
Priority: Trivial (was: Major)
This looks like an autogenerated bug report. I would caution you against this. The code you're flagging is in the test suite. It is NOT performance critical, nor is it run on substantial data volumes. It's probably pointless to change. The human time invested in generating and committing a patch is probably more than the "fix" would ever save.
> Copying data from inputStream to OuputStream needs appropriate buffer size
> --------------------------------------------------------------------------
>
> Key: CHUKWA-583
> URL: https://issues.apache.org/jira/browse/CHUKWA-583
> Project: Chukwa
> Issue Type: Bug
> Components: Data Processors
> Affects Versions: 0.4.0
> Reporter: Xiaoming Shi
> Priority: Trivial
>
> In the file
> ./chukwa-0.4.0/src/test/org/apache/hadoop/chukwa/validationframework/util/DataOperations.java line: 54-58
> In the function copyFile, the buffer size is fixed as 4096 bytes. With the size of the data varies, the performance can be damaged a lot.
> We need an appropriate buffer size which depends on the size of the data to be copied.
> This is the same as the Appache Bug (https://issues.apache.org/bugzilla/show_bug.cgi?id=32546)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CHUKWA-583) Copying data from inputStream to
OuputStream needs appropriate buffer size
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13006248#comment-13006248 ]
Eric Yang commented on CHUKWA-583:
----------------------------------
This bug is in the test code, and we don't push large amount of data over the validation framework. Hence, it is a minor bug. Patch to increase the size is welcome.
> Copying data from inputStream to OuputStream needs appropriate buffer size
> --------------------------------------------------------------------------
>
> Key: CHUKWA-583
> URL: https://issues.apache.org/jira/browse/CHUKWA-583
> Project: Chukwa
> Issue Type: Bug
> Components: Data Processors
> Affects Versions: 0.4.0
> Reporter: Xiaoming Shi
> Priority: Trivial
>
> In the file
> ./chukwa-0.4.0/src/test/org/apache/hadoop/chukwa/validationframework/util/DataOperations.java line: 54-58
> In the function copyFile, the buffer size is fixed as 4096 bytes. With the size of the data varies, the performance can be damaged a lot.
> We need an appropriate buffer size which depends on the size of the data to be copied.
> This is the same as the Appache Bug (https://issues.apache.org/bugzilla/show_bug.cgi?id=32546)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira