You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Manoj Mokashi (Created) (JIRA)" <ji...@apache.org> on 2012/03/07 11:14:57 UTC
[jira] [Created] (IO-305) New copy() method in IOUtils that takes
additional offset, length and buffersize arguments
New copy() method in IOUtils that takes additional offset, length and buffersize arguments
------------------------------------------------------------------------------------------
Key: IO-305
URL: https://issues.apache.org/jira/browse/IO-305
Project: Commons IO
Issue Type: New Feature
Components: Utilities
Reporter: Manoj Mokashi
Priority: Minor
/**
* Copy from input to output stream
* @param is : input stream
* @param os : output stream
* @param offset : number of bytes to skip from input before copying
* -ve values are ignored
* @param len : number of bytes to copy. -1 means all
* @param bufferSize : buffer size to use for copying
* @throws IOException
*/
public static void copy( InputStream is, OutputStream os, int offset, int len, int bufferSize) throws IOException
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (IO-305) New copy() method in IOUtils that takes
additional offset, length and buffersize arguments
Posted by "Sebb (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebb resolved IO-305.
---------------------
Resolution: Fixed
Fix Version/s: 2.2
Added methods for bytes and chars based on copyLarge.
Dropped buffer size parameter as not essential.
> New copy() method in IOUtils that takes additional offset, length and buffersize arguments
> ------------------------------------------------------------------------------------------
>
> Key: IO-305
> URL: https://issues.apache.org/jira/browse/IO-305
> Project: Commons IO
> Issue Type: New Feature
> Components: Utilities
> Reporter: Manoj Mokashi
> Priority: Minor
> Fix For: 2.2
>
> Attachments: IOUtils.java, IOUtilsTest.java
>
>
> /**
> * Copy from input to output stream
> * @param is : input stream
> * @param os : output stream
> * @param offset : number of bytes to skip from input before copying
> * -ve values are ignored
> * @param len : number of bytes to copy. -1 means all
> * @param bufferSize : buffer size to use for copying
> * @throws IOException
> */
> public static void copy( InputStream is, OutputStream os, int offset, int len, int bufferSize) throws IOException
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IO-305) New copy() method in IOUtils that takes
additional offset, length and buffersize arguments
Posted by "Sebb (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231047#comment-13231047 ]
Sebb commented on IO-305:
-------------------------
The code would probably be simplified by using copyLarge for the case where len == -1 (otherwise it has to keep checking for this).
Also, unless the copyLarge methods are enhanced to allow the buffer or its size to be specified, it would be more consistent to use the same buffer size as the copyLarge methods, and omit the size parameter.
> New copy() method in IOUtils that takes additional offset, length and buffersize arguments
> ------------------------------------------------------------------------------------------
>
> Key: IO-305
> URL: https://issues.apache.org/jira/browse/IO-305
> Project: Commons IO
> Issue Type: New Feature
> Components: Utilities
> Reporter: Manoj Mokashi
> Priority: Minor
> Attachments: IOUtils.java, IOUtilsTest.java
>
>
> /**
> * Copy from input to output stream
> * @param is : input stream
> * @param os : output stream
> * @param offset : number of bytes to skip from input before copying
> * -ve values are ignored
> * @param len : number of bytes to copy. -1 means all
> * @param bufferSize : buffer size to use for copying
> * @throws IOException
> */
> public static void copy( InputStream is, OutputStream os, int offset, int len, int bufferSize) throws IOException
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (IO-305) New copy() method in IOUtils that takes
additional offset, length and buffersize arguments
Posted by "Gary D. Gregory (Closed) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary D. Gregory closed IO-305.
------------------------------
Version 2.2 has been released and addresses this issue.
> New copy() method in IOUtils that takes additional offset, length and buffersize arguments
> ------------------------------------------------------------------------------------------
>
> Key: IO-305
> URL: https://issues.apache.org/jira/browse/IO-305
> Project: Commons IO
> Issue Type: New Feature
> Components: Utilities
> Reporter: Manoj Mokashi
> Priority: Minor
> Fix For: 2.2
>
> Attachments: IOUtils.java, IOUtilsTest.java
>
>
> /**
> * Copy from input to output stream
> * @param is : input stream
> * @param os : output stream
> * @param offset : number of bytes to skip from input before copying
> * -ve values are ignored
> * @param len : number of bytes to copy. -1 means all
> * @param bufferSize : buffer size to use for copying
> * @throws IOException
> */
> public static void copy( InputStream is, OutputStream os, int offset, int len, int bufferSize) throws IOException
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IO-305) New copy() method in IOUtils that takes
additional offset, length and buffersize arguments
Posted by "Manoj Mokashi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231869#comment-13231869 ]
Manoj Mokashi commented on IO-305:
----------------------------------
In my humble opinion, the buffer size parameter is important to control performance,
the default buffer size is 4096, and for large files we would need more. If copyLarge should not accept a buffer size parameter,
maybe we should use the copy method. Is the check for len == -1 really a performance issue, especially since we don't read byte-by-byte ?
> New copy() method in IOUtils that takes additional offset, length and buffersize arguments
> ------------------------------------------------------------------------------------------
>
> Key: IO-305
> URL: https://issues.apache.org/jira/browse/IO-305
> Project: Commons IO
> Issue Type: New Feature
> Components: Utilities
> Reporter: Manoj Mokashi
> Priority: Minor
> Fix For: 2.2
>
> Attachments: IOUtils.java, IOUtilsTest.java
>
>
> /**
> * Copy from input to output stream
> * @param is : input stream
> * @param os : output stream
> * @param offset : number of bytes to skip from input before copying
> * -ve values are ignored
> * @param len : number of bytes to copy. -1 means all
> * @param bufferSize : buffer size to use for copying
> * @throws IOException
> */
> public static void copy( InputStream is, OutputStream os, int offset, int len, int bufferSize) throws IOException
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (IO-305) New copy() method in IOUtils that takes
additional offset, length and buffersize arguments
Posted by "Manoj Mokashi (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manoj Mokashi updated IO-305:
-----------------------------
Attachment: IOUtilsTest.java
IOUtils.java
The new method is in the IOUtils class.Note that its my class, not from the apache repository.
The IOUtilsTest is junit test case.
> New copy() method in IOUtils that takes additional offset, length and buffersize arguments
> ------------------------------------------------------------------------------------------
>
> Key: IO-305
> URL: https://issues.apache.org/jira/browse/IO-305
> Project: Commons IO
> Issue Type: New Feature
> Components: Utilities
> Reporter: Manoj Mokashi
> Priority: Minor
> Attachments: IOUtils.java, IOUtilsTest.java
>
>
> /**
> * Copy from input to output stream
> * @param is : input stream
> * @param os : output stream
> * @param offset : number of bytes to skip from input before copying
> * -ve values are ignored
> * @param len : number of bytes to copy. -1 means all
> * @param bufferSize : buffer size to use for copying
> * @throws IOException
> */
> public static void copy( InputStream is, OutputStream os, int offset, int len, int bufferSize) throws IOException
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IO-305) New copy() method in IOUtils that takes
additional offset, length and buffersize arguments
Posted by "Sebb (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224225#comment-13224225 ]
Sebb commented on IO-305:
-------------------------
Should use long for the offset and length.
It would be useful to return the number of bytes actually copied, as per the copyLarge methods.
> New copy() method in IOUtils that takes additional offset, length and buffersize arguments
> ------------------------------------------------------------------------------------------
>
> Key: IO-305
> URL: https://issues.apache.org/jira/browse/IO-305
> Project: Commons IO
> Issue Type: New Feature
> Components: Utilities
> Reporter: Manoj Mokashi
> Priority: Minor
> Attachments: IOUtils.java, IOUtilsTest.java
>
>
> /**
> * Copy from input to output stream
> * @param is : input stream
> * @param os : output stream
> * @param offset : number of bytes to skip from input before copying
> * -ve values are ignored
> * @param len : number of bytes to copy. -1 means all
> * @param bufferSize : buffer size to use for copying
> * @throws IOException
> */
> public static void copy( InputStream is, OutputStream os, int offset, int len, int bufferSize) throws IOException
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IO-305) New copy() method in IOUtils that takes
additional offset, length and buffersize arguments
Posted by "Sebb (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231927#comment-13231927 ]
Sebb commented on IO-305:
-------------------------
The default buffer size of 4096 was chosen because it gives good performance.
Have you any performance tests that show otherwise?
If so, we can consider implementing this for all the copyLarge methods, see: IO-308
bq. Is the check for len == -1 really a performance issue
Code no longer checks the length twice; I reimplemented the loop in order to support returning the copied length.
> New copy() method in IOUtils that takes additional offset, length and buffersize arguments
> ------------------------------------------------------------------------------------------
>
> Key: IO-305
> URL: https://issues.apache.org/jira/browse/IO-305
> Project: Commons IO
> Issue Type: New Feature
> Components: Utilities
> Reporter: Manoj Mokashi
> Priority: Minor
> Fix For: 2.2
>
> Attachments: IOUtils.java, IOUtilsTest.java
>
>
> /**
> * Copy from input to output stream
> * @param is : input stream
> * @param os : output stream
> * @param offset : number of bytes to skip from input before copying
> * -ve values are ignored
> * @param len : number of bytes to copy. -1 means all
> * @param bufferSize : buffer size to use for copying
> * @throws IOException
> */
> public static void copy( InputStream is, OutputStream os, int offset, int len, int bufferSize) throws IOException
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IO-305) New copy() method in IOUtils that takes
additional offset, length and buffersize arguments
Posted by "Manoj Mokashi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13232219#comment-13232219 ]
Manoj Mokashi commented on IO-305:
----------------------------------
I tested copying a 500MB tar archive with diffent buffersizes, and it does make a difference.
e.g. buffersize => time in millis:
4096=>129954,*16=>71734,*64=91328
4096=>120406,*16=>80219,*64=69687
btw, accessing buffer.length inside the loop seems to affect performance for bigger lengths.
As seen in the 1st statistics, the *64 method actually takes longer than *16.
In the 2nd set i have used a buffersize var outside the loop as its constant.
I guess the results will vary as per avail memory, OS, disk types etc.
But it does make a difference to specify buffer size.
wrt IO-308, i agree that passing a buffer would be even better.
> New copy() method in IOUtils that takes additional offset, length and buffersize arguments
> ------------------------------------------------------------------------------------------
>
> Key: IO-305
> URL: https://issues.apache.org/jira/browse/IO-305
> Project: Commons IO
> Issue Type: New Feature
> Components: Utilities
> Reporter: Manoj Mokashi
> Priority: Minor
> Fix For: 2.2
>
> Attachments: IOUtils.java, IOUtilsTest.java
>
>
> /**
> * Copy from input to output stream
> * @param is : input stream
> * @param os : output stream
> * @param offset : number of bytes to skip from input before copying
> * -ve values are ignored
> * @param len : number of bytes to copy. -1 means all
> * @param bufferSize : buffer size to use for copying
> * @throws IOException
> */
> public static void copy( InputStream is, OutputStream os, int offset, int len, int bufferSize) throws IOException
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira