You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by "Holger Hoffstätte (JIRA)" <ji...@apache.org> on 2006/09/02 18:09:29 UTC
[jira] Created: (TRANSACTION-12) Use NIO for file copying/moving
Use NIO for file copying/moving
-------------------------------
Key: TRANSACTION-12
URL: http://issues.apache.org/jira/browse/TRANSACTION-12
Project: Commons Transaction
Issue Type: Wish
Affects Versions: 1.2
Reporter: Holger Hoffstätte
Attachments: moveByRenameAndNIO.txt
The Mule project (http://mule.codehaus.org/) is considering adoption of commons-transaction for (among other things) file transactions. Unfortunately inspection of the code reveals that FileHelper copies files in the slowest possible way (traditional Java I/O via Input/OutputStreams). Since some of our users move lots and/or large files (hundreds of megabytes to gigabytes) we implemented file moving by rename (atomic and obviously very fast when source and target are on the same filesystem) and NIO as backup strategy (a lot less CPU usage).
Attached is a code snippet that outlines the procedure; it would be cool if c-tx could adopt a similar strategy for moving via rename-first and copy via NIO. I do not know if and how this might conflict with c-tx's "transactional" behaviour and recovery but then again it is just a suggestion. :-)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
[jira] Commented: (TRANSACTION-12) Use NIO for file copying/moving
Posted by "Oliver Zeigermann (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/TRANSACTION-12?page=comments#action_12432306 ]
Oliver Zeigermann commented on TRANSACTION-12:
----------------------------------------------
Renaming (rename-first as you call it) is already done by the current implementation.
The current fall back strategy is used as commons transaction only requires Java 1.3 which does not support NIO. It might be an option to have different implementations for pre 1.4 and post 1.4 JDKs though. Could you think of something like that?
> Use NIO for file copying/moving
> -------------------------------
>
> Key: TRANSACTION-12
> URL: http://issues.apache.org/jira/browse/TRANSACTION-12
> Project: Commons Transaction
> Issue Type: Wish
> Affects Versions: 1.2
> Reporter: Holger Hoffstätte
> Attachments: moveByRenameAndNIO.txt
>
>
> The Mule project (http://mule.codehaus.org/) is considering adoption of commons-transaction for (among other things) file transactions. Unfortunately inspection of the code reveals that FileHelper copies files in the slowest possible way (traditional Java I/O via Input/OutputStreams). Since some of our users move lots and/or large files (hundreds of megabytes to gigabytes) we implemented file moving by rename (atomic and obviously very fast when source and target are on the same filesystem) and NIO as backup strategy (a lot less CPU usage).
> Attached is a code snippet that outlines the procedure; it would be cool if c-tx could adopt a similar strategy for moving via rename-first and copy via NIO. I do not know if and how this might conflict with c-tx's "transactional" behaviour and recovery but then again it is just a suggestion. :-)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
[jira] Updated: (TRANSACTION-12) Use NIO for file copying/moving
Posted by "Jörg Heinicke (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/TRANSACTION-12?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jörg Heinicke updated TRANSACTION-12:
-------------------------------------
Fix Version/s: 2.0
For the moment I'd postpone this one after a possible refactoring. But if you provide a more or less backwards compatible way it can go into an earlier release as well ;)
> Use NIO for file copying/moving
> -------------------------------
>
> Key: TRANSACTION-12
> URL: https://issues.apache.org/jira/browse/TRANSACTION-12
> Project: Commons Transaction
> Issue Type: Wish
> Affects Versions: 1.2
> Reporter: Holger Hoffstätte
> Fix For: 2.0
>
> Attachments: moveByRenameAndNIO.txt
>
>
> The Mule project (http://mule.codehaus.org/) is considering adoption of commons-transaction for (among other things) file transactions. Unfortunately inspection of the code reveals that FileHelper copies files in the slowest possible way (traditional Java I/O via Input/OutputStreams). Since some of our users move lots and/or large files (hundreds of megabytes to gigabytes) we implemented file moving by rename (atomic and obviously very fast when source and target are on the same filesystem) and NIO as backup strategy (a lot less CPU usage).
> Attached is a code snippet that outlines the procedure; it would be cool if c-tx could adopt a similar strategy for moving via rename-first and copy via NIO. I do not know if and how this might conflict with c-tx's "transactional" behaviour and recovery but then again it is just a suggestion. :-)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
[jira] Commented: (TRANSACTION-12) Use NIO for file copying/moving
Posted by "Holger Hoffstätte (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/TRANSACTION-12?page=comments#action_12432331 ]
Holger Hoffstätte commented on TRANSACTION-12:
----------------------------------------------
Thanks for feedback - I didn't realize you were shooting for compatibility with JDK 1.3 (bummer ;) or that you already did rename-first (good to know). It's not a dealbreaker since adoption of c-tx is still some way off; when the time has come we may think about providing something like a pluggable FileHelper as extension/patch.
> Use NIO for file copying/moving
> -------------------------------
>
> Key: TRANSACTION-12
> URL: http://issues.apache.org/jira/browse/TRANSACTION-12
> Project: Commons Transaction
> Issue Type: Wish
> Affects Versions: 1.2
> Reporter: Holger Hoffstätte
> Attachments: moveByRenameAndNIO.txt
>
>
> The Mule project (http://mule.codehaus.org/) is considering adoption of commons-transaction for (among other things) file transactions. Unfortunately inspection of the code reveals that FileHelper copies files in the slowest possible way (traditional Java I/O via Input/OutputStreams). Since some of our users move lots and/or large files (hundreds of megabytes to gigabytes) we implemented file moving by rename (atomic and obviously very fast when source and target are on the same filesystem) and NIO as backup strategy (a lot less CPU usage).
> Attached is a code snippet that outlines the procedure; it would be cool if c-tx could adopt a similar strategy for moving via rename-first and copy via NIO. I do not know if and how this might conflict with c-tx's "transactional" behaviour and recovery but then again it is just a suggestion. :-)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org