You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Igor Smereka (JIRA)" <ji...@apache.org> on 2011/03/18 19:59:31 UTC
[jira] Created: (IO-266) FileUtils.copyFile() throws IOException
when copying large files to a shared directory (on Windows)
FileUtils.copyFile() throws IOException when copying large files to a shared directory (on Windows)
---------------------------------------------------------------------------------------------------
Key: IO-266
URL: https://issues.apache.org/jira/browse/IO-266
Project: Commons IO
Issue Type: Bug
Reporter: Igor Smereka
java.io.IOException: Insufficient system resources exist to complete the requested service
at sun.nio.ch.FileDispatcher.pwrite0(Native Method)
at sun.nio.ch.FileDispatcher.pwrite(Unknown Source)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
at sun.nio.ch.IOUtil.write(Unknown Source)
at sun.nio.ch.FileChannelImpl.write(Unknown Source)
at sun.nio.ch.FileChannelImpl.transferFromFileChannel(Unknown Source)
at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:813)
at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:783)
at org.test.igor.TestFileUtils.main(TestFileUtils.java:55)
NOTE: the issue is cased by the function doCopyFile(File srcFile, File destFile, boolean preserveFileDate) using hardcoded data chunks of FIFTY_MB in the transferFrom() call.
Reducing this chunk from 50M to 31M solves the issue for my situation (32M still fails).
Here is a test program to reproduce the issue:
package org.test.igor;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.channels.FileChannel;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
public class TestFileUtils {
public static void main(String[] args){
File src = new File("D:\\2011.1-dev\\test\\test");
File dest = new File("\\\\ismerek1\\Shared");
String filename = "jdk-6u19-windows-x64.exe";
File file = new File(src, filename);
File toFile = new File(dest, filename);
try {
FileUtils.copyFile(file, toFile, true);
System.out.println("Successful copy");
}
catch (IOException e1) {
e1.printStackTrace();
}
}
}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IO-266) FileUtils.copyFile() throws IOException
when copying large files to a shared directory (on Windows)
Posted by "Niall Pemberton (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13018285#comment-13018285 ]
Niall Pemberton commented on IO-266:
------------------------------------
I think we should just reduce the size to fix the problem Igor found - so 30M is the nearest (rounded)
> FileUtils.copyFile() throws IOException when copying large files to a shared directory (on Windows)
> ---------------------------------------------------------------------------------------------------
>
> Key: IO-266
> URL: https://issues.apache.org/jira/browse/IO-266
> Project: Commons IO
> Issue Type: Bug
> Affects Versions: 2.0.1
> Environment: Windows 2003 Server 64-bit
> Reporter: Igor Smereka
>
> java.io.IOException: Insufficient system resources exist to complete the requested service
> at sun.nio.ch.FileDispatcher.pwrite0(Native Method)
> at sun.nio.ch.FileDispatcher.pwrite(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.FileChannelImpl.write(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferFromFileChannel(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
> at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:813)
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:783)
> at org.test.igor.TestFileUtils.main(TestFileUtils.java:55)
> NOTE: the issue is cased by the function doCopyFile(File srcFile, File destFile, boolean preserveFileDate) using hardcoded data chunks of FIFTY_MB in the transferFrom() call.
> Reducing this chunk from 50M to 31M solves the issue for my situation (32M still fails).
> Here is a test program to reproduce the issue:
> package org.test.igor;
> import java.io.File;
> import java.io.FileInputStream;
> import java.io.FileOutputStream;
> import java.io.IOException;
> import java.nio.channels.FileChannel;
> import org.apache.commons.io.FileUtils;
> import org.apache.commons.io.IOUtils;
> public class TestFileUtils {
> public static void main(String[] args){
>
>
> File src = new File("D:\\2011.1-dev\\test\\test");
> File dest = new File("\\\\ismerek1\\Shared");
>
> String filename = "jdk-6u19-windows-x64.exe";
>
> File file = new File(src, filename);
> File toFile = new File(dest, filename);
>
> try {
> FileUtils.copyFile(file, toFile, true);
> System.out.println("Successful copy");
> }
> catch (IOException e1) {
> e1.printStackTrace();
> }
> }
> }
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (IO-266) FileUtils.copyFile() throws IOException
when copying large files to a shared directory (on Windows)
Posted by "Arnaud BRAND (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arnaud BRAND updated IO-266:
----------------------------
Attachment: FileUtils.java.patch
I attached a patch setting the chunk at 30MB since I'm also affected by this issue.
Thanks.
> FileUtils.copyFile() throws IOException when copying large files to a shared directory (on Windows)
> ---------------------------------------------------------------------------------------------------
>
> Key: IO-266
> URL: https://issues.apache.org/jira/browse/IO-266
> Project: Commons IO
> Issue Type: Bug
> Affects Versions: 2.0.1
> Environment: Windows 2003 Server 64-bit
> Reporter: Igor Smereka
> Attachments: FileUtils.java.patch
>
>
> java.io.IOException: Insufficient system resources exist to complete the requested service
> at sun.nio.ch.FileDispatcher.pwrite0(Native Method)
> at sun.nio.ch.FileDispatcher.pwrite(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.FileChannelImpl.write(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferFromFileChannel(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
> at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:813)
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:783)
> at org.test.igor.TestFileUtils.main(TestFileUtils.java:55)
> NOTE: the issue is cased by the function doCopyFile(File srcFile, File destFile, boolean preserveFileDate) using hardcoded data chunks of FIFTY_MB in the transferFrom() call.
> Reducing this chunk from 50M to 31M solves the issue for my situation (32M still fails).
> Here is a test program to reproduce the issue:
> package org.test.igor;
> import java.io.File;
> import java.io.FileInputStream;
> import java.io.FileOutputStream;
> import java.io.IOException;
> import java.nio.channels.FileChannel;
> import org.apache.commons.io.FileUtils;
> import org.apache.commons.io.IOUtils;
> public class TestFileUtils {
> public static void main(String[] args){
>
>
> File src = new File("D:\\2011.1-dev\\test\\test");
> File dest = new File("\\\\ismerek1\\Shared");
>
> String filename = "jdk-6u19-windows-x64.exe";
>
> File file = new File(src, filename);
> File toFile = new File(dest, filename);
>
> try {
> FileUtils.copyFile(file, toFile, true);
> System.out.println("Successful copy");
> }
> catch (IOException e1) {
> e1.printStackTrace();
> }
> }
> }
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (IO-266) FileUtils.copyFile() throws IOException
when copying large files to a shared directory (on Windows)
Posted by "Gary D. Gregory (Closed) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary D. Gregory closed IO-266.
------------------------------
Closing, we released version 2.1.
> FileUtils.copyFile() throws IOException when copying large files to a shared directory (on Windows)
> ---------------------------------------------------------------------------------------------------
>
> Key: IO-266
> URL: https://issues.apache.org/jira/browse/IO-266
> Project: Commons IO
> Issue Type: Bug
> Affects Versions: 2.0.1
> Environment: Windows 2003 Server 64-bit
> Reporter: Igor Smereka
> Fix For: 2.1
>
> Attachments: FileUtils.java.patch
>
>
> java.io.IOException: Insufficient system resources exist to complete the requested service
> at sun.nio.ch.FileDispatcher.pwrite0(Native Method)
> at sun.nio.ch.FileDispatcher.pwrite(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.FileChannelImpl.write(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferFromFileChannel(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
> at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:813)
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:783)
> at org.test.igor.TestFileUtils.main(TestFileUtils.java:55)
> NOTE: the issue is cased by the function doCopyFile(File srcFile, File destFile, boolean preserveFileDate) using hardcoded data chunks of FIFTY_MB in the transferFrom() call.
> Reducing this chunk from 50M to 31M solves the issue for my situation (32M still fails).
> Here is a test program to reproduce the issue:
> package org.test.igor;
> import java.io.File;
> import java.io.FileInputStream;
> import java.io.FileOutputStream;
> import java.io.IOException;
> import java.nio.channels.FileChannel;
> import org.apache.commons.io.FileUtils;
> import org.apache.commons.io.IOUtils;
> public class TestFileUtils {
> public static void main(String[] args){
>
>
> File src = new File("D:\\2011.1-dev\\test\\test");
> File dest = new File("\\\\ismerek1\\Shared");
>
> String filename = "jdk-6u19-windows-x64.exe";
>
> File file = new File(src, filename);
> File toFile = new File(dest, filename);
>
> try {
> FileUtils.copyFile(file, toFile, true);
> System.out.println("Successful copy");
> }
> catch (IOException e1) {
> e1.printStackTrace();
> }
> }
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (IO-266) FileUtils.copyFile() throws IOException
when copying large files to a shared directory (on Windows)
Posted by "Sebb (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017556#comment-13017556 ]
Sebb commented on IO-266:
-------------------------
50MB does seem rather large as a default; I would have thought 16MB would be plenty?
Perhaps we should make the default smaller.
It would be possible to add new methods which allowed the buffer size to be provided; however this would mean adding about 7 new methods.
Unfortunately changing the buffersize to a variable would be thread-hostile: if two threads set different values, there is a timing window between setting the value and starting the copy method. This cannot be avoided.
It's a pity the class methods are all static ...
One other possibility would be to allow the default to be set by using a System Property; this could be thread-safe, but the value could not be changed once set.
> FileUtils.copyFile() throws IOException when copying large files to a shared directory (on Windows)
> ---------------------------------------------------------------------------------------------------
>
> Key: IO-266
> URL: https://issues.apache.org/jira/browse/IO-266
> Project: Commons IO
> Issue Type: Bug
> Affects Versions: 2.0.1
> Environment: Windows 2003 Server 64-bit
> Reporter: Igor Smereka
>
> java.io.IOException: Insufficient system resources exist to complete the requested service
> at sun.nio.ch.FileDispatcher.pwrite0(Native Method)
> at sun.nio.ch.FileDispatcher.pwrite(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.FileChannelImpl.write(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferFromFileChannel(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
> at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:813)
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:783)
> at org.test.igor.TestFileUtils.main(TestFileUtils.java:55)
> NOTE: the issue is cased by the function doCopyFile(File srcFile, File destFile, boolean preserveFileDate) using hardcoded data chunks of FIFTY_MB in the transferFrom() call.
> Reducing this chunk from 50M to 31M solves the issue for my situation (32M still fails).
> Here is a test program to reproduce the issue:
> package org.test.igor;
> import java.io.File;
> import java.io.FileInputStream;
> import java.io.FileOutputStream;
> import java.io.IOException;
> import java.nio.channels.FileChannel;
> import org.apache.commons.io.FileUtils;
> import org.apache.commons.io.IOUtils;
> public class TestFileUtils {
> public static void main(String[] args){
>
>
> File src = new File("D:\\2011.1-dev\\test\\test");
> File dest = new File("\\\\ismerek1\\Shared");
>
> String filename = "jdk-6u19-windows-x64.exe";
>
> File file = new File(src, filename);
> File toFile = new File(dest, filename);
>
> try {
> FileUtils.copyFile(file, toFile, true);
> System.out.println("Successful copy");
> }
> catch (IOException e1) {
> e1.printStackTrace();
> }
> }
> }
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (IO-266) FileUtils.copyFile() throws IOException
when copying large files to a shared directory (on Windows)
Posted by "Sebb (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IO-266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sebb resolved IO-266.
---------------------
Resolution: Fixed
Fix Version/s: 2.1
Changed to 30MB
> FileUtils.copyFile() throws IOException when copying large files to a shared directory (on Windows)
> ---------------------------------------------------------------------------------------------------
>
> Key: IO-266
> URL: https://issues.apache.org/jira/browse/IO-266
> Project: Commons IO
> Issue Type: Bug
> Affects Versions: 2.0.1
> Environment: Windows 2003 Server 64-bit
> Reporter: Igor Smereka
> Fix For: 2.1
>
> Attachments: FileUtils.java.patch
>
>
> java.io.IOException: Insufficient system resources exist to complete the requested service
> at sun.nio.ch.FileDispatcher.pwrite0(Native Method)
> at sun.nio.ch.FileDispatcher.pwrite(Unknown Source)
> at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source)
> at sun.nio.ch.IOUtil.write(Unknown Source)
> at sun.nio.ch.FileChannelImpl.write(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferFromFileChannel(Unknown Source)
> at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
> at org.apache.commons.io.FileUtils.doCopyFile(FileUtils.java:813)
> at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:783)
> at org.test.igor.TestFileUtils.main(TestFileUtils.java:55)
> NOTE: the issue is cased by the function doCopyFile(File srcFile, File destFile, boolean preserveFileDate) using hardcoded data chunks of FIFTY_MB in the transferFrom() call.
> Reducing this chunk from 50M to 31M solves the issue for my situation (32M still fails).
> Here is a test program to reproduce the issue:
> package org.test.igor;
> import java.io.File;
> import java.io.FileInputStream;
> import java.io.FileOutputStream;
> import java.io.IOException;
> import java.nio.channels.FileChannel;
> import org.apache.commons.io.FileUtils;
> import org.apache.commons.io.IOUtils;
> public class TestFileUtils {
> public static void main(String[] args){
>
>
> File src = new File("D:\\2011.1-dev\\test\\test");
> File dest = new File("\\\\ismerek1\\Shared");
>
> String filename = "jdk-6u19-windows-x64.exe";
>
> File file = new File(src, filename);
> File toFile = new File(dest, filename);
>
> try {
> FileUtils.copyFile(file, toFile, true);
> System.out.println("Successful copy");
> }
> catch (IOException e1) {
> e1.printStackTrace();
> }
> }
> }
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira