You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by "Michael Mathers (JIRA)" <ji...@apache.org> on 2009/10/10 02:26:52 UTC
[jira] Created: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
File Component - Rename operations fail sometimes on certain filesystems (Windows)
----------------------------------------------------------------------------------
Key: CAMEL-2058
URL: https://issues.apache.org/activemq/browse/CAMEL-2058
Project: Apache Camel
Issue Type: Bug
Components: camel-core
Affects Versions: 2.0.0
Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
Reporter: Michael Mathers
On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
A couple ways to fix:
1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
2). Attempt a copy instead if the rename fails
- maybe after a brief pause
- maybe even after a number of rename attempts
- maybe watch to see the file is closed prior to a rename attempt.
Rename operation failures may affect other things as well like certain locking schemes.
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CAMEL-2058) File Component - Rename
operations fail sometimes on certain filesystems (Windows)
Posted by "Michael Mathers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54720#action_54720 ]
Michael Mathers edited comment on CAMEL-2058 at 10/12/09 10:59 AM:
-------------------------------------------------------------------
For the record, I did try my route in 2.1-SNAPSHOT before filing the report. But apparently I tried it just for {{localWorkDirectory}} and not for {{tempPrefix}}.
2.1-SNAPSHOT does seem to be fixed for {{tempPrefix}} (although I think I still found an issue there), but +NOT+ fixed for {{localWorkDirectory}}, an option of the FTP component.
\\
h3. Issue #1: {{localWorkDirectory}}
In both 2.0 and 2.1snap, this has the following behavior.
# After creating the "inprogress" file, it fails renaming (only makes a single attempt) to the final name in the working directory.
# After failing rename, it proceeds to consider the message "delivered" and does not attempt to reconsume the file on the next poll (this seems wrong to me, but I'm a newbie).
\\
h3. Issue #2: {{tempPrefix}}
You are right this is fixed with CAMEL-1965 as you indicated. _However_, when using {{tempPrefix}} with the {{fileExist}} option does not behave as expected. Instead, the {{fileExist}} options are applied to the "temp" file and not to the target file.
|| Option Value || Expected Behavior || Actual Behavior when using {{tempPrefix}} ||
| {{Override}} (Default) | If the target file exists, overwrite it | Will overwrite the temp file if it exists but will fail if the target file exists because the rename operation fails |
| {{Fail}} | Fail the exchange immediately if the target file exists | Will fail the exchange immediately if the temp file exists. Will not fail immediately if the target file exists. The exchange still fails ultimately though due to the failing of the rename operation. |
I did not test {{Append}} or {{Ignore}} values but they probably have similar problems. Also, I did not test other File component options that could involve rename operations such as {{readLock:rename}}, {{preMove}}, or {{moveFailed}}.
Let me know if you want me to file a separate issue for #2. Given the number of potential issues with rename and the Java implementation of it (other filesystems?), I'm still thinking that an eventual "ForceCopyInsteadOfRename" option might be a good idea.
was (Author: mbmather):
For the record, I did try my route in 2.1-SNAPSHOT before filing the report. But apparently I tried it just for {{localWorkDirectory}} and not for {{tempPrefix}}.
2.1-SNAPSHOT does seem to be fixed for {{tempPrefix}} (although I think I still found an issue there), but +NOT+ fixed for {{localWorkDirectory}}, an option of the FTP component.
\\
h3. Issue #1: {{localWorkDirectory}}
In both 2.0 and 2.1snap, this has the following behavior.
# After creating the "inprogress" file, it fails renaming (only makes a single attempt) to the final name in the working directory.
# After failing rename, it proceeds to consider the message "delivered" and does not attempt to reconsume the file on the next poll (this seems wrong to me, but I'm a newbie).
\\
h3. Issue #2: {{tempPrefix}}
You are right this is fixed with CAMEL-1965 as you indicated. _However_, when using {{tempPrefix}} with the {{fileExist}} option does behave as expected. Instead, the {{fileExist}} options are applied to the "temp" file and not to the target file.
|| Option Value || Expected Behavior || Actual Behavior when using {{tempPrefix}} ||
| {{Override}} (Default) | If the target file exists, overwrite it | Will overwrite the temp file if it exists but will fail if the target file exists because the rename operation fails |
| {{Fail}} | Fail the exchange immediately if the target file exists | Will fail the exchange immediately if the temp file exists. Will not fail immediately if the target file exists. The exchange still fails ultimately though due to the failing of the rename operation. |
I did not test {{Append}} or {{Ignore}} values but they probably have similar problems. Also, I did not test other File component options that could involve rename operations such as {{readLock:rename}}, {{preMove}}, or {{moveFailed}}.
Let me know if you want me to file a separate issue for #2. Given the number of potential issues with rename and the Java implementation of it (other filesystems?), I'm still thinking that an eventual "ForceCopyInsteadOfRename" option might be a good idea.
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54939#action_54939 ]
Claus Ibsen commented on CAMEL-2058:
------------------------------------
Okay looks like I got the hang of this and have fixes being tested now
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CAMEL-2058) File Component - Rename
operations fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54825#action_54825 ]
Claus Ibsen edited comment on CAMEL-2058 at 10/19/09 7:31 AM:
--------------------------------------------------------------
*Issue #1*
1. I am fixing the rename to be aligned with other code by doing retries to cater for Windows platforms. *DONE*
Commited: 826686.
was (Author: davsclaus):
*Issue #1*
1. I am fixing the rename to be aligned with other code by doing retries to cater for Windows platforms.
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CAMEL-2058) File Component - Rename
operations fail sometimes on certain filesystems (Windows)
Posted by "Michael Mathers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54720#action_54720 ]
Michael Mathers edited comment on CAMEL-2058 at 10/12/09 10:52 AM:
-------------------------------------------------------------------
For the record, I did try my route in 2.1-SNAPSHOT before filing the report. But apparently I tried it just for {{localWorkDirectory}} and not for {{tempPrefix}}.
2.1-SNAPSHOT does seem to be fixed for {{tempPrefix}} (although I think I still found an issue there), but +NOT+ fixed for {{localWorkDirectory}}, an option of the FTP component.
\\
h3. Issue #1: {{localWorkDirectory}}
In both 2.0 and 2.1snap, this has the following behavior.
# After creating the "inprogress" file, it fails renaming (only makes a single attempt) to the final name in the working directory.
# After failing rename, it proceeds to consider the message "delivered" and does not attempt to reconsume the file on the next poll (this seems wrong to me, but I'm a newbie).
\\
h3. Issue #2: {{tempPrefix}}
You are right this is fixed with CAMEL-1965 as you indicated. _However_, when using {{tempPrefix}} with the {{fileExist}} option does behave as expected. Instead, the {{fileExist}} options are applied to the "temp" file and not to the target file.
|| Option Value || Expected Behavior || Actual Behavior when using {{tempPrefix}} ||
| {{Override}} (Default) | If the target file exists, overwrite it | Will overwrite the temp file if it exists but will fail if the target file exists because the rename operation fails |
| {{Fail}} | Fail the exchange immediately if the target file exists | Will fail the exchange immediately if the temp file exists. Will not fail immediately if the target file exists. The exchange still fails ultimately though due to the failing of the rename operation. |
I did not test {{Append}} or {{Ignore}} values but they probably have similar problems. Also, I did not test other File component options that could involve rename operations such as {{readLock:rename}}, {{preMove}}, or {{moveFailed}}.
Let me know if you want me to file a separate issue for #2. Given the number of potential issues with rename and the Java implementation of it (other filesystems?), I'm still thinking that an eventual "ForceCopyInsteadOfRename" option might be a good idea.
was (Author: mbmather):
For the record, I did try my route in 2.1-SNAPSHOT before filing the report. But apparently I tried it just for {{localWorkDirectory}} and not for {{tempPrefix}}.
2.1-SNAPSHOT does seem to be fixed for {{tempPrefix}} (although I think I still found an issue there), but +NOT+ fixed for {{localWorkDirectory}}, an option of the FTP component.
\\
h3. Issue #1: {{localWorkDirectory}}
In both 2.0 and 2.1snap, this has the following behavior.
# After creating the "inprogress" file, it fails renaming (only makes a single attempt) to the final name in the working directory.
# After failing rename, it proceeds to consider the message "delivered" and does not attempt to reconsume the file on the next poll (this seems wrong to me, but I'm a newbie).
\\
h3. Issue #2: {{tempPrefix}}
You are right this is fixed with CAMEL-1965 as you indicated. _However_, when using {{tempPrefix}} with the {{fileExist}} option does behave as expected. Instead, the {{fileExist}} options are applied to the "temp" file and not to the target file.
|| Option Value || Expected Behavior || Actual Behavior when using {{tempPrefix}} ||
| {{Override}} (Default) | If the target file exists, overwrite it | Will overwrite the temp file if it exists but will fail if the target file exists because the rename operation fails |
| {{Fail}} | Fail the exchange immediately if the target file exists | Will fail the exchange immediately if the temp file exists. Will not fail immediately if the target file exists. The exchange still fails ultimately though due to the failing of the rename operation. |
I did not test {{Append}} or {{Ignore}} values but they probably have similar problems. Also, I did not test other File component options that could involve rename operations such as {{readLock:rename}}, {{preMove}}, or {{moveFailed}}.
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Michael Mathers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54952#action_54952 ]
Michael Mathers commented on CAMEL-2058:
----------------------------------------
Sorry, I haven't got back to help you out with this as you requested. I just haven't been able to. I'll try to hit the new changes this weekend.
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54827#action_54827 ]
Claus Ibsen commented on CAMEL-2058:
------------------------------------
Can you post a snippet of the route that you are using to test this?
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-2058.
--------------------------------
Resolution: Fixed
trunk: 828994.
Michael can you test again?
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54712#action_54712 ]
Claus Ibsen commented on CAMEL-2058:
------------------------------------
Michael can you try 2.1-SNAPSHOT as we have fixed improved this. I think the ticket was CAMEL-1963 or the likes (reported by Charles about some transaction stuff)
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CAMEL-2058) File Component - Rename
operations fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54712#action_54712 ]
Claus Ibsen edited comment on CAMEL-2058 at 10/12/09 4:07 AM:
--------------------------------------------------------------
Michael can you try 2.1-SNAPSHOT as we have fixed improved this.
Fixed already by CAMEL-1965.
was (Author: davsclaus):
Michael can you try 2.1-SNAPSHOT as we have fixed improved this. I think the ticket was CAMEL-1963 or the likes (reported by Charles about some transaction stuff)
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-2058:
-------------------------------
Fix Version/s: 2.1.0
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Michael Mathers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54955#action_54955 ]
Michael Mathers commented on CAMEL-2058:
----------------------------------------
I produced a test to test the issue with the {{tempPrefix}} use on Windows when the temporary file pre-exists and {{fileExist}} behavior is set to 'Fail'. This test fails in 2.0.0 and now passes in 2.1-SNAPSHOT. I also tested that when the _actual_ target file pre-exists that it fails correctly (and not failing due to a "rename" failure).
I'd produce the test code here but it is currently intermixed with proprietary stuff. This looks totally resolved to me!
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work started: (CAMEL-2058) File Component - Rename
operations fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on CAMEL-2058 started by Claus Ibsen.
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54825#action_54825 ]
Claus Ibsen commented on CAMEL-2058:
------------------------------------
*Issue #1*
1. I am fixing the rename to be aligned with other code by doing retries to cater for Windows platforms.
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54957#action_54957 ]
Claus Ibsen commented on CAMEL-2058:
------------------------------------
Thanks Michael for reporting and testing it.
We can add your test code to Camel when you have the proprietary removed.
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54938#action_54938 ]
Claus Ibsen commented on CAMEL-2058:
------------------------------------
*Issue #2*
I guess it depends on the OS. On my Mac the rename operation will succeed even if the target file exists. So that could be an Windows OS issue, that wont allow renaming a file when the target filename already exists.
But Camel should of course use the {{fileExist}} option in any case
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Michael Mathers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54720#action_54720 ]
Michael Mathers commented on CAMEL-2058:
----------------------------------------
For the record, I did try my route in 2.1-SNAPSHOT before filing the report. But apparently I tried it just for {{localWorkDirectory}} and not for {{tempPrefix}}.
2.1-SNAPSHOT does seem to be fixed for {{tempPrefix}} (although I think I still found an issue there), but +NOT+ fixed for {{localWorkDirectory}}, an option of the FTP component.
\\
h3. Issue #1: {{localWorkDirectory}}
In both 2.0 and 2.1snap, this has the following behavior.
# After creating the "inprogress" file, it fails renaming (only makes a single attempt) to the final name in the working directory.
# After failing rename, it proceeds to consider the message "delivered" and does not attempt to reconsume the file on the next poll (this seems wrong to me, but I'm a newbie).
\\
h3. Issue #2: {{tempPrefix}}
You are right this is fixed with CAMEL-1965 as you indicated. _However_, when using {{tempPrefix}} with the {{fileExist}} option does behave as expected. Instead, the {{fileExist}} options are applied to the "temp" file and not to the target file.
|| Option Value || Expected Behavior || Actual Behavior when using {{tempPrefix}} ||
| {{Override}} (Default) | If the target file exists, overwrite it | Will overwrite the temp file if it exists but will fail if the target file exists because the rename operation fails |
| {{Fail}} | Fail the exchange immediately if the target file exists | Will fail the exchange immediately if the temp file exists. Will not fail immediately if the target file exists. The exchange still fails ultimately though due to the failing of the rename operation. |
I did not test {{Append}} or {{Ignore}} values but they probably have similar problems. Also, I did not test other File component options that could involve rename operations such as {{readLock:rename}}, {{preMove}}, or {{moveFailed}}.
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CAMEL-2058) File Component - Rename operations
fail sometimes on certain filesystems (Windows)
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen reassigned CAMEL-2058:
----------------------------------
Assignee: Claus Ibsen
> File Component - Rename operations fail sometimes on certain filesystems (Windows)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-2058
> URL: https://issues.apache.org/activemq/browse/CAMEL-2058
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0.0
> Environment: Sun JDK 1.6_16, Eclipse Gallileo, Spring 2.5, Camel 2.0.0, Windows XP SP2, McAfee VirusScan
> Reporter: Michael Mathers
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> On Windows (don't know if there are other platforms that suffer from this problem), the file component fails to successfully rename files with the File.renameTo operation. It fails when the rename is performed immediately after closing a file. On Windows this usually indicates that some other process has the file open. This occurs due to things like Virus scanners which keep the file open for very short periods of time. Given a slight pause the rename would succeed.
> This is a serious problem which effectively makes useless options like "tempPrefix" in the File Producer and "localWorkDirectory" with the FTP Consumer. Workarounds like "disable your virus scanner" don't cut it for everyone (me specifically) as I'm system privilege restricted from doing so, and even then, there's no guarantee that other windows processes might not do similar things (file indexers, etc).
> The Java spec doesn't define the behavior of the rename operation and specifically says that this can vary from implementation to implementation / filesystem to filesystem. Second, rename doesn't say why it fails, it merely returns false which is very unhelpful.
> A couple ways to fix:
> 1). Provide an option to disable this optimization. ie, a "alwaysCopyInsteadOfRename" or something (clean, simple, easy).
> This would be a simple fix. More or less just a few clauses/tests in GenericFileProducer around any "is local" checks....
> 2). Attempt a copy instead if the rename fails
> - maybe after a brief pause
> - maybe even after a number of rename attempts
> - maybe watch to see the file is closed prior to a rename attempt.
> Rename operation failures may affect other things as well like certain locking schemes.
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4167147
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.