You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Edgar Fuß <ef...@math.uni-bonn.de> on 2009/09/23 14:30:15 UTC
Repositories mounted on NFS don't work
I would like to file this as an issue, but the Guidelines urge me to wait for a response on the list. I've already brought up the subject six weeks ago, but still no answer. Should I just go ahead and file an issue?
The problem forces me to stick with 1.4.2, so here it is again:
At least on NetBSD (see Problem Report pkg/40229), Subversion 1.5 doesn't work with repositories residing on NFS-mounted file systems (at least not with a NetBSD 4 NFS server). With 1.4, they do work.
The problem is related to the fix of Issue 1896. Due to NFS cacheing, files that have already been deleted may re-appear after the directory rewinding, causing svn_io_remove_dir2() to fail. Disabling the rewind apperantly "fixes" the problem.
I suppose one could either build a list of all files to be deleted beforehand, as suggested by Greg Hudson in the comments to said Issue; or, one could just ignore any ENOENT errors -- I can't think of any downside of that, but I might be wrong.
I'm not subscribed to the list. In case I am supposed to subscribe when reporting an Issue, please tell me and I will do so. I will also be glad for any workaround. Or is this considered not to be a bug in Subversion?
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2398903
Re: Repositories mounted on NFS don't work
Posted by Stefan Sperling <st...@elego.de>.
On Wed, Sep 23, 2009 at 04:30:15PM +0200, Edgar Fuß wrote:
> I would like to file this as an issue, but the Guidelines urge me to wait for a response on the list. I've already brought up the subject six weeks ago, but still no answer. Should I just go ahead and file an issue?
Yes, please do.
> I'm not subscribed to the list. In case I am supposed to subscribe when reporting an Issue, please tell me and I will do so.
That's not necessary. You can add yourself to the Cc list of the new
issue, if you like.
Stefan
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2398912
Re: Issue 3501: Repositories mounted on NFS don't work
Posted by Peter Samuelson <pe...@p12n.org>.
[Bert Huijben]
> Do we need a backport of this fix to 1.6.x?
Created and nominated branch 1.6.x-r898963.
Peter
Re: Issue 3501: Repositories mounted on NFS don't work
Posted by Peter Samuelson <pe...@p12n.org>.
[Bert Huijben]
> Do we need a backport of this fix to 1.6.x?
Hmmm - yeah - I guess it would be an easy backport to vote on. Even
though, as you say, the fix doesn't literally merge cleanly. I'll try
and remember to propose it tonight, when I have time to do the whole
backport branch thing.
Peter
Re: Issue 3501: Repositories mounted on NFS don't work
Posted by Peter Samuelson <pe...@p12n.org>.
[Bert Huijben]
> Do we need a backport of this fix to 1.6.x?
Hmmm - yeah - I guess it would be an easy backport to vote on. Even
though, as you say, the fix doesn't literally merge cleanly. I'll try
and remember to propose it tonight, when I have time to do the whole
backport branch thing.
Peter
Re: Issue 3501: Repositories mounted on NFS don't work
Posted by Peter Samuelson <pe...@p12n.org>.
[Bert Huijben]
> Do we need a backport of this fix to 1.6.x?
Created and nominated branch 1.6.x-r898963.
Peter
RE: Issue 3501: Repositories mounted on NFS don't work
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: Peter Samuelson [mailto:peter@p12n.org]
> Sent: woensdag 13 januari 2010 23:07
> To: dev@subversion.apache.org; Edgar Fuß
> Subject: Re: Issue 3501: Repositories mounted on NFS don't work
>
>
> [Bert Huijben]
> > I think it is easier to use svn_io_get_dirents2() then this block
> > replicating its implementation.
>
> Indeed it is easier. It took me several days to get around to testing
> and committing - r898963.
Do we need a backport of this fix to 1.6.x?
(It would need a backport branch as one of the functions the new code used
is not available in 1.6)
Bert
RE: Issue 3501: Repositories mounted on NFS don't work
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: Peter Samuelson [mailto:peter@p12n.org]
> Sent: woensdag 13 januari 2010 23:07
> To: dev@subversion.apache.org; Edgar Fuß
> Subject: Re: Issue 3501: Repositories mounted on NFS don't work
>
>
> [Bert Huijben]
> > I think it is easier to use svn_io_get_dirents2() then this block
> > replicating its implementation.
>
> Indeed it is easier. It took me several days to get around to testing
> and committing - r898963.
Do we need a backport of this fix to 1.6.x?
(It would need a backport branch as one of the functions the new code used
is not available in 1.6)
Bert
Re: Issue 3501: Repositories mounted on NFS don't work
Posted by Peter Samuelson <pe...@p12n.org>.
[Bert Huijben]
> I think it is easier to use svn_io_get_dirents2() then this block
> replicating its implementation.
Indeed it is easier. It took me several days to get around to testing
and committing - r898963.
Peter
RE: Issue 3501: Repositories mounted on NFS don't work
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: Peter Samuelson [mailto:peter@p12n.org]
> Sent: donderdag 7 januari 2010 2:30
> To: Edgar Fuß
> Cc: dev@subversion.apache.org
> Subject: Re: Issue 3501: Repositories mounted on NFS don't work
>
>
> [Edgar Fuß]
> > To summarise the problem: recursive directory deletion fails because
> entries
> > re-appear after dir rewinding. The fix may be as easy as ignoring
ENOENT.
>
> Or not calling rewind. Something like the following (untested) against
> 1.6.x. Same patch applies to trunk except for some conflicts involving
> path vs. dirent functions.
> --
> Peter Samuelson | org-tld!p12n!peter | http://p12n.org/
>
> [[[
> * subversion/libsvn_subr/io.c
> (svn_io_remove_dir2): Remove rewinddir() magic (see Issues 1896 and
> 3501). Instead, build linked lists of files and subdirs to delete,
> then delete them in separate loops.
> ]]]
>
> Index: subversion/libsvn_subr/io.c
> ==========================================================
> =========
> --- subversion/libsvn_subr/io.c (revisione 887406)
> +++ subversion/libsvn_subr/io.c (copia locale)
> @@ -1862,17 +1862,13 @@
>
> Similar problem has been observed on FreeBSD.
>
> - See http://subversion.tigris.org/issues/show_bug.cgi?id=1896 for more
> - discussion and an initial solution.
> + The workaround is to collect filenames in the readdir loop, then
> + delete them in a separate loop (two, actually, one for subdirs and one
> + for other files). A previous workaround involving rewinddir is
> + problematic on Win32 and some NFS clients, notably NetBSD.
>
> - To work around the problem, we do a rewinddir after we delete all files
> - and see if there's anything left. We repeat the steps untill there's
> - nothing left to delete.
> -
> - This workaround causes issues on Windows where delete's are
> asynchronous,
> - however, so we never rewind if we're on Windows (the delete says it is
> - complete, we rewind, we see the same file and try to delete it again,
> - we fail.
> + See http://subversion.tigris.org/issues/show_bug.cgi?id=1896 and
> + http://subversion.tigris.org/issues/show_bug.cgi?id=3501.
> */
>
> /* Neither windows nor unix allows us to delete a non-empty
> @@ -1890,7 +1886,8 @@
> apr_pool_t *subpool;
> apr_int32_t flags = APR_FINFO_TYPE | APR_FINFO_NAME;
> const char *path_apr;
> - int need_rewind;
> + struct path_list { char *path; struct path_list *next; }
> + *del_files = NULL, *del_dirs = NULL, *del_tmp;
>
> /* Check for pending cancellation request.
> If we need to bail out, do so early. */
> @@ -1922,72 +1919,36 @@
>
> subpool = svn_pool_create(pool);
>
> - do
> + while ((status = apr_dir_read(&this_entry, flags, this_dir)) ==
> APR_SUCCESS)
> {
> - need_rewind = FALSE;
> + const char *entry_utf8;
> + if ((this_entry.filetype == APR_DIR)
> + && ((this_entry.name[0] == '.')
> + && ((this_entry.name[1] == '\0')
> + || ((this_entry.name[1] == '.')
> + && (this_entry.name[2] == '\0')))))
> + {
> + /* skip "." and ".." */
> + continue;
> + }
> + SVN_ERR(entry_name_to_utf8(&entry_utf8, this_entry.name,
> + path_apr, subpool));
>
> - for (status = apr_dir_read(&this_entry, flags, this_dir);
> - status == APR_SUCCESS;
> - status = apr_dir_read(&this_entry, flags, this_dir))
> - {
> - svn_pool_clear(subpool);
> - if ((this_entry.filetype == APR_DIR)
> - && ((this_entry.name[0] == '.')
> - && ((this_entry.name[1] == '\0')
> - || ((this_entry.name[1] == '.')
> - && (this_entry.name[2] == '\0')))))
> - {
> - continue;
> - }
> - else /* something other than "." or "..", so proceed */
> - {
> - const char *fullpath, *entry_utf8;
> + del_tmp = apr_palloc(subpool, sizeof *del_tmp);
> + del_tmp->path = svn_path_join(path, entry_utf8, subpool);
>
> -#ifndef WIN32
> - need_rewind = TRUE;
> -#endif
> -
> - SVN_ERR(entry_name_to_utf8(&entry_utf8, this_entry.name,
> - path_apr, subpool));
> -
> - fullpath = svn_path_join(path, entry_utf8, subpool);
> -
> - if (this_entry.filetype == APR_DIR)
> - {
> - /* Don't check for cancellation, the callee
> - will immediately do so */
> - SVN_ERR(svn_io_remove_dir2(fullpath, FALSE,
> - cancel_func, cancel_baton,
> - subpool));
> - }
> - else
> - {
> - svn_error_t *err;
> -
> - if (cancel_func)
> - SVN_ERR((*cancel_func)(cancel_baton));
> -
> - err = svn_io_remove_file(fullpath, subpool);
> - if (err)
> - return svn_error_createf
> - (err->apr_err, err, _("Can't remove '%s'"),
> - svn_path_local_style(fullpath, subpool));
> - }
> - }
> - }
> -
> - if (need_rewind)
> - {
> - status = apr_dir_rewind(this_dir);
> - if (status)
> - return svn_error_wrap_apr(status, _("Can't rewind directory
'%s'"),
> - svn_path_local_style (path, pool));
> - }
I think it is easier to use svn_io_get_dirents2() then this block
replicating its implementation.
/** Read all of the disk entries in directory @a path, a utf8-encoded
* path. Set @a *dirents to a hash mapping dirent names (<tt>char *</tt>)
to
* #svn_io_dirent_t structures, allocated in @a pool.
*
* @note The `.' and `..' directories normally returned by
* apr_dir_read() are NOT returned in the hash.
*
* @note The kind field in the @a dirents is set according to the mapping
* as documented for svn_io_check_path()
*
* @since New in 1.3.
*/
svn_error_t *
svn_io_get_dirents2(apr_hash_t **dirents,
const char *path,
apr_pool_t *pool);
> + if (this_entry.filetype == APR_DIR)
> + {
> + del_tmp->next = del_dirs;
> + del_dirs = del_tmp;
> + }
> + else
> + {
> + del_tmp->next = del_files;
> + del_files = del_tmp;
> + }
> }
> - while (need_rewind);
>
> - svn_pool_destroy(subpool);
> -
> if (!APR_STATUS_IS_ENOENT(status))
> return svn_error_wrap_apr(status, _("Can't read directory '%s'"),
> svn_path_local_style(path, pool));
> @@ -1997,6 +1958,31 @@
> return svn_error_wrap_apr(status, _("Error closing directory '%s'"),
> svn_path_local_style(path, pool));
>
> + for (; del_dirs; del_dirs = del_dirs->next)
> + {
> + /* Don't check for cancellation, the callee will immediately do so
*/
> + SVN_ERR(svn_io_remove_dir2(del_dirs->path, FALSE,
> + cancel_func, cancel_baton,
> + subpool));
> + }
> + for (; del_files; del_files = del_files->next)
> + {
> + svn_error_t *err;
> +
> + if (cancel_func)
> + SVN_ERR((*cancel_func)(cancel_baton));
> +
> + err = svn_io_remove_file2(del_files->path, FALSE, subpool);
> + if (err)
> + {
> + return svn_error_createf
> + (err->apr_err, err, _("Can't remove '%s'"),
> + svn_path_local_style(del_files->path, subpool));
> + }
> + }
> +
> + svn_pool_destroy(subpool);
> +
> status = apr_dir_remove(path_apr, pool);
> WIN32_RETRY_LOOP(status, apr_dir_remove(path_apr, pool));
> if (status)
RE: Issue 3501: Repositories mounted on NFS don't work
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: Peter Samuelson [mailto:peter@p12n.org]
> Sent: donderdag 7 januari 2010 2:30
> To: Edgar Fuß
> Cc: dev@subversion.apache.org
> Subject: Re: Issue 3501: Repositories mounted on NFS don't work
>
>
> [Edgar Fuß]
> > To summarise the problem: recursive directory deletion fails because
> entries
> > re-appear after dir rewinding. The fix may be as easy as ignoring
ENOENT.
>
> Or not calling rewind. Something like the following (untested) against
> 1.6.x. Same patch applies to trunk except for some conflicts involving
> path vs. dirent functions.
> --
> Peter Samuelson | org-tld!p12n!peter | http://p12n.org/
>
> [[[
> * subversion/libsvn_subr/io.c
> (svn_io_remove_dir2): Remove rewinddir() magic (see Issues 1896 and
> 3501). Instead, build linked lists of files and subdirs to delete,
> then delete them in separate loops.
> ]]]
>
> Index: subversion/libsvn_subr/io.c
> ==========================================================
> =========
> --- subversion/libsvn_subr/io.c (revisione 887406)
> +++ subversion/libsvn_subr/io.c (copia locale)
> @@ -1862,17 +1862,13 @@
>
> Similar problem has been observed on FreeBSD.
>
> - See http://subversion.tigris.org/issues/show_bug.cgi?id=1896 for more
> - discussion and an initial solution.
> + The workaround is to collect filenames in the readdir loop, then
> + delete them in a separate loop (two, actually, one for subdirs and one
> + for other files). A previous workaround involving rewinddir is
> + problematic on Win32 and some NFS clients, notably NetBSD.
>
> - To work around the problem, we do a rewinddir after we delete all files
> - and see if there's anything left. We repeat the steps untill there's
> - nothing left to delete.
> -
> - This workaround causes issues on Windows where delete's are
> asynchronous,
> - however, so we never rewind if we're on Windows (the delete says it is
> - complete, we rewind, we see the same file and try to delete it again,
> - we fail.
> + See http://subversion.tigris.org/issues/show_bug.cgi?id=1896 and
> + http://subversion.tigris.org/issues/show_bug.cgi?id=3501.
> */
>
> /* Neither windows nor unix allows us to delete a non-empty
> @@ -1890,7 +1886,8 @@
> apr_pool_t *subpool;
> apr_int32_t flags = APR_FINFO_TYPE | APR_FINFO_NAME;
> const char *path_apr;
> - int need_rewind;
> + struct path_list { char *path; struct path_list *next; }
> + *del_files = NULL, *del_dirs = NULL, *del_tmp;
>
> /* Check for pending cancellation request.
> If we need to bail out, do so early. */
> @@ -1922,72 +1919,36 @@
>
> subpool = svn_pool_create(pool);
>
> - do
> + while ((status = apr_dir_read(&this_entry, flags, this_dir)) ==
> APR_SUCCESS)
> {
> - need_rewind = FALSE;
> + const char *entry_utf8;
> + if ((this_entry.filetype == APR_DIR)
> + && ((this_entry.name[0] == '.')
> + && ((this_entry.name[1] == '\0')
> + || ((this_entry.name[1] == '.')
> + && (this_entry.name[2] == '\0')))))
> + {
> + /* skip "." and ".." */
> + continue;
> + }
> + SVN_ERR(entry_name_to_utf8(&entry_utf8, this_entry.name,
> + path_apr, subpool));
>
> - for (status = apr_dir_read(&this_entry, flags, this_dir);
> - status == APR_SUCCESS;
> - status = apr_dir_read(&this_entry, flags, this_dir))
> - {
> - svn_pool_clear(subpool);
> - if ((this_entry.filetype == APR_DIR)
> - && ((this_entry.name[0] == '.')
> - && ((this_entry.name[1] == '\0')
> - || ((this_entry.name[1] == '.')
> - && (this_entry.name[2] == '\0')))))
> - {
> - continue;
> - }
> - else /* something other than "." or "..", so proceed */
> - {
> - const char *fullpath, *entry_utf8;
> + del_tmp = apr_palloc(subpool, sizeof *del_tmp);
> + del_tmp->path = svn_path_join(path, entry_utf8, subpool);
>
> -#ifndef WIN32
> - need_rewind = TRUE;
> -#endif
> -
> - SVN_ERR(entry_name_to_utf8(&entry_utf8, this_entry.name,
> - path_apr, subpool));
> -
> - fullpath = svn_path_join(path, entry_utf8, subpool);
> -
> - if (this_entry.filetype == APR_DIR)
> - {
> - /* Don't check for cancellation, the callee
> - will immediately do so */
> - SVN_ERR(svn_io_remove_dir2(fullpath, FALSE,
> - cancel_func, cancel_baton,
> - subpool));
> - }
> - else
> - {
> - svn_error_t *err;
> -
> - if (cancel_func)
> - SVN_ERR((*cancel_func)(cancel_baton));
> -
> - err = svn_io_remove_file(fullpath, subpool);
> - if (err)
> - return svn_error_createf
> - (err->apr_err, err, _("Can't remove '%s'"),
> - svn_path_local_style(fullpath, subpool));
> - }
> - }
> - }
> -
> - if (need_rewind)
> - {
> - status = apr_dir_rewind(this_dir);
> - if (status)
> - return svn_error_wrap_apr(status, _("Can't rewind directory
'%s'"),
> - svn_path_local_style (path, pool));
> - }
I think it is easier to use svn_io_get_dirents2() then this block
replicating its implementation.
/** Read all of the disk entries in directory @a path, a utf8-encoded
* path. Set @a *dirents to a hash mapping dirent names (<tt>char *</tt>)
to
* #svn_io_dirent_t structures, allocated in @a pool.
*
* @note The `.' and `..' directories normally returned by
* apr_dir_read() are NOT returned in the hash.
*
* @note The kind field in the @a dirents is set according to the mapping
* as documented for svn_io_check_path()
*
* @since New in 1.3.
*/
svn_error_t *
svn_io_get_dirents2(apr_hash_t **dirents,
const char *path,
apr_pool_t *pool);
> + if (this_entry.filetype == APR_DIR)
> + {
> + del_tmp->next = del_dirs;
> + del_dirs = del_tmp;
> + }
> + else
> + {
> + del_tmp->next = del_files;
> + del_files = del_tmp;
> + }
> }
> - while (need_rewind);
>
> - svn_pool_destroy(subpool);
> -
> if (!APR_STATUS_IS_ENOENT(status))
> return svn_error_wrap_apr(status, _("Can't read directory '%s'"),
> svn_path_local_style(path, pool));
> @@ -1997,6 +1958,31 @@
> return svn_error_wrap_apr(status, _("Error closing directory '%s'"),
> svn_path_local_style(path, pool));
>
> + for (; del_dirs; del_dirs = del_dirs->next)
> + {
> + /* Don't check for cancellation, the callee will immediately do so
*/
> + SVN_ERR(svn_io_remove_dir2(del_dirs->path, FALSE,
> + cancel_func, cancel_baton,
> + subpool));
> + }
> + for (; del_files; del_files = del_files->next)
> + {
> + svn_error_t *err;
> +
> + if (cancel_func)
> + SVN_ERR((*cancel_func)(cancel_baton));
> +
> + err = svn_io_remove_file2(del_files->path, FALSE, subpool);
> + if (err)
> + {
> + return svn_error_createf
> + (err->apr_err, err, _("Can't remove '%s'"),
> + svn_path_local_style(del_files->path, subpool));
> + }
> + }
> +
> + svn_pool_destroy(subpool);
> +
> status = apr_dir_remove(path_apr, pool);
> WIN32_RETRY_LOOP(status, apr_dir_remove(path_apr, pool));
> if (status)
[PATCH] Issue 3501: Repositories mounted on NFS don't work
Posted by Peter Samuelson <pe...@p12n.org>.
[Peter Samuelson]
> Something like the following (untested) against 1.6.x.
Well, it almost worked. Here's one, also for 1.6.x, that seems to pass
the tests. Edgar, can you test this on your NFS client?
I'll commit the trunk version of this soon if no one objects. It
shouldn't need a new test; issue 3501 causes lots of test failures
already.
--
Peter Samuelson | org-tld!p12n!peter | http://p12n.org/
[[[
Fix issue #3501: svn_io_remove_dir2 failing due to NFS caching.
* subversion/libsvn_subr/io.c
(svn_io_remove_dir2): Remove rewinddir() magic from Issue 1896
fix. Instead, build linked lists of files and subdirs to delete,
then delete them in separate loops.
]]]
--- subversion/libsvn_subr/io.c
+++ subversion/libsvn_subr/io.c
@@ -1862,17 +1862,13 @@
Similar problem has been observed on FreeBSD.
- See http://subversion.tigris.org/issues/show_bug.cgi?id=1896 for more
- discussion and an initial solution.
+ The workaround is to collect filenames in the readdir loop, then
+ delete them in a separate loop (two, actually, one for subdirs and one
+ for other files). A previous workaround involving rewinddir is
+ problematic on Win32 and some NFS clients, notably NetBSD.
- To work around the problem, we do a rewinddir after we delete all files
- and see if there's anything left. We repeat the steps untill there's
- nothing left to delete.
-
- This workaround causes issues on Windows where delete's are asynchronous,
- however, so we never rewind if we're on Windows (the delete says it is
- complete, we rewind, we see the same file and try to delete it again,
- we fail.
+ See http://subversion.tigris.org/issues/show_bug.cgi?id=1896 and
+ http://subversion.tigris.org/issues/show_bug.cgi?id=3501.
*/
/* Neither windows nor unix allows us to delete a non-empty
@@ -1890,7 +1886,8 @@
apr_pool_t *subpool;
apr_int32_t flags = APR_FINFO_TYPE | APR_FINFO_NAME;
const char *path_apr;
- int need_rewind;
+ struct path_list { char *path; struct path_list *next; }
+ *del_files = NULL, *del_dirs = NULL, *del_tmp;
/* Check for pending cancellation request.
If we need to bail out, do so early. */
@@ -1922,72 +1919,36 @@
subpool = svn_pool_create(pool);
- do
+ while ((status = apr_dir_read(&this_entry, flags, this_dir)) == APR_SUCCESS)
{
- need_rewind = FALSE;
-
- for (status = apr_dir_read(&this_entry, flags, this_dir);
- status == APR_SUCCESS;
- status = apr_dir_read(&this_entry, flags, this_dir))
+ const char *entry_utf8;
+ if ((this_entry.filetype == APR_DIR)
+ && ((this_entry.name[0] == '.')
+ && ((this_entry.name[1] == '\0')
+ || ((this_entry.name[1] == '.')
+ && (this_entry.name[2] == '\0')))))
{
- svn_pool_clear(subpool);
- if ((this_entry.filetype == APR_DIR)
- && ((this_entry.name[0] == '.')
- && ((this_entry.name[1] == '\0')
- || ((this_entry.name[1] == '.')
- && (this_entry.name[2] == '\0')))))
- {
- continue;
- }
- else /* something other than "." or "..", so proceed */
- {
- const char *fullpath, *entry_utf8;
+ /* skip "." and ".." */
+ continue;
+ }
+ SVN_ERR(entry_name_to_utf8(&entry_utf8, this_entry.name,
+ path_apr, subpool));
-#ifndef WIN32
- need_rewind = TRUE;
-#endif
+ del_tmp = apr_palloc(subpool, sizeof *del_tmp);
+ del_tmp->path = svn_path_join(path, entry_utf8, subpool);
- SVN_ERR(entry_name_to_utf8(&entry_utf8, this_entry.name,
- path_apr, subpool));
-
- fullpath = svn_path_join(path, entry_utf8, subpool);
-
- if (this_entry.filetype == APR_DIR)
- {
- /* Don't check for cancellation, the callee
- will immediately do so */
- SVN_ERR(svn_io_remove_dir2(fullpath, FALSE,
- cancel_func, cancel_baton,
- subpool));
- }
- else
- {
- svn_error_t *err;
-
- if (cancel_func)
- SVN_ERR((*cancel_func)(cancel_baton));
-
- err = svn_io_remove_file(fullpath, subpool);
- if (err)
- return svn_error_createf
- (err->apr_err, err, _("Can't remove '%s'"),
- svn_path_local_style(fullpath, subpool));
- }
- }
+ if (this_entry.filetype == APR_DIR)
+ {
+ del_tmp->next = del_dirs;
+ del_dirs = del_tmp;
}
-
- if (need_rewind)
+ else
{
- status = apr_dir_rewind(this_dir);
- if (status)
- return svn_error_wrap_apr(status, _("Can't rewind directory '%s'"),
- svn_path_local_style (path, pool));
+ del_tmp->next = del_files;
+ del_files = del_tmp;
}
}
- while (need_rewind);
- svn_pool_destroy(subpool);
-
if (!APR_STATUS_IS_ENOENT(status))
return svn_error_wrap_apr(status, _("Can't read directory '%s'"),
svn_path_local_style(path, pool));
@@ -1997,6 +1958,29 @@
return svn_error_wrap_apr(status, _("Error closing directory '%s'"),
svn_path_local_style(path, pool));
+ for (; del_dirs; del_dirs = del_dirs->next)
+ {
+ /* Don't check for cancellation, the callee will immediately do so */
+ SVN_ERR(svn_io_remove_dir2(del_dirs->path, FALSE,
+ cancel_func, cancel_baton,
+ subpool));
+ }
+ for (; del_files; del_files = del_files->next)
+ {
+ svn_error_t *err;
+
+ if (cancel_func)
+ SVN_ERR((*cancel_func)(cancel_baton));
+
+ err = svn_io_remove_file(del_files->path, subpool);
+ if (err)
+ return svn_error_createf
+ (err->apr_err, err, _("Can't remove '%s'"),
+ svn_path_local_style(del_files->path, subpool));
+ }
+
+ svn_pool_destroy(subpool);
+
status = apr_dir_remove(path_apr, pool);
WIN32_RETRY_LOOP(status, apr_dir_remove(path_apr, pool));
if (status)
Re: Issue 3501: Repositories mounted on NFS don't work
Posted by Peter Samuelson <pe...@p12n.org>.
[Edgar Fuß]
> To summarise the problem: recursive directory deletion fails because entries
> re-appear after dir rewinding. The fix may be as easy as ignoring ENOENT.
Or not calling rewind. Something like the following (untested) against
1.6.x. Same patch applies to trunk except for some conflicts involving
path vs. dirent functions.
--
Peter Samuelson | org-tld!p12n!peter | http://p12n.org/
[[[
* subversion/libsvn_subr/io.c
(svn_io_remove_dir2): Remove rewinddir() magic (see Issues 1896 and
3501). Instead, build linked lists of files and subdirs to delete,
then delete them in separate loops.
]]]
Index: subversion/libsvn_subr/io.c
===================================================================
--- subversion/libsvn_subr/io.c (revisione 887406)
+++ subversion/libsvn_subr/io.c (copia locale)
@@ -1862,17 +1862,13 @@
Similar problem has been observed on FreeBSD.
- See http://subversion.tigris.org/issues/show_bug.cgi?id=1896 for more
- discussion and an initial solution.
+ The workaround is to collect filenames in the readdir loop, then
+ delete them in a separate loop (two, actually, one for subdirs and one
+ for other files). A previous workaround involving rewinddir is
+ problematic on Win32 and some NFS clients, notably NetBSD.
- To work around the problem, we do a rewinddir after we delete all files
- and see if there's anything left. We repeat the steps untill there's
- nothing left to delete.
-
- This workaround causes issues on Windows where delete's are asynchronous,
- however, so we never rewind if we're on Windows (the delete says it is
- complete, we rewind, we see the same file and try to delete it again,
- we fail.
+ See http://subversion.tigris.org/issues/show_bug.cgi?id=1896 and
+ http://subversion.tigris.org/issues/show_bug.cgi?id=3501.
*/
/* Neither windows nor unix allows us to delete a non-empty
@@ -1890,7 +1886,8 @@
apr_pool_t *subpool;
apr_int32_t flags = APR_FINFO_TYPE | APR_FINFO_NAME;
const char *path_apr;
- int need_rewind;
+ struct path_list { char *path; struct path_list *next; }
+ *del_files = NULL, *del_dirs = NULL, *del_tmp;
/* Check for pending cancellation request.
If we need to bail out, do so early. */
@@ -1922,72 +1919,36 @@
subpool = svn_pool_create(pool);
- do
+ while ((status = apr_dir_read(&this_entry, flags, this_dir)) == APR_SUCCESS)
{
- need_rewind = FALSE;
+ const char *entry_utf8;
+ if ((this_entry.filetype == APR_DIR)
+ && ((this_entry.name[0] == '.')
+ && ((this_entry.name[1] == '\0')
+ || ((this_entry.name[1] == '.')
+ && (this_entry.name[2] == '\0')))))
+ {
+ /* skip "." and ".." */
+ continue;
+ }
+ SVN_ERR(entry_name_to_utf8(&entry_utf8, this_entry.name,
+ path_apr, subpool));
- for (status = apr_dir_read(&this_entry, flags, this_dir);
- status == APR_SUCCESS;
- status = apr_dir_read(&this_entry, flags, this_dir))
- {
- svn_pool_clear(subpool);
- if ((this_entry.filetype == APR_DIR)
- && ((this_entry.name[0] == '.')
- && ((this_entry.name[1] == '\0')
- || ((this_entry.name[1] == '.')
- && (this_entry.name[2] == '\0')))))
- {
- continue;
- }
- else /* something other than "." or "..", so proceed */
- {
- const char *fullpath, *entry_utf8;
+ del_tmp = apr_palloc(subpool, sizeof *del_tmp);
+ del_tmp->path = svn_path_join(path, entry_utf8, subpool);
-#ifndef WIN32
- need_rewind = TRUE;
-#endif
-
- SVN_ERR(entry_name_to_utf8(&entry_utf8, this_entry.name,
- path_apr, subpool));
-
- fullpath = svn_path_join(path, entry_utf8, subpool);
-
- if (this_entry.filetype == APR_DIR)
- {
- /* Don't check for cancellation, the callee
- will immediately do so */
- SVN_ERR(svn_io_remove_dir2(fullpath, FALSE,
- cancel_func, cancel_baton,
- subpool));
- }
- else
- {
- svn_error_t *err;
-
- if (cancel_func)
- SVN_ERR((*cancel_func)(cancel_baton));
-
- err = svn_io_remove_file(fullpath, subpool);
- if (err)
- return svn_error_createf
- (err->apr_err, err, _("Can't remove '%s'"),
- svn_path_local_style(fullpath, subpool));
- }
- }
- }
-
- if (need_rewind)
- {
- status = apr_dir_rewind(this_dir);
- if (status)
- return svn_error_wrap_apr(status, _("Can't rewind directory '%s'"),
- svn_path_local_style (path, pool));
- }
+ if (this_entry.filetype == APR_DIR)
+ {
+ del_tmp->next = del_dirs;
+ del_dirs = del_tmp;
+ }
+ else
+ {
+ del_tmp->next = del_files;
+ del_files = del_tmp;
+ }
}
- while (need_rewind);
- svn_pool_destroy(subpool);
-
if (!APR_STATUS_IS_ENOENT(status))
return svn_error_wrap_apr(status, _("Can't read directory '%s'"),
svn_path_local_style(path, pool));
@@ -1997,6 +1958,31 @@
return svn_error_wrap_apr(status, _("Error closing directory '%s'"),
svn_path_local_style(path, pool));
+ for (; del_dirs; del_dirs = del_dirs->next)
+ {
+ /* Don't check for cancellation, the callee will immediately do so */
+ SVN_ERR(svn_io_remove_dir2(del_dirs->path, FALSE,
+ cancel_func, cancel_baton,
+ subpool));
+ }
+ for (; del_files; del_files = del_files->next)
+ {
+ svn_error_t *err;
+
+ if (cancel_func)
+ SVN_ERR((*cancel_func)(cancel_baton));
+
+ err = svn_io_remove_file2(del_files->path, FALSE, subpool);
+ if (err)
+ {
+ return svn_error_createf
+ (err->apr_err, err, _("Can't remove '%s'"),
+ svn_path_local_style(del_files->path, subpool));
+ }
+ }
+
+ svn_pool_destroy(subpool);
+
status = apr_dir_remove(path_apr, pool);
WIN32_RETRY_LOOP(status, apr_dir_remove(path_apr, pool));
if (status)
Re: Issue 3501: Repositories mounted on NFS don't work
Posted by Peter Samuelson <pe...@p12n.org>.
[Edgar Fuß]
> To summarise the problem: recursive directory deletion fails because entries
> re-appear after dir rewinding. The fix may be as easy as ignoring ENOENT.
Or not calling rewind. Something like the following (untested) against
1.6.x. Same patch applies to trunk except for some conflicts involving
path vs. dirent functions.
--
Peter Samuelson | org-tld!p12n!peter | http://p12n.org/
[[[
* subversion/libsvn_subr/io.c
(svn_io_remove_dir2): Remove rewinddir() magic (see Issues 1896 and
3501). Instead, build linked lists of files and subdirs to delete,
then delete them in separate loops.
]]]
Index: subversion/libsvn_subr/io.c
===================================================================
--- subversion/libsvn_subr/io.c (revisione 887406)
+++ subversion/libsvn_subr/io.c (copia locale)
@@ -1862,17 +1862,13 @@
Similar problem has been observed on FreeBSD.
- See http://subversion.tigris.org/issues/show_bug.cgi?id=1896 for more
- discussion and an initial solution.
+ The workaround is to collect filenames in the readdir loop, then
+ delete them in a separate loop (two, actually, one for subdirs and one
+ for other files). A previous workaround involving rewinddir is
+ problematic on Win32 and some NFS clients, notably NetBSD.
- To work around the problem, we do a rewinddir after we delete all files
- and see if there's anything left. We repeat the steps untill there's
- nothing left to delete.
-
- This workaround causes issues on Windows where delete's are asynchronous,
- however, so we never rewind if we're on Windows (the delete says it is
- complete, we rewind, we see the same file and try to delete it again,
- we fail.
+ See http://subversion.tigris.org/issues/show_bug.cgi?id=1896 and
+ http://subversion.tigris.org/issues/show_bug.cgi?id=3501.
*/
/* Neither windows nor unix allows us to delete a non-empty
@@ -1890,7 +1886,8 @@
apr_pool_t *subpool;
apr_int32_t flags = APR_FINFO_TYPE | APR_FINFO_NAME;
const char *path_apr;
- int need_rewind;
+ struct path_list { char *path; struct path_list *next; }
+ *del_files = NULL, *del_dirs = NULL, *del_tmp;
/* Check for pending cancellation request.
If we need to bail out, do so early. */
@@ -1922,72 +1919,36 @@
subpool = svn_pool_create(pool);
- do
+ while ((status = apr_dir_read(&this_entry, flags, this_dir)) == APR_SUCCESS)
{
- need_rewind = FALSE;
+ const char *entry_utf8;
+ if ((this_entry.filetype == APR_DIR)
+ && ((this_entry.name[0] == '.')
+ && ((this_entry.name[1] == '\0')
+ || ((this_entry.name[1] == '.')
+ && (this_entry.name[2] == '\0')))))
+ {
+ /* skip "." and ".." */
+ continue;
+ }
+ SVN_ERR(entry_name_to_utf8(&entry_utf8, this_entry.name,
+ path_apr, subpool));
- for (status = apr_dir_read(&this_entry, flags, this_dir);
- status == APR_SUCCESS;
- status = apr_dir_read(&this_entry, flags, this_dir))
- {
- svn_pool_clear(subpool);
- if ((this_entry.filetype == APR_DIR)
- && ((this_entry.name[0] == '.')
- && ((this_entry.name[1] == '\0')
- || ((this_entry.name[1] == '.')
- && (this_entry.name[2] == '\0')))))
- {
- continue;
- }
- else /* something other than "." or "..", so proceed */
- {
- const char *fullpath, *entry_utf8;
+ del_tmp = apr_palloc(subpool, sizeof *del_tmp);
+ del_tmp->path = svn_path_join(path, entry_utf8, subpool);
-#ifndef WIN32
- need_rewind = TRUE;
-#endif
-
- SVN_ERR(entry_name_to_utf8(&entry_utf8, this_entry.name,
- path_apr, subpool));
-
- fullpath = svn_path_join(path, entry_utf8, subpool);
-
- if (this_entry.filetype == APR_DIR)
- {
- /* Don't check for cancellation, the callee
- will immediately do so */
- SVN_ERR(svn_io_remove_dir2(fullpath, FALSE,
- cancel_func, cancel_baton,
- subpool));
- }
- else
- {
- svn_error_t *err;
-
- if (cancel_func)
- SVN_ERR((*cancel_func)(cancel_baton));
-
- err = svn_io_remove_file(fullpath, subpool);
- if (err)
- return svn_error_createf
- (err->apr_err, err, _("Can't remove '%s'"),
- svn_path_local_style(fullpath, subpool));
- }
- }
- }
-
- if (need_rewind)
- {
- status = apr_dir_rewind(this_dir);
- if (status)
- return svn_error_wrap_apr(status, _("Can't rewind directory '%s'"),
- svn_path_local_style (path, pool));
- }
+ if (this_entry.filetype == APR_DIR)
+ {
+ del_tmp->next = del_dirs;
+ del_dirs = del_tmp;
+ }
+ else
+ {
+ del_tmp->next = del_files;
+ del_files = del_tmp;
+ }
}
- while (need_rewind);
- svn_pool_destroy(subpool);
-
if (!APR_STATUS_IS_ENOENT(status))
return svn_error_wrap_apr(status, _("Can't read directory '%s'"),
svn_path_local_style(path, pool));
@@ -1997,6 +1958,31 @@
return svn_error_wrap_apr(status, _("Error closing directory '%s'"),
svn_path_local_style(path, pool));
+ for (; del_dirs; del_dirs = del_dirs->next)
+ {
+ /* Don't check for cancellation, the callee will immediately do so */
+ SVN_ERR(svn_io_remove_dir2(del_dirs->path, FALSE,
+ cancel_func, cancel_baton,
+ subpool));
+ }
+ for (; del_files; del_files = del_files->next)
+ {
+ svn_error_t *err;
+
+ if (cancel_func)
+ SVN_ERR((*cancel_func)(cancel_baton));
+
+ err = svn_io_remove_file2(del_files->path, FALSE, subpool);
+ if (err)
+ {
+ return svn_error_createf
+ (err->apr_err, err, _("Can't remove '%s'"),
+ svn_path_local_style(del_files->path, subpool));
+ }
+ }
+
+ svn_pool_destroy(subpool);
+
status = apr_dir_remove(path_apr, pool);
WIN32_RETRY_LOOP(status, apr_dir_remove(path_apr, pool));
if (status)
Issue 3501: Repositories mounted on NFS don't work
Posted by Edgar Fuß <ef...@math.uni-bonn.de>.
Hello,
may I humbly ask what the state of affairs rergarding Issue #3501 is?
For me, ths problem forces us to stay with Subversion 1.4.
Is the issue considered fixed, irrelevant, non-reproducable, a NetBSD problem,
a non-issue or not severe enough to be fixed?
To summarise the problem: recursive directory deletion fails because entries
re-appear after dir rewinding. The fix may be as easy as ignoring ENOENT.
Thanks.