You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@subversion.apache.org by st...@apache.org on 2011/11/01 19:26:55 UTC
svn commit: r1196191 - in /subversion/trunk/subversion/libsvn_wc:
wc-queries.sql wc_db.c
Author: stsp
Date: Tue Nov 1 18:26:54 2011
New Revision: 1196191
URL: http://svn.apache.org/viewvc?rev=1196191&view=rev
Log:
During 'svn rm', sort the delete-list in C, rather than in SQL.
This speeds up 'svn rm dir/*'.
Also fix notification in case of cancellation. We were omitting notification
for some nodes in the delete-list if the user cancelled early. This is wrong
because all nodes in the delete-list are already deleted at this point.
* subversion/libsvn_wc/wc-queries.sql
(STMT_SELECT_DELETE_LIST): Don't ORDER BY.
* subversion/libsvn_wc/wc_db.c
(do_delete_notify): Sort the delete-list in C, if it is not already sorted.
Notify for all paths in the delete-list before allowing cancellation.
Modified:
subversion/trunk/subversion/libsvn_wc/wc-queries.sql
subversion/trunk/subversion/libsvn_wc/wc_db.c
Modified: subversion/trunk/subversion/libsvn_wc/wc-queries.sql
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/wc-queries.sql?rev=1196191&r1=1196190&r2=1196191&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/wc-queries.sql (original)
+++ subversion/trunk/subversion/libsvn_wc/wc-queries.sql Tue Nov 1 18:26:54 2011
@@ -1208,7 +1208,6 @@ WHERE wc_id = ?1
-- STMT_SELECT_DELETE_LIST
SELECT local_relpath FROM delete_list
-ORDER BY local_relpath
-- STMT_FINALIZE_DELETE
DROP TABLE IF EXISTS delete_list
Modified: subversion/trunk/subversion/libsvn_wc/wc_db.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/wc_db.c?rev=1196191&r1=1196190&r2=1196191&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/wc_db.c (original)
+++ subversion/trunk/subversion/libsvn_wc/wc_db.c Tue Nov 1 18:26:54 2011
@@ -32,6 +32,7 @@
#include "svn_dirent_uri.h"
#include "svn_path.h"
#include "svn_hash.h"
+#include "svn_sorts.h"
#include "svn_wc.h"
#include "svn_checksum.h"
#include "svn_pools.h"
@@ -6559,38 +6560,71 @@ do_delete_notify(void *baton,
svn_sqlite__stmt_t *stmt;
svn_boolean_t have_row;
apr_pool_t *iterpool;
+ apr_array_header_t *delete_list;
+ svn_boolean_t sorted;
+ int i;
SVN_ERR(svn_sqlite__get_statement(&stmt, wcroot->sdb,
STMT_SELECT_DELETE_LIST));
SVN_ERR(svn_sqlite__step(&have_row, stmt));
- iterpool = svn_pool_create(scratch_pool);
+ /* Build a sorted list of deleted nodes. STMT_SELECT_DELETE_LIST
+ * doesn't sort because sorting in SQL is more expensive than in C. */
+ delete_list = apr_array_make(scratch_pool, 1, sizeof(const char *));
+ sorted = TRUE;
+ i = 0;
while (have_row)
{
const char *notify_relpath;
const char *notify_abspath;
-
- svn_pool_clear(iterpool);
-
- if (cancel_func)
- SVN_ERR(cancel_func(cancel_baton));
+ const char *prev_abspath;
notify_relpath = svn_sqlite__column_text(stmt, 0, NULL);
notify_abspath = svn_dirent_join(wcroot->abspath,
notify_relpath,
- iterpool);
+ scratch_pool);
+ if (i == 0)
+ {
+ APR_ARRAY_PUSH(delete_list, const char *) = notify_abspath;
+ i++;
+ SVN_ERR(svn_sqlite__step(&have_row, stmt));
+ continue;
+ }
+ prev_abspath = APR_ARRAY_IDX(delete_list, i - 1, const char *);
+ if (sorted)
+ sorted = (svn_path_compare_paths(prev_abspath, notify_abspath) <= 0);
+ APR_ARRAY_PUSH(delete_list, const char *) = notify_abspath;
+ i++;
+ SVN_ERR(svn_sqlite__step(&have_row, stmt));
+ }
+
+ SVN_ERR(svn_sqlite__reset(stmt));
+
+ if (!sorted)
+ qsort(delete_list->elts, delete_list->nelts, delete_list->elt_size,
+ svn_sort_compare_paths);
+
+ iterpool = svn_pool_create(scratch_pool);
+ for (i = 0; i < delete_list->nelts; i++)
+ {
+ svn_pool_clear(iterpool);
notify_func(notify_baton,
- svn_wc_create_notify(notify_abspath,
+ svn_wc_create_notify(APR_ARRAY_IDX(delete_list, i,
+ const char *),
svn_wc_notify_delete,
iterpool),
iterpool);
-
- SVN_ERR(svn_sqlite__step(&have_row, stmt));
}
svn_pool_destroy(iterpool);
- return svn_error_trace(svn_sqlite__reset(stmt));
+ /* We only allow cancellation after notification for all deleted nodes
+ * has happened. The nodes are already deleted so we should notify for
+ * all of them. */
+ if (cancel_func)
+ SVN_ERR(cancel_func(cancel_baton));
+
+ return SVN_NO_ERROR;
}
Re: svn commit: r1196191 - in /subversion/trunk/subversion/libsvn_wc:
wc-queries.sql wc_db.c
Posted by Hyrum K Wright <hy...@wandisco.com>.
On Tue, Nov 1, 2011 at 1:53 PM, Philip Martin <ph...@wandisco.com>wrote:
> Stefan Sperling <st...@elego.de> writes:
>
> > On Tue, Nov 01, 2011 at 06:36:26PM +0000, Philip Martin wrote:
> >> Is sorting in C better than putting UNIQUE on the local_relpath column?
> >
> > I don't know. I don't mind either way. Let's pick whichever is faster.
>
> I'd expect performance to be similar. I have a small preference for
> UNIQUE because it leaves sorting and memory allocation to the database.
Same here, assuming the performance is similar.
-Hyrum
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com/
Re: svn commit: r1196191 - in /subversion/trunk/subversion/libsvn_wc: wc-queries.sql wc_db.c
Posted by Philip Martin <ph...@wandisco.com>.
Stefan Sperling <st...@elego.de> writes:
> On Tue, Nov 01, 2011 at 06:36:26PM +0000, Philip Martin wrote:
>> Is sorting in C better than putting UNIQUE on the local_relpath column?
>
> I don't know. I don't mind either way. Let's pick whichever is faster.
I'd expect performance to be similar. I have a small preference for
UNIQUE because it leaves sorting and memory allocation to the database.
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com
Re: svn commit: r1196191 - in
/subversion/trunk/subversion/libsvn_wc: wc-queries.sql wc_db.c
Posted by Stefan Sperling <st...@elego.de>.
On Wed, Nov 02, 2011 at 01:41:29PM +0000, Philip Martin wrote:
> I've been timing the performance on NFS working copies and I can see no
> significant speed difference between using an SQLite index and sorting
> in C. So I propose to switch to the index as I believe letting the DB
> do the work is better.
I agree.
Re: svn commit: r1196191 - in /subversion/trunk/subversion/libsvn_wc: wc-queries.sql wc_db.c
Posted by Philip Martin <ph...@wandisco.com>.
Stefan Sperling <st...@elego.de> writes:
> On Wed, Nov 02, 2011 at 12:59:31PM +0000, Philip Martin wrote:
>> Stefan Sperling <st...@elego.de> writes:
>>
>> > On Tue, Nov 01, 2011 at 06:36:26PM +0000, Philip Martin wrote:
>> >> Is sorting in C better than putting UNIQUE on the local_relpath column?
>> >
>> > I don't know. I don't mind either way. Let's pick whichever is faster.
>> >
>> > The notification/cancellation fix is needed in either case.
>>
>> I think the cacellation change is wrong. This is happening inside an
>> SQLite transaction. When the user cancels a delete operation during
>> notification the cancel callback will return an svn_error_t causing the
>> transaction to abort. That means that no change will have been made to
>> the database. I don't see how postponing that abort until all the
>> notifications have been made is an improvement.
>
> The transaction is already done when notification happens.
> See wc_db.c:with_finalization() -- notification is performed by
> the work_cb() callback, not the txn_cb() callback.
OK.
I've been timing the performance on NFS working copies and I can see no
significant speed difference between using an SQLite index and sorting
in C. So I propose to switch to the index as I believe letting the DB
do the work is better.
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com
Re: svn commit: r1196191 - in
/subversion/trunk/subversion/libsvn_wc: wc-queries.sql wc_db.c
Posted by Stefan Sperling <st...@elego.de>.
On Wed, Nov 02, 2011 at 12:59:31PM +0000, Philip Martin wrote:
> Stefan Sperling <st...@elego.de> writes:
>
> > On Tue, Nov 01, 2011 at 06:36:26PM +0000, Philip Martin wrote:
> >> Is sorting in C better than putting UNIQUE on the local_relpath column?
> >
> > I don't know. I don't mind either way. Let's pick whichever is faster.
> >
> > The notification/cancellation fix is needed in either case.
>
> I think the cacellation change is wrong. This is happening inside an
> SQLite transaction. When the user cancels a delete operation during
> notification the cancel callback will return an svn_error_t causing the
> transaction to abort. That means that no change will have been made to
> the database. I don't see how postponing that abort until all the
> notifications have been made is an improvement.
The transaction is already done when notification happens.
See wc_db.c:with_finalization() -- notification is performed by
the work_cb() callback, not the txn_cb() callback.
Re: svn commit: r1196191 - in /subversion/trunk/subversion/libsvn_wc: wc-queries.sql wc_db.c
Posted by Philip Martin <ph...@wandisco.com>.
Stefan Sperling <st...@elego.de> writes:
> On Tue, Nov 01, 2011 at 06:36:26PM +0000, Philip Martin wrote:
>> Is sorting in C better than putting UNIQUE on the local_relpath column?
>
> I don't know. I don't mind either way. Let's pick whichever is faster.
>
> The notification/cancellation fix is needed in either case.
I think the cacellation change is wrong. This is happening inside an
SQLite transaction. When the user cancels a delete operation during
notification the cancel callback will return an svn_error_t causing the
transaction to abort. That means that no change will have been made to
the database. I don't see how postponing that abort until all the
notifications have been made is an improvement.
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com
Re: svn commit: r1196191 - in
/subversion/trunk/subversion/libsvn_wc: wc-queries.sql wc_db.c
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Nov 01, 2011 at 06:36:26PM +0000, Philip Martin wrote:
> Is sorting in C better than putting UNIQUE on the local_relpath column?
I don't know. I don't mind either way. Let's pick whichever is faster.
The notification/cancellation fix is needed in either case.
Re: svn commit: r1196191 - in /subversion/trunk/subversion/libsvn_wc: wc-queries.sql wc_db.c
Posted by Philip Martin <ph...@wandisco.com>.
stsp@apache.org writes:
> Author: stsp
> Date: Tue Nov 1 18:26:54 2011
> New Revision: 1196191
>
> URL: http://svn.apache.org/viewvc?rev=1196191&view=rev
> Log:
> During 'svn rm', sort the delete-list in C, rather than in SQL.
> This speeds up 'svn rm dir/*'.
>
> Also fix notification in case of cancellation. We were omitting notification
> for some nodes in the delete-list if the user cancelled early. This is wrong
> because all nodes in the delete-list are already deleted at this point.
>
> * subversion/libsvn_wc/wc-queries.sql
> (STMT_SELECT_DELETE_LIST): Don't ORDER BY.
>
> * subversion/libsvn_wc/wc_db.c
> (do_delete_notify): Sort the delete-list in C, if it is not already sorted.
> Notify for all paths in the delete-list before allowing cancellation.
Is sorting in C better than putting UNIQUE on the local_relpath column?
--
Philip