You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@subversion.apache.org by st...@apache.org on 2010/11/28 01:52:57 UTC
svn commit: r1039808 - in /subversion/trunk/subversion: include/svn_wc.h
libsvn_client/prop_commands.c libsvn_wc/props.c libsvn_wc/wc-queries.sql
libsvn_wc/wc_db.c libsvn_wc/wc_db.h
Author: stsp
Date: Sun Nov 28 00:52:56 2010
New Revision: 1039808
URL: http://svn.apache.org/viewvc?rev=1039808&view=rev
Log:
Add a new wc-ng API for recursive property listing.
Make svn_client_propset3() use the new API for proplist with depth=infinity.
With this API the working copy library "crawls" nodes using DB queries,
without requiring callers to manually crawl the working copy with
svn_wc__node_walk_children() and call svn_wc_prop_list2() on each node.
A callback is invoked for each node found in the working copy that
has properties. Each callback invocation corresponds to one result
row of the queries issued and one sqlite step.
The API is not fully implemented yet -- the only depth currently supported
is 'infinity'. I hope to fix this soon. It will require tweaking the queries
a little (and my SQL is a bit rusty...).
gprof says this change speeds up "svn proplist -R svn-trunk-wc >/dev/null"
on my box from about 0.35s to about 0.11s. During profiled runs, the function
sqlite3Step() went from using 27% of the total time down to 15%. The time spent
within and below svn_client_proplist3() went down from 51% to 36%.
The profiler overhead accounted for about 31% of the total run time
before this change and about 45% after.
I hope this is going in the right direction -- I'm a bit detached from
current wc-ng developments so I cannot be sure if this commit is in
accordance with The Plan. If it is wrong for some reason just let me know :)
I'm currently planning to create a similar interface for getting pristine
props, and looking into making recursive propget use a similar approach.
* subversion/include/svn_wc.h
(svn_wc_prop_list_receiver_func_t): Declare new callback type.
(svn_wc_prop_list_recursive): Declare.
* subversion/libsvn_wc/props.c
(svn_wc_prop_list_recursive): New API described above.
* subversion/libsvn_wc/wc-queries.sql
(STMT_SELECT_NODE_PROPS_RECURSIVE,
STMT_SELECT_ACTUAL_PROPS_RECURSIVE): New queries to read properties
from the NODES and ACTUAL_NODE tables. The queries filter nodes that
svn_client_proplist3() isn't interested in (e.g. nodes that are locally
deleted). Each query also filters nodes selected by the other query,
so they can be run one after the other to produce a valid property listing.
* subversion/libsvn_wc/wc_db.c
(read_props_recursive_run_query): New. Issues a given query for properties
within a working copy subtree and invokes a receiver callback for
each node with properties.
(svn_wc__db_read_props_recursive): New. Low-level interface invoked by
svn_wc_prop_list_recursive().
* subversion/libsvn_wc/wc_db.h
(svn_wc__db_read_props_recursive): Declare.
* subversion/libsvn_client/prop_commands.c
(recursive_proplist_receiver_baton): New. Very similar to the existing
propget_walk_baton but kept separate in anticipation of the latter
baton's eventual removal.
(recursive_proplist_receiver): New. Similar to propget_walk_cb() but a lot
simpler because the working copy library does the bulk of the work now.
(svn_client_proplist3): Call svn_wc_prop_list_recursive() for the case it
already supports (depth=infinity).
Modified:
subversion/trunk/subversion/include/svn_wc.h
subversion/trunk/subversion/libsvn_client/prop_commands.c
subversion/trunk/subversion/libsvn_wc/props.c
subversion/trunk/subversion/libsvn_wc/wc-queries.sql
subversion/trunk/subversion/libsvn_wc/wc_db.c
subversion/trunk/subversion/libsvn_wc/wc_db.h
Modified: subversion/trunk/subversion/include/svn_wc.h
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/include/svn_wc.h?rev=1039808&r1=1039807&r2=1039808&view=diff
==============================================================================
--- subversion/trunk/subversion/include/svn_wc.h (original)
+++ subversion/trunk/subversion/include/svn_wc.h Sun Nov 28 00:52:56 2010
@@ -5538,6 +5538,35 @@ svn_wc_prop_list(apr_hash_t **props,
svn_wc_adm_access_t *adm_access,
apr_pool_t *pool);
+/** A callback invoked by svn_wc_prop_list_recursive() */
+typedef svn_error_t *(*svn_wc_prop_list_receiver_func_t)(
+ void *baton,
+ const char *local_abspath,
+ apr_hash_t *props,
+ apr_pool_t *scratch_pool);
+
+/** Call @a receiver_func, passing @a receiver_baton, an absolute path, and
+ * a hash table mapping <tt>char *</tt> names onto <tt>svn_string_t *</tt>
+ * values for all the regular properties of the node at @a local_abspath
+ * and any node beneath @a local_abspath within the specified @a depth.
+ *
+ * If a node has no properties, @a receiver_func is not called for the node.
+ *
+ * Use @a wc_ctx to access the working copy, and @a scratch_pool for
+ * temporary allocations.
+ *
+ * If the the node at @a local_abspath does not exist,
+ * #SVN_ERR_WC_PATH_NOT_FOUND is returned.
+ *
+ * @since New in 1.7.
+ */
+svn_error_t *
+svn_wc_prop_list_recursive(svn_wc_context_t *wc_ctx,
+ const char *local_abspath,
+ svn_depth_t depth,
+ svn_wc_prop_list_receiver_func_t receiver_func,
+ void *receiver_baton,
+ apr_pool_t *scratch_pool);
/** Return the set of "pristine" properties for @a local_abspath.
*
Modified: subversion/trunk/subversion/libsvn_client/prop_commands.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_client/prop_commands.c?rev=1039808&r1=1039807&r2=1039808&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_client/prop_commands.c (original)
+++ subversion/trunk/subversion/libsvn_client/prop_commands.c Sun Nov 28 00:52:56 2010
@@ -1219,6 +1219,45 @@ proplist_walk_cb(const char *local_abspa
scratch_pool);
}
+struct recursive_proplist_receiver_baton
+{
+ apr_hash_t *changelist_hash; /* Keys are changelists to filter on. */
+ svn_wc_context_t *wc_ctx; /* Working copy context. */
+ svn_proplist_receiver_t receiver; /* Proplist receiver to call. */
+ void *receiver_baton; /* Baton for the proplist receiver. */
+
+ /* Anchor, anchor_abspath pair for converting to relative paths */
+ const char *anchor;
+ const char *anchor_abspath;
+};
+
+static svn_error_t *
+recursive_proplist_receiver(void *baton,
+ const char *local_abspath,
+ apr_hash_t *props,
+ apr_pool_t *scratch_pool)
+{
+ struct recursive_proplist_receiver_baton *b = baton;
+ const char *path;
+
+ /* If the node doesn't pass changelist filtering, get outta here. */
+ if (! svn_wc__changelist_match(b->wc_ctx, local_abspath,
+ b->changelist_hash, scratch_pool))
+ return SVN_NO_ERROR;
+
+ if (b->anchor && b->anchor_abspath)
+ {
+ path = svn_dirent_join(b->anchor,
+ svn_dirent_skip_ancestor(b->anchor_abspath,
+ local_abspath),
+ scratch_pool);
+ }
+ else
+ path = local_abspath;
+
+ return call_receiver(path, props, b->receiver, b->receiver_baton,
+ scratch_pool);
+}
/* Note: this implementation is very similar to svn_client_propget3(). */
svn_error_t *
@@ -1280,6 +1319,36 @@ svn_client_proplist3(const char *path_or
{
struct proplist_walk_baton wb;
+ /* ### The new svn_wc_prop_list_recursive() interface currently
+ * ### only supports depth infinity.
+ * ### And there will be a separate API for getting pristine props. */
+ if (depth == svn_depth_infinity && ! pristine)
+ {
+ struct recursive_proplist_receiver_baton b;
+
+ b.wc_ctx = ctx->wc_ctx;
+ b.changelist_hash = changelist_hash;
+ b.receiver = receiver;
+ b.receiver_baton = receiver_baton;
+
+ if (strcmp(path_or_url, local_abspath) != 0)
+ {
+ b.anchor = path_or_url;
+ b.anchor_abspath = local_abspath;
+ }
+ else
+ {
+ b.anchor = NULL;
+ b.anchor_abspath = NULL;
+ }
+
+ return svn_error_return(svn_wc_prop_list_recursive(
+ ctx->wc_ctx, local_abspath,
+ svn_depth_infinity,
+ recursive_proplist_receiver,
+ &b, pool));
+ }
+
wb.wc_ctx = ctx->wc_ctx;
wb.pristine = pristine;
wb.changelist_hash = changelist_hash;
Modified: subversion/trunk/subversion/libsvn_wc/props.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/props.c?rev=1039808&r1=1039807&r2=1039808&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/props.c (original)
+++ subversion/trunk/subversion/libsvn_wc/props.c Sun Nov 28 00:52:56 2010
@@ -1668,6 +1668,22 @@ svn_wc_prop_list2(apr_hash_t **props,
scratch_pool));
}
+svn_error_t *
+svn_wc_prop_list_recursive(svn_wc_context_t *wc_ctx,
+ const char *local_abspath,
+ svn_depth_t depth,
+ svn_wc_prop_list_receiver_func_t receiver_func,
+ void *receiver_baton,
+ apr_pool_t *scratch_pool)
+{
+ SVN_ERR_ASSERT(depth == svn_depth_infinity); /* ### TODO other depths */
+
+ return svn_error_return(svn_wc__db_read_props_recursive(wc_ctx->db,
+ local_abspath,
+ receiver_func,
+ receiver_baton,
+ scratch_pool));
+}
svn_error_t *
svn_wc__get_pristine_props(apr_hash_t **props,
Modified: subversion/trunk/subversion/libsvn_wc/wc-queries.sql
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/wc-queries.sql?rev=1039808&r1=1039807&r2=1039808&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/wc-queries.sql (original)
+++ subversion/trunk/subversion/libsvn_wc/wc-queries.sql Sun Nov 28 00:52:56 2010
@@ -149,10 +149,30 @@ SELECT properties, presence FROM nodes
WHERE wc_id = ?1 AND local_relpath = ?2
ORDER BY op_depth DESC;
+-- STMT_SELECT_NODE_PROPS_RECURSIVE
+SELECT properties, local_relpath FROM nodes
+WHERE wc_id = ?1
+ AND local_relpath NOT IN (
+ SELECT actual_node.local_relpath FROM actual_node
+ WHERE actual_node.wc_id = ?1
+ )
+ AND (presence = 'normal' OR presence = 'incomplete')
+GROUP BY local_relpath
+ORDER BY op_depth DESC;
+
-- STMT_SELECT_ACTUAL_PROPS
SELECT properties FROM actual_node
WHERE wc_id = ?1 AND local_relpath = ?2;
+-- STMT_SELECT_ACTUAL_PROPS_RECURSIVE
+SELECT properties, local_relpath FROM actual_node
+WHERE wc_id = ?1
+ AND local_relpath NOT IN (
+ SELECT nodes.local_relpath FROM nodes
+ WHERE nodes.wc_id = ?1
+ AND nodes.presence != 'normal' AND nodes.presence != 'incomplete'
+ );
+
-- STMT_UPDATE_NODE_BASE_PROPS
UPDATE nodes SET properties = ?3
WHERE wc_id = ?1 AND local_relpath = ?2 AND op_depth = 0;
Modified: subversion/trunk/subversion/libsvn_wc/wc_db.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/wc_db.c?rev=1039808&r1=1039807&r2=1039808&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/wc_db.c (original)
+++ subversion/trunk/subversion/libsvn_wc/wc_db.c Sun Nov 28 00:52:56 2010
@@ -5587,6 +5587,81 @@ svn_wc__db_read_props(apr_hash_t **props
return SVN_NO_ERROR;
}
+static svn_error_t *
+read_props_recursive_run_query(int stmt_idx,
+ svn_wc__db_pdh_t *pdh,
+ svn_wc_prop_list_receiver_func_t receiver_func,
+ void *receiver_baton,
+ apr_pool_t *scratch_pool)
+{
+ svn_sqlite__stmt_t *stmt;
+ svn_boolean_t have_row;
+
+ SVN_ERR(svn_sqlite__get_statement(&stmt, pdh->wcroot->sdb, stmt_idx));
+ SVN_ERR(svn_sqlite__bindf(stmt, "i", pdh->wcroot->wc_id));
+ SVN_ERR(svn_sqlite__step(&have_row, stmt));
+ while (have_row && !svn_sqlite__column_is_null(stmt, 0))
+ {
+ const char *local_relpath;
+ const char *local_abspath;
+ const char *prop_data;
+ svn_skel_t *prop_skel;
+ apr_size_t len;
+
+ prop_data = svn_sqlite__column_blob(stmt, 0, &len, NULL);
+ if (prop_data)
+ {
+ local_relpath = svn_sqlite__column_text(stmt, 1, scratch_pool);
+ SVN_ERR_ASSERT(local_relpath);
+
+ local_abspath = svn_dirent_join(pdh->wcroot->abspath,
+ local_relpath,
+ scratch_pool);
+ prop_skel = svn_skel__parse(prop_data, len, scratch_pool);
+ if (svn_skel__list_length(prop_skel) > 0)
+ {
+ apr_hash_t *props;
+
+ SVN_ERR(svn_skel__parse_proplist(&props, prop_skel,
+ scratch_pool));
+ if (apr_hash_count(props) > 0 && receiver_func)
+ (*receiver_func)(receiver_baton, local_abspath, props,
+ scratch_pool);
+ }
+ }
+
+ SVN_ERR(svn_sqlite__step(&have_row, stmt));
+ }
+
+ return svn_error_return(svn_sqlite__reset(stmt));
+}
+
+
+svn_error_t *
+svn_wc__db_read_props_recursive(svn_wc__db_t *db,
+ const char *local_abspath,
+ svn_wc_prop_list_receiver_func_t receiver_func,
+ void *receiver_baton,
+ apr_pool_t *scratch_pool)
+{
+ svn_wc__db_pdh_t *pdh;
+ const char *local_relpath;
+
+ SVN_ERR_ASSERT(svn_dirent_is_absolute(local_abspath));
+ SVN_ERR(svn_wc__db_pdh_parse_local_abspath(&pdh, &local_relpath, db,
+ local_abspath, svn_sqlite__mode_readwrite,
+ scratch_pool, scratch_pool));
+ VERIFY_USABLE_PDH(pdh);
+ SVN_ERR(read_props_recursive_run_query(STMT_SELECT_ACTUAL_PROPS_RECURSIVE,
+ pdh, receiver_func, receiver_baton,
+ scratch_pool));
+ SVN_ERR(read_props_recursive_run_query(STMT_SELECT_NODE_PROPS_RECURSIVE,
+ pdh, receiver_func, receiver_baton,
+ scratch_pool));
+
+ return SVN_NO_ERROR;
+}
+
static svn_error_t *
db_read_pristine_props(apr_hash_t **props,
Modified: subversion/trunk/subversion/libsvn_wc/wc_db.h
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/wc_db.h?rev=1039808&r1=1039807&r2=1039808&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/wc_db.h (original)
+++ subversion/trunk/subversion/libsvn_wc/wc_db.h Sun Nov 28 00:52:56 2010
@@ -1567,6 +1567,19 @@ svn_wc__db_read_props(apr_hash_t **props
apr_pool_t *result_pool,
apr_pool_t *scratch_pool);
+/* Call RECEIVER_FUNC, passing RECEIVER_BATON, an absolute path, and
+ * a hash table mapping <tt>char *</tt> names onto svn_string_t *
+ * values for all the regular properties of the node at LOCAL_ABSPATH
+ * and any node beneath LOCAL_ABSPATH.
+ *
+ * ### TODO: Add and implement DEPTH parameter.
+ */
+svn_error_t *
+svn_wc__db_read_props_recursive(svn_wc__db_t *db,
+ const char *local_abspath,
+ svn_wc_prop_list_receiver_func_t receiver_func,
+ void *receiver_baton,
+ apr_pool_t *scratch_pool);
/* Set *PROPS to the properties of the node LOCAL_ABSPATH in the WORKING
tree (looking through to the BASE tree as required).