You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@subversion.apache.org by st...@apache.org on 2015/08/06 14:54:36 UTC
svn commit: r1694489 [1/2] - in /subversion/trunk: ./
subversion/include/private/ subversion/libsvn_fs_fs/
subversion/libsvn_fs_x/ subversion/libsvn_subr/ subversion/tests/libsvn_subr/
Author: stefan2
Date: Thu Aug 6 12:54:36 2015
New Revision: 1694489
URL: http://svn.apache.org/r1694489
Log:
Merge the 1.10-cache-improvements development branch to trunk and remove
the BRANCH-README file. There were no conflicts.
Changes brought by this branch:
- Introduce a NULL cache implementation to unify / simplify cache usage
- Space and runtime efficiency improvements to the recently introduced
full key comparison in membuffer caches.
Added:
subversion/trunk/subversion/libsvn_subr/cache-null.c
- copied unchanged from r1694487, subversion/branches/1.10-cache-improvements/subversion/libsvn_subr/cache-null.c
Modified:
subversion/trunk/ (props changed)
subversion/trunk/subversion/include/private/svn_cache.h
subversion/trunk/subversion/libsvn_fs_fs/caching.c
subversion/trunk/subversion/libsvn_fs_x/ (props changed)
subversion/trunk/subversion/libsvn_fs_x/cached_data.c
subversion/trunk/subversion/libsvn_fs_x/caching.c
subversion/trunk/subversion/libsvn_fs_x/tree.c
subversion/trunk/subversion/libsvn_subr/cache-membuffer.c
subversion/trunk/subversion/tests/libsvn_subr/cache-test.c
Propchange: subversion/trunk/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Thu Aug 6 12:54:36 2015
@@ -1,4 +1,4 @@
-/subversion/branches/1.10-cache-improvements:1675666-1677522,1679679,1679681,1679859
+/subversion/branches/1.10-cache-improvements:1669168-1694487
/subversion/branches/1.5.x-r30215:870312
/subversion/branches/1.7.x-fs-verify:1146708,1161180
/subversion/branches/1.9-cache-improvements:1678948-1679863
Modified: subversion/trunk/subversion/include/private/svn_cache.h
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/include/private/svn_cache.h?rev=1694489&r1=1694488&r2=1694489&view=diff
==============================================================================
--- subversion/trunk/subversion/include/private/svn_cache.h (original)
+++ subversion/trunk/subversion/include/private/svn_cache.h Thu Aug 6 12:54:36 2015
@@ -356,7 +356,12 @@ svn_cache__membuffer_cache_create(svn_me
*
* If @a thread_safe is true, and APR is compiled with threads, all
* accesses to the cache will be protected with a mutex, if the shared
- * @a memcache has also been created with thread_safe flag set.
+ * @a membuffer has also been created with thread_safe flag set.
+ *
+ * If @a short_lived is set, assume that the data stored through this
+ * cache will probably only be needed for a short period of time.
+ * Typically, some UUID is used as part of the prefix in that scenario.
+ * This flag is a mere hint and does not affect functionality.
*
* These caches do not support svn_cache__iter.
*/
@@ -369,10 +374,24 @@ svn_cache__create_membuffer_cache(svn_ca
const char *prefix,
apr_uint32_t priority,
svn_boolean_t thread_safe,
+ svn_boolean_t short_lived,
apr_pool_t *result_pool,
apr_pool_t *scratch_pool);
/**
+ * Creates a null-cache instance in @a *cache_p, allocated from
+ * @a result_pool. The given @c id is the only data stored in it and can
+ * be retrieved through svn_cache__get_info().
+ *
+ * The cache object will immediately evict (reject) any data being added
+ * to it and will always report as empty.
+ */
+svn_error_t *
+svn_cache__create_null(svn_cache__t **cache_p,
+ const char *id,
+ apr_pool_t *result_pool);
+
+/**
* Sets @a handler to be @a cache's error handling routine. If any
* error is returned from a call to svn_cache__get or svn_cache__set, @a
* handler will be called with @a baton and the error, and the
Modified: subversion/trunk/subversion/libsvn_fs_fs/caching.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_fs_fs/caching.c?rev=1694489&r1=1694488&r2=1694489&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_fs_fs/caching.c (original)
+++ subversion/trunk/subversion/libsvn_fs_fs/caching.c Thu Aug 6 12:54:36 2015
@@ -274,7 +274,8 @@ init_callbacks(svn_cache__t *cache,
* MEMBUFFER is not NULL. Fallbacks to inprocess cache if MEMCACHE and
* MEMBUFFER are NULL and pages is non-zero. Sets *CACHE_P to NULL
* otherwise. Use the given PRIORITY class for the new cache. If it
- * is 0, then use the default priority class.
+ * is 0, then use the default priority class. HAS_NAMESPACE indicates
+ * whether we prefixed this cache instance with a namespace.
*
* Unless NO_HANDLER is true, register an error handler that reports errors
* as warnings to the FS warning callback.
@@ -292,6 +293,7 @@ create_cache(svn_cache__t **cache_p,
apr_ssize_t klen,
const char *prefix,
apr_uint32_t priority,
+ svn_boolean_t has_namespace,
svn_fs_t *fs,
svn_boolean_t no_handler,
apr_pool_t *result_pool,
@@ -314,9 +316,12 @@ create_cache(svn_cache__t **cache_p,
}
else if (membuffer)
{
+ /* We assume caches with namespaces to be relatively short-lived,
+ * i.e. their data will not be needed after a while. */
SVN_ERR(svn_cache__create_membuffer_cache(
cache_p, membuffer, serializer, deserializer,
- klen, prefix, priority, FALSE, result_pool, scratch_pool));
+ klen, prefix, priority, FALSE, has_namespace,
+ result_pool, scratch_pool));
}
else if (pages)
{
@@ -349,6 +354,7 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
svn_boolean_t cache_txdeltas;
svn_boolean_t cache_fulltexts;
const char *cache_namespace;
+ svn_boolean_t has_namespace;
/* Evaluating the cache configuration. */
SVN_ERR(read_config(&cache_namespace,
@@ -358,6 +364,7 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
pool));
prefix = apr_pstrcat(pool, "ns:", cache_namespace, ":", prefix, SVN_VA_NULL);
+ has_namespace = strlen(cache_namespace) > 0;
membuffer = svn_cache__get_global_membuffer_cache();
@@ -386,34 +393,35 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
* commands, this is only going to contain a few entries (svnadmin
* dump/verify is an exception here), so to reduce overhead let's
* try to keep it to just one page. I estimate each entry has about
- * 72 bytes of overhead (svn_revnum_t key, svn_fs_id_t +
- * id_private_t + 3 strings for value, and the cache_entry); the
- * default pool size is 8192, so about a hundred should fit
- * comfortably. */
+ * 130 bytes of overhead (svn_revnum_t key, ID struct, and the cache_entry);
+ * the default pool size is 8192, so about a fifty should fit comfortably.
+ */
SVN_ERR(create_cache(&(ffd->rev_root_id_cache),
NULL,
membuffer,
- 1, 100,
+ 1, 50,
svn_fs_fs__serialize_id,
svn_fs_fs__deserialize_id,
sizeof(svn_revnum_t),
apr_pstrcat(pool, prefix, "RRI", SVN_VA_NULL),
0,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
- /* Rough estimate: revision DAG nodes have size around 320 bytes, so
- * let's put 16 on a page. */
+ /* Rough estimate: revision DAG nodes have size around 1kBytes, so
+ * let's put 8 on a page. */
SVN_ERR(create_cache(&(ffd->rev_node_cache),
NULL,
membuffer,
- 1024, 16,
+ 1, 8,
svn_fs_fs__dag_serialize,
svn_fs_fs__dag_deserialize,
APR_HASH_KEY_STRING,
apr_pstrcat(pool, prefix, "DAG", SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_LOW_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -425,28 +433,30 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
SVN_ERR(create_cache(&(ffd->dir_cache),
NULL,
membuffer,
- 1024, 8,
+ 1, 8,
svn_fs_fs__serialize_dir_entries,
svn_fs_fs__deserialize_dir_entries,
sizeof(pair_cache_key_t),
apr_pstrcat(pool, prefix, "DIR", SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
- /* Only 16 bytes per entry (a revision number + the corresponding offset).
- Since we want ~8k pages, that means 512 entries per page. */
+ /* 8 kBytes per entry (1000 revs / shared, one file offset per rev).
+ Covering about 8 pack files gives us an "o.k." hit rate. */
SVN_ERR(create_cache(&(ffd->packed_offset_cache),
NULL,
membuffer,
- 32, 1,
+ 8, 1,
svn_fs_fs__serialize_manifest,
svn_fs_fs__deserialize_manifest,
sizeof(svn_revnum_t),
apr_pstrcat(pool, prefix, "PACK-MANIFEST",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -455,12 +465,13 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
SVN_ERR(create_cache(&(ffd->node_revision_cache),
NULL,
membuffer,
- 32, 32, /* ~200 byte / entry; 1k entries total */
+ 2, 16, /* ~500 byte / entry; 32 entries total */
svn_fs_fs__serialize_node_revision,
svn_fs_fs__deserialize_node_revision,
sizeof(pair_cache_key_t),
apr_pstrcat(pool, prefix, "NODEREVS", SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -469,12 +480,13 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
SVN_ERR(create_cache(&(ffd->rep_header_cache),
NULL,
membuffer,
- 1, 1000, /* ~8 bytes / entry; 1k entries total */
+ 1, 200, /* ~40 bytes / entry; 200 entries total */
svn_fs_fs__serialize_rep_header,
svn_fs_fs__deserialize_rep_header,
sizeof(pair_cache_key_t),
apr_pstrcat(pool, prefix, "REPHEADER", SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -489,6 +501,7 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
sizeof(svn_revnum_t),
apr_pstrcat(pool, prefix, "CHANGES", SVN_VA_NULL),
0,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -499,12 +512,13 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
SVN_ERR(create_cache(&(ffd->fulltext_cache),
ffd->memcache,
membuffer,
- 0, 0, /* Do not use inprocess cache */
+ 0, 0, /* Do not use the inprocess cache */
/* Values are svn_stringbuf_t */
NULL, NULL,
sizeof(pair_cache_key_t),
apr_pstrcat(pool, prefix, "TEXT", SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -512,13 +526,14 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
SVN_ERR(create_cache(&(ffd->properties_cache),
NULL,
membuffer,
- 0, 0, /* Do not use inprocess cache */
+ 0, 0, /* Do not use the inprocess cache */
svn_fs_fs__serialize_properties,
svn_fs_fs__deserialize_properties,
sizeof(pair_cache_key_t),
apr_pstrcat(pool, prefix, "PROP",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -526,13 +541,14 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
SVN_ERR(create_cache(&(ffd->mergeinfo_cache),
NULL,
membuffer,
- 0, 0, /* Do not use inprocess cache */
+ 0, 0, /* Do not use the inprocess cache */
svn_fs_fs__serialize_mergeinfo,
svn_fs_fs__deserialize_mergeinfo,
APR_HASH_KEY_STRING,
apr_pstrcat(pool, prefix, "MERGEINFO",
SVN_VA_NULL),
0,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -540,13 +556,14 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
SVN_ERR(create_cache(&(ffd->mergeinfo_existence_cache),
NULL,
membuffer,
- 0, 0, /* Do not use inprocess cache */
+ 0, 0, /* Do not use the inprocess cache */
/* Values are svn_stringbuf_t */
NULL, NULL,
APR_HASH_KEY_STRING,
apr_pstrcat(pool, prefix, "HAS_MERGEINFO",
SVN_VA_NULL),
0,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -565,13 +582,14 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
SVN_ERR(create_cache(&(ffd->raw_window_cache),
NULL,
membuffer,
- 0, 0, /* Do not use inprocess cache */
+ 0, 0, /* Do not use the inprocess cache */
svn_fs_fs__serialize_raw_window,
svn_fs_fs__deserialize_raw_window,
sizeof(window_cache_key_t),
apr_pstrcat(pool, prefix, "RAW_WINDOW",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_LOW_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -579,13 +597,14 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
SVN_ERR(create_cache(&(ffd->txdelta_window_cache),
NULL,
membuffer,
- 0, 0, /* Do not use inprocess cache */
+ 0, 0, /* Do not use the inprocess cache */
svn_fs_fs__serialize_txdelta_window,
svn_fs_fs__deserialize_txdelta_window,
sizeof(window_cache_key_t),
apr_pstrcat(pool, prefix, "TXDELTA_WINDOW",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_LOW_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -593,13 +612,14 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
SVN_ERR(create_cache(&(ffd->combined_window_cache),
NULL,
membuffer,
- 0, 0, /* Do not use inprocess cache */
+ 0, 0, /* Do not use the inprocess cache */
/* Values are svn_stringbuf_t */
NULL, NULL,
sizeof(window_cache_key_t),
apr_pstrcat(pool, prefix, "COMBINED_WINDOW",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_LOW_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -613,28 +633,34 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
SVN_ERR(create_cache(&(ffd->l2p_header_cache),
NULL,
membuffer,
- 64, 16, /* entry size varies but we must cover
- a reasonable number of revisions (1k) */
+ 8, 16, /* entry size varies but we must cover a
+ reasonable number of rev / pack files
+ to allow for delta chains to be walked
+ efficiently etc. */
svn_fs_fs__serialize_l2p_header,
svn_fs_fs__deserialize_l2p_header,
sizeof(pair_cache_key_t),
apr_pstrcat(pool, prefix, "L2P_HEADER",
(char *)NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
SVN_ERR(create_cache(&(ffd->l2p_page_cache),
NULL,
membuffer,
- 64, 16, /* entry size varies but we must cover
- a reasonable number of revisions (1k) */
+ 8, 16, /* entry size varies but we must cover a
+ reasonable number of rev / pack files
+ to allow for delta chains to be walked
+ efficiently etc. */
svn_fs_fs__serialize_l2p_page,
svn_fs_fs__deserialize_l2p_page,
sizeof(svn_fs_fs__page_cache_key_t),
apr_pstrcat(pool, prefix, "L2P_PAGE",
(char *)NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -648,19 +674,21 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
apr_pstrcat(pool, prefix, "P2L_HEADER",
(char *)NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
SVN_ERR(create_cache(&(ffd->p2l_page_cache),
NULL,
membuffer,
- 4, 16, /* Variably sized entries. Rarely used. */
+ 4, 1, /* Variably sized entries. Rarely used. */
svn_fs_fs__serialize_p2l_page,
svn_fs_fs__deserialize_p2l_page,
sizeof(svn_fs_fs__page_cache_key_t),
apr_pstrcat(pool, prefix, "P2L_PAGE",
(char *)NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
no_handler,
fs->pool, pool));
@@ -807,6 +835,7 @@ svn_fs_fs__initialize_txn_caches(svn_fs_
apr_pstrcat(pool, prefix, "TXNDIR",
SVN_VA_NULL),
0,
+ TRUE, /* The TXN-ID is our namespace. */
fs,
TRUE,
pool, pool));
Propchange: subversion/trunk/subversion/libsvn_fs_x/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Thu Aug 6 12:54:36 2015
@@ -1,3 +1,4 @@
+/subversion/branches/1.10-cache-improvements/subversion/libsvn_fs_x:1669168-1694487
/subversion/branches/1.5.x-r30215/subversion/libsvn_fs_x:870312
/subversion/branches/1.7.x-fs-verify/subversion/libsvn_fs_x:1146708,1161180
/subversion/branches/10Gb/subversion/libsvn_fs_x:1388102,1388163-1388190,1388195,1388202,1388205,1388211,1388276,1388362,1388375,1388394,1388636,1388639-1388640,1388643-1388644,1388654,1388720,1388789,1388795,1388801,1388805,1388807,1388810,1388816,1389044,1389276,1389289,1389662,1389867,1390017,1390209,1390216,1390407,1390409,1390414,1390419,1390955
Modified: subversion/trunk/subversion/libsvn_fs_x/cached_data.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_fs_x/cached_data.c?rev=1694489&r1=1694488&r2=1694489&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_fs_x/cached_data.c (original)
+++ subversion/trunk/subversion/libsvn_fs_x/cached_data.c Thu Aug 6 12:54:36 2015
@@ -355,16 +355,13 @@ get_node_revision_body(svn_fs_x__noderev
/* Not found or not applicable. Try a noderev cache lookup.
* If that succeeds, we are done here. */
- if (ffd->node_revision_cache)
- {
- SVN_ERR(svn_cache__get((void **) noderev_p,
- &is_cached,
- ffd->node_revision_cache,
- &key,
- result_pool));
- if (is_cached)
- return SVN_NO_ERROR;
- }
+ SVN_ERR(svn_cache__get((void **) noderev_p,
+ &is_cached,
+ ffd->node_revision_cache,
+ &key,
+ result_pool));
+ if (is_cached)
+ return SVN_NO_ERROR;
/* block-read will parse the whole block and will also return
the one noderev that we need right now. */
@@ -655,7 +652,7 @@ create_rep_state_body(rep_state_t **rep_
: NULL;
/* cache lookup, i.e. skip reading the rep header if possible */
- if (ffd->rep_header_cache && SVN_IS_VALID_REVNUM(revision))
+ if (SVN_IS_VALID_REVNUM(revision))
SVN_ERR(svn_cache__get((void **) &rh, &is_cached,
ffd->rep_header_cache, &key, result_pool));
@@ -743,9 +740,8 @@ create_rep_state_body(rep_state_t **rep_
{
SVN_ERR(block_read(NULL, fs, &rs->rep_id, rs->sfile->rfile,
result_pool, scratch_pool));
- if (ffd->rep_header_cache)
- SVN_ERR(svn_cache__set(ffd->rep_header_cache, &key, rh,
- scratch_pool));
+ SVN_ERR(svn_cache__set(ffd->rep_header_cache, &key, rh,
+ scratch_pool));
}
}
@@ -1075,22 +1071,14 @@ get_cached_window_sizes(window_sizes_t *
svn_boolean_t *is_cached,
apr_pool_t *pool)
{
- if (! rs->window_cache)
- {
- /* txdelta window has not been enabled */
- *is_cached = FALSE;
- }
- else
- {
- svn_fs_x__window_cache_key_t key = { 0 };
- SVN_ERR(svn_cache__get_partial((void **)sizes,
- is_cached,
- rs->window_cache,
- get_window_key(&key, rs),
- get_cached_window_sizes_func,
- NULL,
- pool));
- }
+ svn_fs_x__window_cache_key_t key = { 0 };
+ SVN_ERR(svn_cache__get_partial((void **)sizes,
+ is_cached,
+ rs->window_cache,
+ get_window_key(&key, rs),
+ get_cached_window_sizes_func,
+ NULL,
+ pool));
return SVN_NO_ERROR;
}
@@ -1103,33 +1091,25 @@ get_cached_window(svn_txdelta_window_t *
apr_pool_t *result_pool,
apr_pool_t *scratch_pool)
{
- if (! rs->window_cache)
- {
- /* txdelta window has not been enabled */
- *is_cached = FALSE;
- }
- else
- {
- /* ask the cache for the desired txdelta window */
- svn_fs_x__txdelta_cached_window_t *cached_window;
- svn_fs_x__window_cache_key_t key = { 0 };
- get_window_key(&key, rs);
- key.chunk_index = chunk_index;
- SVN_ERR(svn_cache__get((void **) &cached_window,
- is_cached,
- rs->window_cache,
- &key,
- result_pool));
-
- if (*is_cached)
- {
- /* found it. Pass it back to the caller. */
- *window_p = cached_window->window;
-
- /* manipulate the RS as if we just read the data */
- rs->current = cached_window->end_offset;
- rs->chunk_index = chunk_index;
- }
+ /* ask the cache for the desired txdelta window */
+ svn_fs_x__txdelta_cached_window_t *cached_window;
+ svn_fs_x__window_cache_key_t key = { 0 };
+ get_window_key(&key, rs);
+ key.chunk_index = chunk_index;
+ SVN_ERR(svn_cache__get((void **) &cached_window,
+ is_cached,
+ rs->window_cache,
+ &key,
+ result_pool));
+
+ if (*is_cached)
+ {
+ /* found it. Pass it back to the caller. */
+ *window_p = cached_window->window;
+
+ /* manipulate the RS as if we just read the data */
+ rs->current = cached_window->end_offset;
+ rs->chunk_index = chunk_index;
}
return SVN_NO_ERROR;
@@ -1145,23 +1125,20 @@ set_cached_window(svn_txdelta_window_t *
apr_off_t start_offset,
apr_pool_t *scratch_pool)
{
- if (rs->window_cache)
- {
- /* store the window and the first offset _past_ it */
- svn_fs_x__txdelta_cached_window_t cached_window;
- svn_fs_x__window_cache_key_t key = {0};
-
- cached_window.window = window;
- cached_window.start_offset = start_offset - rs->start;
- cached_window.end_offset = rs->current;
-
- /* but key it with the start offset because that is the known state
- * when we will look it up */
- SVN_ERR(svn_cache__set(rs->window_cache,
- get_window_key(&key, rs),
- &cached_window,
- scratch_pool));
- }
+ /* store the window and the first offset _past_ it */
+ svn_fs_x__txdelta_cached_window_t cached_window;
+ svn_fs_x__window_cache_key_t key = {0};
+
+ cached_window.window = window;
+ cached_window.start_offset = start_offset - rs->start;
+ cached_window.end_offset = rs->current;
+
+ /* but key it with the start offset because that is the known state
+ * when we will look it up */
+ SVN_ERR(svn_cache__set(rs->window_cache,
+ get_window_key(&key, rs),
+ &cached_window,
+ scratch_pool));
return SVN_NO_ERROR;
}
@@ -1178,23 +1155,13 @@ get_cached_combined_window(svn_stringbuf
svn_boolean_t *is_cached,
apr_pool_t *pool)
{
- if (! rs->combined_cache)
- {
- /* txdelta window has not been enabled */
- *is_cached = FALSE;
- }
- else
- {
- /* ask the cache for the desired txdelta window */
- svn_fs_x__window_cache_key_t key = { 0 };
- return svn_cache__get((void **)window_p,
- is_cached,
- rs->combined_cache,
- get_window_key(&key, rs),
- pool);
- }
-
- return SVN_NO_ERROR;
+ /* ask the cache for the desired txdelta window */
+ svn_fs_x__window_cache_key_t key = { 0 };
+ return svn_cache__get((void **)window_p,
+ is_cached,
+ rs->combined_cache,
+ get_window_key(&key, rs),
+ pool);
}
/* Store the WINDOW read for the rep state RS in the current FSX session's
@@ -1205,18 +1172,13 @@ set_cached_combined_window(svn_stringbuf
rep_state_t *rs,
apr_pool_t *scratch_pool)
{
- if (rs->combined_cache)
- {
- /* but key it with the start offset because that is the known state
- * when we will look it up */
- svn_fs_x__window_cache_key_t key = { 0 };
- return svn_cache__set(rs->combined_cache,
- get_window_key(&key, rs),
- window,
- scratch_pool);
- }
-
- return SVN_NO_ERROR;
+ /* but key it with the start offset because that is the known state
+ * when we will look it up */
+ svn_fs_x__window_cache_key_t key = { 0 };
+ return svn_cache__set(rs->combined_cache,
+ get_window_key(&key, rs),
+ window,
+ scratch_pool);
}
/* Build an array of rep_state structures in *LIST giving the delta
@@ -1458,24 +1420,20 @@ read_container_window(svn_stringbuf_t **
svn_fs_x__data_t *ffd = fs->fsap_data;
svn_fs_x__pair_cache_key_t key;
svn_revnum_t revision = svn_fs_x__get_revnum(rs->rep_id.change_set);
+ svn_boolean_t is_cached = FALSE;
+ svn_fs_x__reps_baton_t baton;
SVN_ERR(auto_set_start_offset(rs, scratch_pool));
key.revision = svn_fs_x__packed_base_rev(fs, revision);
key.second = rs->start;
/* already in cache? */
- if (ffd->reps_container_cache)
- {
- svn_boolean_t is_cached = FALSE;
- svn_fs_x__reps_baton_t baton;
- baton.fs = fs;
- baton.idx = rs->sub_item;
-
- SVN_ERR(svn_cache__get_partial((void**)&extractor, &is_cached,
- ffd->reps_container_cache, &key,
- svn_fs_x__reps_get_func, &baton,
- result_pool));
- }
+ baton.fs = fs;
+ baton.idx = rs->sub_item;
+ SVN_ERR(svn_cache__get_partial((void**)&extractor, &is_cached,
+ ffd->reps_container_cache, &key,
+ svn_fs_x__reps_get_func, &baton,
+ result_pool));
/* read from disk, if necessary */
if (extractor == NULL)
@@ -1742,18 +1700,13 @@ read_rep_header(svn_fs_x__rep_header_t *
svn_fs_x__data_t *ffd = fs->fsap_data;
svn_boolean_t is_cached = FALSE;
- if (ffd->rep_header_cache)
- {
- SVN_ERR(svn_cache__get((void**)rep_header, &is_cached,
- ffd->rep_header_cache, key, pool));
- if (is_cached)
- return SVN_NO_ERROR;
- }
+ SVN_ERR(svn_cache__get((void**)rep_header, &is_cached,
+ ffd->rep_header_cache, key, pool));
+ if (is_cached)
+ return SVN_NO_ERROR;
SVN_ERR(svn_fs_x__read_rep_header(rep_header, stream, pool, pool));
-
- if (ffd->rep_header_cache)
- SVN_ERR(svn_cache__set(ffd->rep_header_cache, key, *rep_header, pool));
+ SVN_ERR(svn_cache__set(ffd->rep_header_cache, key, *rep_header, pool));
return SVN_NO_ERROR;
}
@@ -2186,7 +2139,7 @@ svn_fs_x__get_contents(svn_stream_t **co
/* Make the stream attempt fulltext cache lookups if the fulltext
* is cacheable. If it is not, then also don't try to buffer and
* cache it. */
- if (ffd->fulltext_cache && cache_fulltext
+ if ( cache_fulltext
&& SVN_IS_VALID_REVNUM(revision)
&& fulltext_size_is_cachable(ffd, len))
{
@@ -2257,8 +2210,7 @@ svn_fs_x__try_process_file_contents(svn_
fulltext_cache_key.revision = svn_fs_x__get_revnum(rep->id.change_set);
fulltext_cache_key.second = rep->id.number;
- if (ffd->fulltext_cache
- && SVN_IS_VALID_REVNUM(fulltext_cache_key.revision)
+ if ( SVN_IS_VALID_REVNUM(fulltext_cache_key.revision)
&& fulltext_size_is_cachable(ffd, rep->expanded_size))
{
cache_access_wrapper_baton_t wrapper_baton;
@@ -2344,12 +2296,11 @@ svn_fs_x__get_file_delta_stream(svn_txde
svn_stream_t *source_stream, *target_stream;
rep_state_t *rep_state;
svn_fs_x__rep_header_t *rep_header;
- svn_fs_x__data_t *ffd = fs->fsap_data;
/* Try a shortcut: if the target is stored as a delta against the source,
then just use that delta. However, prefer using the fulltext cache
whenever that is available. */
- if (target->data_rep && (source || !ffd->fulltext_cache))
+ if (target->data_rep && source)
{
/* Read target's base rep if any. */
SVN_ERR(create_rep_state(&rep_state, &rep_header, NULL,
@@ -2654,23 +2605,19 @@ svn_fs_x__rep_contents_dir(apr_array_hea
/* find the cache we may use */
svn_cache__t *cache = locate_dir_cache(fs, &key, noderev);
- if (cache)
- {
- svn_boolean_t found;
+ svn_boolean_t found;
- SVN_ERR(svn_cache__get((void **)entries_p, &found, cache, &key,
- result_pool));
- if (found)
- return SVN_NO_ERROR;
- }
+ SVN_ERR(svn_cache__get((void **)entries_p, &found, cache, &key,
+ result_pool));
+ if (found)
+ return SVN_NO_ERROR;
/* Read in the directory contents. */
SVN_ERR(get_dir_contents(entries_p, fs, noderev, result_pool,
scratch_pool));
/* Update the cache, if we are to use one. */
- if (cache)
- SVN_ERR(svn_cache__set(cache, &key, *entries_p, scratch_pool));
+ SVN_ERR(svn_cache__set(cache, &key, *entries_p, scratch_pool));
return SVN_NO_ERROR;
}
@@ -2699,25 +2646,22 @@ svn_fs_x__rep_contents_dir_entry(svn_fs_
/* find the cache we may use */
svn_fs_x__id_t key;
svn_cache__t *cache = locate_dir_cache(fs, &key, noderev);
- if (cache)
- {
- svn_fs_x__ede_baton_t baton;
- baton.hint = *hint;
- baton.name = name;
-
- /* Cache lookup. */
- SVN_ERR(svn_cache__get_partial((void **)dirent,
- &found,
- cache,
- &key,
- svn_fs_x__extract_dir_entry,
- &baton,
- result_pool));
-
- /* Remember the new clue only if we found something at that spot. */
- if (found)
- *hint = baton.hint;
- }
+ svn_fs_x__ede_baton_t baton;
+ baton.hint = *hint;
+ baton.name = name;
+
+ /* Cache lookup. */
+ SVN_ERR(svn_cache__get_partial((void **)dirent,
+ &found,
+ cache,
+ &key,
+ svn_fs_x__extract_dir_entry,
+ &baton,
+ result_pool));
+
+ /* Remember the new clue only if we found something at that spot. */
+ if (found)
+ *hint = baton.hint;
/* fetch data from disk if we did not find it in the cache */
if (! found)
@@ -2778,7 +2722,7 @@ svn_fs_x__get_proplist(apr_hash_t **prop
key.revision = svn_fs_x__get_revnum(rep->id.change_set);
key.second = rep->id.number;
- if (ffd->properties_cache && SVN_IS_VALID_REVNUM(key.revision))
+ if (SVN_IS_VALID_REVNUM(key.revision))
{
svn_boolean_t is_cached;
SVN_ERR(svn_cache__get((void **) proplist_p, &is_cached,
@@ -2794,7 +2738,7 @@ svn_fs_x__get_proplist(apr_hash_t **prop
result_pool));
SVN_ERR(svn_stream_close(stream));
- if (ffd->properties_cache && SVN_IS_VALID_REVNUM(rep->id.change_set))
+ if (SVN_IS_VALID_REVNUM(rep->id.change_set))
SVN_ERR(svn_cache__set(ffd->properties_cache, &key, proplist,
scratch_pool));
}
@@ -2834,7 +2778,7 @@ svn_fs_x__get_changes(apr_array_header_t
/* try cache lookup first */
- if (ffd->changes_container_cache && svn_fs_x__is_packed_rev(fs, rev))
+ if (svn_fs_x__is_packed_rev(fs, rev))
{
apr_off_t offset;
apr_uint32_t sub_item;
@@ -2850,15 +2794,11 @@ svn_fs_x__get_changes(apr_array_header_t
svn_fs_x__changes_get_list_func,
&sub_item, result_pool));
}
- else if (ffd->changes_cache)
+ else
{
SVN_ERR(svn_cache__get((void **) changes, &found, ffd->changes_cache,
&rev, result_pool));
}
- else
- {
- found = FALSE;
- }
if (!found)
{
@@ -2890,15 +2830,11 @@ block_read_contents(svn_fs_t *fs,
apr_off_t max_offset,
apr_pool_t *scratch_pool)
{
- svn_fs_x__data_t *ffd = fs->fsap_data;
svn_fs_x__representation_cache_key_t header_key = { 0 };
rep_state_t rs = { 0 };
svn_filesize_t fulltext_len;
svn_fs_x__rep_header_t *rep_header;
- if (!ffd->txdelta_window_cache || !ffd->combined_window_cache)
- return SVN_NO_ERROR;
-
header_key.revision = (apr_int32_t)key->revision;
header_key.is_packed = svn_fs_x__is_packed_rev(fs, header_key.revision);
header_key.item_index = key->second;
@@ -2976,14 +2912,13 @@ block_read_changes(apr_array_header_t **
svn_fs_x__data_t *ffd = fs->fsap_data;
svn_stream_t *stream;
svn_revnum_t revision = svn_fs_x__get_revnum(entry->items[0].change_set);
- if (!must_read && !ffd->changes_cache)
- return SVN_NO_ERROR;
+ apr_size_t estimated_size;
/* we don't support containers, yet */
SVN_ERR_ASSERT(entry->item_count == 1);
/* already in cache? */
- if (!must_read && ffd->changes_cache)
+ if (!must_read)
{
svn_boolean_t is_cached = FALSE;
SVN_ERR(svn_cache__has_key(&is_cached, ffd->changes_cache, &revision,
@@ -2995,24 +2930,20 @@ block_read_changes(apr_array_header_t **
SVN_ERR(read_item(&stream, fs, rev_file, entry, scratch_pool));
/* read changes from revision file */
-
SVN_ERR(svn_fs_x__read_changes(changes, stream, result_pool, scratch_pool));
/* cache for future reference */
- if (ffd->changes_cache)
- {
- /* Guesstimate for the size of the in-cache representation. */
- apr_size_t estimated_size = (apr_size_t)250 * (*changes)->nelts;
+ /* Guesstimate for the size of the in-cache representation. */
+ estimated_size = (apr_size_t)250 * (*changes)->nelts;
- /* Don't even serialize data that probably won't fit into the
- * cache. This often implies that either CHANGES is very
- * large, memory is scarce or both. Having a huge temporary
- * copy would not be a good thing in either case. */
- if (svn_cache__is_cachable(ffd->changes_cache, estimated_size))
- SVN_ERR(svn_cache__set(ffd->changes_cache, &revision, *changes,
- scratch_pool));
- }
+ /* Don't even serialize data that probably won't fit into the
+ * cache. This often implies that either CHANGES is very
+ * large, memory is scarce or both. Having a huge temporary
+ * copy would not be a good thing in either case. */
+ if (svn_cache__is_cachable(ffd->changes_cache, estimated_size))
+ SVN_ERR(svn_cache__set(ffd->changes_cache, &revision, *changes,
+ scratch_pool));
return SVN_NO_ERROR;
}
@@ -3037,7 +2968,7 @@ block_read_changes_container(apr_array_h
key.second = entry->offset;
/* already in cache? */
- if (!must_read && ffd->changes_container_cache)
+ if (!must_read)
{
svn_boolean_t is_cached = FALSE;
SVN_ERR(svn_cache__has_key(&is_cached, ffd->changes_container_cache,
@@ -3058,10 +2989,8 @@ block_read_changes_container(apr_array_h
if (must_read)
SVN_ERR(svn_fs_x__changes_get_list(changes, container, sub_item,
result_pool));
-
- if (ffd->changes_container_cache)
- SVN_ERR(svn_cache__set(ffd->changes_container_cache, &key, container,
- scratch_pool));
+ SVN_ERR(svn_cache__set(ffd->changes_container_cache, &key, container,
+ scratch_pool));
return SVN_NO_ERROR;
}
@@ -3078,14 +3007,12 @@ block_read_noderev(svn_fs_x__noderev_t *
{
svn_fs_x__data_t *ffd = fs->fsap_data;
svn_stream_t *stream;
- if (!must_read && !ffd->node_revision_cache)
- return SVN_NO_ERROR;
/* we don't support containers, yet */
SVN_ERR_ASSERT(entry->item_count == 1);
/* already in cache? */
- if (!must_read && ffd->node_revision_cache)
+ if (!must_read)
{
svn_boolean_t is_cached = FALSE;
SVN_ERR(svn_cache__has_key(&is_cached, ffd->node_revision_cache, key,
@@ -3100,9 +3027,8 @@ block_read_noderev(svn_fs_x__noderev_t *
SVN_ERR(svn_fs_x__read_noderev(noderev_p, stream, result_pool,
scratch_pool));
- if (ffd->node_revision_cache)
- SVN_ERR(svn_cache__set(ffd->node_revision_cache, key, *noderev_p,
- scratch_pool));
+ SVN_ERR(svn_cache__set(ffd->node_revision_cache, key, *noderev_p,
+ scratch_pool));
return SVN_NO_ERROR;
}
@@ -3127,7 +3053,7 @@ block_read_noderevs_container(svn_fs_x__
key.second = entry->offset;
/* already in cache? */
- if (!must_read && ffd->noderevs_container_cache)
+ if (!must_read)
{
svn_boolean_t is_cached = FALSE;
SVN_ERR(svn_cache__has_key(&is_cached, ffd->noderevs_container_cache,
@@ -3147,9 +3073,8 @@ block_read_noderevs_container(svn_fs_x__
SVN_ERR(svn_fs_x__noderevs_get(noderev_p, container, sub_item,
result_pool));
- if (ffd->noderevs_container_cache)
- SVN_ERR(svn_cache__set(ffd->noderevs_container_cache, &key, container,
- scratch_pool));
+ SVN_ERR(svn_cache__set(ffd->noderevs_container_cache, &key, container,
+ scratch_pool));
return SVN_NO_ERROR;
}
@@ -3174,7 +3099,7 @@ block_read_reps_container(svn_fs_x__rep_
key.second = entry->offset;
/* already in cache? */
- if (!must_read && ffd->reps_container_cache)
+ if (!must_read)
{
svn_boolean_t is_cached = FALSE;
SVN_ERR(svn_cache__has_key(&is_cached, ffd->reps_container_cache,
@@ -3195,9 +3120,8 @@ block_read_reps_container(svn_fs_x__rep_
SVN_ERR(svn_fs_x__reps_get(extractor, fs, container, sub_item,
result_pool));
- if (ffd->noderevs_container_cache)
- SVN_ERR(svn_cache__set(ffd->reps_container_cache, &key, container,
- scratch_pool));
+ SVN_ERR(svn_cache__set(ffd->reps_container_cache, &key, container,
+ scratch_pool));
return SVN_NO_ERROR;
}
@@ -3289,11 +3213,10 @@ block_read(void **result,
break;
case SVN_FS_X__ITEM_TYPE_NODEREV:
- if (ffd->node_revision_cache || is_result)
- SVN_ERR(block_read_noderev((svn_fs_x__noderev_t **)&item,
- fs, revision_file,
- entry, &key, is_result,
- pool, iterpool));
+ SVN_ERR(block_read_noderev((svn_fs_x__noderev_t **)&item,
+ fs, revision_file,
+ entry, &key, is_result,
+ pool, iterpool));
break;
case SVN_FS_X__ITEM_TYPE_CHANGES:
Modified: subversion/trunk/subversion/libsvn_fs_x/caching.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_fs_x/caching.c?rev=1694489&r1=1694488&r2=1694489&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_fs_x/caching.c (original)
+++ subversion/trunk/subversion/libsvn_fs_x/caching.c Thu Aug 6 12:54:36 2015
@@ -260,42 +260,42 @@ init_callbacks(svn_cache__t *cache,
svn_cache__error_handler_t error_handler,
apr_pool_t *pool)
{
- if (cache != NULL)
- {
#ifdef SVN_DEBUG_CACHE_DUMP_STATS
- /* schedule printing the access statistics upon pool cleanup,
- * i.e. end of FSX session.
- */
- dump_cache_baton_t *baton;
-
- baton = apr_palloc(pool, sizeof(*baton));
- baton->pool = pool;
- baton->cache = cache;
-
- apr_pool_cleanup_register(pool,
- baton,
- dump_cache_statistics,
- apr_pool_cleanup_null);
-#endif
+ /* schedule printing the access statistics upon pool cleanup,
+ * i.e. end of FSX session.
+ */
+ dump_cache_baton_t *baton;
- if (error_handler)
- SVN_ERR(svn_cache__set_error_handler(cache,
- error_handler,
- fs,
- pool));
+ baton = apr_palloc(pool, sizeof(*baton));
+ baton->pool = pool;
+ baton->cache = cache;
+
+ apr_pool_cleanup_register(pool,
+ baton,
+ dump_cache_statistics,
+ apr_pool_cleanup_null);
+#endif
- }
+ if (error_handler)
+ SVN_ERR(svn_cache__set_error_handler(cache,
+ error_handler,
+ fs,
+ pool));
return SVN_NO_ERROR;
}
/* Sets *CACHE_P to cache instance based on provided options.
- * Creates memcache if MEMCACHE is not NULL. Creates membuffer cache if
- * MEMBUFFER is not NULL. Fallbacks to inprocess cache if MEMCACHE and
- * MEMBUFFER are NULL and pages is non-zero. Sets *CACHE_P to NULL
- * otherwise. Use the given PRIORITY class for the new cache. If it
- * is 0, then use the default priority class.
+ *
+ * If DUMMY_CACHE is set, create a null cache. Otherwise, creates a memcache
+ * if MEMCACHE is not NULL or a membuffer cache if MEMBUFFER is not NULL.
+ * Falls back to inprocess cache if no other cache type has been selected
+ * and PAGES is not 0. Create a null cache otherwise.
+ *
+ * Use the given PRIORITY class for the new cache. If PRIORITY is 0, then
+ * use the default priority class. HAS_NAMESPACE indicates whether we
+ * prefixed this cache instance with a namespace.
*
* Unless NO_HANDLER is true, register an error handler that reports errors
* as warnings to the FS warning callback.
@@ -313,8 +313,10 @@ create_cache(svn_cache__t **cache_p,
apr_ssize_t klen,
const char *prefix,
apr_uint32_t priority,
+ svn_boolean_t has_namespace,
svn_fs_t *fs,
svn_boolean_t no_handler,
+ svn_boolean_t dummy_cache,
apr_pool_t *result_pool,
apr_pool_t *scratch_pool)
{
@@ -324,7 +326,11 @@ create_cache(svn_cache__t **cache_p,
if (priority == 0)
priority = SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY;
- if (memcache)
+ if (dummy_cache)
+ {
+ SVN_ERR(svn_cache__create_null(cache_p, prefix, result_pool));
+ }
+ else if (memcache)
{
SVN_ERR(svn_cache__create_memcache(cache_p, memcache,
serializer, deserializer, klen,
@@ -335,9 +341,12 @@ create_cache(svn_cache__t **cache_p,
}
else if (membuffer)
{
+ /* We assume caches with namespaces to be relatively short-lived,
+ * i.e. their data will not be needed after a while. */
SVN_ERR(svn_cache__create_membuffer_cache(
cache_p, membuffer, serializer, deserializer,
- klen, prefix, priority, FALSE, result_pool, scratch_pool));
+ klen, prefix, priority, FALSE, has_namespace,
+ result_pool, scratch_pool));
}
else if (pages)
{
@@ -347,7 +356,7 @@ create_cache(svn_cache__t **cache_p,
}
else
{
- *cache_p = NULL;
+ SVN_ERR(svn_cache__create_null(cache_p, prefix, result_pool));
}
SVN_ERR(init_callbacks(*cache_p, fs, error_handler, result_pool));
@@ -372,6 +381,7 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
svn_boolean_t cache_fulltexts;
svn_boolean_t cache_revprops;
const char *cache_namespace;
+ svn_boolean_t has_namespace;
/* Evaluating the cache configuration. */
SVN_ERR(read_config(&cache_namespace,
@@ -383,6 +393,7 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
prefix = apr_pstrcat(scratch_pool, "ns:", cache_namespace, ":", prefix,
SVN_VA_NULL);
+ has_namespace = strlen(cache_namespace) > 0;
membuffer = svn_cache__get_global_membuffer_cache();
@@ -418,8 +429,9 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
APR_HASH_KEY_STRING,
apr_pstrcat(scratch_pool, prefix, "DAG", SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_LOW_PRIORITY,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
/* 1st level DAG node cache */
@@ -435,8 +447,9 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
sizeof(svn_fs_x__id_t),
apr_pstrcat(scratch_pool, prefix, "DIR", SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
/* Only 16 bytes per entry (a revision number + the corresponding offset).
@@ -451,8 +464,9 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
apr_pstrcat(scratch_pool, prefix, "PACK-MANIFEST",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
/* initialize node revision cache, if caching has been enabled */
@@ -466,8 +480,9 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
apr_pstrcat(scratch_pool, prefix, "NODEREVS",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
/* initialize representation header cache, if caching has been enabled */
@@ -481,8 +496,9 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
apr_pstrcat(scratch_pool, prefix, "REPHEADER",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
/* initialize node change list cache, if caching has been enabled */
@@ -496,136 +512,120 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
apr_pstrcat(scratch_pool, prefix, "CHANGES",
SVN_VA_NULL),
0,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
/* if enabled, cache fulltext and other derived information */
- if (cache_fulltexts)
- {
- SVN_ERR(create_cache(&(ffd->fulltext_cache),
- ffd->memcache,
- membuffer,
- 0, 0, /* Do not use inprocess cache */
- /* Values are svn_stringbuf_t */
- NULL, NULL,
- sizeof(svn_fs_x__pair_cache_key_t),
- apr_pstrcat(scratch_pool, prefix, "TEXT",
- SVN_VA_NULL),
- SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
- fs,
- no_handler,
- fs->pool, scratch_pool));
-
- SVN_ERR(create_cache(&(ffd->properties_cache),
- NULL,
- membuffer,
- 0, 0, /* Do not use inprocess cache */
- svn_fs_x__serialize_properties,
- svn_fs_x__deserialize_properties,
- sizeof(svn_fs_x__pair_cache_key_t),
- apr_pstrcat(scratch_pool, prefix, "PROP",
- SVN_VA_NULL),
- SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
- fs,
- no_handler,
- fs->pool, scratch_pool));
-
- SVN_ERR(create_cache(&(ffd->mergeinfo_cache),
- NULL,
- membuffer,
- 0, 0, /* Do not use inprocess cache */
- svn_fs_x__serialize_mergeinfo,
- svn_fs_x__deserialize_mergeinfo,
- APR_HASH_KEY_STRING,
- apr_pstrcat(scratch_pool, prefix, "MERGEINFO",
- SVN_VA_NULL),
- 0,
- fs,
- no_handler,
- fs->pool, scratch_pool));
-
- SVN_ERR(create_cache(&(ffd->mergeinfo_existence_cache),
- NULL,
- membuffer,
- 0, 0, /* Do not use inprocess cache */
- /* Values are svn_stringbuf_t */
- NULL, NULL,
- APR_HASH_KEY_STRING,
- apr_pstrcat(scratch_pool, prefix, "HAS_MERGEINFO",
- SVN_VA_NULL),
- 0,
- fs,
- no_handler,
- fs->pool, scratch_pool));
- }
- else
- {
- ffd->fulltext_cache = NULL;
- ffd->properties_cache = NULL;
- ffd->mergeinfo_cache = NULL;
- ffd->mergeinfo_existence_cache = NULL;
- }
+ SVN_ERR(create_cache(&(ffd->fulltext_cache),
+ ffd->memcache,
+ membuffer,
+ 0, 0, /* Do not use inprocess cache */
+ /* Values are svn_stringbuf_t */
+ NULL, NULL,
+ sizeof(svn_fs_x__pair_cache_key_t),
+ apr_pstrcat(scratch_pool, prefix, "TEXT",
+ SVN_VA_NULL),
+ SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
+ has_namespace,
+ fs,
+ no_handler, !cache_fulltexts,
+ fs->pool, scratch_pool));
+
+ SVN_ERR(create_cache(&(ffd->properties_cache),
+ NULL,
+ membuffer,
+ 0, 0, /* Do not use inprocess cache */
+ svn_fs_x__serialize_properties,
+ svn_fs_x__deserialize_properties,
+ sizeof(svn_fs_x__pair_cache_key_t),
+ apr_pstrcat(scratch_pool, prefix, "PROP",
+ SVN_VA_NULL),
+ SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
+ has_namespace,
+ fs,
+ no_handler, !cache_fulltexts,
+ fs->pool, scratch_pool));
+
+ SVN_ERR(create_cache(&(ffd->mergeinfo_cache),
+ NULL,
+ membuffer,
+ 0, 0, /* Do not use inprocess cache */
+ svn_fs_x__serialize_mergeinfo,
+ svn_fs_x__deserialize_mergeinfo,
+ APR_HASH_KEY_STRING,
+ apr_pstrcat(scratch_pool, prefix, "MERGEINFO",
+ SVN_VA_NULL),
+ 0,
+ has_namespace,
+ fs,
+ no_handler, !cache_fulltexts,
+ fs->pool, scratch_pool));
+
+ SVN_ERR(create_cache(&(ffd->mergeinfo_existence_cache),
+ NULL,
+ membuffer,
+ 0, 0, /* Do not use inprocess cache */
+ /* Values are svn_stringbuf_t */
+ NULL, NULL,
+ APR_HASH_KEY_STRING,
+ apr_pstrcat(scratch_pool, prefix, "HAS_MERGEINFO",
+ SVN_VA_NULL),
+ 0,
+ has_namespace,
+ fs,
+ no_handler, !cache_fulltexts,
+ fs->pool, scratch_pool));
/* if enabled, cache revprops */
- if (cache_revprops)
- {
- SVN_ERR(create_cache(&(ffd->revprop_cache),
- NULL,
- membuffer,
- 0, 0, /* Do not use inprocess cache */
- svn_fs_x__serialize_properties,
- svn_fs_x__deserialize_properties,
- sizeof(svn_fs_x__pair_cache_key_t),
- apr_pstrcat(scratch_pool, prefix, "REVPROP",
- SVN_VA_NULL),
- SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
- fs,
- no_handler,
- fs->pool, scratch_pool));
- }
- else
- {
- ffd->revprop_cache = NULL;
- }
+ SVN_ERR(create_cache(&(ffd->revprop_cache),
+ NULL,
+ membuffer,
+ 0, 0, /* Do not use inprocess cache */
+ svn_fs_x__serialize_properties,
+ svn_fs_x__deserialize_properties,
+ sizeof(svn_fs_x__pair_cache_key_t),
+ apr_pstrcat(scratch_pool, prefix, "REVPROP",
+ SVN_VA_NULL),
+ SVN_CACHE__MEMBUFFER_DEFAULT_PRIORITY,
+ has_namespace,
+ fs,
+ no_handler, !cache_revprops,
+ fs->pool, scratch_pool));
/* if enabled, cache text deltas and their combinations */
- if (cache_txdeltas)
- {
- SVN_ERR(create_cache(&(ffd->txdelta_window_cache),
- NULL,
- membuffer,
- 0, 0, /* Do not use inprocess cache */
- svn_fs_x__serialize_txdelta_window,
- svn_fs_x__deserialize_txdelta_window,
- sizeof(svn_fs_x__window_cache_key_t),
- apr_pstrcat(scratch_pool, prefix, "TXDELTA_WINDOW",
- SVN_VA_NULL),
- SVN_CACHE__MEMBUFFER_LOW_PRIORITY,
- fs,
- no_handler,
- fs->pool, scratch_pool));
-
- SVN_ERR(create_cache(&(ffd->combined_window_cache),
- NULL,
- membuffer,
- 0, 0, /* Do not use inprocess cache */
- /* Values are svn_stringbuf_t */
- NULL, NULL,
- sizeof(svn_fs_x__window_cache_key_t),
- apr_pstrcat(scratch_pool, prefix, "COMBINED_WINDOW",
- SVN_VA_NULL),
- SVN_CACHE__MEMBUFFER_LOW_PRIORITY,
- fs,
- no_handler,
- fs->pool, scratch_pool));
- }
- else
- {
- ffd->txdelta_window_cache = NULL;
- ffd->combined_window_cache = NULL;
- }
+ SVN_ERR(create_cache(&(ffd->txdelta_window_cache),
+ NULL,
+ membuffer,
+ 0, 0, /* Do not use inprocess cache */
+ svn_fs_x__serialize_txdelta_window,
+ svn_fs_x__deserialize_txdelta_window,
+ sizeof(svn_fs_x__window_cache_key_t),
+ apr_pstrcat(scratch_pool, prefix, "TXDELTA_WINDOW",
+ SVN_VA_NULL),
+ SVN_CACHE__MEMBUFFER_LOW_PRIORITY,
+ has_namespace,
+ fs,
+ no_handler, !cache_txdeltas,
+ fs->pool, scratch_pool));
+
+ SVN_ERR(create_cache(&(ffd->combined_window_cache),
+ NULL,
+ membuffer,
+ 0, 0, /* Do not use inprocess cache */
+ /* Values are svn_stringbuf_t */
+ NULL, NULL,
+ sizeof(svn_fs_x__window_cache_key_t),
+ apr_pstrcat(scratch_pool, prefix, "COMBINED_WINDOW",
+ SVN_VA_NULL),
+ SVN_CACHE__MEMBUFFER_LOW_PRIORITY,
+ has_namespace,
+ fs,
+ no_handler, !cache_txdeltas,
+ fs->pool, scratch_pool));
+ /* Caches for our various container types. */
SVN_ERR(create_cache(&(ffd->noderevs_container_cache),
NULL,
membuffer,
@@ -636,8 +636,9 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
apr_pstrcat(scratch_pool, prefix, "NODEREVSCNT",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
SVN_ERR(create_cache(&(ffd->changes_container_cache),
NULL,
@@ -649,8 +650,9 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
apr_pstrcat(scratch_pool, prefix, "CHANGESCNT",
SVN_VA_NULL),
0,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
SVN_ERR(create_cache(&(ffd->reps_container_cache),
NULL,
@@ -662,10 +664,12 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
apr_pstrcat(scratch_pool, prefix, "REPSCNT",
SVN_VA_NULL),
0,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
+ /* Cache index info. */
SVN_ERR(create_cache(&(ffd->l2p_header_cache),
NULL,
membuffer,
@@ -677,8 +681,9 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
apr_pstrcat(scratch_pool, prefix, "L2P_HEADER",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
SVN_ERR(create_cache(&(ffd->l2p_page_cache),
NULL,
@@ -691,8 +696,9 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
apr_pstrcat(scratch_pool, prefix, "L2P_PAGE",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
SVN_ERR(create_cache(&(ffd->p2l_header_cache),
NULL,
@@ -704,8 +710,9 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
apr_pstrcat(scratch_pool, prefix, "P2L_HEADER",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
SVN_ERR(create_cache(&(ffd->p2l_page_cache),
NULL,
@@ -717,8 +724,9 @@ svn_fs_x__initialize_caches(svn_fs_t *fs
apr_pstrcat(scratch_pool, prefix, "P2L_PAGE",
SVN_VA_NULL),
SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+ has_namespace,
fs,
- no_handler,
+ no_handler, FALSE,
fs->pool, scratch_pool));
return SVN_NO_ERROR;
Modified: subversion/trunk/subversion/libsvn_fs_x/tree.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_fs_x/tree.c?rev=1694489&r1=1694488&r2=1694489&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_fs_x/tree.c (original)
+++ subversion/trunk/subversion/libsvn_fs_x/tree.c Thu Aug 6 12:54:36 2015
@@ -4084,16 +4084,13 @@ get_mergeinfo_for_path(svn_mergeinfo_t *
cache_key = mergeinfo_cache_key(path, rev_root, inherit,
adjust_inherited_mergeinfo, scratch_pool);
- if (ffd->mergeinfo_existence_cache)
- {
- SVN_ERR(svn_cache__get((void **)&mergeinfo_exists, &found,
- ffd->mergeinfo_existence_cache,
- cache_key, result_pool));
- if (found && mergeinfo_exists->data[0] == '1')
- SVN_ERR(svn_cache__get((void **)mergeinfo, &found,
- ffd->mergeinfo_cache,
- cache_key, result_pool));
- }
+ SVN_ERR(svn_cache__get((void **)&mergeinfo_exists, &found,
+ ffd->mergeinfo_existence_cache,
+ cache_key, result_pool));
+ if (found && mergeinfo_exists->data[0] == '1')
+ SVN_ERR(svn_cache__get((void **)mergeinfo, &found,
+ ffd->mergeinfo_cache,
+ cache_key, result_pool));
if (! found)
{
@@ -4101,16 +4098,13 @@ get_mergeinfo_for_path(svn_mergeinfo_t *
inherit,
adjust_inherited_mergeinfo,
result_pool, scratch_pool));
- if (ffd->mergeinfo_existence_cache)
- {
- mergeinfo_exists = svn_stringbuf_create(*mergeinfo ? "1" : "0",
- scratch_pool);
- SVN_ERR(svn_cache__set(ffd->mergeinfo_existence_cache,
- cache_key, mergeinfo_exists, scratch_pool));
- if (*mergeinfo)
- SVN_ERR(svn_cache__set(ffd->mergeinfo_cache,
- cache_key, *mergeinfo, scratch_pool));
- }
+ mergeinfo_exists = svn_stringbuf_create(*mergeinfo ? "1" : "0",
+ scratch_pool);
+ SVN_ERR(svn_cache__set(ffd->mergeinfo_existence_cache,
+ cache_key, mergeinfo_exists, scratch_pool));
+ if (*mergeinfo)
+ SVN_ERR(svn_cache__set(ffd->mergeinfo_cache,
+ cache_key, *mergeinfo, scratch_pool));
}
return SVN_NO_ERROR;
Re: svn commit: r1694489 [1/2] - in /subversion/trunk: ./
subversion/include/private/ subversion/libsvn_fs_fs/ subversion/libsvn_fs_x/
subversion/libsvn_subr/ subversion/tests/libsvn_subr/
Posted by Stefan Fuhrmann <st...@wandisco.com>.
On Fri, Aug 7, 2015 at 2:57 PM, Stefan Fuhrmann <
stefan.fuhrmann@wandisco.com> wrote:
> On Thu, Aug 6, 2015 at 3:59 PM, Ivan Zhakov <iv...@visualsvn.com> wrote:
>
>> On 6 August 2015 at 15:54, <st...@apache.org> wrote:
>> > Author: stefan2
>> > Date: Thu Aug 6 12:54:36 2015
>> > New Revision: 1694489
>> >
>> > URL: http://svn.apache.org/r1694489
>> > Log:
>> > Merge the 1.10-cache-improvements development branch to trunk and remove
>> > the BRANCH-README file. There were no conflicts.
>> >
>> > Changes brought by this branch:
>> > - Introduce a NULL cache implementation to unify / simplify cache usage
>> > - Space and runtime efficiency improvements to the recently introduced
>> > full key comparison in membuffer caches.
>
>
> Also after this change fs-test.exe fails with assertion in Windows
>> 32-bit Debug build:
>> [[
>> START: fs-test.exe
>> Assertion failed: !group || !group->header.used, file
>> ..\..\..\subversion\libsvn_subr\cache-membuffer.c, line 1031
>> ]]
>>
>
> This one is odd because the merge did not touch anything
> obviously linked to spare bucket handling. Could be a long-
> standing issue.
>
> I failed to reproduce it with a 32 bit debug build on Win64.
> Does it fail for you in serial execution mode? Is it deterministic
> in that case? Does the attached patch prevent it?
>
Tweaking the code a bit I was finally able to reproduce the problem.
It has been introduced in 1.9 and is fixed by r1695022.
The merge above only triggered it by enlarging the cache index
structure, in turn reducing the capacity of individual entry groups -
making them more likely to overflow and trigger the allocation of
spare groups.
-- Stefan^2.
Re: svn commit: r1694489 [1/2] - in /subversion/trunk: ./
subversion/include/private/ subversion/libsvn_fs_fs/ subversion/libsvn_fs_x/
subversion/libsvn_subr/ subversion/tests/libsvn_subr/
Posted by Stefan Fuhrmann <st...@wandisco.com>.
On Thu, Aug 6, 2015 at 3:59 PM, Ivan Zhakov <iv...@visualsvn.com> wrote:
> On 6 August 2015 at 15:54, <st...@apache.org> wrote:
> > Author: stefan2
> > Date: Thu Aug 6 12:54:36 2015
> > New Revision: 1694489
> >
> > URL: http://svn.apache.org/r1694489
> > Log:
> > Merge the 1.10-cache-improvements development branch to trunk and remove
> > the BRANCH-README file. There were no conflicts.
> >
> > Changes brought by this branch:
> > - Introduce a NULL cache implementation to unify / simplify cache usage
> > - Space and runtime efficiency improvements to the recently introduced
> > full key comparison in membuffer caches.
> >
> Stefan,
>
> Could you please write full log message for this change as required by
> our community guide?
>
This has been discussed before on more than one occasion.
The latest instance that I could find starts here:
http://mail-archives.apache.org/mod_mbox/subversion-dev/201501.mbox/%3CCABw-3YetbY57PngpBGnVbAMkApMd-v4ZGzsAfHe20ZYWi2i%2BTA%40mail.gmail.com%3E
Bottom line: 'svn log -g -r1694489' if you really need to know
the details. For convenience, there is a short summary in the
commit message. This is something people had asked for in
the past.
> Also after this change fs-test.exe fails with assertion in Windows
> 32-bit Debug build:
> [[
> START: fs-test.exe
> Assertion failed: !group || !group->header.used, file
> ..\..\..\subversion\libsvn_subr\cache-membuffer.c, line 1031
> ]]
>
This one is odd because the merge did not touch anything
obviously linked to spare bucket handling. Could be a long-
standing issue.
I failed to reproduce it with a 32 bit debug build on Win64.
Does it fail for you in serial execution mode? Is it deterministic
in that case? Does the attached patch prevent it?
I'm also getting random assertion like this in other tests.
>
That would be expected as this is low-level functionality
which will be used whenever you access repo data.
This commit mixes different unrelated changes, so it's very hard to
> find which part caused assertion.
>
You might try a bisection on the former 1.10-cache-improvements
branch, starting with HEAD (to see whether the branch
or the combination of trunk & branch is to blame).
-- Stefan^2.
Re: svn commit: r1694489 [1/2] - in /subversion/trunk: ./
subversion/include/private/ subversion/libsvn_fs_fs/ subversion/libsvn_fs_x/
subversion/libsvn_subr/ subversion/tests/libsvn_subr/
Posted by Ivan Zhakov <iv...@visualsvn.com>.
On 6 August 2015 at 15:54, <st...@apache.org> wrote:
> Author: stefan2
> Date: Thu Aug 6 12:54:36 2015
> New Revision: 1694489
>
> URL: http://svn.apache.org/r1694489
> Log:
> Merge the 1.10-cache-improvements development branch to trunk and remove
> the BRANCH-README file. There were no conflicts.
>
> Changes brought by this branch:
> - Introduce a NULL cache implementation to unify / simplify cache usage
> - Space and runtime efficiency improvements to the recently introduced
> full key comparison in membuffer caches.
>
Stefan,
Could you please write full log message for this change as required by
our community guide?
Also after this change fs-test.exe fails with assertion in Windows
32-bit Debug build:
[[
START: fs-test.exe
Assertion failed: !group || !group->header.used, file
..\..\..\subversion\libsvn_subr\cache-membuffer.c, line 1031
]]
I'm also getting random assertion like this in other tests.
This commit mixes different unrelated changes, so it's very hard to
find which part caused assertion.
--
Ivan Zhakov