You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Daniel Berlin <da...@dberlin.org> on 2002/02/04 16:46:45 UTC
[PATCH]: Automatic log file removal
Gee, it looked formatted okay in emacs (i used svn-dev.el and all)
I blame something else.
Anyway, this patch adds a thread that auto-removes log files every minute.
This is the right speed for my use, which currently consists of lots of
large cvs repository conversions (which are repeated commits, rather than
imports).
It's more the number of revisions than the size of the repository. In
this case, it's a 16 meg cvs repository, but it's got 2056 seperate
"revisions" in it.
This generates 99 log files (989 meg):
989MB 754KB 745B Log bytes written (772841 bytes).
0 Log bytes written since last checkpoint.
40500 Total log file writes.
899 Total log file write due to overflow.
40021 Total log file flushes.
99 Current log file number.
10388385 Current log file offset.
99 On-disk log file number.
10388385 On-disk log file offset.
1 Max commits in a log flush.
1 Min commits in a log flush.
36435 Number of log flushes containing a transaction commit.
The example i took it from in the berkeley docs had the sleep time at 5
minutes, just for reference.
This patch also ups the log file buffer from 32k to 256k to improve
throughput. This was based on statistics showing it was impeding
performance somewhat. With it at 256k, we don't overflow before a commit
anywhere near as often (the above is from a 256k log file buffer, a 32k
would show that we were consistently filling the log file buffer multiple
times per commit).
The removal thread will exit itself when the fs->env variable it was
passed the address to goes NULL, or when it can't remove a log file, or
the log_archive command returns an error.
The only thing changed from the version in the manual is the timeout, the
formatting, and the check to see when the fs->env we were passed goes
NULL.
I've assumed the berkeley guys know what they are doing when they wrote
the example.
:)
I think this enhancement might have been on the bite-sized task list, i'm
not sure.
Index: ./fs.c
===================================================================
--- ./fs.c
+++ ./fs.c Mon Feb 4 11:06:08 2002
@@ -38,6 +38,42 @@
#include "svn_private_config.h"
+static void *
+logfile_thread (void *arg)
+{
+ DB_ENV **argenv;
+ DB_ENV *dbenv;
+ int ret;
+ char **begin, **list;
+ argenv = arg;
+ dbenv = *argenv;
+
+ /* Check once every minute. */
+ for (;; sleep(60))
+ {
+ if (!*argenv)
+ pthread_exit(NULL);
+ /* Get the list of log files. */
+ if ((ret = dbenv->log_archive(dbenv, &list, DB_ARCH_ABS)) != 0)
+ {
+ dbenv->err(dbenv, ret, "DB_ENV->log_archive");
+ pthread_exit (NULL);
+ }
+
+ /* Remove the log files. */
+ if (list != NULL)
+ {
+ for (begin = list; *list != NULL; ++list)
+ if ((ret = remove(*list)) != 0)
+ {
+ dbenv->err(dbenv, ret, "remove %s", *list);
+ pthread_exit (NULL);
+ }
+ free (begin);
+ }
+ }
+}
+
/* Checking for return values, and reporting errors. */
@@ -355,7 +393,11 @@
from those participating in the deadlock. */
SVN_ERR (DB_WRAP (fs, "setting deadlock detection policy",
fs->env->set_lk_detect (fs->env, DB_LOCK_RANDOM)));
-
+ /* For our purposes, 32k is too small of a log file buffer.
+ Kick it up to 256k to increase throughput. */
+ SVN_ERR (DB_WRAP (fs, "setting log file buffer size",
+ fs->env->set_lg_bsize (fs->env, 256 * 1024)));
+
return SVN_NO_ERROR;
}
@@ -441,8 +483,9 @@
svn_error_t *
svn_fs_open_berkeley (svn_fs_t *fs, const char *path)
{
+ pthread_t ptid;
svn_error_t *svn_err;
-
+ int ret;
SVN_ERR (check_already_open (fs));
/* Initialize paths. */
@@ -459,8 +502,12 @@
| DB_INIT_MPOOL
| DB_INIT_TXN),
0666));
+ if (svn_err) goto error;
+ ret = pthread_create (&ptid, NULL, logfile_thread, (void *)(&fs->env));
+ if (ret != 0)
+ svn_err = svn_error_createf (SVN_ERR_BERKELEY_DB, 0, 0, fs->pool, "spawning log file removal thread");
if (svn_err) goto error;
-
+
/* Open the various databases. */
svn_err = DB_WRAP (fs, "opening `nodes' table",
svn_fs__open_nodes_table (&fs->nodes, fs->env, 0));
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Daniel, I applied this portion of your patch, thanks for investigating
this. If you ever feel like posting all your stats on various log
buffer sizes, that would be interesting, but we needn't wait for that.
The patch isn't likely to do any harm, and you reported improvement.
> Index: ./fs.c
> ===================================================================
> --- ./fs.c
> +++ ./fs.c Mon Feb 4 11:06:08 2002
> @@ -355,7 +393,11 @@
> from those participating in the deadlock. */
> SVN_ERR (DB_WRAP (fs, "setting deadlock detection policy",
> fs->env->set_lk_detect (fs->env, DB_LOCK_RANDOM)));
> -
> + /* For our purposes, 32k is too small of a log file buffer.
> + Kick it up to 256k to increase throughput. */
> + SVN_ERR (DB_WRAP (fs, "setting log file buffer size",
> + fs->env->set_lg_bsize (fs->env, 256 * 1024)));
> +
> return SVN_NO_ERROR;
> }
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Albert Chin <su...@thewrittenword.com>.
On Mon, Feb 04, 2002 at 11:46:45AM -0500, Daniel Berlin wrote:
> Gee, it looked formatted okay in emacs (i used svn-dev.el and all)
> I blame something else.
>
> Anyway, this patch adds a thread that auto-removes log files every minute.
> This is the right speed for my use, which currently consists of lots of
> large cvs repository conversions (which are repeated commits, rather than
> imports).
Would it be good enough to run log_archive() whenever the database is
closed and then remove() the results?
--
albert chin (china@thewrittenword.com)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Daniel, thanks so much! This is the sort of work Subversion really
needs on the repository side...
We'll start looking at these asap; for future reference, it will help
if you can supply a log message for the change (see the HACKING file
for guidelines on this and other stuff).
Thanks,
-Karl
Daniel Berlin <da...@dberlin.org> writes:
> Gee, it looked formatted okay in emacs (i used svn-dev.el and all)
> I blame something else.
>
> Anyway, this patch adds a thread that auto-removes log files every minute.
> This is the right speed for my use, which currently consists of lots of
> large cvs repository conversions (which are repeated commits, rather than
> imports).
>
> It's more the number of revisions than the size of the repository. In
> this case, it's a 16 meg cvs repository, but it's got 2056 seperate
> "revisions" in it.
> This generates 99 log files (989 meg):
>
> 989MB 754KB 745B Log bytes written (772841 bytes).
> 0 Log bytes written since last checkpoint.
> 40500 Total log file writes.
> 899 Total log file write due to overflow.
> 40021 Total log file flushes.
> 99 Current log file number.
> 10388385 Current log file offset.
> 99 On-disk log file number.
> 10388385 On-disk log file offset.
> 1 Max commits in a log flush.
> 1 Min commits in a log flush.
> 36435 Number of log flushes containing a transaction commit.
>
>
> The example i took it from in the berkeley docs had the sleep time at 5
> minutes, just for reference.
>
> This patch also ups the log file buffer from 32k to 256k to improve
> throughput. This was based on statistics showing it was impeding
> performance somewhat. With it at 256k, we don't overflow before a commit
> anywhere near as often (the above is from a 256k log file buffer, a 32k
> would show that we were consistently filling the log file buffer multiple
> times per commit).
>
> The removal thread will exit itself when the fs->env variable it was
> passed the address to goes NULL, or when it can't remove a log file, or
> the log_archive command returns an error.
>
> The only thing changed from the version in the manual is the timeout, the
> formatting, and the check to see when the fs->env we were passed goes
> NULL.
>
> I've assumed the berkeley guys know what they are doing when they wrote
> the example.
> :)
>
>
> I think this enhancement might have been on the bite-sized task list, i'm
> not sure.
>
>
> Index: ./fs.c
> ===================================================================
> --- ./fs.c
> +++ ./fs.c Mon Feb 4 11:06:08 2002
> @@ -38,6 +38,42 @@
> #include "svn_private_config.h"
>
>
> +static void *
> +logfile_thread (void *arg)
> +{
> + DB_ENV **argenv;
> + DB_ENV *dbenv;
> + int ret;
> + char **begin, **list;
> + argenv = arg;
> + dbenv = *argenv;
> +
> + /* Check once every minute. */
> + for (;; sleep(60))
> + {
> + if (!*argenv)
> + pthread_exit(NULL);
> + /* Get the list of log files. */
> + if ((ret = dbenv->log_archive(dbenv, &list, DB_ARCH_ABS)) != 0)
> + {
> + dbenv->err(dbenv, ret, "DB_ENV->log_archive");
> + pthread_exit (NULL);
> + }
> +
> + /* Remove the log files. */
> + if (list != NULL)
> + {
> + for (begin = list; *list != NULL; ++list)
> + if ((ret = remove(*list)) != 0)
> + {
> + dbenv->err(dbenv, ret, "remove %s", *list);
> + pthread_exit (NULL);
> + }
> + free (begin);
> + }
> + }
> +}
> +
> /* Checking for return values, and reporting errors. */
>
>
> @@ -355,7 +393,11 @@
> from those participating in the deadlock. */
> SVN_ERR (DB_WRAP (fs, "setting deadlock detection policy",
> fs->env->set_lk_detect (fs->env, DB_LOCK_RANDOM)));
> -
> + /* For our purposes, 32k is too small of a log file buffer.
> + Kick it up to 256k to increase throughput. */
> + SVN_ERR (DB_WRAP (fs, "setting log file buffer size",
> + fs->env->set_lg_bsize (fs->env, 256 * 1024)));
> +
> return SVN_NO_ERROR;
> }
>
> @@ -441,8 +483,9 @@
> svn_error_t *
> svn_fs_open_berkeley (svn_fs_t *fs, const char *path)
> {
> + pthread_t ptid;
> svn_error_t *svn_err;
> -
> + int ret;
> SVN_ERR (check_already_open (fs));
>
> /* Initialize paths. */
> @@ -459,8 +502,12 @@
> | DB_INIT_MPOOL
> | DB_INIT_TXN),
> 0666));
> + if (svn_err) goto error;
> + ret = pthread_create (&ptid, NULL, logfile_thread, (void *)(&fs->env));
> + if (ret != 0)
> + svn_err = svn_error_createf (SVN_ERR_BERKELEY_DB, 0, 0, fs->pool, "spawning log file removal thread");
> if (svn_err) goto error;
> -
> +
> /* Open the various databases. */
> svn_err = DB_WRAP (fs, "opening `nodes' table",
> svn_fs__open_nodes_table (&fs->nodes, fs->env, 0));
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Feb 05, 2002 at 11:33:09AM -0600, Albert Chin wrote:
> On Mon, Feb 04, 2002 at 11:46:45AM -0500, Daniel Berlin wrote:
> > Anyway, this patch adds a thread that auto-removes log files every minute.
> > This is the right speed for my use, which currently consists of lots of
> > large cvs repository conversions (which are repeated commits, rather than
> > imports).
>
> BTW, HP-UX 10.20 doesn't support POSIX threads so the pthread_create()
> will fail. What do we do for non-thread-aware platforms?
>
> For IRIX 6.2, you need a patch to enable threads so out of the box it
> probably won't work there either.
The original patch was (gently) rejected due to its use of threads for
exactly this reason.
We're (going to be) using an external script to clean the log files.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Daniel Berlin <da...@dberlin.org>.
On Tue, 5 Feb 2002, Albert Chin wrote:
> On Mon, Feb 04, 2002 at 11:46:45AM -0500, Daniel Berlin wrote:
> > Anyway, this patch adds a thread that auto-removes log files every minute.
> > This is the right speed for my use, which currently consists of lots of
> > large cvs repository conversions (which are repeated commits, rather than
> > imports).
>
> BTW, HP-UX 10.20 doesn't support POSIX threads so the pthread_create()
> will fail. What do we do for non-thread-aware platforms?
>
I sent a follow up patch that uses apr threads, which should be mappable
on HP-UX's dce thread interface.
> For IRIX 6.2, you need a patch to enable threads so out of the box it
> probably won't work there either.
>
>
IRIX 4, 5, and 6 implement setcontext/getcontext/makecontext/swapcontext,
even if only 6.3+ include pthreads out of the box
You could use these to implement user level threads if you have to.
But that's way overboard, in any case.
--Dan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Albert Chin <su...@thewrittenword.com>.
On Mon, Feb 04, 2002 at 11:46:45AM -0500, Daniel Berlin wrote:
> Anyway, this patch adds a thread that auto-removes log files every minute.
> This is the right speed for my use, which currently consists of lots of
> large cvs repository conversions (which are repeated commits, rather than
> imports).
BTW, HP-UX 10.20 doesn't support POSIX threads so the pthread_create()
will fail. What do we do for non-thread-aware platforms?
For IRIX 6.2, you need a patch to enable threads so out of the box it
probably won't work there either.
--
albert chin (china@thewrittenword.com)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Greg Stein <gs...@lyra.org>.
On Mon, Feb 04, 2002 at 03:12:02PM -0500, Daniel Berlin wrote:
> On Mon, 4 Feb 2002, Greg Stein wrote:
>...
> > 3) Apache opens/closes the FS on each request;
> Really?
No. I'm lying. ;-)
Yes, really.
>...
> > this thread would barely get
> > started before it died; even worse, it would introduce context switching
> > on every request, when we opened the repos
>
> This could be fixed easily, by having the log file archiver open the db
> itself anyway when the module starts.
> etc.
Yup. That's why I prefaced this part with "needs more work" :-) There is an
Apache hook called "child_init", IIRC, which would do this. However, we're
really want it as a post-config hook to start the reaper in just one
process.
I like Mike's suggestion of just putting this into the post-commit hook.
But even that may not be sufficient. When somebody does an "svn update", we
construct an SVN transaction. I bet that transaction gets tossed into the
log files.
[ an easy test is to just issue 1000 updates against a server and see if the
logs grow ]
>... [ excellent explanation ] ...
> Keeping every log file by default is solving a problem that is already
> solved in multiple ways.
Good points all. Okay.. you're right: we should toss them by default, and
allow an administrator to keep them with a bit of work.
>...
> keeping them for some time interval. But requiring a separate script just
> adds another step most people have to take to set up subversion.
Right. I would hope that we can refine our system to make it easy to set up.
We're still pre-alpha right now, after all :-)
I don't believe our server will ever be quite as easy as CVS, though. Some
of our capabilities come at a cost (threaded, high capacity server; robust
database; etc). But we can endeavor to make it as simple as possible.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Branko Čibej <br...@xbc.nu>.
Greg Stein wrote:
>4) people might want to *keep* those logs; they can be used to replay the
> database back to a fixed point, or they can be shipped off to another
> machine to replicate the database (thanks to Kirby for these ideas)
>
Not just that -- storing logs as they're freed gives you a hot
incremental backup of the database, which is probably better for
production environments than copying the whole repository on every
commit. :-)
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
cmpilato@collab.net writes:
> So give it to them via the default post-commit hook script. Have the
> default post-commit hook do the removal of the archivable logs.
+1
I think this is the best solution so far, because it defaults to the
space-saving way, while giving administrators a clear
point-of-tweakness to go to if they want to change the behavior (for
example, instead of rm'ing the log files, it could back them up to
somewhere else!)
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by cm...@collab.net.
Karl Fogel <kf...@newton.ch.collab.net> writes:
> IMHO, what most sites want is a repository that behaves well
> space-wise, which they're accustomed to from CVS. But a repository
> administrator should have the option of turning this off, in case they
> want to do the stuff you talk about above.
So give it to them via the default post-commit hook script. Have the
default post-commit hook do the removal of the archivable logs.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Greg Stein <gs...@lyra.org> writes:
> 4) people might want to *keep* those logs; they can be used to replay the
> database back to a fixed point, or they can be shipped off to another
> machine to replicate the database (thanks to Kirby for these ideas)
The question is, what should the default behavior be? Grow until
admin decides to clean, or be small unless/until the admin requests
log preservation?
IMHO, what most sites want is a repository that behaves well
space-wise, which they're accustomed to from CVS. But a repository
administrator should have the option of turning this off, in case they
want to do the stuff you talk about above.
Whether Daniel's approach is the best one, I don't know (I'm more
interested in the part of his patch that ups log file buffer to 256k).
But I don't think that having repositories keep old log files around
is a behavior out-of-the-box Subversion should have.
Either way, +1 on having a log-killer.sh (or more general
clean-repos.sh) script in tools/.
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Daniel Berlin <da...@dberlin.org>.
On Mon, 4 Feb 2002, Greg Stein wrote:
> On Mon, Feb 04, 2002 at 12:02:44PM -0500, Daniel Berlin wrote:
> > Of course, I really meant this patch, instead, which uses the apr_thread
> > functions. :)
>
> I don't believe the server should be doing this, and for a couple reasons:
>
> 1) threads are not always available
....
> 2) there is potentially more work than this since Apache can open the FS
> multiple times (from different processes); we'd want to ensure there is
> only one "log killer" running for any repository
> 3) Apache opens/closes the FS on each request;
Really?
Eek.
> this thread would barely get
> started before it died; even worse, it would introduce context switching
> on every request, when we opened the repos
This could be fixed easily, by having the log file archiver open the db
itself anyway when the module starts.
etc.
> 4) people might want to *keep* those logs; they can be used to replay the
> database back to a fixed point, or they can be shipped off to another
> machine to replicate the database (thanks to Kirby for these ideas)
Except, it's really not as feasible as it sounds to keep the logs on the
same machine, due to disk space issues (they really just add up too
quickly).
Replication is your answer here, and it doesn't require keeping the log
files on your main machine.
Why?
Because you can set up a replicated client that is a log file only client,
specifically for this purpose.
People who need to keep log files in case a disk blows up are *much* more
likely to do this, and simply have this machine archive the log files
somewhere, than to risk running out of disk space on an important server
that may be doing other tasks.
In fact, specifically, this is one of the reasons replication was added
to Berkeley DB.
The only reason to keep absolutely every log file is if you can't lose a
second worth of work after your disk blows up completely.
And, BTW, it also assumes you aren't in some kind of redundant array where
a disk blowing up is just fine, which, if you can't afford a seconds worth
of loss, you probably already have.
Keeping every log file by default is solving a problem that is already
solved in multiple ways.
It's something users are going to complain about.
Especially since users include more than just people who want to use the
server portion. Their will be plenty of people who use it for local
repositories who will be completely turned off by this behavior.
> I'm perfectly happy to put a log-killer.sh script into /trunk/tools that
> people can run (feed it a list of repositories to keep clean).
> Administrators can use that script if they choose.
As you wish.
I still think it's the wrong answer for the majority of people.
Its one thing to have it configurable, defaulting to not keeping them or
keeping them for some time interval. But requiring a separate script just
adds another step most people have to take to set up subversion.
>
> Cheers,
> -g
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Greg Stein <gs...@lyra.org>.
On Mon, Feb 04, 2002 at 12:02:44PM -0500, Daniel Berlin wrote:
> Of course, I really meant this patch, instead, which uses the apr_thread
> functions. :)
I don't believe the server should be doing this, and for a couple reasons:
1) threads are not always available
2) there is potentially more work than this since Apache can open the FS
multiple times (from different processes); we'd want to ensure there is
only one "log killer" running for any repository
3) Apache opens/closes the FS on each request; this thread would barely get
started before it died; even worse, it would introduce context switching
on every request, when we opened the repos
4) people might want to *keep* those logs; they can be used to replay the
database back to a fixed point, or they can be shipped off to another
machine to replicate the database (thanks to Kirby for these ideas)
I'm perfectly happy to put a log-killer.sh script into /trunk/tools that
people can run (feed it a list of repositories to keep clean).
Administrators can use that script if they choose.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: [PATCH]: Automatic log file removal
Posted by Daniel Berlin <da...@dberlin.org>.
Of course, I really meant this patch, instead, which uses the apr_thread
functions. :)
Index: ./fs.c
===================================================================
--- ./fs.c
+++ ./fs.c Mon Feb 4 11:58:26 2002
@@ -38,6 +38,42 @@
#include "svn_private_config.h"
+static void *
+logfile_thread (apr_thread_t *thd, void *arg)
+{
+ DB_ENV **argenv;
+ DB_ENV *dbenv;
+ int ret;
+ char **begin, **list;
+ argenv = arg;
+ dbenv = *argenv;
+
+ /* Check once every minute. */
+ for (;; sleep(60))
+ {
+ if (!*argenv)
+ apr_thread_exit(thd, 1);
+ /* Get the list of log files. */
+ if ((ret = dbenv->log_archive(dbenv, &list, DB_ARCH_ABS)) != 0)
+ {
+ dbenv->err(dbenv, ret, "DB_ENV->log_archive");
+ apr_thread_exit(thd, 1);
+ }
+
+ /* Remove the log files. */
+ if (list != NULL)
+ {
+ for (begin = list; *list != NULL; ++list)
+ if ((ret = remove(*list)) != 0)
+ {
+ dbenv->err(dbenv, ret, "remove %s", *list);
+ apr_thread_exit(thd, 1);
+ }
+ free (begin);
+ }
+ }
+}
+
/* Checking for return values, and reporting errors. */
@@ -355,7 +391,11 @@
from those participating in the deadlock. */
SVN_ERR (DB_WRAP (fs, "setting deadlock detection policy",
fs->env->set_lk_detect (fs->env, DB_LOCK_RANDOM)));
-
+ /* For our purposes, 32k is too small of a log file buffer.
+ Kick it up to 256k to increase throughput. */
+ SVN_ERR (DB_WRAP (fs, "setting log file buffer size",
+ fs->env->set_lg_bsize (fs->env, 256 * 1024)));
+
return SVN_NO_ERROR;
}
@@ -441,8 +481,9 @@
svn_error_t *
svn_fs_open_berkeley (svn_fs_t *fs, const char *path)
{
+ apr_thread_t *tid;
svn_error_t *svn_err;
-
+ int ret;
SVN_ERR (check_already_open (fs));
/* Initialize paths. */
@@ -459,8 +500,12 @@
| DB_INIT_MPOOL
| DB_INIT_TXN),
0666));
+ if (svn_err) goto error;
+ ret = apr_thread_create (&tid, NULL, logfile_thread, (void *)(&fs->env), fs->pool);
+ if (ret != 0)
+ svn_err = svn_error_createf (SVN_ERR_BERKELEY_DB, 0, 0, fs->pool, "spawning log file removal thread");
if (svn_err) goto error;
-
+
/* Open the various databases. */
svn_err = DB_WRAP (fs, "opening `nodes' table",
svn_fs__open_nodes_table (&fs->nodes, fs->env, 0));
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org