[Keith Bostic ] Re: [#6254] log optimization for record append

[cm...@collab.net]

For those who might be interesed, remember long ago when Greg Stein
switched our Berkeley `strings' table over to a duplicate-key
database?  The reason was that we were getting exponential log growth
and slow access rates using the append-another-chunk-of-data-to-
an-existing-row approach that we *had* been using.

Well, here's some insider info on why we were seeing exponential
growth for append operations (something we felt we shouldn't be
seeing).

------- Start of forwarded message -------
Date: Thu, 11 Jul 2002 21:41:17 -0400 (EDT)
From: Keith Bostic <bo...@sleepycat.com>
Message-Id: <20...@abyssinian.sleepycat.com>
To: cmpilato@collab.net
Subject: Re: [#6254] log optimization for record append
Cc: striker@apache.org, support@sleepycat.com

> I'll make an attempt.  
>
> Basically we were populating a single row's value in chunks -- a
> looping codepath that wrote so many bytes of data to the end of what
> had already been written.  If memory serves, this was done with
> regular DB->put calls using a DBT with the DB_DBT_PARTIAL flag.  So,
> for example, an unrolled loop might look like:

I think we understand now -- Berkeley DB does not do any kind
of prefix or suffix logging optimizations for overflow records,
that is, records too large to fit on the standard Btree pages.

So, yes, I agree this scenario would result in enormous log
files.

Thank you for explaining this one to us!

Regards,
--keith

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Keith Bostic
Sleepycat Software Inc.		bostic@sleepycat.com
118 Tower Rd.			+1-781-259-3139
Lincoln, MA 01773		http://www.sleepycat.com

------- End of forwarded message -------

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org