You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@subversion.apache.org by "Kevin D. Bond" <kd...@korl.com> on 2002/10/14 00:29:09 UTC

Re: need advice building svn 0.14.3 under OS X

Please note that there is a bug with MUTEXs in Berkeley DB 4.0 under 
Mac OS X.  In fact I had problems with everything from 3.3 to 4.0.  4.1 
includes a fix, you can get the mutex change from 4.1 and include it in 
4.0.  If unfixed, this bug will cause database corruption during 
concurrent writes to the database.

Otherwise, most of the OS X problems are changes in the gcc.  You could 
use gcc_select to switch back to 2.95 to try the configure and build.

-kevin

On Friday, September 27, 2002, at 01:24  PM, Hamilton Link wrote:

> Thanks for your suggestion, Justin. I rearranged the system as you
> suggested, and am still getting the same error.
>
> The only place I can find "-install_name" is in svn-build/libtool, and
> hey, there's -dynamiclib too, presubably being left out of the result:
>
> archive_cmds="\$nonopt \$(test x\$module = xyes && echo -bundle || echo
> -dynamiclib) \$allow_undefined_flag -o \$lib \$libobjs
> \$deplibs\$linker_flags -install_name \$rpath/\$soname \$verstring"
>
> Does this line indicate to anyone why -dynamiclib isn't being included?
> Thanks.
>
>
>
> Here's what I did to address your concerns about neon being in
> /usr/local:
>
> I removed db 4.1 and made sure I had db 4.0 properly installed.
>
> I rebuilt httpd with the following config:
>
> ./configure --enable-dav --enable-so --prefix=/usr/local/apache2
> --with-dbm=db4 --with-berkeley-db=/usr/local/BerkeleyDB.4.0
> --enable-maintainer-mode
>
> I don't remember if I explicitly installed expat by hand but I probably
> did (I certainly have expat stuff in /usr/local, which because
> "apr-config --libs" just returns "-lm" under OS X is where I seem to
> need to put it).
>
> I reinstalled neon into /usr/local/neon (and scrubbed /usr/local and
> it's subdirs of all the files that were in /usr/local/neon). Here's how
> I configured neon:
>
> ./configure --prefix=/usr/local/neon --with-apr=/usr/local/apache2
> --with-apr-util=/usr/local/apache2 --with-expat --enable-shared
>
> Then I cleared out my svn-build directory and tried building that 
> again,
> I tried a couple of different configurations...
>
> 1)
>
> ../svn/configure --with-apr=/usr/local/apache2
> --with-apr-util=/usr/local/apache2
> --with-berkeley-db=/usr/local/BerkeleyDB.4.0 
> --with-neon=/usr/local/neon
> --with-apxs=/usr/local/apache2/bin/apxs --enable-maintainer-mode
>
> In this case, I think because "apr-config --libs" just returns "-lm",
> the make process dies complaining that
> /Users/helink/subversion/httpd-2.0/usr/local/apache2/lib doesn't exist.
> I guess this is an issue with libtool? Anyway it doesn't get very far.
>
> 2)
>
> ../svn/configure --with-neon=/usr/local/neon
> --with-apr=/Users/helink/subversion/httpd-2.0/srclib/apr
> --with-apr-util=/Users/helink/subversion/httpd-2.0/srclib/apr-util
> --with-berkeley-db=/usr/local/BerkeleyDB.4.0
> --with-apxs=/usr/local/apache2/bin/apxs --enable-maintainer-mode
>
> This time it could find everything, but gave me the same error as
> before:
>
> "cd subversion/mod_dav_svn && /bin/sh /Users/helink/subversion/svn-
> build/libtool --silent --mode=link  gcc -DDARWIN
> -DSIGPROCMASK_SETS_THREAD_MASK -no-cpp-precomp  -g -Wall
> -Wmissing-prototypes -Wstrict-prototypes -Wmissing-declarations
> -Wpointer-arith -Wwrite-strings -Wshadow -DSVN_DEBUG -DAP_DEBUG
> -I../svn/subversion/include -I. -I/usr/local/include/neon
> -I/usr/local/BerkeleyDB.4.0/include  -I/usr/local/apache2/include
> -I/usr/local/apache2/include -I/usr/local/BerkeleyDB.4.0/include
> -I/usr/local/include -L/usr/local/BerkeleyDB.4.0/lib -L/usr/local/lib
> -rpath /usr/local/apache2/modules -o mod_dav_svn.la -avoid-version
> -module activity.lo deadprops.lo liveprops.lo log.lo merge.lo
> mod_dav_svn.lo repos.lo update.lo util.lo version.lo
> ../../subversion/libsvn_repos/libsvn_repos-1.la
> ../../subversion/libsvn_fs/libsvn_fs-1.la
> ../../subversion/libsvn_delta/libsvn_delta-1.la
> ../../subversion/libsvn_subr/libsvn_subr-1.la
>
> gcc: -install_name only allowed with -dynamiclib
> make: *** [subversion/mod_dav_svn/mod_dav_svn.la] Error 1"
>
> thanks,
> hamilton
>
> Justin Erenkrantz wrote:
>>
>> On Thu, Sep 26, 2002 at 04:36:22PM -0600, Hamilton Link wrote:
>>> ../svn/configure --with-apr=/usr/local/apache2 --with-apr-
>>> util=/usr/local/apache2 --with-berkeley-db=/usr/local/BerkeleyDB.4.1
>>> --with-neon=/usr/local --with-apxs=/usr/local/apache2/bin/apxs
>>> --enable-maintainer-mode
>>
>> Try installing neon to a different directory than /usr/local.
>> It confuses the heck out of the compiler as it messes up the
>> ordering of the header files (/usr/local is an implicit directory,
>> by reordering them, you may subtly break things).  I'd try that and
>> see what happens.
>>
>> Note that berkeley db-4.1 isn't supported by SVN.  It shouldn't
>> have even compiled against it since the API changed - it's likely
>> you have an older version stashed in /usr/local.  So, I bet the
>> inclusion of /usr/local is breaking some things.  -- justin
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1 WAS: RE: need advice building svn 0.14.3 under OS X

Posted by Olaf Hering <ol...@suse.de>.

begin  On Mon, Oct 14, Sander Striker wrote:

> Any reasons why we shouldn't move to 4.1 and stick with 4.0?

Is there a way to use gdbm instead of db?


Gruss Olaf

-- 
 $ man Notes

BUGS
       You need http://www.sauerstoff-laden.de/ soon...

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Garrett Rooney <ro...@electricjellyfish.net>.

Justin Erenkrantz wrote:
> --On Monday, October 14, 2002 3:48 AM -0700 Greg Stein <gs...@lyra.org> 
> wrote:
> 
>> DB4.0 isn't all that common yet, let alone 4.1.
>>
>> IMO, the ideal situation is to allow 4.0 *or* 4.1, and tweak our
>> calls accordingly. In addition, if you're on MacOS X, then require
>> 4.1.
>>
>> So... +0 on requiring 4.1, +1 if coded as above.
> 
> 
> BTW, the repositories seem to be forward-compatible - that is, 4.1 can 
> read 4.0 repositories without any user changes.
> 
> Nevertheless, I think we're seeing enough traffic that I'll try to take 
> the time to conditionally code up this patch so it can use both APIs.  
> But, it's going to be downright ugly.  -- justin

i'd just like to add another +1 to being able to work with 4.1.  the 
darwinports tree only has packages for 4.1, and that's kept me from 
having a subversion port that does ra_local.

if we can possibly be able to detect and work with both 4.0 and 4.1, 
that would be best, but we will need to make this jump eventually, so we 
might as well do it now.

-garrett


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Justin Erenkrantz <je...@apache.org>.

--On Monday, October 14, 2002 3:48 AM -0700 Greg Stein 
<gs...@lyra.org> wrote:

> DB4.0 isn't all that common yet, let alone 4.1.
>
> IMO, the ideal situation is to allow 4.0 *or* 4.1, and tweak our
> calls accordingly. In addition, if you're on MacOS X, then require
> 4.1.
>
> So... +0 on requiring 4.1, +1 if coded as above.

BTW, the repositories seem to be forward-compatible - that is, 4.1 
can read 4.0 repositories without any user changes.

Nevertheless, I think we're seeing enough traffic that I'll try to 
take the time to conditionally code up this patch so it can use both 
APIs.  But, it's going to be downright ugly.  -- justin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Zack Weinberg <za...@codesourcery.com>.

On Wed, Oct 16, 2002 at 11:27:02AM -0700, Justin Erenkrantz wrote:
> Well, some of the open calls do have their own transaction. 
> Therefore, any such macro would have to realize when a NULL txn is 
> passed that you have to then pass in DB_AUTO_COMMIT as well.
> 
> # if BDB_VERSION > 4001
> # define BDB_OPEN_ARGS(db, txn, cr) \
>  db, txn, #db, (cr ? (DB_CREAT | DB_EXCL) : 0) | \
>  (txn ? 0 : DB_AUTO_COMMIT), 0666
> # else
> # define BDB_OPEN_ARGS(db, txn, cr) \
>    db, #db, (cr ? (DB_CREAT | DB_EXCL) : 0), 0666
> # endif
> 
> This scores really low on my beauty meter.  -- justin

It is, however, prettier than having equivalent logic all over
libsvn_fs.

The question is whether compatibility with DB 4.0 is worth a certain
amount of ugliness -- I would say yes, because (as other people have
pointed out) 4.1 is not common yet.

zw

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Justin Erenkrantz <je...@apache.org>.

--On Wednesday, October 16, 2002 11:10 AM -0700 Zack Weinberg 
<za...@codesourcery.com> wrote:

> Can't you wrap things up in a compatibility header file?  Like this:
>
># define BDB_VERSION (DB_VERSION_MAJOR * 1000 + DB_VERSION_MINOR)
>
># if BDB_VERSION > 4001
># define BDB_OPEN_ARGS(db, cr) \
>    db, NULL, #db, (cr ? (DB_CREAT | DB_EXCL) : 0) | DB_AUTO_COMMIT,
> 0666
># else
># define BDB_OPEN_ARGS(db, cr) \
>    db, #db, (cr ? (DB_CREAT | DB_EXCL) : 0), 0666
># endif

Well, some of the open calls do have their own transaction. 
Therefore, any such macro would have to realize when a NULL txn is 
passed that you have to then pass in DB_AUTO_COMMIT as well.

# if BDB_VERSION > 4001
# define BDB_OPEN_ARGS(db, txn, cr) \
  db, txn, #db, (cr ? (DB_CREAT | DB_EXCL) : 0) | \
  (txn ? 0 : DB_AUTO_COMMIT), 0666
# else
# define BDB_OPEN_ARGS(db, txn, cr) \
    db, #db, (cr ? (DB_CREAT | DB_EXCL) : 0), 0666
# endif

This scores really low on my beauty meter.  -- justin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Zack Weinberg <za...@codesourcery.com>.

On Wed, Oct 16, 2002 at 10:01:09AM -0700, Justin Erenkrantz wrote:
> --On Wednesday, October 16, 2002 12:52 PM +0200 brane@xbc.nu wrote:
> 
> To give you an idea what *every* open call is going to look like, 
> here's the code to open the changes table and to maintain backwards 
> compatibility if you rely on the header files:
> 
> DB_ERR (changes->open (changes,
> #if defined(DB_VERSION_MAJOR) && (DB_VERSION_MAJOR == 4) && \
>    defined(DB_VERSION_MINOR) && (DB_VERSION_MINOR >= 1)
>                       NULL,
> #endif
>                        "changes", 0, DB_BTREE,
>                        (create ? (DB_CREATE | DB_EXCL) : 0)
> #if defined(DB_VERSION_MAJOR) && (DB_VERSION_MAJOR == 4) && \
>    defined(DB_VERSION_MINOR) && (DB_VERSION_MINOR >= 1)
>                       | DB_AUTO_COMMIT
> #endif
>                        , 0666));

Can't you wrap things up in a compatibility header file?  Like this:

#define BDB_VERSION (DB_VERSION_MAJOR * 1000 + DB_VERSION_MINOR)

#if BDB_VERSION > 4001
#define BDB_OPEN_ARGS(db, cr) \
   db, NULL, #db, (cr ? (DB_CREAT | DB_EXCL) : 0) | DB_AUTO_COMMIT, 0666
#else
#define BDB_OPEN_ARGS(db, cr) \
   db, #db, (cr ? (DB_CREAT | DB_EXCL) : 0), 0666
#endif

...

  DB_ERR (changes->open (BDB_OPEN_ARGS (changes, create)));

zw

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Justin Erenkrantz <je...@apache.org>.

--On Wednesday, October 16, 2002 8:26 PM +0200 
"=?UTF-8?B?QnJhbmtvIMSMaWJlag==?=" <br...@xbc.nu> wrote:

> Um, no, the _code_ has to remember the version numbers at build
> time, and check them again at runtime. That's not something you can
> do in autoconf, anyway.

But, that's not how we do it now.  Please take a look at
subversion/libsvn_fs/fs.c:62.  Those values come explicitly from 
autoconf not from the BDB headers.  This is why I said I think we 
need to have a SVN_FS_GOT_DB_* to parallel SV_FS_WANT_DB_*.  We could 
toss the SVN_FS_WANT_DB_* entirely and just hope that the includes 
are right so we can use the BDB header info.  But, ISTR that people 
have had problems with the BDB headers misreporting the version 
number (rather the compiler getting the wrong includes, but the 
linker links against the right library).

> I'm fairly sure I could come up with an even uglier hack. But I
> could also come up with something much cleaner, e.g.:

I think someone said before that we don't want to use functions to 
access BDB.  But, we're sort of tilting that way anyway...

Yes, we could create a DB layer here by abstracting all BDB calls.
Then, I'd say that we should just fix up APR-util's dbm API to fully 
support SVN and be done with it.  The biggest thing it'd need is 
transactional support as well as version-query support.

> /* Just assume we have the DB_VERSION_MAJOR and DB_VERSION_MINOR;
>    if we don't, we're using the wrong version of BDB anyway. */
># if (DB_VERSION_MAJOR == 4) && (DB_VERSION_MINOR >= 1)
># define SVN_HAS_DB_41 1
># else
># define SVN_HAS_DB_41 0
># endif
>
> extern const int svn_bdb_version_major;
> extern const int svn_bdb_version_minor;

Part of the problem is that there is no common header file that all 
of the BDB-using files include that makes sense to include this.  So, 
I guess we'd have to add a new file?  *sigh*

> Sure we don't. I for one don't believe that you should obfuscate C
> code just because you can

I guess my preference is to keep the code as close to where it is as 
used as possible.  And, I just don't have the time to go create a new 
DB abstraction layer...

> That's as may be, but I don't think that's a good enough reason to
> force people to install yet another nonstandard package, given how
> trivial it is to be compatible with both versions.

I don't think it's as trivial as it should have been.  This migration 
path is sucky.  Any change to the API like this should have been in a 
major version not a minor version.  *sigh*

> I'd be very surprised ifi it were a bug. Berkeley's structures are
> sure to have changed, and the environment files are just shared mem
> regions.

Then, is it always recommended to run 'db_recover -ve' whenever the 
BDB version changes?  I couldn't find anything along that line in 
their readmes.

I guess part of my concern is that if we *can* wedge the repos with 
4.0, but not with 4.1, why are we doing anyone a favor by supporting 
a known broken BDB version?  -- justin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by "Edward S. Marshall" <es...@logic.net>.

On Thu, 2002-10-17 at 14:14, Branko Čibej wrote:
> Not really. You need APR for Apache 2.0; every distro that comes with 
> that will have APR.

Not that it'll necessarily be usable. ;-) The Red Hat 8.0 packaging of
Apache 2.0 skipped out on instdso.sh, apr-config, and apu-config.
(There's a bugzilla entry open on it though, hopefully they'll have a
repackaged version soon. A real 1.0 of APR would be even better, so they
could break it out into a separate package and get more applications
making use of it...)

-- 
Edward S. Marshall <es...@logic.net>
http://esm.logic.net/

Re: Upgrade to db 4.1

Posted by Branko Čibej <br...@xbc.nu>.

Jeff Stuart wrote:

>On Wed, 2002-10-16 at 14:26, Branko Čibej wrote:
>  
>
>>That's as may be, but I don't think that's a good enough reason to force 
>>people to install yet another nonstandard package, given how trivial it 
>>is to be compatible with both versions.
>>    
>>
>
>Uhh... Subversion depends on APR which is DEFINITELY non standard. :)
>
Not really. You need APR for Apache 2.0; every distro that comes with 
that will have APR.

>Hell, subversion ITSELF is currently nonstandard. :)  While I'd probably
>grumble about having to upgrade to DB 4.1... If it fixes a lot of
>problems/possible problems, then I'd say GO for it!  (Note: ONLY so long
>as there's a conversion program if needed from 4.0 to 4.1.)
>
I'd _really_ like to see us support both. Maybe it's time to dust of my 
Azure Staff of Extended Hacking and beat on Justin's patch until it does 
what I want. :-)

>As far as I know, there's no "official" packaging of subversion yet.  
>
There's no 1.0 of Subversion, either, but we have packages for quite a 
few systems.

-- 
Brane Čibej   <br...@xbc.nu>   http://www.xbc.nu/brane/


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Dave Rolsky <au...@urth.org>.

On 17 Oct 2002, Jeff Stuart wrote:

> As far as I know, there's no "official" packaging of subversion yet.

Debian has packages for subversion, libsvn, libsvn-dev, python-subversion
bindings, libapr, libapr-dev, and libdb4.0 in unstable.  Some of those
packages are also in testing.


-dave

/*==================
www.urth.org
we await the New Sun
==================*/


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Jeff Stuart <js...@computer-city.net>.

On Wed, 2002-10-16 at 14:26, Branko Čibej wrote:
> That's as may be, but I don't think that's a good enough reason to force 
> people to install yet another nonstandard package, given how trivial it 
> is to be compatible with both versions.

Uhh... Subversion depends on APR which is DEFINITELY non standard. :) 
Hell, subversion ITSELF is currently nonstandard. :)  While I'd probably
grumble about having to upgrade to DB 4.1... If it fixes a lot of
problems/possible problems, then I'd say GO for it!  (Note: ONLY so long
as there's a conversion program if needed from 4.0 to 4.1.)

As far as I know, there's no "official" packaging of subversion yet.  

-- 
Jeff Stuart <js...@computer-city.net>

Re: Upgrade to db 4.1

Posted by Branko Čibej <br...@xbc.nu>.

Justin Erenkrantz wrote:

> --On Wednesday, October 16, 2002 12:52 PM +0200 brane@xbc.nu wrote:
>
>> We already have autocode that extracts the BDB version number;
>> we're checking for 4.0.14 right now, but there's no reason we
>> couldn't check for 4.1.x, too. And any part of the code that uses
>> BDB will pull in the BDB headers, so that you could ifdef on
>> DB_MAJOR and DB_MINOR where necessary. I don't think we need any
>> new or different autoconfigury at all.
>
>
> Nope, all it does is guarantee we have a minimum version - we don't 
> have macros to determine what version we have.  We have to also tweak 
> the run-time checks to ensure that we run against the version that we 
> *built* against not the minimum version we requested.  So, I believe 
> that means that the autoconf checks have to extract what we have at 
> configure-time.

Um, no, the _code_ has to remember the version numbers at build time, 
and check them again at runtime. That's not something you can do in 
autoconf, anyway.

> To give you an idea what *every* open call is going to look like, 
> here's the code to open the changes table and to maintain backwards 
> compatibility if you rely on the header files:
>
> DB_ERR (changes->open (changes,
> #if defined(DB_VERSION_MAJOR) && (DB_VERSION_MAJOR == 4) && \
>    defined(DB_VERSION_MINOR) && (DB_VERSION_MINOR >= 1)
>                       NULL,
> #endif
>                        "changes", 0, DB_BTREE,
>                        (create ? (DB_CREATE | DB_EXCL) : 0)
> #if defined(DB_VERSION_MAJOR) && (DB_VERSION_MAJOR == 4) && \
>    defined(DB_VERSION_MINOR) && (DB_VERSION_MINOR >= 1)
>                       | DB_AUTO_COMMIT
> #endif
>                        , 0666));

I'm fairly sure I could come up with an even uglier hack. But I could 
also come up with something much cleaner, e.g.:

/* Just assume we have the DB_VERSION_MAJOR and DB_VERSION_MINOR;
   if we don't, we're using the wrong version of BDB anyway. */
#if (DB_VERSION_MAJOR == 4) && (DB_VERSION_MINOR >= 1)
#define SVN_HAS_DB_41 1
#else
#define SVN_HAS_DB_41 0
#endif

extern const int svn_bdb_version_major;
extern const int svn_bdb_version_minor;

...

const int svn_bdb_version_major = DB_VERSION_MAJOR;
const int svn_bdb_version_minor = DB_VERSION_MINOR;

svn_error_t *svn_bdb_open(DB *db, const char *file, DBTYPE type,
                          u_int32_t flags, int mode)
{
  int major, minor;
  db_version(&major, &minor, NULL):
  if (major != svn_bdb_version_major || minor != svn_bdb_version_minor)
    return svn_error_create(/* something appropriate */);

#if SVN_HAS_DB_41
  DB_ERR (db->open(db, NULL, file, 0, type, flags | DB_AUTO_COMMIT, mode));
#else
  DB_ERR (return db->open(db, file, 0, type, flags, mode));
#endif
  return SVN_NO_ERROR;
}


> If you like that strategy and think it promotes clean code, well, we 
> don't see alike.

Sure we don't. I for one don't believe that you should obfuscate C code 
just because you can

> I think that's ugly and outweighs the few distributions that ship BDB 
> 4.0.  They'll be adding BDB 4.1 soon enough due to the reliability 
> enhancements... 

That's as may be, but I don't think that's a good enough reason to force 
people to install yet another nonstandard package, given how trivial it 
is to be compatible with both versions.

> As an aside, I had to run db_recover -ve (aka 'svnadmin recover') on 
> all my databases when upgrading to 4.1 - it seems that the databases 
> are forward-compatible, but not the environments.  I wonder if that's 
> a bug in BDB 4.1.  If so, perhaps we should drop a line to Keith and 
> see if they can resolve that. 

I'd be very surprised ifi it were a bug. Berkeley's structures are sure 
to have changed, and the environment files are just shared mem regions.

> But, now, I can't manage to wedge the repository with BDB 4.1.  Does 
> anybody have a sure-fire repro case for wedging repositories?  (I've 
> tried Ctrl+C, kill -9, etc.)  -- justin

That's very encouraging!

-- 
Brane Čibej   <br...@xbc.nu>   http://www.xbc.nu/brane/


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Noel Yap <ya...@yahoo.com>.

--- Philip Martin <ph...@codematters.co.uk> wrote:
> Justin Erenkrantz <je...@apache.org> writes:
> 
> > But, now, I can't manage to wedge the repository
> with BDB 4.1.  Does
> > anybody have a sure-fire repro case for wedging
> repositories?  (I've
> > tried Ctrl+C, kill -9, etc.)  -- justin
> 
> It wedges about nine times out of ten if I interrupt
> the stress.pl
> script during the "Transmitting file data" part of
> the first commit.
> 
> $ ../svn/tools/dev/stress.pl -c
> [stuff]
> Transmitting file data ...^C
> $ svnlook repostress/
> 
> 2002-10-16 23:38:51 +0100 (Wed, 16 Oct 2002)
> 0
> 
> It's hung.

I've consistently wedged (if I understand the meaning)
it by doing "svn add -R; svn ci" on a LARGE directory
structure, one of whose directories contains on the
order of 1000 files.  The funny thing is that the
transaction processes OK and I don't have to
db_recover.  OTOH, I had to dump the working
directory.  Come to think of it, maybe it was just in
the middle of updating the working directory info? 
Although, it was chugging for about 30 minutes before
I killed it.

HTH,
Noel

__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Philip Martin <ph...@codematters.co.uk>.

Justin Erenkrantz <je...@apache.org> writes:

> But, now, I can't manage to wedge the repository with BDB 4.1.  Does
> anybody have a sure-fire repro case for wedging repositories?  (I've
> tried Ctrl+C, kill -9, etc.)  -- justin

It wedges about nine times out of ten if I interrupt the stress.pl
script during the "Transmitting file data" part of the first commit.

$ ../svn/tools/dev/stress.pl -c
[stuff]
Transmitting file data ...^C
$ svnlook repostress/

2002-10-16 23:38:51 +0100 (Wed, 16 Oct 2002)
0

It's hung.


-- 
Philip Martin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Karl Fogel <kf...@newton.ch.collab.net>.

Nuutti Kotivuori <na...@iki.fi> writes:
> 'svnadmin dump' - Ctrl+C. Wedges 100% of the times for me.
> 
> Or, do you mean by wedging the same thing as I do - having all
> subsequent SVN commands hang when trying to access the repository?

I think he does, but the thing is, your recipe is a great way to make
subsequent commands wedge :-).

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Nuutti Kotivuori <na...@iki.fi>.

Justin Erenkrantz wrote:
> But, now, I can't manage to wedge the repository with BDB 4.1.  Does
> anybody have a sure-fire repro case for wedging repositories?  (I've
> tried Ctrl+C, kill -9, etc.)  -- justin

'svnadmin dump' - Ctrl+C. Wedges 100% of the times for me.

Or, do you mean by wedging the same thing as I do - having all
subsequent SVN commands hang when trying to access the repository?

-- Naked

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Justin Erenkrantz <je...@apache.org>.

--On Wednesday, October 16, 2002 12:52 PM +0200 brane@xbc.nu wrote:

> We already have autocode that extracts the BDB version number;
> we're checking for 4.0.14 right now, but there's no reason we
> couldn't check for 4.1.x, too. And any part of the code that uses
> BDB will pull in the BDB headers, so that you could ifdef on
> DB_MAJOR and DB_MINOR where necessary. I don't think we need any
> new or different autoconfigury at all.

Nope, all it does is guarantee we have a minimum version - we don't 
have macros to determine what version we have.  We have to also tweak 
the run-time checks to ensure that we run against the version that we 
*built* against not the minimum version we requested.  So, I believe 
that means that the autoconf checks have to extract what we have at 
configure-time.

To give you an idea what *every* open call is going to look like, 
here's the code to open the changes table and to maintain backwards 
compatibility if you rely on the header files:

DB_ERR (changes->open (changes,
#if defined(DB_VERSION_MAJOR) && (DB_VERSION_MAJOR == 4) && \
    defined(DB_VERSION_MINOR) && (DB_VERSION_MINOR >= 1)
                       NULL,
#endif
                        "changes", 0, DB_BTREE,
                        (create ? (DB_CREATE | DB_EXCL) : 0)
#if defined(DB_VERSION_MAJOR) && (DB_VERSION_MAJOR == 4) && \
    defined(DB_VERSION_MINOR) && (DB_VERSION_MINOR >= 1)
                       | DB_AUTO_COMMIT
#endif
                        , 0666));

If you like that strategy and think it promotes clean code, well, we 
don't see alike.  I think that's ugly and outweighs the few 
distributions that ship BDB 4.0.  They'll be adding BDB 4.1 soon 
enough due to the reliability enhancements...

As an aside, I had to run db_recover -ve (aka 'svnadmin recover') on 
all my databases when upgrading to 4.1 - it seems that the databases 
are forward-compatible, but not the environments.  I wonder if that's 
a bug in BDB 4.1.  If so, perhaps we should drop a line to Keith and 
see if they can resolve that.

But, now, I can't manage to wedge the repository with BDB 4.1.  Does 
anybody have a sure-fire repro case for wedging repositories?  (I've 
tried Ctrl+C, kill -9, etc.)  -- justin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by br...@xbc.nu.

Quoting Justin Erenkrantz <je...@apache.org>:

> --On Monday, October 14, 2002 3:48 AM -0700 Greg Stein 
> <gs...@lyra.org> wrote:
> 
> > DB4.0 isn't all that common yet, let alone 4.1.
> >
> > IMO, the ideal situation is to allow 4.0 *or* 4.1, and tweak our
> > calls accordingly. In addition, if you're on MacOS X, then require
> > 4.1.
> >
> > So... +0 on requiring 4.1, +1 if coded as above.
> 
> The fact that 4.0 isn't all the common yet I believe is all the more 
> reason to just require BDB 4.1 and be done with it.
> 
> I'm looking at trying to get autoconf to detect the right version of 
> BDB (need to export some SVN_FS_GOT_DB_MAJOR variants), and getting 
> the checks just right probably exceeds my amount of free time that I 
> can dedicate to this.
> 
> I realize that you're +0 on requiring 4.1 - do we have enough +1s on 
> requiring 4.1 that we don't need to keep 4.0 support?  Or, does 
> someone with enough autoconf-fu feel like adding that component to my 
> patches?  Otherwise, it'll have to wait for my schedule to clear a 
> bit.  -- justin

We already have autocode that extracts the BDB version number; we're checking
for 4.0.14 right now, but there's no reason we couldn't check for 4.1.x, too.
And any part of the code that uses BDB will pull in the BDB headers, so that you
could ifdef on DB_MAJOR and DB_MINOR where necessary. I don't think we need any
new or different autoconfigury at all.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Blair Zajac <bl...@orcaware.com>.

Justin Erenkrantz wrote:
> 
> The fact that 4.0 isn't all the common yet I believe is all the more
> reason to just require BDB 4.1 and be done with it.

Fyi, RedHat 8.0 ships with 4.0.

> I'm looking at trying to get autoconf to detect the right version of
> BDB (need to export some SVN_FS_GOT_DB_MAJOR variants), and getting
> the checks just right probably exceeds my amount of free time that I
> can dedicate to this.
> 
> I realize that you're +0 on requiring 4.1 - do we have enough +1s on
> requiring 4.1 that we don't need to keep 4.0 support?  Or, does
> someone with enough autoconf-fu feel like adding that component to my
> patches?  Otherwise, it'll have to wait for my schedule to clear a
> bit.  -- justin

+1.

Blair

-- 
Blair Zajac <bl...@orcaware.com>
Web and OS performance plots - http://www.orcaware.com/orca/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by "Stephen C. Tweedie" <sc...@redhat.com>.

Hi,

On Tue, Oct 15, 2002 at 06:30:12PM -0700, Justin Erenkrantz wrote:

> > DB4.0 isn't all that common yet, let alone 4.1.
> >
> > IMO, the ideal situation is to allow 4.0 *or* 4.1, and tweak our
> > calls accordingly. In addition, if you're on MacOS X, then require
> > 4.1.
> >
> > So... +0 on requiring 4.1, +1 if coded as above.
> 
> The fact that 4.0 isn't all the common yet I believe is all the more 
> reason to just require BDB 4.1 and be done with it.

Red Hat Linux 8.0 uses 4.0.14.  4.0 and 4.1 libs ought to be able to
coexist on the box at the same time, though.

--Stephen

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Philip Martin <ph...@codematters.co.uk>.

Greg Stein <gs...@lyra.org> writes:

> The part about not
> being able to wedge 4.1 is interesting, though. It might be interesting to
> get Philip or C.Mike to throw the stress tool at a 4.1-based SVN and see if
> it can wedge. (and double-check that it *can* still wedge against 4.0)
> Assuming 4.1 stops all potential wedging, then I'd suggest we support both
> for now, and "just before 1.0" flip the switch to force 4.1.

Last time I ran the stress.pl script I committed a couple of thousand
revsions, using 2, 3, 5, and 10 concurrent working copies; I didn't
see a hung repository.

However, interrupting the script while commiting over ra_local is more
than likely to hang a 4.0 repository.

-- 
Philip Martin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Greg Stein <gs...@lyra.org>.

On Wed, Oct 16, 2002 at 11:46:45AM +0000, Gustavo Niemeyer wrote:
> > In my experience, 4.0 and 4.1 are *equally* uncommon.  I'd rather just
> > keep things simple and require 4.1.
> 
> Conectiva, UnitedLinux, SuSE, and RedHat, have 4.0 in their
> active/development distribution, while none have 4.1. That's not a
> showstopper, but it's certainly an extra step towards subversion's
> adoption.

Agreed. This is exactly why I'm only +0 on forcing 4.1. The part about not
being able to wedge 4.1 is interesting, though. It might be interesting to
get Philip or C.Mike to throw the stress tool at a 4.1-based SVN and see if
it can wedge. (and double-check that it *can* still wedge against 4.0)
Assuming 4.1 stops all potential wedging, then I'd suggest we support both
for now, and "just before 1.0" flip the switch to force 4.1.

We did a similar dual-support for 3.x a long while back. After a while, we
found the new version was significantly better, so we just forced that.

>...
> expecting-svndb-soon-ly y'rs

Aaaahhh!!! The Tim Peters meme invades Subversion...

:-)

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Gustavo Niemeyer <ni...@conectiva.com>.

> In my experience, 4.0 and 4.1 are *equally* uncommon.  I'd rather just
> keep things simple and require 4.1.

Conectiva, UnitedLinux, SuSE, and RedHat, have 4.0 in their
active/development distribution, while none have 4.1. That's not a
showstopper, but it's certainly an extra step towards subversion's
adoption. Also, in most cases, there are other software requiring 4.0
already, and probably not yet compatible with 4.1. Thus, to keep
supporting subversion (something we'll do anyway) they have to include
yet another parallel berkley db package. Not that bad, for sure, but
that remembers me about the issue we used to have with rpm, where we
always had a specific db version just to satisfy rpm's needs. In rpm's
case, it was so annoying that they ended up bundling bdb inside the rpm
tarball, and calling it rpmdb.

expecting-svndb-soon-ly y'rs

-- 
Gustavo Niemeyer

[ 2AAC 7928 0FBF 0299 5EB5  60E2 2253 B29A 6664 3A0C ]

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Ben Collins-Sussman <su...@collab.net>.

Justin Erenkrantz <je...@apache.org> writes:

> --On Monday, October 14, 2002 3:48 AM -0700 Greg Stein
> <gs...@lyra.org> wrote:
> 
> > DB4.0 isn't all that common yet, let alone 4.1.
> >
> > IMO, the ideal situation is to allow 4.0 *or* 4.1, and tweak our
> > calls accordingly. In addition, if you're on MacOS X, then require
> > 4.1.
> >
> > So... +0 on requiring 4.1, +1 if coded as above.
> 
> The fact that 4.0 isn't all the common yet I believe is all the more
> reason to just require BDB 4.1 and be done with it.

+1

In my experience, 4.0 and 4.1 are *equally* uncommon.  I'd rather just
keep things simple and require 4.1.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Justin Erenkrantz <je...@apache.org>.

--On Monday, October 14, 2002 3:48 AM -0700 Greg Stein 
<gs...@lyra.org> wrote:

> DB4.0 isn't all that common yet, let alone 4.1.
>
> IMO, the ideal situation is to allow 4.0 *or* 4.1, and tweak our
> calls accordingly. In addition, if you're on MacOS X, then require
> 4.1.
>
> So... +0 on requiring 4.1, +1 if coded as above.

The fact that 4.0 isn't all the common yet I believe is all the more 
reason to just require BDB 4.1 and be done with it.

I'm looking at trying to get autoconf to detect the right version of 
BDB (need to export some SVN_FS_GOT_DB_MAJOR variants), and getting 
the checks just right probably exceeds my amount of free time that I 
can dedicate to this.

I realize that you're +0 on requiring 4.1 - do we have enough +1s on 
requiring 4.1 that we don't need to keep 4.0 support?  Or, does 
someone with enough autoconf-fu feel like adding that component to my 
patches?  Otherwise, it'll have to wait for my schedule to clear a 
bit.  -- justin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Upgrade to db 4.1

Posted by Greg Stein <gs...@lyra.org>.

On Mon, Oct 14, 2002 at 11:45:33AM +0200, Sander Striker wrote:
> > From: Nicholas Riley [mailto:njriley@uiuc.edu]
> > Sent: 14 October 2002 11:25
> 
> > On Sun, Oct 13, 2002 at 08:29:09PM -0400, Kevin D. Bond wrote:
> > > Please note that there is a bug with MUTEXs in Berkeley DB 4.0 under 
> > > Mac OS X.  In fact I had problems with everything from 3.3 to 4.0.  4.1 
> > > includes a fix, you can get the mutex change from 4.1 and include it in 
> > > 4.0.  If unfixed, this bug will cause database corruption during 
> > > concurrent writes to the database.
> > 
> > This explains why I can't run a Subversion server on OS X for more
> > than about 5 commits before the database explodes.  I've spent days
> > trying to track down this problem already, but didn't get anywhere
> > (probably because I was looking inside Subversion instead of BDB for
> > the problem).
> > 
> > Do you have a patch for 4.0.14 to fix the mutex problems?  If so I can
> > test it out... I've got plenty of test cases - for example, all I need
> > to do is create a new repository, import a dumpfile, and I get errors
> > with the version I'm using.
> 
> If this is the case then I think it is a very valid reason for us to consider
> making db 4.1 the required version.  Justin already submitted a patch.
> 
> Any reasons why we shouldn't move to 4.1 and stick with 4.0?

DB4.0 isn't all that common yet, let alone 4.1.

IMO, the ideal situation is to allow 4.0 *or* 4.1, and tweak our calls
accordingly. In addition, if you're on MacOS X, then require 4.1.

So... +0 on requiring 4.1, +1 if coded as above.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

still problems with svn and ra_local under OS X

Posted by Nicholas Riley <nj...@uiuc.edu>.

OK, I applied the mutex patch and rebuilt db-4.0.14.  I'm still having
the same problem.  Here's what happens in a nutshell:

% REPOS=/var/svn/repos/dev; sudo apachectl stop; sudo rm -rf "$REPOS"; sudo svnadmin create "$REPOS"; sudo chown -R svn:svn "$REPOS"; sudo chmod -R go+rX "$REPOS"; pushd "$REPOS"; sudo chown -R www db dav; sudo chmod -R g+w db dav; popd
% sudo -u svn svnadmin load /var/svn/repos/dev < dev\#11-2002.10.07-15.10
<<< Started new txn, based on original revision 1
[...]
------- Committed new rev 11 (loaded from original rev 11) >>>

% svnadmin youngest /var/svn/repos/dev
11
zsh: 22406 abort      svnadmin youngest /var/svn/repos/dev
% svnadmin youngest /var/svn/repos/dev
svn: Berkeley DB error
svn: Berkeley DB error while opening environment for filesystem /var/svn/repos/dev/db:
DB_RUNRECOVERY: Fatal error, run database recovery

At this point, I can perform a db_recover:

% sudo -u svn db_recover -v -h /var/svn/repos/dev/db
db_recover: Finding last valid log LSN: file: 2 offset 355840
db_recover: Checkpoint at: [2][355792]
db_recover: Checkpoint LSN: [2][355792]
db_recover: Previous checkpoint: [2][64885]
db_recover: Checkpoint at: [2][64885]
db_recover: Checkpoint LSN: [2][64885]
db_recover: Previous checkpoint: [1][62490]
db_recover: Recovery starting from [2][64885]
db_recover: Recovery complete at Tue Oct 15 23:07:18 2002
db_recover: Maximum transaction ID 8000025e Recovery checkpoint [2][355888]
db_recover: Recovery complete at Tue Oct 15 23:07:18 2002
db_recover: Maximum transaction id 80000000 Recovery checkpoint [2][355888]

and the cycle repeats again:

% svnadmin youngest /var/svn/repos/dev                                   
11
zsh: 22419 abort      svnadmin youngest /var/svn/repos/dev
% svnadmin youngest /var/svn/repos/dev
svn: Berkeley DB error
svn: Berkeley DB error while opening environment for filesystem /var/svn/repos/dev/db:
DB_RUNRECOVERY: Fatal error, run database recovery

The problem is not the dumpfile, as I sent my dumpfile to Ben
Collins-Sussman and he was able to import it and run svnadmin youngest
to his heart's content.  I've had this problem with many versions of
Subversion over the past few months, but this test was performed with
r3373.

If anyone else running OS X wants to try the above procedure, the
dumpfile is at <http://web.sabi.net/dev-dump.tgz>.

-- 
=Nicholas Riley <nj...@uiuc.edu> | <http://www.uiuc.edu/ph/www/njriley>
        Pablo Research Group, Department of Computer Science and
  Medical Scholars Program, University of Illinois at Urbana-Champaign




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org