You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Sachin Patil <sa...@fastvdo.com> on 2005/08/23 21:10:04 UTC
Repeated database/logfile corruption
HI,
My organization has adopted SVN for revision control over other popular
choices like CVS.
I have successfully maintained repositories on a local(LAN) server providing
access to the
clients using svnserve as a demon process for about 2 months.
But now, the new server is located at a geographically distant location and
hence there is a need for
a more secure access method. We moved from using the svnserve process to
using svn over a ssh tunnel.
Ever since I started access using ssh, my repository seems to get corrupted.
When I run 'svnadmin recover', the recovery process itself fails because the
log file
is in an inconsistant state.
If I remove the log file(delete -- an incorrect thing to do), the recovery
process works fine.
I can checkout one copy of the code and the next immediate attempt to
checkout/update/checkin
or any access to the repository fails since the database goes into a
corrputed state.(wedged)
I have read through a lot of websites/forums trying to find some query
matching my problems.
I have also read the svn-bean-book but have not found anything specific in
there either.
The errors are listed below.
Error Berkeley DB error while checkpointing after Berkeley DB transaction of
filesystem /svnrepo/myproj/db:
Error Invalid argument
Error bdb: DB_ENV->log_flush:LSN of 51/193480 past current end-of-log of
1/10488
Error bdb: Database environment corrupt; the wrong log files may have been
removed or incompatible database files imported from another environment.
These errors keep coming back even after a clean database recovery.
Could anyone please tell me what is causing this error/corruption?
Have I set the incorrect access permissions to the reposotory? I don't think
so.
Also, this error happens irrespective of whether different users are
accessing the repo or the same user is accessing the repo multiple times.
I am completely stuck and have run out of ideas. Any help/clues/pointers are
appreciated.
regards,
Sachin Patil
FastVDO LLC.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Repeated database/logfile corruption
Posted by Joshua Varner <jl...@gmail.com>.
On 8/25/05, Branko Čibej <br...@xbc.nu> wrote:> Joshua Varner wrote:> > >On 8/24/05, Sachin Patil <sa...@fastvdo.com> wrote:> >> >> >>Thanks for the suggestions.> >>I moved my repositories from BDB to FSFS and all the problems have gone> >>away. ( atleast for now ! )> >>Are there any known issues in using FSFS?> >>> >>-Sachin.> >>> >>> >>> >> >There is a limitation with APR 0.9 with files greater than 2GB.> >> >> This is not strictly true. You can recompile apr-0.9 with largefile> support. The problem is that the resulting libraries won't be binary> compatible with applications compiled against the non-largefile versions.> > >So exceptionally large commits with fsfs backends can hit this> >limit. See> >http://subversion.tigris.org/servlets/ReadMsg?listName=users&msgNo=35936> >and others.> >> >There are several workarounds, recompile with APR 0.10 or> >break up a large commit into several.> >> >> Um. You mean apr-1.0.> > -- Brane> It's sad that 9:52 is already too late for me to think straight. Thanksfor the corrections.
Josh
Re: Repeated database/logfile corruption
Posted by Branko Čibej <br...@xbc.nu>.
Joshua Varner wrote:
>On 8/24/05, Sachin Patil <sa...@fastvdo.com> wrote:
>
>
>>Thanks for the suggestions.
>>I moved my repositories from BDB to FSFS and all the problems have gone
>>away. ( atleast for now ! )
>>Are there any known issues in using FSFS?
>>
>>-Sachin.
>>
>>
>>
>
>There is a limitation with APR 0.9 with files greater than 2GB.
>
>
This is not strictly true. You can recompile apr-0.9 with largefile
support. The problem is that the resulting libraries won't be binary
compatible with applications compiled against the non-largefile versions.
>So exceptionally large commits with fsfs backends can hit this
>limit. See
>http://subversion.tigris.org/servlets/ReadMsg?listName=users&msgNo=35936
>and others.
>
>There are several workarounds, recompile with APR 0.10 or
>break up a large commit into several.
>
>
Um. You mean apr-1.0.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Repeated database/logfile corruption
Posted by Joshua Varner <jl...@gmail.com>.
On 8/24/05, Sachin Patil <sa...@fastvdo.com> wrote:
> Thanks for the suggestions.
> I moved my repositories from BDB to FSFS and all the problems have gone
> away. ( atleast for now ! )
> Are there any known issues in using FSFS?
>
> -Sachin.
>
There is a limitation with APR 0.9 with files greater than 2GB.
So exceptionally large commits with fsfs backends can hit this
limit. See
http://subversion.tigris.org/servlets/ReadMsg?listName=users&msgNo=35936
and others.
There are several workarounds, recompile with APR 0.10 or
break up a large commit into several.
The bdb backend is stable, it just has specific requirements of
the filesystem that when not satisfied cause this behavior. If
those conditions are satisfied there are no problems, and many
are using the bdb backend without difficulty.
Josh
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Repeated database/logfile corruption
Posted by Chris Church <fl...@gmail.com>.
I second the previous comments. Switch from BDB to FSFS.
Since upgrading to 1.2, Subversion crashed and locked the repository
about once a week on average while I was using BDB, and it took a
combination of Berkley's db_recover and svnadmin recover to get it
back. It was a pain especially when I was out of town or on vacation
and had to keep everybody back in the office happy.
I've had no problems since switching to FSFS a few weeks back.
On 8/23/05, Dave Camp <da...@thinbits.com> wrote:
>
> On Aug 23, 2005, at 5:03 PM, Ryan Schmidt wrote:
>
> On 23.08.2005, at 23:10, Sachin Patil wrote:
> Ever since I started access using ssh, my repository seems to get corrupted.
>
> Error Berkeley DB error while checkpointing after Berkeley DB transaction of
> filesystem /svnrepo/myproj/db:
> Error Invalid argument
> Error bdb: DB_ENV->log_flush:LSN of 51/193480 past current end-of-log of
> 1/10488
> Error bdb: Database environment corrupt; the wrong log files may have been
> removed or incompatible database files imported from another environment.
>
> I cannot tell you why this is happening, but I can recommend that you put
> the latest Subversion on your server (currently 1.2.1; soon 1.2.3) and
> convert your repository from BDB to FSFS. In 6 months on this list, I have
> seen a fairly constant stream of posts about problems with BDB repository
> wedging, and very few problems with FSFS repositories. This is not because
> of inherent problems with BDB, but rather because of inherent problems with
> the way in which Subversion uses BDB. Since the Subversion and BDB teams
> have yet to completely resolve these issues, I suspect converting will help
> you.
> Yes, get off BDB as fast as possible. General consensus among users on the
> list is that it will eventually get corrupted. In every case moving to FSFS
> has apparently made all the problems go away. It has for me...
>
> Dave
--
Chris Church
flyingfred0@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Repeated database/logfile corruption
Posted by Sachin Patil <sa...@fastvdo.com>.
-----Original Message-----
From: Dave Camp [mailto:dave@thinbits.com]
Sent: Tuesday, August 23, 2005 8:40 PM
To: Ryan Schmidt
Cc: Sachin Patil; users@subversion.tigris.org
Subject: Re: Repeated database/logfile corruption
On Aug 23, 2005, at 5:03 PM, Ryan Schmidt wrote:
On 23.08.2005, at 23:10, Sachin Patil wrote:
Ever since I started access using ssh, my repository seems to get
corrupted.
Error Berkeley DB error while checkpointing after Berkeley DB
transaction of
filesystem /svnrepo/myproj/db:
Error Invalid argument
Error bdb: DB_ENV->log_flush:LSN of 51/193480 past current
end-of-log of
1/10488
Error bdb: Database environment corrupt; the wrong log files may
have been
removed or incompatible database files imported from another
environment.
I cannot tell you why this is happening, but I can recommend that you
put the latest Subversion on your server (currently 1.2.1; soon 1.2.3) and
convert your repository from BDB to FSFS. In 6 months on this list, I have
seen a fairly constant stream of posts about problems with BDB repository
wedging, and very few problems with FSFS repositories. This is not because
of inherent problems with BDB, but rather because of inherent problems with
the way in which Subversion uses BDB. Since the Subversion and BDB teams
have yet to completely resolve these issues, I suspect converting will help
you.
Yes, get off BDB as fast as possible. General consensus among users on
the list is that it will eventually get corrupted. In every case moving to
FSFS has apparently made all the problems go away. It has for me...
Dave
[Sachin Patil]
Thanks for the suggestions.
I moved my repositories from BDB to FSFS and all the problems have gone
away. ( atleast for now ! )
Are there any known issues in using FSFS?
-Sachin.
Re: Repeated database/logfile corruption
Posted by Dave Camp <da...@thinbits.com>.
On Aug 23, 2005, at 5:03 PM, Ryan Schmidt wrote:
> On 23.08.2005, at 23:10, Sachin Patil wrote:
>> Ever since I started access using ssh, my repository seems to get
>> corrupted.
>>
>> Error Berkeley DB error while checkpointing after Berkeley DB
>> transaction of
>> filesystem /svnrepo/myproj/db:
>> Error Invalid argument
>> Error bdb: DB_ENV->log_flush:LSN of 51/193480 past current end-of-
>> log of
>> 1/10488
>> Error bdb: Database environment corrupt; the wrong log files may
>> have been
>> removed or incompatible database files imported from another
>> environment.
>>
>
> I cannot tell you why this is happening, but I can recommend that
> you put the latest Subversion on your server (currently 1.2.1; soon
> 1.2.3) and convert your repository from BDB to FSFS. In 6 months on
> this list, I have seen a fairly constant stream of posts about
> problems with BDB repository wedging, and very few problems with
> FSFS repositories. This is not because of inherent problems with
> BDB, but rather because of inherent problems with the way in which
> Subversion uses BDB. Since the Subversion and BDB teams have yet to
> completely resolve these issues, I suspect converting will help you.
Yes, get off BDB as fast as possible. General consensus among users
on the list is that it will eventually get corrupted. In every case
moving to FSFS has apparently made all the problems go away. It has
for me...
Dave
Re: Repeated database/logfile corruption
Posted by Ryan Schmidt <su...@ryandesign.com>.
On 23.08.2005, at 23:10, Sachin Patil wrote:
> Ever since I started access using ssh, my repository seems to get
> corrupted.
> Error Berkeley DB error while checkpointing after Berkeley DB
> transaction of
> filesystem /svnrepo/myproj/db:
> Error Invalid argument
> Error bdb: DB_ENV->log_flush:LSN of 51/193480 past current end-of-
> log of
> 1/10488
> Error bdb: Database environment corrupt; the wrong log files may
> have been
> removed or incompatible database files imported from another
> environment.
I cannot tell you why this is happening, but I can recommend that you
put the latest Subversion on your server (currently 1.2.1; soon
1.2.3) and convert your repository from BDB to FSFS. In 6 months on
this list, I have seen a fairly constant stream of posts about
problems with BDB repository wedging, and very few problems with FSFS
repositories. This is not because of inherent problems with BDB, but
rather because of inherent problems with the way in which Subversion
uses BDB. Since the Subversion and BDB teams have yet to completely
resolve these issues, I suspect converting will help you.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org