You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Mark Phippard <ma...@gmail.com> on 2011/07/27 20:49:34 UTC
Forcing the SQLite db to close?
Noticed a problem in Eclipse. When someone wants to remove a project from
Eclipse, the .svn folder cannot be deleted because the wc.db file is in use.
This is because, unlike the command line, when we are using JavaHL the
process does not end until someone closes Eclipse.
Is there an API we could add to JavaHL or something to tell SVN to close the
DB handles so that the file is not locked?
Bert/Stefan, how are you dealing with this problem in your GUI tools? I
know you have access to more API than we have for JavaH, so maybe there is
already an API and it just needs to be exposed?
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Forcing the SQLite db to close?
Posted by Mark Phippard <ma...@gmail.com>.
On Wed, Jul 27, 2011 at 3:36 PM, C. Michael Pilato <cm...@collab.net>wrote:
> On 07/27/2011 03:28 PM, Mark Phippard wrote:
> > On Wed, Jul 27, 2011 at 3:22 PM, Philip Martin <
> philip.martin@wandisco.com
> > <ma...@wandisco.com>> wrote:
> >
> > Mark Phippard <markphip@gmail.com <ma...@gmail.com>>
> writes:
> >
> > > Noticed a problem in Eclipse. When someone wants to remove a
> project from
> > > Eclipse, the .svn folder cannot be deleted because the wc.db file
> is
> > in use.
> > > This is because, unlike the command line, when we are using JavaHL
> the
> > > process does not end until someone closes Eclipse.
> > >
> > > Is there an API we could add to JavaHL or something to tell SVN to
> > close the
> > > DB handles so that the file is not locked?
> > >
> > > Bert/Stefan, how are you dealing with this problem in your GUI
> tools? I
> > > know you have access to more API than we have for JavaH, so maybe
> there is
> > > already an API and it just needs to be exposed?
> >
> > There is a private function svn_wc__db_drop_root, I think that does
> what
> > you want. What do you want the public API to look like?
> >
> >
> > Don't know really. JavaHL uses libsvn_client. Not sure what it should
> be
> > called. Maybe disconnect or close? and takes a path within a WC as arg?
>
> Should this closure be tied to a cleanup of the pool which holds the client
> context baton?
>
We probably need some more explicit way to do it. Due to garbage collection
in Java, there could be many JavaHL objects still waiting to be closed. I
am not sure how JavaHL even handles the multiple instances of the class when
it comes to the pools and their lifetimes.
JavaHL has a dispose() method but if you call it, you currently get a
segfault when the same object is later finalized by the garbage collector.
So I am not calling it and just letting GC handle it. Again, not sure if
that would even help here.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Forcing the SQLite db to close?
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 07/27/2011 03:28 PM, Mark Phippard wrote:
> On Wed, Jul 27, 2011 at 3:22 PM, Philip Martin <philip.martin@wandisco.com
> <ma...@wandisco.com>> wrote:
>
> Mark Phippard <markphip@gmail.com <ma...@gmail.com>> writes:
>
> > Noticed a problem in Eclipse. When someone wants to remove a project from
> > Eclipse, the .svn folder cannot be deleted because the wc.db file is
> in use.
> > This is because, unlike the command line, when we are using JavaHL the
> > process does not end until someone closes Eclipse.
> >
> > Is there an API we could add to JavaHL or something to tell SVN to
> close the
> > DB handles so that the file is not locked?
> >
> > Bert/Stefan, how are you dealing with this problem in your GUI tools? I
> > know you have access to more API than we have for JavaH, so maybe there is
> > already an API and it just needs to be exposed?
>
> There is a private function svn_wc__db_drop_root, I think that does what
> you want. What do you want the public API to look like?
>
>
> Don't know really. JavaHL uses libsvn_client. Not sure what it should be
> called. Maybe disconnect or close? and takes a path within a WC as arg?
Should this closure be tied to a cleanup of the pool which holds the client
context baton?
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Re: Forcing the SQLite db to close?
Posted by Mark Phippard <ma...@gmail.com>.
On Wed, Jul 27, 2011 at 3:22 PM, Philip Martin
<ph...@wandisco.com>wrote:
> Mark Phippard <ma...@gmail.com> writes:
>
> > Noticed a problem in Eclipse. When someone wants to remove a project
> from
> > Eclipse, the .svn folder cannot be deleted because the wc.db file is in
> use.
> > This is because, unlike the command line, when we are using JavaHL the
> > process does not end until someone closes Eclipse.
> >
> > Is there an API we could add to JavaHL or something to tell SVN to close
> the
> > DB handles so that the file is not locked?
> >
> > Bert/Stefan, how are you dealing with this problem in your GUI tools? I
> > know you have access to more API than we have for JavaH, so maybe there
> is
> > already an API and it just needs to be exposed?
>
> There is a private function svn_wc__db_drop_root, I think that does what
> you want. What do you want the public API to look like?
>
Don't know really. JavaHL uses libsvn_client. Not sure what it should be
called. Maybe disconnect or close? and takes a path within a WC as arg?
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Forcing the SQLite db to close?
Posted by Hyrum K Wright <hy...@wandisco.com>.
On Thu, Aug 4, 2011 at 6:25 PM, Mark Phippard <ma...@gmail.com> wrote:
> On Thu, Aug 4, 2011 at 7:11 PM, Mark Phippard <ma...@gmail.com> wrote:
>>
>> On Thu, Aug 4, 2011 at 5:29 PM, Hyrum K Wright <hy...@wandisco.com>
>> wrote:
>>>
>>> On Thu, Aug 4, 2011 at 12:02 PM, Mark Phippard <ma...@gmail.com>
>>> wrote:
>>>
>>>
>>> Does r1154023 help at all? It's fairly simple, but "feels" like the
>>> right place to be destroying the wc_ctx of a client context when
>>> disposing of the SVNClient object. The tests still pass, but I didn't
>>> test this specific functionality.
>>>
>>> (Even if it doesn't improve the situation, I'd like to leave it in,
>>> since it improves the memory situation a bit.)
>>>
>>
>> Testing on Windows. Will let you know.
>
> OK, I came up with a reliable recipe on Windows using Subclipse that
> produces the problem. After merging this change to the branch the problem
> is fixed!
> Assuming you nominate this for backport, please add my +1
Great! I'll get this nominated and backported in time for beta3.
-Hyrum
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com/
Re: Forcing the SQLite db to close?
Posted by Mark Phippard <ma...@gmail.com>.
On Thu, Aug 4, 2011 at 7:11 PM, Mark Phippard <ma...@gmail.com> wrote:
> On Thu, Aug 4, 2011 at 5:29 PM, Hyrum K Wright <hy...@wandisco.com>wrote:
>
>> On Thu, Aug 4, 2011 at 12:02 PM, Mark Phippard <ma...@gmail.com>
>> wrote:
>>
>>
>> Does r1154023 help at all? It's fairly simple, but "feels" like the
>> right place to be destroying the wc_ctx of a client context when
>> disposing of the SVNClient object. The tests still pass, but I didn't
>> test this specific functionality.
>>
>> (Even if it doesn't improve the situation, I'd like to leave it in,
>> since it improves the memory situation a bit.)
>>
>>
> Testing on Windows. Will let you know.
>
OK, I came up with a reliable recipe on Windows using Subclipse that
produces the problem. After merging this change to the branch the problem
is fixed!
Assuming you nominate this for backport, please add my +1
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Forcing the SQLite db to close?
Posted by Mark Phippard <ma...@gmail.com>.
On Thu, Aug 4, 2011 at 5:29 PM, Hyrum K Wright <hy...@wandisco.com>wrote:
> On Thu, Aug 4, 2011 at 12:02 PM, Mark Phippard <ma...@gmail.com> wrote:
> > On Thu, Aug 4, 2011 at 9:22 AM, Mark Phippard <ma...@gmail.com>
> wrote:
> >>
> >> On Thu, Aug 4, 2011 at 5:14 AM, Bert Huijben <be...@qqmail.nl> wrote:
> >>>
> >>> The Javahl problem caused by calling dispose() should be fixed in
> >>> r1153799.
> >>>
> >>>
> >>>
> >>> @Mark: Can you verify if this solves your problem?
> >>>
> >>>
> >>>
> >>> If this resolves the problem this fix might need backport to 1.6.
> >>
> >> I merged this commit and the followup to my 1.7.x branch and tested.
> >> This fix definitely seems to fix the crash problem. If you have not
> >> nominated this yet for backport to 1.7.x please do so and add my +1.
> >> From what I can tell, this also fixes the problem with the locks being
> >> left on the database. I probably need to do a little more testing on
> this
> >> to be sure, but from what I see it seems to solve the problem. Now that
> >> dispose works, I also have to resume my task in Subclipse of being sure
> to
> >> call it everywhere I release the object.
> >
> > Now that I have had a chance to test this on Windows, I do not think it
> > fixes the lock problem. The segfaults are fixed though so that is good
> > because we can now start calling that as we intended.
> > Do you have any ideas for how we can solve the lock problem? Maybe we
> can
> > add some kind of call in the dispose method that removes the locks before
> > doing the dispose? I will be happy to test patches.
>
> Does r1154023 help at all? It's fairly simple, but "feels" like the
> right place to be destroying the wc_ctx of a client context when
> disposing of the SVNClient object. The tests still pass, but I didn't
> test this specific functionality.
>
> (Even if it doesn't improve the situation, I'd like to leave it in,
> since it improves the memory situation a bit.)
>
>
Testing on Windows. Will let you know.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Forcing the SQLite db to close?
Posted by Hyrum K Wright <hy...@wandisco.com>.
On Thu, Aug 4, 2011 at 12:02 PM, Mark Phippard <ma...@gmail.com> wrote:
> On Thu, Aug 4, 2011 at 9:22 AM, Mark Phippard <ma...@gmail.com> wrote:
>>
>> On Thu, Aug 4, 2011 at 5:14 AM, Bert Huijben <be...@qqmail.nl> wrote:
>>>
>>> The Javahl problem caused by calling dispose() should be fixed in
>>> r1153799.
>>>
>>>
>>>
>>> @Mark: Can you verify if this solves your problem?
>>>
>>>
>>>
>>> If this resolves the problem this fix might need backport to 1.6.
>>
>> I merged this commit and the followup to my 1.7.x branch and tested.
>> This fix definitely seems to fix the crash problem. If you have not
>> nominated this yet for backport to 1.7.x please do so and add my +1.
>> From what I can tell, this also fixes the problem with the locks being
>> left on the database. I probably need to do a little more testing on this
>> to be sure, but from what I see it seems to solve the problem. Now that
>> dispose works, I also have to resume my task in Subclipse of being sure to
>> call it everywhere I release the object.
>
> Now that I have had a chance to test this on Windows, I do not think it
> fixes the lock problem. The segfaults are fixed though so that is good
> because we can now start calling that as we intended.
> Do you have any ideas for how we can solve the lock problem? Maybe we can
> add some kind of call in the dispose method that removes the locks before
> doing the dispose? I will be happy to test patches.
Does r1154023 help at all? It's fairly simple, but "feels" like the
right place to be destroying the wc_ctx of a client context when
disposing of the SVNClient object. The tests still pass, but I didn't
test this specific functionality.
(Even if it doesn't improve the situation, I'd like to leave it in,
since it improves the memory situation a bit.)
-Hyrum
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com/
Re: Forcing the SQLite db to close?
Posted by Mark Phippard <ma...@gmail.com>.
On Thu, Aug 4, 2011 at 9:22 AM, Mark Phippard <ma...@gmail.com> wrote:
> On Thu, Aug 4, 2011 at 5:14 AM, Bert Huijben <be...@qqmail.nl> wrote:
>
>> The Javahl problem caused by calling dispose() should be fixed in
>> r1153799.****
>>
>> ** **
>>
>> @Mark: Can you verify if this solves your problem?****
>>
>> ** **
>>
>> If this resolves the problem this fix might need backport to 1.6.
>>
>
>
> I merged this commit and the followup to my 1.7.x branch and tested.
>
> This fix definitely seems to fix the crash problem. If you have not
> nominated this yet for backport to 1.7.x please do so and add my +1.
>
> From what I can tell, this also fixes the problem with the locks being left
> on the database. I probably need to do a little more testing on this to be
> sure, but from what I see it seems to solve the problem. Now that dispose
> works, I also have to resume my task in Subclipse of being sure to call it
> everywhere I release the object.
>
Now that I have had a chance to test this on Windows, I do not think it
fixes the lock problem. The segfaults are fixed though so that is good
because we can now start calling that as we intended.
Do you have any ideas for how we can solve the lock problem? Maybe we can
add some kind of call in the dispose method that removes the locks before
doing the dispose? I will be happy to test patches.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Forcing the SQLite db to close?
Posted by Mark Phippard <ma...@gmail.com>.
On Thu, Aug 4, 2011 at 5:14 AM, Bert Huijben <be...@qqmail.nl> wrote:
> The Javahl problem caused by calling dispose() should be fixed in r1153799.
> ****
>
> ** **
>
> @Mark: Can you verify if this solves your problem?****
>
> ** **
>
> If this resolves the problem this fix might need backport to 1.6.
>
I merged this commit and the followup to my 1.7.x branch and tested.
This fix definitely seems to fix the crash problem. If you have not
nominated this yet for backport to 1.7.x please do so and add my +1.
>From what I can tell, this also fixes the problem with the locks being left
on the database. I probably need to do a little more testing on this to be
sure, but from what I see it seems to solve the problem. Now that dispose
works, I also have to resume my task in Subclipse of being sure to call it
everywhere I release the object.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
RE: Forcing the SQLite db to close?
Posted by Bert Huijben <be...@qqmail.nl>.
The Javahl problem caused by calling dispose() should be fixed in r1153799.
@Mark: Can you verify if this solves your problem?
If this resolves the problem this fix might need backport to 1.6.
Bert
From: Mark Phippard [mailto:markphip@gmail.com]
Sent: woensdag 3 augustus 2011 20:45
To: Bert Huijben
Cc: Philip Martin; Subversion Development
Subject: Re: Forcing the SQLite db to close?
On Wed, Jul 27, 2011 at 3:35 PM, Bert Huijben <be...@qqmail.nl> wrote:
> -----Original Message-----
> From: Philip Martin [mailto:philip.martin@wandisco.com]
> Sent: woensdag 27 juli 2011 21:23
> To: Mark Phippard
> Cc: Subversion Development
> Subject: Re: Forcing the SQLite db to close?
>
> Mark Phippard <ma...@gmail.com> writes:
>
> > Noticed a problem in Eclipse. When someone wants to remove a project
> from
> > Eclipse, the .svn folder cannot be deleted because the wc.db file is in
use.
> > This is because, unlike the command line, when we are using JavaHL the
> > process does not end until someone closes Eclipse.
> >
> > Is there an API we could add to JavaHL or something to tell SVN to close
the
> > DB handles so that the file is not locked?
> >
> > Bert/Stefan, how are you dealing with this problem in your GUI tools? I
> > know you have access to more API than we have for JavaH, so maybe there
> is
> > already an API and it just needs to be exposed?
>
> There is a private function svn_wc__db_drop_root, I think that does what
> you want. What do you want the public API to look like?
Or you can close the wc context with svn_wc_context_destroy(). This closes
all contexts at once.
(Clearing the pool with svn_client_ctx_t / svn_wc_context_t has the same
effect).
I see that JavaHL has a ClientContext class and the Java code creates a
private instance of this class. From what I can see in ClientContext.cpp it
stores the svn_client_ctx_t in the global pool.
/* Create a long-lived client context object in the global pool. */
SVN_JNI_ERR(svn_client_create_context(&persistentCtx,
JNIUtil::getPool()),
);
I wonder if we could simply add a method to JavaHL to destroy this context?
I do not see anywhere that the wc_context is created in JavaHL native code
so not sure if it would be possible to clear the pool used to store those.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Forcing the SQLite db to close?
Posted by Mark Phippard <ma...@gmail.com>.
On Wed, Jul 27, 2011 at 3:35 PM, Bert Huijben <be...@qqmail.nl> wrote:
>
>
> > -----Original Message-----
> > From: Philip Martin [mailto:philip.martin@wandisco.com]
> > Sent: woensdag 27 juli 2011 21:23
> > To: Mark Phippard
> > Cc: Subversion Development
> > Subject: Re: Forcing the SQLite db to close?
> >
> > Mark Phippard <ma...@gmail.com> writes:
> >
> > > Noticed a problem in Eclipse. When someone wants to remove a project
> > from
> > > Eclipse, the .svn folder cannot be deleted because the wc.db file is in
> use.
> > > This is because, unlike the command line, when we are using JavaHL the
> > > process does not end until someone closes Eclipse.
> > >
> > > Is there an API we could add to JavaHL or something to tell SVN to
> close
> the
> > > DB handles so that the file is not locked?
> > >
> > > Bert/Stefan, how are you dealing with this problem in your GUI tools?
> I
> > > know you have access to more API than we have for JavaH, so maybe there
> > is
> > > already an API and it just needs to be exposed?
> >
> > There is a private function svn_wc__db_drop_root, I think that does what
> > you want. What do you want the public API to look like?
>
> Or you can close the wc context with svn_wc_context_destroy(). This closes
> all contexts at once.
> (Clearing the pool with svn_client_ctx_t / svn_wc_context_t has the same
> effect).
>
I see that JavaHL has a ClientContext class and the Java code creates a
private instance of this class. From what I can see in ClientContext.cpp it
stores the svn_client_ctx_t in the global pool.
/* Create a long-lived client context object in the global pool. */
SVN_JNI_ERR(svn_client_create_context(&persistentCtx,
JNIUtil::getPool()),
);
I wonder if we could simply add a method to JavaHL to destroy this context?
I do not see anywhere that the wc_context is created in JavaHL native code
so not sure if it would be possible to clear the pool used to store those.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
RE: Forcing the SQLite db to close?
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: Philip Martin [mailto:philip.martin@wandisco.com]
> Sent: woensdag 27 juli 2011 21:23
> To: Mark Phippard
> Cc: Subversion Development
> Subject: Re: Forcing the SQLite db to close?
>
> Mark Phippard <ma...@gmail.com> writes:
>
> > Noticed a problem in Eclipse. When someone wants to remove a project
> from
> > Eclipse, the .svn folder cannot be deleted because the wc.db file is in
use.
> > This is because, unlike the command line, when we are using JavaHL the
> > process does not end until someone closes Eclipse.
> >
> > Is there an API we could add to JavaHL or something to tell SVN to close
the
> > DB handles so that the file is not locked?
> >
> > Bert/Stefan, how are you dealing with this problem in your GUI tools? I
> > know you have access to more API than we have for JavaH, so maybe there
> is
> > already an API and it just needs to be exposed?
>
> There is a private function svn_wc__db_drop_root, I think that does what
> you want. What do you want the public API to look like?
Or you can close the wc context with svn_wc_context_destroy(). This closes
all contexts at once.
(Clearing the pool with svn_client_ctx_t / svn_wc_context_t has the same
effect).
Answering Mark's question:
In SharpSvn I added a boolean on the SvnClient. With the boolean set to
false (the default) I create a new svn_wc_context_t for every command
invocation and clear the pool containing that temporary context when the
command is done. (svn_client_ctx_t lives much longer to cache passwords,
etc.)
This keeps everything compatible with the old version and users that need
the extra performance can switch the boolean.
In AnkhSVN with this per command closing of the database 1.7 is still much
faster than 1.6, so I haven't even looked at cases where I want to keep the
db open.
Bert
Re: Forcing the SQLite db to close?
Posted by Philip Martin <ph...@wandisco.com>.
Mark Phippard <ma...@gmail.com> writes:
> Noticed a problem in Eclipse. When someone wants to remove a project from
> Eclipse, the .svn folder cannot be deleted because the wc.db file is in use.
> This is because, unlike the command line, when we are using JavaHL the
> process does not end until someone closes Eclipse.
>
> Is there an API we could add to JavaHL or something to tell SVN to close the
> DB handles so that the file is not locked?
>
> Bert/Stefan, how are you dealing with this problem in your GUI tools? I
> know you have access to more API than we have for JavaH, so maybe there is
> already an API and it just needs to be exposed?
There is a private function svn_wc__db_drop_root, I think that does what
you want. What do you want the public API to look like?
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com
Re: Forcing the SQLite db to close?
Posted by Mark Phippard <ma...@gmail.com>.
On Wed, Jul 27, 2011 at 3:31 PM, Stefan Küng <to...@gmail.com> wrote:
> On 27.07.2011 20:49, Mark Phippard wrote:
>
>> Noticed a problem in Eclipse. When someone wants to remove a project
>> from Eclipse, the .svn folder cannot be deleted because the wc.db file
>> is in use. This is because, unlike the command line, when we are using
>> JavaHL the process does not end until someone closes Eclipse.
>>
>> Is there an API we could add to JavaHL or something to tell SVN to close
>> the DB handles so that the file is not locked?
>>
>> Bert/Stefan, how are you dealing with this problem in your GUI tools? I
>> know you have access to more API than we have for JavaH, so maybe there
>> is already an API and it just needs to be exposed?
>>
>
> All you need to do is to close the wc context:
> svn_wc_context_destroy()
> and that will close the db as well.
>
>
Still looking for a way to do this from Java. I was hoping that the Java
garbage collection and finalization would at least do it, but it does not
seem to. I searched the JavaHL native code and do not see where this method
is called. Should things like the dispose and finalize methods not do stuff
like this to cleanup resource usage?
Anyone have any ideas how I can do this from Java with the limited stuff
that has been exposed to the API?
I guess ideally, I would like the dispose() method to free these resources,
but we would also need the segfault fixed that happens when GC runs and
frees the objects you have disposed.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Re: Forcing the SQLite db to close?
Posted by Stefan Küng <to...@gmail.com>.
On 27.07.2011 20:49, Mark Phippard wrote:
> Noticed a problem in Eclipse. When someone wants to remove a project
> from Eclipse, the .svn folder cannot be deleted because the wc.db file
> is in use. This is because, unlike the command line, when we are using
> JavaHL the process does not end until someone closes Eclipse.
>
> Is there an API we could add to JavaHL or something to tell SVN to close
> the DB handles so that the file is not locked?
>
> Bert/Stefan, how are you dealing with this problem in your GUI tools? I
> know you have access to more API than we have for JavaH, so maybe there
> is already an API and it just needs to be exposed?
All you need to do is to close the wc context:
svn_wc_context_destroy()
and that will close the db as well.
Stefan
--
___
oo // \\ "De Chelonian Mobile"
(_,\/ \_/ \ TortoiseSVN
\ \_/_\_/> The coolest Interface to (Sub)Version Control
/_/ \_\ http://tortoisesvn.net