You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Mark Phippard <ma...@gmail.com> on 2007/11/26 14:37:32 UTC
JavaHL: Thread safety and Progress Notifications
I have been getting some crashes lately in Subclipse (using 1.5
trunk). Kind of sporadic, not obvious recipe. I finally bothered to
take the time to look at the crash output (this is on OSX). I saw
this:
Thread 22 Crashed:
0 libjvm.dylib 0x9bc16fb8 JVM_MonitorWait + 11444
1 libjvm.dylib 0x9bcd98bb JVM_IsInterrupted + 1797
2 libsvnjavahl-1.0.dylib 0x50cf8f6c
JNIEnv_::CallVoidMethod(_jobject*, _jmethodID*, ...) + 52 (jni.h:1034)
3 libsvnjavahl-1.0.dylib 0x50cde64d
ProgressListener::onProgress(long long, long long, apr_pool_t*) + 495
(ProgressListener.cpp:128)
4 libsvnjavahl-1.0.dylib 0x50cde6c6
ProgressListener::progress(long long, long long, void*, apr_pool_t*) +
82 (ProgressListener.cpp:77)
5 libsvn_ra_neon-1.0.dylib 0x50c946f0 ra_neon_neonprogress + 99
6 libneon.26.dylib 0x515e2f0f ne_read_response_block + 145
7 libneon.26.dylib 0x515e33b5 ne_discard_response + 34
8 libneon.26.dylib 0x515e45aa ne_request_dispatch + 46
9 libsvn_ra_neon-1.0.dylib 0x50c98c18 svn_ra_neon__request_dispatch + 238
10 libsvn_ra_neon-1.0.dylib 0x50c98598 parsed_request + 577
11 libsvn_ra_neon-1.0.dylib 0x50c987ba svn_ra_neon__parsed_request + 114
12 libsvn_ra_neon-1.0.dylib 0x50c8a71c reporter_finish_report + 539
13 libsvn_client-1.0.dylib 0x512e3190 reporter_finish_report + 433
14 libsvn_wc-1.0.dylib 0x50ffeff0 svn_wc_crawl_revisions3 + 1613
15 libsvn_client-1.0.dylib 0x512e380a svn_client_status3 + 1605
16 libsvnjavahl-1.0.dylib 0x50ce74c5 SVNClient::status(char
const*, svn_depth_t, bool, bool, bool, bool, StatusCallback*) + 335
(SVNClient.cpp:172)
17 libsvnjavahl-1.0.dylib 0x50cef3b1
Java_org_tigris_subversion_javahl_SVNClient_status + 323
(org_tigris_subversion_javahl_SVNClient.cpp:179)
I think I know what the problem is. We recently started using the new
progress notifications. I think what happens is I take an option to
do an Update. We set the progress listener so that we can provide
feedback. At some point during this process, Eclipse kicks off a
status refresh in another thread (also normal). There is only one
JavaHL object. When the status operation starts, the progress
listener is still set because update is running. When update ends, it
is nulling out the progress listener.
SVNClient.setProgressListener(null);
Status goes to send a progress notification, and something is now null.
I have changed Subclipse so that we no longer null out the listener,
and instead just swallow the notifications that we do not want inside
our object. I wonder if there is some kind of safety checks that can
be added in the JavaHL C++ code though so that it does not crash. I
do not think what I am doing is that abnormal so someone else might
run into this.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: JavaHL: Thread safety and Progress Notifications
Posted by Mark Phippard <ma...@gmail.com>.
On Nov 26, 2007 8:15 PM, Mark Phippard <ma...@gmail.com> wrote:
> <dlr> Have you used Java's ThreadLocal class before?
> <markphip> no
> <dlr> It can be used to achieve almost "magical" behavior.
> <dlr> It's really easy to mis-use, too, and results in code that's
> harder for newcomers to follow.
> <dlr> But for the original author, it's quite easy to use.
> <dlr> Basically, it gives you a hashtable that's associated with the
> "current" thread (whichever that may be).
> <dlr> So when you call SVNClient.setProgressListener(), it would in
> turn call ThreadLocal.set().
> <dlr> ...associating the supplied ProgressListener with the current thread.
I read this article about ThreadLocal:
http://www-128.ibm.com/developerworks/java/library/j-threads3.html
I do not see how it would fix this problem. It would let me have a
separate ProgressListener on each thread, but the problem was that
there is just a single SVNClient, so there would still be a problem of
calling SVNClient.setProgressListener(null) on another thread.
Actually, my current code would be creating a unique ProgressListener
per thread anyway.
I could see where using ThreadLocal to create one SVNClient per thread
could work (I am a little concerned about performance based on that
article though). It probably is not an issue in a desktop app like
Eclipse though. A problem with this approach is that I do not see how
I could call dispose(). I could use finalize() and that might be OK,
but most text books do not encourage its use. I cannot think of
another way to call dispose() though.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: JavaHL: Thread safety and Progress Notifications
Posted by Mark Phippard <ma...@gmail.com>.
On Nov 26, 2007 9:37 AM, Mark Phippard <ma...@gmail.com> wrote:
> I have been getting some crashes lately in Subclipse (using 1.5
> trunk). Kind of sporadic, not obvious recipe. I finally bothered to
> take the time to look at the crash output (this is on OSX). I saw
> this:
>
> Thread 22 Crashed:
> 0 libjvm.dylib 0x9bc16fb8 JVM_MonitorWait + 11444
> 1 libjvm.dylib 0x9bcd98bb JVM_IsInterrupted + 1797
> 2 libsvnjavahl-1.0.dylib 0x50cf8f6c
> JNIEnv_::CallVoidMethod(_jobject*, _jmethodID*, ...) + 52 (jni.h:1034)
> 3 libsvnjavahl-1.0.dylib 0x50cde64d
> ProgressListener::onProgress(long long, long long, apr_pool_t*) + 495
> (ProgressListener.cpp:128)
> 4 libsvnjavahl-1.0.dylib 0x50cde6c6
> ProgressListener::progress(long long, long long, void*, apr_pool_t*) +
> 82 (ProgressListener.cpp:77)
> 5 libsvn_ra_neon-1.0.dylib 0x50c946f0 ra_neon_neonprogress + 99
> 6 libneon.26.dylib 0x515e2f0f ne_read_response_block + 145
> 7 libneon.26.dylib 0x515e33b5 ne_discard_response + 34
> 8 libneon.26.dylib 0x515e45aa ne_request_dispatch + 46
> 9 libsvn_ra_neon-1.0.dylib 0x50c98c18 svn_ra_neon__request_dispatch + 238
> 10 libsvn_ra_neon-1.0.dylib 0x50c98598 parsed_request + 577
> 11 libsvn_ra_neon-1.0.dylib 0x50c987ba svn_ra_neon__parsed_request + 114
> 12 libsvn_ra_neon-1.0.dylib 0x50c8a71c reporter_finish_report + 539
> 13 libsvn_client-1.0.dylib 0x512e3190 reporter_finish_report + 433
> 14 libsvn_wc-1.0.dylib 0x50ffeff0 svn_wc_crawl_revisions3 + 1613
> 15 libsvn_client-1.0.dylib 0x512e380a svn_client_status3 + 1605
> 16 libsvnjavahl-1.0.dylib 0x50ce74c5 SVNClient::status(char
> const*, svn_depth_t, bool, bool, bool, bool, StatusCallback*) + 335
> (SVNClient.cpp:172)
> 17 libsvnjavahl-1.0.dylib 0x50cef3b1
> Java_org_tigris_subversion_javahl_SVNClient_status + 323
> (org_tigris_subversion_javahl_SVNClient.cpp:179)
>
> I think I know what the problem is. We recently started using the new
> progress notifications. I think what happens is I take an option to
> do an Update. We set the progress listener so that we can provide
> feedback. At some point during this process, Eclipse kicks off a
> status refresh in another thread (also normal). There is only one
> JavaHL object. When the status operation starts, the progress
> listener is still set because update is running. When update ends, it
> is nulling out the progress listener.
>
> SVNClient.setProgressListener(null);
>
> Status goes to send a progress notification, and something is now null.
>
> I have changed Subclipse so that we no longer null out the listener,
> and instead just swallow the notifications that we do not want inside
> our object. I wonder if there is some kind of safety checks that can
> be added in the JavaHL C++ code though so that it does not crash. I
> do not think what I am doing is that abnormal so someone else might
> run into this.
Dan and I discussed this on IRC. Here is transcript for archives:
<dlr> markphip: Around? I wanted to understand more about that
threading issue you ran into w/ usage of the JavaHL API.
<dlr> The Subclipse usage was nuking svn_client_ctx_t->progress_func
while it was being used by multiple threads.
<markphip> that was my guess
<dlr> AIUI, the C API assumes one svn_client_ctx_t per thread.
<dlr> That would mean one SVNClient per thread.
<markphip> there is only one instance of SVNClient and one setProgress
and we set it to null
<markphip> hmm, we talked about this once before and that is what had
me change it to be like this
<markphip> we used to create a new SVNClient everytime we wanted to do something
<dlr> I recall
<markphip> we do not directly deal with threads in Eclipse, but I
guess maybe we could determine the current thread and have a hashmap
or something
<dlr> For operations which don't change the state of the
svn_client_ctx_t, that should work fine.
<markphip> I do not think we would know when the threads end though
<dlr> But, if you've got multiple threads using the same
svn_client_ctx_t, you can't change it and expect things to work.
<markphip> what is svn_client_ctx_t in layman's terms?
<dlr> In Java terms, it's an Interface.
<dlr> ...implemented by clients, to provide Subversion behaviors
custom to those clients.
<markphip> but in SVN terms, what things are part of it? The
configuration area type stuff?
<dlr> markphip: http://svn.collab.net/svn-doxygen/structsvn__client__ctx__t.html
<markphip> the change I made will probably work then, I just did not
know if this could also be considered a bug ... sounds like not
<dlr> markphip: It contains a bunch of callbacks (methods on the Java
interface). So, even more than carrying configuration, it dictates
behavior.
<dlr> When you reset the progress_func field, you're overwriting one
of those behaviors, mid-operation.
<markphip> hmm, this is going to require some thought then ... sounds
like problems waiting to happen
<markphip> a lot of what is in that struct we could fairly safely not
change, but some of the things we do regularly
<dlr> If you don't write to it, like I said, you should be good.
<dlr> ok
<dlr> What else, other than progress_func and progress_baton?
<markphip> well, we add the conflict stuff during merge .. then remove it
<markphip> I think the log message callback probably only gets set
during that operation
<markphip> I suppose in both of those cases though, another thread
would not be changing it
<dlr> I don't quite understand why you need to remove the callbacks.
<dlr> You're using objects for callbacks; do these objects contain
some state that's specific to a single invocation of the callback?
<markphip> in the case of the merge one, it is related to the UI and
the callback knowing which merge it pertains to etc...
<markphip> the merge one might contain some state. The others ...
probably not so much
<dlr> then why remove them?
<markphip> the progress one has some info about what operation it was
running so it know when to clear the counters etc...
<dlr> okay, so a few callbacks have state
<markphip> I doubt we remove things like the log message callback
<markphip> but we probably replace it
<dlr> Yeah, I don't see a need there.
<dlr> That could theoretically be a problem, too.
<markphip> I'd have to look, it could always be the same object
<dlr> (see Java's double-checked locking)
<dlr> What you could do, for the guys you need to add state for, is
use a SVNClient extension that uses thread-local storage for the
callbacks.
<markphip> So if we eventually create more than one SVNClient, the
thing that started this many months ago, is that we should call
dispose() when we are done with it?
<dlr> Ideally, no matter how many you create, you should call dispose().
<dlr> Have you used Java's ThreadLocal class before?
<markphip> no
<dlr> It can be used to achieve almost "magical" behavior.
<dlr> It's really easy to mis-use, too, and results in code that's
harder for newcomers to follow.
<dlr> But for the original author, it's quite easy to use.
<dlr> Basically, it gives you a hashtable that's associated with the
"current" thread (whichever that may be).
<dlr> So when you call SVNClient.setProgressListener(), it would in
turn call ThreadLocal.set().
<dlr> ...associating the supplied ProgressListener with the current thread.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: JavaHL: Thread safety and Progress Notifications
Posted by Mark Phippard <ma...@gmail.com>.
On Nov 26, 2007 10:37 PM, David Glasser <gl...@davidglasser.net> wrote:
>
> On Nov 26, 2007 6:37 AM, Mark Phippard <ma...@gmail.com> wrote:
> > I have been getting some crashes lately in Subclipse (using 1.5
> > trunk). Kind of sporadic, not obvious recipe. I finally bothered to
> > take the time to look at the crash output (this is on OSX). I saw
> > this:
> >
> > Thread 22 Crashed:
> > 0 libjvm.dylib 0x9bc16fb8 JVM_MonitorWait + 11444
> > 1 libjvm.dylib 0x9bcd98bb JVM_IsInterrupted + 1797
> > 2 libsvnjavahl-1.0.dylib 0x50cf8f6c
> > JNIEnv_::CallVoidMethod(_jobject*, _jmethodID*, ...) + 52 (jni.h:1034)
> > 3 libsvnjavahl-1.0.dylib 0x50cde64d
> > ProgressListener::onProgress(long long, long long, apr_pool_t*) + 495
> > (ProgressListener.cpp:128)
> > 4 libsvnjavahl-1.0.dylib 0x50cde6c6
> > ProgressListener::progress(long long, long long, void*, apr_pool_t*) +
> > 82 (ProgressListener.cpp:77)
> > 5 libsvn_ra_neon-1.0.dylib 0x50c946f0 ra_neon_neonprogress + 99
> > 6 libneon.26.dylib 0x515e2f0f ne_read_response_block + 145
> > 7 libneon.26.dylib 0x515e33b5 ne_discard_response + 34
> > 8 libneon.26.dylib 0x515e45aa ne_request_dispatch + 46
> > 9 libsvn_ra_neon-1.0.dylib 0x50c98c18 svn_ra_neon__request_dispatch + 238
> > 10 libsvn_ra_neon-1.0.dylib 0x50c98598 parsed_request + 577
> > 11 libsvn_ra_neon-1.0.dylib 0x50c987ba svn_ra_neon__parsed_request + 114
> > 12 libsvn_ra_neon-1.0.dylib 0x50c8a71c reporter_finish_report + 539
> > 13 libsvn_client-1.0.dylib 0x512e3190 reporter_finish_report + 433
> > 14 libsvn_wc-1.0.dylib 0x50ffeff0 svn_wc_crawl_revisions3 + 1613
> > 15 libsvn_client-1.0.dylib 0x512e380a svn_client_status3 + 1605
> > 16 libsvnjavahl-1.0.dylib 0x50ce74c5 SVNClient::status(char
> > const*, svn_depth_t, bool, bool, bool, bool, StatusCallback*) + 335
> > (SVNClient.cpp:172)
> > 17 libsvnjavahl-1.0.dylib 0x50cef3b1
> > Java_org_tigris_subversion_javahl_SVNClient_status + 323
> > (org_tigris_subversion_javahl_SVNClient.cpp:179)
> >
> > I think I know what the problem is. We recently started using the new
> > progress notifications. I think what happens is I take an option to
> > do an Update. We set the progress listener so that we can provide
> > feedback. At some point during this process, Eclipse kicks off a
> > status refresh in another thread (also normal). There is only one
> > JavaHL object. When the status operation starts, the progress
> > listener is still set because update is running. When update ends, it
> > is nulling out the progress listener.
> >
> > SVNClient.setProgressListener(null);
> >
> > Status goes to send a progress notification, and something is now null.
> >
> > I have changed Subclipse so that we no longer null out the listener,
> > and instead just swallow the notifications that we do not want inside
> > our object. I wonder if there is some kind of safety checks that can
> > be added in the JavaHL C++ code though so that it does not crash. I
> > do not think what I am doing is that abnormal so someone else might
> > run into this.
>
> Is .setProgressListener new in 1.5?
Yes.
> I'm not very familiar with JavaHL, but it seems like the problem is
> having a static progress listener in the first place. Is there really
> no context object or something that can get the progress listener?
The method is allowing us to provide the implementation of the
callback. See the IRC thread for more details from Dan. I do not
think there is anything static here, but since I am using a single
SVNClient object in multiple threads, I have to be careful to not
change the context.
Keep in mind that when I say it used in multiple threads, the
overwhelming majority of the time it is just one thing running (and we
also have some of our own synchronization via the Eclipse job
management support). But, due to the highly threaded nature of
Eclipse, it is definitely possible for things like status refreshes to
happen while other operations are running.
In this case, I do not think this is too hard to solve. We really
only need one progress listener, and we really did not have any need
to set it and null it around each operation. So we can just set it
once and forget it. I am a little worried about other things that are
stored in the context where maybe this will not always be the case.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: JavaHL: Thread safety and Progress Notifications
Posted by David Glasser <gl...@davidglasser.net>.
On Nov 26, 2007 6:37 AM, Mark Phippard <ma...@gmail.com> wrote:
> I have been getting some crashes lately in Subclipse (using 1.5
> trunk). Kind of sporadic, not obvious recipe. I finally bothered to
> take the time to look at the crash output (this is on OSX). I saw
> this:
>
> Thread 22 Crashed:
> 0 libjvm.dylib 0x9bc16fb8 JVM_MonitorWait + 11444
> 1 libjvm.dylib 0x9bcd98bb JVM_IsInterrupted + 1797
> 2 libsvnjavahl-1.0.dylib 0x50cf8f6c
> JNIEnv_::CallVoidMethod(_jobject*, _jmethodID*, ...) + 52 (jni.h:1034)
> 3 libsvnjavahl-1.0.dylib 0x50cde64d
> ProgressListener::onProgress(long long, long long, apr_pool_t*) + 495
> (ProgressListener.cpp:128)
> 4 libsvnjavahl-1.0.dylib 0x50cde6c6
> ProgressListener::progress(long long, long long, void*, apr_pool_t*) +
> 82 (ProgressListener.cpp:77)
> 5 libsvn_ra_neon-1.0.dylib 0x50c946f0 ra_neon_neonprogress + 99
> 6 libneon.26.dylib 0x515e2f0f ne_read_response_block + 145
> 7 libneon.26.dylib 0x515e33b5 ne_discard_response + 34
> 8 libneon.26.dylib 0x515e45aa ne_request_dispatch + 46
> 9 libsvn_ra_neon-1.0.dylib 0x50c98c18 svn_ra_neon__request_dispatch + 238
> 10 libsvn_ra_neon-1.0.dylib 0x50c98598 parsed_request + 577
> 11 libsvn_ra_neon-1.0.dylib 0x50c987ba svn_ra_neon__parsed_request + 114
> 12 libsvn_ra_neon-1.0.dylib 0x50c8a71c reporter_finish_report + 539
> 13 libsvn_client-1.0.dylib 0x512e3190 reporter_finish_report + 433
> 14 libsvn_wc-1.0.dylib 0x50ffeff0 svn_wc_crawl_revisions3 + 1613
> 15 libsvn_client-1.0.dylib 0x512e380a svn_client_status3 + 1605
> 16 libsvnjavahl-1.0.dylib 0x50ce74c5 SVNClient::status(char
> const*, svn_depth_t, bool, bool, bool, bool, StatusCallback*) + 335
> (SVNClient.cpp:172)
> 17 libsvnjavahl-1.0.dylib 0x50cef3b1
> Java_org_tigris_subversion_javahl_SVNClient_status + 323
> (org_tigris_subversion_javahl_SVNClient.cpp:179)
>
> I think I know what the problem is. We recently started using the new
> progress notifications. I think what happens is I take an option to
> do an Update. We set the progress listener so that we can provide
> feedback. At some point during this process, Eclipse kicks off a
> status refresh in another thread (also normal). There is only one
> JavaHL object. When the status operation starts, the progress
> listener is still set because update is running. When update ends, it
> is nulling out the progress listener.
>
> SVNClient.setProgressListener(null);
>
> Status goes to send a progress notification, and something is now null.
>
> I have changed Subclipse so that we no longer null out the listener,
> and instead just swallow the notifications that we do not want inside
> our object. I wonder if there is some kind of safety checks that can
> be added in the JavaHL C++ code though so that it does not crash. I
> do not think what I am doing is that abnormal so someone else might
> run into this.
Is .setProgressListener new in 1.5?
I'm not very familiar with JavaHL, but it seems like the problem is
having a static progress listener in the first place. Is there really
no context object or something that can get the progress listener?
--dave
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org