You are viewing a plain text version of this content. The canonical link for it is here.
Posted to slide-dev@jakarta.apache.org by James Mason <ma...@apache.org> on 2004/07/30 02:00:31 UTC
More on deadlocks
A description of the behavior I'm seeing can be found at
http://www.mail-archive.com/slide-dev@jakarta.apache.org/msg11007.html.
I haven't tracked down the source of this problem yet, but I have some
more information and I'm hoping something will jump out at someone more
familiar with the code.
Here are two snips from MySQL's general log file. The first is a failed
PUT of a single file using a CVS checkout from the beginning of the
week. The second is a successful PUT of the same file using 2.1M1. I
don't think the versions are significant, I just couldn't get my
checkout install to start working again after I broke it ;).
---------------- failed -------------------
3 Query SELECT 1 FROM VERSION_HISTORY vh, URI u WHERE vh.URI_ID =
u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO = '1.0'
3 Query commit
3 Query rollback
3 Query SET autocommit=1
3 Query SET autocommit=0
3 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
3 Query select vc.CONTENT from VERSION_CONTENT vc, VERSION_HISTORY
vh, URI u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID
and u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
4 Connect slide@localhost on slide
4 Init DB slide
4 Query select round('inf'), round('-inf'), round('nan')
4 Query SHOW VARIABLES
4 Query SET autocommit=1
4 Query SET autocommit=0
4 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
4 Query select 1 from VERSION_CONTENT vc, VERSION_HISTORY vh, URI
u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID and
u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
4 Query delete VERSION_CONTENT from VERSION_CONTENT vc,
VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
vh.REVISION_NO = '1.0' and vh.URI_ID=u.URI_ID AND
u.URI_STRING='/files/2nd Floor.jpg'
4 Query rollback
4 Query rollback
4 Query SET autocommit=1
---------------- successful -------------------
3 Query SELECT 1 FROM VERSION_HISTORY vh, URI u WHERE vh.URI_ID =
u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO = '1.0'
3 Query commit
3 Query rollback
3 Query SET autocommit=1
3 Query SET autocommit=0
3 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
3 Query select 1 from VERSION_CONTENT vc, VERSION_HISTORY vh, URI
u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID and
u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
3 Query delete VERSION_CONTENT from VERSION_CONTENT vc,
VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
vh.REVISION_NO = '1.0' and vh.URI_ID=u.URI_ID AND
u.URI_STRING='/files/2nd Floor.jpg'
3 Query select 1 from VERSION v, URI u where v.URI_ID = u.URI_ID
and u.URI_STRING = '/files/2nd Floor.jpg'
3 Query select 1 from VERSION_HISTORY vh, URI u where vh.URI_ID =
u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO = '1.0'
3 Query insert into VERSION_CONTENT (VERSION_ID, CONTENT) select
vh.VERSION_ID, '[... some binary junk ...]’
----------- end logs --------------
Right off I'm seeing that the failed operation opens a second connection
(the first column is the connection number) to the database to store the
content *without* committing the first transaction. I'm pretty sure this
is the cause of the deadlock.
The other odd behavior is that the failed PUT calls
retrieveRevisionContent just before it opens the second connection to
store the content. This is strange, because the file being uploaded
doesn't exist, so there's nothing in the database to retrieve. I've also
noticed in Slide's logs that on PUTs where a deadlock occurs GetMethod
throws an IllegalStateException while trying to set the size of the
response buffer (see the linked to message above).
-James
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
Not quite sure, I still think it would, as the stores will dynamically
join the transaction, right? Maybe I am missing something here...
Oliver
Andreas Probst wrote:
> Hi Oliver,
>
> for me it sounds reasonable to execute all statements inside the
> transactions. I think this would also clear the source code as
> enlisting and delisting of stores wouldn't be necessary any
> more, would it?
>
> Regards,
>
> Andreas
>
> On 2 Aug 2004 at 9:08, Oliver Zeigermann wrote:
>
>
>>Referring to GET this is correct and intentional. The original idea was
>>that there is no need to have a pure read request inside of a
>>transaction. However, this design has been made up before I joined the
>>Slide community and I no longer feel it makes any sense. I would opt for
>>having all requests inside of transactions. I remember there was an
>>issue with automatic user creation which failed in a read request, as it
>>was not part of a transaction.
>>
>>Would do you people think? Shall we have all methods executed inside of
>>transactions? Shall I start a vote?
>>
>>Oliver
>>
>>James Mason wrote:
>>
>>
>>>I'm see in AbstractWebdavMethod that setForceStoreEnlistment is only
>>>being set to true during an external transaction. Is this intentional?
>>>
>>>-James
>>>
>>>James Mason wrote:
>>>
>>>
>>>>Well, I think I know what's happening now. Here's a stack trace from a
>>>>failed GET:
>>>>
>>>>java.lang.Throwable
>>>> at
>>>>org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
>>>>
>>>> at
>>>>org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
>>>>
>>>> at
>>>>org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
>>>>
>>>> at
>>>>org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
>>>>
>>>> at
>>>>org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
>>>> at
>>>>org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
>>>> at
>>>>org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
>>>>
>>>>
>>>>If you compare line numbers with the code you'll notice that this
>>>>request is happening outside of a transaction. Line 1287 in
>>>>AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
>>>>ends up happening is all the classes assume someone else will end up
>>>>closing the connection, and it never gets closed. If I set the max
>>>>pool size to 1 GetMethod never returns.
>>>>
>>>>I'll look into why SlideToken doesn't think it's part of a
>>>>transaction, but I figured you might be able to find it faster :).
>>>>
>>>>-James
>>>>
>>>>Oliver Zeigermann wrote:
>>>>
>>>>
>>>>>James Mason wrote:
>>>>>
>>>>>
>>>>>>More below.
>>>>>>
>>>>>>Oliver Zeigermann wrote:
>>>>>>
>>>>>>
>>>>>>>Another thing you may want to try. In
>>>>>>>org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is
>>>>>>>a read only request outside of a transaction a new connection is
>>>>>>>retrieved from the pool, the request is done and the connection
>>>>>>>closed. Maybe MySQL needs a commit or rollback before the
>>>>>>>connection close? Maybe it starts a transaction even with a read
>>>>>>>request? I do not know. To be on the save side try to add it and do
>>>>>>>it quick replace:
>>>>>>>
>>>>>>>
>>>>>>>> connection.close();
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>with
>>>>>>>
>>>>>>>
>>>>>>>> connection.commit();
>>>>>>>> connection.close();
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>But leave sequence methods as they are.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>I modified retrieveObject() in AbstractRDBMSStore. Is this the
>>>>>>method you meant? I haven't noticed any change in behavior.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>Try this modification with *all* connection.close() statements in
>>>>>AbstractRDBMSStore, except the ones for sequences.
>>>>>
>>>>>
>>>>>>>Oliver
>>>>>>>
>>>>>>>Oliver Zeigermann wrote:
>>>>>>>
>>>>>>>
>>>>>>>>All this is in one transaction? Only a single client? Then a
>>>>>>>>*real* deadlock is indeed unlikely.
>>>>>>>>
>>>>>>>>There is a pre-check outside of the real transaction to see if the
>>>>>>>>resource you request is a collection. If so a HTML page will be
>>>>>>>>generated. Maybe this is the source of the problem. To check,
>>>>>>>>please comment out that check and see if it works. The check is
>>>>>>>>done in org.apache.slide.webdav.WebdavServlet in line 153 with
>>>>>>>>isCollection(req). Try replacing it with false for the test.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>Tried this too. Didn't help.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>Do not mean to let you down, but this was my last idea. Isn't there
>>>>>anyone out there experiencing the same? You can't be the only one
>>>>>using MySQL...
>>>>>
>>>>>
>>>>>>I'm going to try to figure out when/where the second connection is
>>>>>>getting opened. Let me know if you think of anything else I could try.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>Please tell if there is anything I can do to help you with this.
>>>>>
>>>>>Oliver
>>>>>
>>>>>---------------------------------------------------------------------
>>>>>To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>>>>For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>>>For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>>
>>>>
>>>>
>>>
>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>>For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Andreas Probst <an...@gmx.net>.
Hi Oliver,
for me it sounds reasonable to execute all statements inside the
transactions. I think this would also clear the source code as
enlisting and delisting of stores wouldn't be necessary any
more, would it?
Regards,
Andreas
On 2 Aug 2004 at 9:08, Oliver Zeigermann wrote:
> Referring to GET this is correct and intentional. The original idea was
> that there is no need to have a pure read request inside of a
> transaction. However, this design has been made up before I joined the
> Slide community and I no longer feel it makes any sense. I would opt for
> having all requests inside of transactions. I remember there was an
> issue with automatic user creation which failed in a read request, as it
> was not part of a transaction.
>
> Would do you people think? Shall we have all methods executed inside of
> transactions? Shall I start a vote?
>
> Oliver
>
> James Mason wrote:
>
> > I'm see in AbstractWebdavMethod that setForceStoreEnlistment is only
> > being set to true during an external transaction. Is this intentional?
> >
> > -James
> >
> > James Mason wrote:
> >
> >> Well, I think I know what's happening now. Here's a stack trace from a
> >> failed GET:
> >>
> >> java.lang.Throwable
> >> at
> >> org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
> >>
> >> at
> >> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
> >>
> >> at
> >> org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
> >>
> >> at
> >> org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
> >>
> >> at
> >> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
> >> at
> >> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
> >> at
> >> org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
> >>
> >>
> >> If you compare line numbers with the code you'll notice that this
> >> request is happening outside of a transaction. Line 1287 in
> >> AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
> >> ends up happening is all the classes assume someone else will end up
> >> closing the connection, and it never gets closed. If I set the max
> >> pool size to 1 GetMethod never returns.
> >>
> >> I'll look into why SlideToken doesn't think it's part of a
> >> transaction, but I figured you might be able to find it faster :).
> >>
> >> -James
> >>
> >> Oliver Zeigermann wrote:
> >>
> >>> James Mason wrote:
> >>>
> >>>> More below.
> >>>>
> >>>> Oliver Zeigermann wrote:
> >>>>
> >>>>> Another thing you may want to try. In
> >>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is
> >>>>> a read only request outside of a transaction a new connection is
> >>>>> retrieved from the pool, the request is done and the connection
> >>>>> closed. Maybe MySQL needs a commit or rollback before the
> >>>>> connection close? Maybe it starts a transaction even with a read
> >>>>> request? I do not know. To be on the save side try to add it and do
> >>>>> it quick replace:
> >>>>>
> >>>>>> connection.close();
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> with
> >>>>>
> >>>>>> connection.commit();
> >>>>>> connection.close();
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> But leave sequence methods as they are.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> I modified retrieveObject() in AbstractRDBMSStore. Is this the
> >>>> method you meant? I haven't noticed any change in behavior.
> >>>
> >>>
> >>>
> >>>
> >>> Try this modification with *all* connection.close() statements in
> >>> AbstractRDBMSStore, except the ones for sequences.
> >>>
> >>>>>
> >>>>> Oliver
> >>>>>
> >>>>> Oliver Zeigermann wrote:
> >>>>>
> >>>>>> All this is in one transaction? Only a single client? Then a
> >>>>>> *real* deadlock is indeed unlikely.
> >>>>>>
> >>>>>> There is a pre-check outside of the real transaction to see if the
> >>>>>> resource you request is a collection. If so a HTML page will be
> >>>>>> generated. Maybe this is the source of the problem. To check,
> >>>>>> please comment out that check and see if it works. The check is
> >>>>>> done in org.apache.slide.webdav.WebdavServlet in line 153 with
> >>>>>> isCollection(req). Try replacing it with false for the test.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Tried this too. Didn't help.
> >>>
> >>>
> >>>
> >>>
> >>> Do not mean to let you down, but this was my last idea. Isn't there
> >>> anyone out there experiencing the same? You can't be the only one
> >>> using MySQL...
> >>>
> >>>> I'm going to try to figure out when/where the second connection is
> >>>> getting opened. Let me know if you think of anything else I could try.
> >>>
> >>>
> >>>
> >>>
> >>> Please tell if there is anything I can do to help you with this.
> >>>
> >>> Oliver
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> >>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >>>
> >>>
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> >> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >>
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
This should be done with my latest commit. Could anyone check if I have
got them all?
Thanks,
Oliver
Oliver Zeigermann wrote:
> Hmmm, thinking about it. I should really call commit on temporary
> connections before closing to be on the save side...
>
> Oliver
>
> Oliver Zeigermann wrote:
>
>> So, you implemented your own connection pool? Cool stuff... If it
>> works what should be the problem.
>>
>> I just checked DBCP the default connection pool for Slide and it
>> actually seems connections do not get committed or rolled back before
>> they are returned to the pool. However, I understand that no new
>> transaction is started on a connection when there is no write request
>> in it, right?
>>
>> Oliver
>>
>> Andreas Probst wrote:
>>
>>> on 2 Aug 2004 at 17:30, Oliver Zeigermann wrote:
>>>
>>>>>>> I think a vote would be good. Since with the RDBMS stores a read
>>>>>>> request can lock the database it should probably happen in a
>>>>>>> transaction. I'm still learning the intricacies of transactions,
>>>>>>> though, so don't take my word on it :).
>>>>>>
>>>>>>
>>>>>>
>>>>>> Not quite sure what you mean here. If the read request is not part
>>>>>> of a transaction the locks will only be there for the duration of
>>>>>> this single read statement, not for the duration of the whole
>>>>>> transaction. This is better, isn't is?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> With implicit transactions there will be database transactions
>>>>> started automatically. These will last until the Connection.close().
>>>>> Actually one should avoid closing connections. With a connection
>>>>> pool a connection will never be closed so the transaction will last
>>>>> until a commit or rollback.
>>>>
>>>>
>>>>
>>>> Not quite sure what your intention is here. Could you please clarify?
>>>>
>>>
>>>
>>> For our project I'm bound to a Slide head version of late November
>>> 2003, where there wasn't a database connection pool yet. I think now
>>> there is one somehow.
>>> Within a few minutes there were hundreds of connections opened and
>>> closed. This works for MS SQL Server on a Windows Server machine.
>>> However, on Windows XP the closed connections were not freed fast
>>> enough. So I got NoFreeSocket or so inside the SQLException. With
>>> Oracle opening and closing connections all the time would not work at
>>> all, no matter which OS. (This is what our database expert says.)
>>> That's why I implemented a database connection pool. A call to
>>> getConnection() or getNewConnection() or so in AbstractDBMSStore or
>>> so (I don't know the names by heart and now I'm on vacation ;)
>>> returns a connection from the pool of connections. The connections
>>> are bound to the current thread. Within a single thread the same
>>> connection is returned. Within another thread another connection is
>>> returned. When close() is called on the connection wrapper object, it
>>> is not really closed but put back to the pool of connections.
>>>
>>> With this pool I have a fixed number of connection during the whole
>>> runtime of the server. In my tests I archived more than 1 million
>>> documents in one run. So this works.
>>>
>>> As there is always the same connection returned within the same
>>> thread, all statements live within the transaction, which was started
>>> in the WebDAV layer. Even if the stores are not enlisted in the
>>> transaction, they really are, because they work on the same database
>>> connection.
>>>
>>> I hope this does not cause any problems. So far it works fine for the
>>> plain WebDAV operations.
>>>
>>> I hope this explains what I meant...
>>>
>>> Andreas
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
Hmmm, thinking about it. I should really call commit on temporary
connections before closing to be on the save side...
Oliver
Oliver Zeigermann wrote:
> So, you implemented your own connection pool? Cool stuff... If it works
> what should be the problem.
>
> I just checked DBCP the default connection pool for Slide and it
> actually seems connections do not get committed or rolled back before
> they are returned to the pool. However, I understand that no new
> transaction is started on a connection when there is no write request in
> it, right?
>
> Oliver
>
> Andreas Probst wrote:
>
>> on 2 Aug 2004 at 17:30, Oliver Zeigermann wrote:
>>
>>>>>> I think a vote would be good. Since with the RDBMS stores a read
>>>>>> request can lock the database it should probably happen in a
>>>>>> transaction. I'm still learning the intricacies of transactions,
>>>>>> though, so don't take my word on it :).
>>>>>
>>>>>
>>>>> Not quite sure what you mean here. If the read request is not part
>>>>> of a transaction the locks will only be there for the duration of
>>>>> this single read statement, not for the duration of the whole
>>>>> transaction. This is better, isn't is?
>>>>
>>>>
>>>>
>>>> With implicit transactions there will be database transactions
>>>> started automatically. These will last until the Connection.close().
>>>> Actually one should avoid closing connections. With a connection
>>>> pool a connection will never be closed so the transaction will last
>>>> until a commit or rollback.
>>>
>>>
>>> Not quite sure what your intention is here. Could you please clarify?
>>>
>>
>>
>> For our project I'm bound to a Slide head version of late November
>> 2003, where there wasn't a database connection pool yet. I think now
>> there is one somehow.
>> Within a few minutes there were hundreds of connections opened and
>> closed. This works for MS SQL Server on a Windows Server machine.
>> However, on Windows XP the closed connections were not freed fast
>> enough. So I got NoFreeSocket or so inside the SQLException. With
>> Oracle opening and closing connections all the time would not work at
>> all, no matter which OS. (This is what our database expert says.)
>> That's why I implemented a database connection pool. A call to
>> getConnection() or getNewConnection() or so in AbstractDBMSStore or so
>> (I don't know the names by heart and now I'm on vacation ;) returns a
>> connection from the pool of connections. The connections are bound to
>> the current thread. Within a single thread the same connection is
>> returned. Within another thread another connection is returned. When
>> close() is called on the connection wrapper object, it is not really
>> closed but put back to the pool of connections.
>>
>> With this pool I have a fixed number of connection during the whole
>> runtime of the server. In my tests I archived more than 1 million
>> documents in one run. So this works.
>>
>> As there is always the same connection returned within the same
>> thread, all statements live within the transaction, which was started
>> in the WebDAV layer. Even if the stores are not enlisted in the
>> transaction, they really are, because they work on the same database
>> connection.
>>
>> I hope this does not cause any problems. So far it works fine for the
>> plain WebDAV operations.
>>
>> I hope this explains what I meant...
>>
>> Andreas
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
So, you implemented your own connection pool? Cool stuff... If it works
what should be the problem.
I just checked DBCP the default connection pool for Slide and it
actually seems connections do not get committed or rolled back before
they are returned to the pool. However, I understand that no new
transaction is started on a connection when there is no write request in
it, right?
Oliver
Andreas Probst wrote:
> on 2 Aug 2004 at 17:30, Oliver Zeigermann wrote:
>
>>>>>I think a vote would be good. Since with the RDBMS stores a read request
>>>>>can lock the database it should probably happen in a transaction. I'm
>>>>>still learning the intricacies of transactions, though, so don't take my
>>>>>word on it :).
>>>>
>>>>Not quite sure what you mean here. If the read request is not part of a
>>>>transaction the locks will only be there for the duration of this single
>>>>read statement, not for the duration of the whole transaction. This is
>>>>better, isn't is?
>>>
>>>
>>>With implicit transactions there will be database transactions
>>>started automatically. These will last until the
>>>Connection.close().
>>>
>>>Actually one should avoid closing connections. With a connection
>>>pool a connection will never be closed so the transaction will
>>>last until a commit or rollback.
>>
>>Not quite sure what your intention is here. Could you please clarify?
>>
>
>
> For our project I'm bound to a Slide head version of late
> November 2003, where there wasn't a database connection pool
> yet. I think now there is one somehow.
>
> Within a few minutes there were hundreds of connections opened
> and closed. This works for MS SQL Server on a Windows Server
> machine. However, on Windows XP the closed connections were not
> freed fast enough. So I got NoFreeSocket or so inside the
> SQLException. With Oracle opening and closing connections all
> the time would not work at all, no matter which OS. (This is
> what our database expert says.)
>
> That's why I implemented a database connection pool. A call to
> getConnection() or getNewConnection() or so in AbstractDBMSStore
> or so (I don't know the names by heart and now I'm on vacation
> ;) returns a connection from the pool of connections. The
> connections are bound to the current thread. Within a single
> thread the same connection is returned. Within another thread
> another connection is returned. When close() is called on the
> connection wrapper object, it is not really closed but put back
> to the pool of connections.
>
> With this pool I have a fixed number of connection during the
> whole runtime of the server. In my tests I archived more than 1
> million documents in one run. So this works.
>
> As there is always the same connection returned within the same
> thread, all statements live within the transaction, which was
> started in the WebDAV layer. Even if the stores are not enlisted
> in the transaction, they really are, because they work on the
> same database connection.
>
> I hope this does not cause any problems. So far it works fine
> for the plain WebDAV operations.
>
> I hope this explains what I meant...
>
> Andreas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Andreas Probst <an...@gmx.net>.
on 2 Aug 2004 at 17:30, Oliver Zeigermann wrote:
> >>>I think a vote would be good. Since with the RDBMS stores a read request
> >>>can lock the database it should probably happen in a transaction. I'm
> >>>still learning the intricacies of transactions, though, so don't take my
> >>>word on it :).
> >>
> >>Not quite sure what you mean here. If the read request is not part of a
> >>transaction the locks will only be there for the duration of this single
> >>read statement, not for the duration of the whole transaction. This is
> >>better, isn't is?
> >
> >
> > With implicit transactions there will be database transactions
> > started automatically. These will last until the
> > Connection.close().
> >
> > Actually one should avoid closing connections. With a connection
> > pool a connection will never be closed so the transaction will
> > last until a commit or rollback.
>
> Not quite sure what your intention is here. Could you please clarify?
>
For our project I'm bound to a Slide head version of late
November 2003, where there wasn't a database connection pool
yet. I think now there is one somehow.
Within a few minutes there were hundreds of connections opened
and closed. This works for MS SQL Server on a Windows Server
machine. However, on Windows XP the closed connections were not
freed fast enough. So I got NoFreeSocket or so inside the
SQLException. With Oracle opening and closing connections all
the time would not work at all, no matter which OS. (This is
what our database expert says.)
That's why I implemented a database connection pool. A call to
getConnection() or getNewConnection() or so in AbstractDBMSStore
or so (I don't know the names by heart and now I'm on vacation
;) returns a connection from the pool of connections. The
connections are bound to the current thread. Within a single
thread the same connection is returned. Within another thread
another connection is returned. When close() is called on the
connection wrapper object, it is not really closed but put back
to the pool of connections.
With this pool I have a fixed number of connection during the
whole runtime of the server. In my tests I archived more than 1
million documents in one run. So this works.
As there is always the same connection returned within the same
thread, all statements live within the transaction, which was
started in the WebDAV layer. Even if the stores are not enlisted
in the transaction, they really are, because they work on the
same database connection.
I hope this does not cause any problems. So far it works fine
for the plain WebDAV operations.
I hope this explains what I meant...
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
Andreas Probst wrote:
> Please read between the lines:
>
> On 2 Aug 2004 at 9:46, Oliver Zeigermann wrote:
>
>
>>James Mason wrote:
>>
>>
>>>I think a vote would be good. Since with the RDBMS stores a read request
>>>can lock the database it should probably happen in a transaction. I'm
>>>still learning the intricacies of transactions, though, so don't take my
>>>word on it :).
>>
>>Not quite sure what you mean here. If the read request is not part of a
>>transaction the locks will only be there for the duration of this single
>>read statement, not for the duration of the whole transaction. This is
>>better, isn't is?
>
>
> With implicit transactions there will be database transactions
> started automatically. These will last until the
> Connection.close().
>
> Actually one should avoid closing connections. With a connection
> pool a connection will never be closed so the transaction will
> last until a commit or rollback.
Not quite sure what your intention is here. Could you please clarify?
Thanks,
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Andreas Probst <an...@gmx.net>.
Please read between the lines:
On 2 Aug 2004 at 9:46, Oliver Zeigermann wrote:
> James Mason wrote:
>
> > I think a vote would be good. Since with the RDBMS stores a read request
> > can lock the database it should probably happen in a transaction. I'm
> > still learning the intricacies of transactions, though, so don't take my
> > word on it :).
>
> Not quite sure what you mean here. If the read request is not part of a
> transaction the locks will only be there for the duration of this single
> read statement, not for the duration of the whole transaction. This is
> better, isn't is?
With implicit transactions there will be database transactions
started automatically. These will last until the
Connection.close().
Actually one should avoid closing connections. With a connection
pool a connection will never be closed so the transaction will
last until a commit or rollback.
> However, the drawback is things may change between two
> reads inside one and the same GET request, which *might* lead to
> inconsistent data. This is another pro to have GET start its own
> transaction.
>
> > As for the error, how do you feel about retrieveRevisionContent() in
> > StandardRDBMSAdapter calling NodeRevisionContent.setContent() with a
> > zero length byte array when the content length is equal to zero? This
> > would save a little time, since a database query would be unnecessary,
> > and it would allow the connection to be closed immediately.
>
> This generally makes sense, but I suspect you want to remove the symptom
> of the problem you discovered, not the real cause ;)
>
> Oliver
>
> > -James
> >
> > Oliver Zeigermann wrote:
> >
> >> Referring to GET this is correct and intentional. The original idea
> >> was that there is no need to have a pure read request inside of a
> >> transaction. However, this design has been made up before I joined the
> >> Slide community and I no longer feel it makes any sense. I would opt
> >> for having all requests inside of transactions. I remember there was
> >> an issue with automatic user creation which failed in a read request,
> >> as it was not part of a transaction.
> >>
> >> Would do you people think? Shall we have all methods executed inside
> >> of transactions? Shall I start a vote?
> >>
> >> Oliver
> >>
> >> James Mason wrote:
> >>
> >>> I'm see in AbstractWebdavMethod that setForceStoreEnlistment is only
> >>> being set to true during an external transaction. Is this intentional?
> >>>
> >>> -James
> >>>
> >>> James Mason wrote:
> >>>
> >>>> Well, I think I know what's happening now. Here's a stack trace from
> >>>> a failed GET:
> >>>>
> >>>> java.lang.Throwable
> >>>> at
> >>>> org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
> >>>>
> >>>> at
> >>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
> >>>>
> >>>> at
> >>>> org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
> >>>>
> >>>> at
> >>>> org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
> >>>>
> >>>> at
> >>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
> >>>> at
> >>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
> >>>> at
> >>>> org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
> >>>>
> >>>>
> >>>> If you compare line numbers with the code you'll notice that this
> >>>> request is happening outside of a transaction. Line 1287 in
> >>>> AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
> >>>> ends up happening is all the classes assume someone else will end up
> >>>> closing the connection, and it never gets closed. If I set the max
> >>>> pool size to 1 GetMethod never returns.
> >>>>
> >>>> I'll look into why SlideToken doesn't think it's part of a
> >>>> transaction, but I figured you might be able to find it faster :).
> >>>>
> >>>> -James
> >>>>
> >>>> Oliver Zeigermann wrote:
> >>>>
> >>>>> James Mason wrote:
> >>>>>
> >>>>>> More below.
> >>>>>>
> >>>>>> Oliver Zeigermann wrote:
> >>>>>>
> >>>>>>> Another thing you may want to try. In
> >>>>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there
> >>>>>>> is a read only request outside of a transaction a new connection
> >>>>>>> is retrieved from the pool, the request is done and the
> >>>>>>> connection closed. Maybe MySQL needs a commit or rollback before
> >>>>>>> the connection close? Maybe it starts a transaction even with a
> >>>>>>> read request? I do not know. To be on the save side try to add it
> >>>>>>> and do it quick replace:
> >>>>>>>
> >>>>>>>> connection.close();
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> with
> >>>>>>>
> >>>>>>>> connection.commit();
> >>>>>>>> connection.close();
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> But leave sequence methods as they are.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> I modified retrieveObject() in AbstractRDBMSStore. Is this the
> >>>>>> method you meant? I haven't noticed any change in behavior.
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> Try this modification with *all* connection.close() statements in
> >>>>> AbstractRDBMSStore, except the ones for sequences.
> >>>>>
> >>>>>>>
> >>>>>>> Oliver
> >>>>>>>
> >>>>>>> Oliver Zeigermann wrote:
> >>>>>>>
> >>>>>>>> All this is in one transaction? Only a single client? Then a
> >>>>>>>> *real* deadlock is indeed unlikely.
> >>>>>>>>
> >>>>>>>> There is a pre-check outside of the real transaction to see if
> >>>>>>>> the resource you request is a collection. If so a HTML page will
> >>>>>>>> be generated. Maybe this is the source of the problem. To check,
> >>>>>>>> please comment out that check and see if it works. The check is
> >>>>>>>> done in org.apache.slide.webdav.WebdavServlet in line 153 with
> >>>>>>>> isCollection(req). Try replacing it with false for the test.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Tried this too. Didn't help.
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> Do not mean to let you down, but this was my last idea. Isn't there
> >>>>> anyone out there experiencing the same? You can't be the only one
> >>>>> using MySQL...
> >>>>>
> >>>>>> I'm going to try to figure out when/where the second connection is
> >>>>>> getting opened. Let me know if you think of anything else I could
> >>>>>> try.
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> Please tell if there is anything I can do to help you with this.
> >>>>>
> >>>>> Oliver
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> >>>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> >>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> >>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> >> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >>
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: slide-dev-help@jakarta.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
James Mason wrote:
> I think a vote would be good. Since with the RDBMS stores a read request
> can lock the database it should probably happen in a transaction. I'm
> still learning the intricacies of transactions, though, so don't take my
> word on it :).
Not quite sure what you mean here. If the read request is not part of a
transaction the locks will only be there for the duration of this single
read statement, not for the duration of the whole transaction. This is
better, isn't is? However, the drawback is things may change between two
reads inside one and the same GET request, which *might* lead to
inconsistent data. This is another pro to have GET start its own
transaction.
> As for the error, how do you feel about retrieveRevisionContent() in
> StandardRDBMSAdapter calling NodeRevisionContent.setContent() with a
> zero length byte array when the content length is equal to zero? This
> would save a little time, since a database query would be unnecessary,
> and it would allow the connection to be closed immediately.
This generally makes sense, but I suspect you want to remove the symptom
of the problem you discovered, not the real cause ;)
Oliver
> -James
>
> Oliver Zeigermann wrote:
>
>> Referring to GET this is correct and intentional. The original idea
>> was that there is no need to have a pure read request inside of a
>> transaction. However, this design has been made up before I joined the
>> Slide community and I no longer feel it makes any sense. I would opt
>> for having all requests inside of transactions. I remember there was
>> an issue with automatic user creation which failed in a read request,
>> as it was not part of a transaction.
>>
>> Would do you people think? Shall we have all methods executed inside
>> of transactions? Shall I start a vote?
>>
>> Oliver
>>
>> James Mason wrote:
>>
>>> I'm see in AbstractWebdavMethod that setForceStoreEnlistment is only
>>> being set to true during an external transaction. Is this intentional?
>>>
>>> -James
>>>
>>> James Mason wrote:
>>>
>>>> Well, I think I know what's happening now. Here's a stack trace from
>>>> a failed GET:
>>>>
>>>> java.lang.Throwable
>>>> at
>>>> org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
>>>>
>>>> at
>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
>>>>
>>>> at
>>>> org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
>>>>
>>>> at
>>>> org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
>>>>
>>>> at
>>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
>>>> at
>>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
>>>> at
>>>> org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
>>>>
>>>>
>>>> If you compare line numbers with the code you'll notice that this
>>>> request is happening outside of a transaction. Line 1287 in
>>>> AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
>>>> ends up happening is all the classes assume someone else will end up
>>>> closing the connection, and it never gets closed. If I set the max
>>>> pool size to 1 GetMethod never returns.
>>>>
>>>> I'll look into why SlideToken doesn't think it's part of a
>>>> transaction, but I figured you might be able to find it faster :).
>>>>
>>>> -James
>>>>
>>>> Oliver Zeigermann wrote:
>>>>
>>>>> James Mason wrote:
>>>>>
>>>>>> More below.
>>>>>>
>>>>>> Oliver Zeigermann wrote:
>>>>>>
>>>>>>> Another thing you may want to try. In
>>>>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there
>>>>>>> is a read only request outside of a transaction a new connection
>>>>>>> is retrieved from the pool, the request is done and the
>>>>>>> connection closed. Maybe MySQL needs a commit or rollback before
>>>>>>> the connection close? Maybe it starts a transaction even with a
>>>>>>> read request? I do not know. To be on the save side try to add it
>>>>>>> and do it quick replace:
>>>>>>>
>>>>>>>> connection.close();
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> with
>>>>>>>
>>>>>>>> connection.commit();
>>>>>>>> connection.close();
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> But leave sequence methods as they are.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> I modified retrieveObject() in AbstractRDBMSStore. Is this the
>>>>>> method you meant? I haven't noticed any change in behavior.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Try this modification with *all* connection.close() statements in
>>>>> AbstractRDBMSStore, except the ones for sequences.
>>>>>
>>>>>>>
>>>>>>> Oliver
>>>>>>>
>>>>>>> Oliver Zeigermann wrote:
>>>>>>>
>>>>>>>> All this is in one transaction? Only a single client? Then a
>>>>>>>> *real* deadlock is indeed unlikely.
>>>>>>>>
>>>>>>>> There is a pre-check outside of the real transaction to see if
>>>>>>>> the resource you request is a collection. If so a HTML page will
>>>>>>>> be generated. Maybe this is the source of the problem. To check,
>>>>>>>> please comment out that check and see if it works. The check is
>>>>>>>> done in org.apache.slide.webdav.WebdavServlet in line 153 with
>>>>>>>> isCollection(req). Try replacing it with false for the test.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Tried this too. Didn't help.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Do not mean to let you down, but this was my last idea. Isn't there
>>>>> anyone out there experiencing the same? You can't be the only one
>>>>> using MySQL...
>>>>>
>>>>>> I'm going to try to figure out when/where the second connection is
>>>>>> getting opened. Let me know if you think of anything else I could
>>>>>> try.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Please tell if there is anything I can do to help you with this.
>>>>>
>>>>> Oliver
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>>
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by James Mason <ma...@apache.org>.
I think a vote would be good. Since with the RDBMS stores a read request
can lock the database it should probably happen in a transaction. I'm
still learning the intricacies of transactions, though, so don't take my
word on it :).
As for the error, how do you feel about retrieveRevisionContent() in
StandardRDBMSAdapter calling NodeRevisionContent.setContent() with a
zero length byte array when the content length is equal to zero? This
would save a little time, since a database query would be unnecessary,
and it would allow the connection to be closed immediately.
-James
Oliver Zeigermann wrote:
> Referring to GET this is correct and intentional. The original idea was
> that there is no need to have a pure read request inside of a
> transaction. However, this design has been made up before I joined the
> Slide community and I no longer feel it makes any sense. I would opt for
> having all requests inside of transactions. I remember there was an
> issue with automatic user creation which failed in a read request, as it
> was not part of a transaction.
>
> Would do you people think? Shall we have all methods executed inside of
> transactions? Shall I start a vote?
>
> Oliver
>
> James Mason wrote:
>
>> I'm see in AbstractWebdavMethod that setForceStoreEnlistment is only
>> being set to true during an external transaction. Is this intentional?
>>
>> -James
>>
>> James Mason wrote:
>>
>>> Well, I think I know what's happening now. Here's a stack trace from
>>> a failed GET:
>>>
>>> java.lang.Throwable
>>> at
>>> org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
>>>
>>> at
>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
>>>
>>> at
>>> org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
>>>
>>> at
>>> org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
>>>
>>> at
>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
>>> at
>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
>>> at
>>> org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
>>>
>>>
>>> If you compare line numbers with the code you'll notice that this
>>> request is happening outside of a transaction. Line 1287 in
>>> AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
>>> ends up happening is all the classes assume someone else will end up
>>> closing the connection, and it never gets closed. If I set the max
>>> pool size to 1 GetMethod never returns.
>>>
>>> I'll look into why SlideToken doesn't think it's part of a
>>> transaction, but I figured you might be able to find it faster :).
>>>
>>> -James
>>>
>>> Oliver Zeigermann wrote:
>>>
>>>> James Mason wrote:
>>>>
>>>>> More below.
>>>>>
>>>>> Oliver Zeigermann wrote:
>>>>>
>>>>>> Another thing you may want to try. In
>>>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is
>>>>>> a read only request outside of a transaction a new connection is
>>>>>> retrieved from the pool, the request is done and the connection
>>>>>> closed. Maybe MySQL needs a commit or rollback before the
>>>>>> connection close? Maybe it starts a transaction even with a read
>>>>>> request? I do not know. To be on the save side try to add it and
>>>>>> do it quick replace:
>>>>>>
>>>>>>> connection.close();
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> with
>>>>>>
>>>>>>> connection.commit();
>>>>>>> connection.close();
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> But leave sequence methods as they are.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I modified retrieveObject() in AbstractRDBMSStore. Is this the
>>>>> method you meant? I haven't noticed any change in behavior.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Try this modification with *all* connection.close() statements in
>>>> AbstractRDBMSStore, except the ones for sequences.
>>>>
>>>>>>
>>>>>> Oliver
>>>>>>
>>>>>> Oliver Zeigermann wrote:
>>>>>>
>>>>>>> All this is in one transaction? Only a single client? Then a
>>>>>>> *real* deadlock is indeed unlikely.
>>>>>>>
>>>>>>> There is a pre-check outside of the real transaction to see if
>>>>>>> the resource you request is a collection. If so a HTML page will
>>>>>>> be generated. Maybe this is the source of the problem. To check,
>>>>>>> please comment out that check and see if it works. The check is
>>>>>>> done in org.apache.slide.webdav.WebdavServlet in line 153 with
>>>>>>> isCollection(req). Try replacing it with false for the test.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Tried this too. Didn't help.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Do not mean to let you down, but this was my last idea. Isn't there
>>>> anyone out there experiencing the same? You can't be the only one
>>>> using MySQL...
>>>>
>>>>> I'm going to try to figure out when/where the second connection is
>>>>> getting opened. Let me know if you think of anything else I could try.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Please tell if there is anything I can do to help you with this.
>>>>
>>>> Oliver
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>>
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
Referring to GET this is correct and intentional. The original idea was
that there is no need to have a pure read request inside of a
transaction. However, this design has been made up before I joined the
Slide community and I no longer feel it makes any sense. I would opt for
having all requests inside of transactions. I remember there was an
issue with automatic user creation which failed in a read request, as it
was not part of a transaction.
Would do you people think? Shall we have all methods executed inside of
transactions? Shall I start a vote?
Oliver
James Mason wrote:
> I'm see in AbstractWebdavMethod that setForceStoreEnlistment is only
> being set to true during an external transaction. Is this intentional?
>
> -James
>
> James Mason wrote:
>
>> Well, I think I know what's happening now. Here's a stack trace from a
>> failed GET:
>>
>> java.lang.Throwable
>> at
>> org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
>>
>> at
>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
>>
>> at
>> org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
>>
>> at
>> org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
>>
>> at
>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
>> at
>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
>> at
>> org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
>>
>>
>> If you compare line numbers with the code you'll notice that this
>> request is happening outside of a transaction. Line 1287 in
>> AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
>> ends up happening is all the classes assume someone else will end up
>> closing the connection, and it never gets closed. If I set the max
>> pool size to 1 GetMethod never returns.
>>
>> I'll look into why SlideToken doesn't think it's part of a
>> transaction, but I figured you might be able to find it faster :).
>>
>> -James
>>
>> Oliver Zeigermann wrote:
>>
>>> James Mason wrote:
>>>
>>>> More below.
>>>>
>>>> Oliver Zeigermann wrote:
>>>>
>>>>> Another thing you may want to try. In
>>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is
>>>>> a read only request outside of a transaction a new connection is
>>>>> retrieved from the pool, the request is done and the connection
>>>>> closed. Maybe MySQL needs a commit or rollback before the
>>>>> connection close? Maybe it starts a transaction even with a read
>>>>> request? I do not know. To be on the save side try to add it and do
>>>>> it quick replace:
>>>>>
>>>>>> connection.close();
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> with
>>>>>
>>>>>> connection.commit();
>>>>>> connection.close();
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> But leave sequence methods as they are.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> I modified retrieveObject() in AbstractRDBMSStore. Is this the
>>>> method you meant? I haven't noticed any change in behavior.
>>>
>>>
>>>
>>>
>>> Try this modification with *all* connection.close() statements in
>>> AbstractRDBMSStore, except the ones for sequences.
>>>
>>>>>
>>>>> Oliver
>>>>>
>>>>> Oliver Zeigermann wrote:
>>>>>
>>>>>> All this is in one transaction? Only a single client? Then a
>>>>>> *real* deadlock is indeed unlikely.
>>>>>>
>>>>>> There is a pre-check outside of the real transaction to see if the
>>>>>> resource you request is a collection. If so a HTML page will be
>>>>>> generated. Maybe this is the source of the problem. To check,
>>>>>> please comment out that check and see if it works. The check is
>>>>>> done in org.apache.slide.webdav.WebdavServlet in line 153 with
>>>>>> isCollection(req). Try replacing it with false for the test.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Tried this too. Didn't help.
>>>
>>>
>>>
>>>
>>> Do not mean to let you down, but this was my last idea. Isn't there
>>> anyone out there experiencing the same? You can't be the only one
>>> using MySQL...
>>>
>>>> I'm going to try to figure out when/where the second connection is
>>>> getting opened. Let me know if you think of anything else I could try.
>>>
>>>
>>>
>>>
>>> Please tell if there is anything I can do to help you with this.
>>>
>>> Oliver
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by James Mason <ma...@apache.org>.
I'm see in AbstractWebdavMethod that setForceStoreEnlistment is only
being set to true during an external transaction. Is this intentional?
-James
James Mason wrote:
> Well, I think I know what's happening now. Here's a stack trace from a
> failed GET:
>
> java.lang.Throwable
> at
> org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
>
> at
> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
>
> at
> org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
>
> at
> org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
>
> at org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
> at org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
> at
> org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
>
> If you compare line numbers with the code you'll notice that this
> request is happening outside of a transaction. Line 1287 in
> AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
> ends up happening is all the classes assume someone else will end up
> closing the connection, and it never gets closed. If I set the max pool
> size to 1 GetMethod never returns.
>
> I'll look into why SlideToken doesn't think it's part of a transaction,
> but I figured you might be able to find it faster :).
>
> -James
>
> Oliver Zeigermann wrote:
>
>> James Mason wrote:
>>
>>> More below.
>>>
>>> Oliver Zeigermann wrote:
>>>
>>>> Another thing you may want to try. In
>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is a
>>>> read only request outside of a transaction a new connection is
>>>> retrieved from the pool, the request is done and the connection
>>>> closed. Maybe MySQL needs a commit or rollback before the connection
>>>> close? Maybe it starts a transaction even with a read request? I do
>>>> not know. To be on the save side try to add it and do it quick replace:
>>>>
>>>>> connection.close();
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> with
>>>>
>>>>> connection.commit();
>>>>> connection.close();
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> But leave sequence methods as they are.
>>>
>>>
>>>
>>>
>>> I modified retrieveObject() in AbstractRDBMSStore. Is this the method
>>> you meant? I haven't noticed any change in behavior.
>>
>>
>>
>> Try this modification with *all* connection.close() statements in
>> AbstractRDBMSStore, except the ones for sequences.
>>
>>>>
>>>> Oliver
>>>>
>>>> Oliver Zeigermann wrote:
>>>>
>>>>> All this is in one transaction? Only a single client? Then a *real*
>>>>> deadlock is indeed unlikely.
>>>>>
>>>>> There is a pre-check outside of the real transaction to see if the
>>>>> resource you request is a collection. If so a HTML page will be
>>>>> generated. Maybe this is the source of the problem. To check,
>>>>> please comment out that check and see if it works. The check is
>>>>> done in org.apache.slide.webdav.WebdavServlet in line 153 with
>>>>> isCollection(req). Try replacing it with false for the test.
>>>
>>>
>>>
>>>
>>> Tried this too. Didn't help.
>>
>>
>>
>> Do not mean to let you down, but this was my last idea. Isn't there
>> anyone out there experiencing the same? You can't be the only one
>> using MySQL...
>>
>>> I'm going to try to figure out when/where the second connection is
>>> getting opened. Let me know if you think of anything else I could try.
>>
>>
>>
>> Please tell if there is anything I can do to help you with this.
>>
>> Oliver
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
One more argument for all requests in transactions :)
Oliver
James Mason wrote:
> Well, since putting all requests into transactions I've been unable to
> cause a deadlock. I'm now trying to track down random 404 errors while
> uploading files, but the deadlock seems to have gone away.
>
> -James
>
> Oliver Zeigermann wrote:
>
>> James Mason wrote:
>>
>>> Oliver Zeigermann wrote:
>>>
>>>> Hmm, first of all I notice our line number do not match. I suppose
>>>> this is because of the removal of author tags and not relevant to
>>>> our problem, do you agree?
>>>
>>>
>>>
>>>
>>> Agreed. I've been adding debugging statements and I may have
>>> forgetten to remove a few.
>>>
>>>>
>>>> I see what you mean and I really wonder why the connection never is
>>>> closed. Are you using compression mode?
>>>
>>>
>>>
>>>
>>> I'm not using compression at the moment. I don't suppose the fact
>>> that there's no content could be affecting the way the input stream
>>> is handled? I looked, and it doesn't seem like it. The reason I
>>> mention this is WebFolders seems to do one of two things when
>>> uploading a file. It either executes a HEAD then PUT, or it executes
>>> PUT, PROPFIND, GET then PUT with the first PUT having no body.
>>
>>
>>
>> I do think an empty input stream should be any different from a
>> non-empty one, but I really do not know. A possible scenario might be
>> the JDBC driver sees the BLOB is empty and does not request it from
>> the DB in the fist place. Who knows? I guess in those complex
>> scenarios almost everything is possible.
>>
>> Oliver
>>
>>> -James
>>>
>>>> When looking at method retrieveRevisionContent in
>>>> StandardRDBMSAdapter (the one MySQLAdapter uses as well) the
>>>> connection gets closed by
>>>>
>>>>> if (temporaryConnection) {
>>>>> // XXX is needed, as calling store does not
>>>>> know if connection should be closed now
>>>>> connection.close();
>>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> If you do not use compression the connection will be closed inside
>>>> of JDBCAwareInputStream which will be copied in GetMethod.copy
>>>>
>>>> So, this seems to be alright (at least for me).
>>>>
>>>> I, however, notice the original input stream from the BLOB never
>>>> gets closed in compression mode. In MS SQLServer (my default DB)
>>>> this seems to be done when the statement or resultset gets closed,
>>>> but maybe not with MySQL. Maybe MySQL can not free the connection as
>>>> long as the input stream has not been closed? Simply do not know.
>>>> Please test by switchting over to non compressed mode (if you use it
>>>> in the first place).
>>>>
>>>> Hope this helps and we get closed to get this solved...
>>>>
>>>> Oliver
>>>>
>>>>
>>>> James Mason wrote:
>>>>
>>>>> Well, I think I know what's happening now. Here's a stack trace
>>>>> from a failed GET:
>>>>>
>>>>> java.lang.Throwable
>>>>> at
>>>>> org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
>>>>>
>>>>> at
>>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
>>>>>
>>>>> at
>>>>> org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
>>>>>
>>>>> at
>>>>> org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
>>>>>
>>>>> at
>>>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
>>>>> at
>>>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
>>>>> at
>>>>> org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
>>>>>
>>>>>
>>>>> If you compare line numbers with the code you'll notice that this
>>>>> request is happening outside of a transaction. Line 1287 in
>>>>> AbstractStore (isForceStoreEnlistment(uri)) is returning false.
>>>>> What ends up happening is all the classes assume someone else will
>>>>> end up closing the connection, and it never gets closed. If I set
>>>>> the max pool size to 1 GetMethod never returns.
>>>>>
>>>>> I'll look into why SlideToken doesn't think it's part of a
>>>>> transaction, but I figured you might be able to find it faster :).
>>>>>
>>>>> -James
>>>>>
>>>>> Oliver Zeigermann wrote:
>>>>>
>>>>>> James Mason wrote:
>>>>>>
>>>>>>> More below.
>>>>>>>
>>>>>>> Oliver Zeigermann wrote:
>>>>>>>
>>>>>>>> Another thing you may want to try. In
>>>>>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there
>>>>>>>> is a read only request outside of a transaction a new connection
>>>>>>>> is retrieved from the pool, the request is done and the
>>>>>>>> connection closed. Maybe MySQL needs a commit or rollback before
>>>>>>>> the connection close? Maybe it starts a transaction even with a
>>>>>>>> read request? I do not know. To be on the save side try to add
>>>>>>>> it and do it quick replace:
>>>>>>>>
>>>>>>>>> connection.close();
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> with
>>>>>>>>
>>>>>>>>> connection.commit();
>>>>>>>>> connection.close();
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> But leave sequence methods as they are.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I modified retrieveObject() in AbstractRDBMSStore. Is this the
>>>>>>> method you meant? I haven't noticed any change in behavior.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Try this modification with *all* connection.close() statements in
>>>>>> AbstractRDBMSStore, except the ones for sequences.
>>>>>>
>>>>>>>>
>>>>>>>> Oliver
>>>>>>>>
>>>>>>>> Oliver Zeigermann wrote:
>>>>>>>>
>>>>>>>>> All this is in one transaction? Only a single client? Then a
>>>>>>>>> *real* deadlock is indeed unlikely.
>>>>>>>>>
>>>>>>>>> There is a pre-check outside of the real transaction to see if
>>>>>>>>> the resource you request is a collection. If so a HTML page
>>>>>>>>> will be generated. Maybe this is the source of the problem. To
>>>>>>>>> check, please comment out that check and see if it works. The
>>>>>>>>> check is done in org.apache.slide.webdav.WebdavServlet in line
>>>>>>>>> 153 with isCollection(req). Try replacing it with false for the
>>>>>>>>> test.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Tried this too. Didn't help.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Do not mean to let you down, but this was my last idea. Isn't
>>>>>> there anyone out there experiencing the same? You can't be the
>>>>>> only one using MySQL...
>>>>>>
>>>>>>> I'm going to try to figure out when/where the second connection
>>>>>>> is getting opened. Let me know if you think of anything else I
>>>>>>> could try.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Please tell if there is anything I can do to help you with this.
>>>>>>
>>>>>> Oliver
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by James Mason <ma...@apache.org>.
Well, since putting all requests into transactions I've been unable to
cause a deadlock. I'm now trying to track down random 404 errors while
uploading files, but the deadlock seems to have gone away.
-James
Oliver Zeigermann wrote:
> James Mason wrote:
>
>> Oliver Zeigermann wrote:
>>
>>> Hmm, first of all I notice our line number do not match. I suppose
>>> this is because of the removal of author tags and not relevant to our
>>> problem, do you agree?
>>
>>
>>
>> Agreed. I've been adding debugging statements and I may have forgetten
>> to remove a few.
>>
>>>
>>> I see what you mean and I really wonder why the connection never is
>>> closed. Are you using compression mode?
>>
>>
>>
>> I'm not using compression at the moment. I don't suppose the fact that
>> there's no content could be affecting the way the input stream is
>> handled? I looked, and it doesn't seem like it. The reason I mention
>> this is WebFolders seems to do one of two things when uploading a
>> file. It either executes a HEAD then PUT, or it executes PUT,
>> PROPFIND, GET then PUT with the first PUT having no body.
>
>
> I do think an empty input stream should be any different from a
> non-empty one, but I really do not know. A possible scenario might be
> the JDBC driver sees the BLOB is empty and does not request it from the
> DB in the fist place. Who knows? I guess in those complex scenarios
> almost everything is possible.
>
> Oliver
>
>> -James
>>
>>> When looking at method retrieveRevisionContent in
>>> StandardRDBMSAdapter (the one MySQLAdapter uses as well) the
>>> connection gets closed by
>>>
>>>> if (temporaryConnection) {
>>>> // XXX is needed, as calling store does not
>>>> know if connection should be closed now
>>>> connection.close();
>>>> }
>>>
>>>
>>>
>>>
>>> If you do not use compression the connection will be closed inside of
>>> JDBCAwareInputStream which will be copied in GetMethod.copy
>>>
>>> So, this seems to be alright (at least for me).
>>>
>>> I, however, notice the original input stream from the BLOB never gets
>>> closed in compression mode. In MS SQLServer (my default DB) this
>>> seems to be done when the statement or resultset gets closed, but
>>> maybe not with MySQL. Maybe MySQL can not free the connection as long
>>> as the input stream has not been closed? Simply do not know. Please
>>> test by switchting over to non compressed mode (if you use it in the
>>> first place).
>>>
>>> Hope this helps and we get closed to get this solved...
>>>
>>> Oliver
>>>
>>>
>>> James Mason wrote:
>>>
>>>> Well, I think I know what's happening now. Here's a stack trace from
>>>> a failed GET:
>>>>
>>>> java.lang.Throwable
>>>> at
>>>> org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
>>>>
>>>> at
>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
>>>>
>>>> at
>>>> org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
>>>>
>>>> at
>>>> org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
>>>>
>>>> at
>>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
>>>> at
>>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
>>>> at
>>>> org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
>>>>
>>>>
>>>> If you compare line numbers with the code you'll notice that this
>>>> request is happening outside of a transaction. Line 1287 in
>>>> AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
>>>> ends up happening is all the classes assume someone else will end up
>>>> closing the connection, and it never gets closed. If I set the max
>>>> pool size to 1 GetMethod never returns.
>>>>
>>>> I'll look into why SlideToken doesn't think it's part of a
>>>> transaction, but I figured you might be able to find it faster :).
>>>>
>>>> -James
>>>>
>>>> Oliver Zeigermann wrote:
>>>>
>>>>> James Mason wrote:
>>>>>
>>>>>> More below.
>>>>>>
>>>>>> Oliver Zeigermann wrote:
>>>>>>
>>>>>>> Another thing you may want to try. In
>>>>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there
>>>>>>> is a read only request outside of a transaction a new connection
>>>>>>> is retrieved from the pool, the request is done and the
>>>>>>> connection closed. Maybe MySQL needs a commit or rollback before
>>>>>>> the connection close? Maybe it starts a transaction even with a
>>>>>>> read request? I do not know. To be on the save side try to add it
>>>>>>> and do it quick replace:
>>>>>>>
>>>>>>>> connection.close();
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> with
>>>>>>>
>>>>>>>> connection.commit();
>>>>>>>> connection.close();
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> But leave sequence methods as they are.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> I modified retrieveObject() in AbstractRDBMSStore. Is this the
>>>>>> method you meant? I haven't noticed any change in behavior.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Try this modification with *all* connection.close() statements in
>>>>> AbstractRDBMSStore, except the ones for sequences.
>>>>>
>>>>>>>
>>>>>>> Oliver
>>>>>>>
>>>>>>> Oliver Zeigermann wrote:
>>>>>>>
>>>>>>>> All this is in one transaction? Only a single client? Then a
>>>>>>>> *real* deadlock is indeed unlikely.
>>>>>>>>
>>>>>>>> There is a pre-check outside of the real transaction to see if
>>>>>>>> the resource you request is a collection. If so a HTML page will
>>>>>>>> be generated. Maybe this is the source of the problem. To check,
>>>>>>>> please comment out that check and see if it works. The check is
>>>>>>>> done in org.apache.slide.webdav.WebdavServlet in line 153 with
>>>>>>>> isCollection(req). Try replacing it with false for the test.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Tried this too. Didn't help.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Do not mean to let you down, but this was my last idea. Isn't there
>>>>> anyone out there experiencing the same? You can't be the only one
>>>>> using MySQL...
>>>>>
>>>>>> I'm going to try to figure out when/where the second connection is
>>>>>> getting opened. Let me know if you think of anything else I could
>>>>>> try.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Please tell if there is anything I can do to help you with this.
>>>>>
>>>>> Oliver
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
James Mason wrote:
> Oliver Zeigermann wrote:
>
>> Hmm, first of all I notice our line number do not match. I suppose
>> this is because of the removal of author tags and not relevant to our
>> problem, do you agree?
>
>
> Agreed. I've been adding debugging statements and I may have forgetten
> to remove a few.
>
>>
>> I see what you mean and I really wonder why the connection never is
>> closed. Are you using compression mode?
>
>
> I'm not using compression at the moment. I don't suppose the fact that
> there's no content could be affecting the way the input stream is
> handled? I looked, and it doesn't seem like it. The reason I mention
> this is WebFolders seems to do one of two things when uploading a file.
> It either executes a HEAD then PUT, or it executes PUT, PROPFIND, GET
> then PUT with the first PUT having no body.
I do think an empty input stream should be any different from a
non-empty one, but I really do not know. A possible scenario might be
the JDBC driver sees the BLOB is empty and does not request it from the
DB in the fist place. Who knows? I guess in those complex scenarios
almost everything is possible.
Oliver
> -James
>
>> When looking at method retrieveRevisionContent in StandardRDBMSAdapter
>> (the one MySQLAdapter uses as well) the connection gets closed by
>>
>>> if (temporaryConnection) {
>>> // XXX is needed, as calling store does not
>>> know if connection should be closed now
>>> connection.close();
>>> }
>>
>>
>>
>> If you do not use compression the connection will be closed inside of
>> JDBCAwareInputStream which will be copied in GetMethod.copy
>>
>> So, this seems to be alright (at least for me).
>>
>> I, however, notice the original input stream from the BLOB never gets
>> closed in compression mode. In MS SQLServer (my default DB) this seems
>> to be done when the statement or resultset gets closed, but maybe not
>> with MySQL. Maybe MySQL can not free the connection as long as the
>> input stream has not been closed? Simply do not know. Please test by
>> switchting over to non compressed mode (if you use it in the first
>> place).
>>
>> Hope this helps and we get closed to get this solved...
>>
>> Oliver
>>
>>
>> James Mason wrote:
>>
>>> Well, I think I know what's happening now. Here's a stack trace from
>>> a failed GET:
>>>
>>> java.lang.Throwable
>>> at
>>> org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
>>>
>>> at
>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
>>>
>>> at
>>> org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
>>>
>>> at
>>> org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
>>>
>>> at
>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
>>> at
>>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
>>> at
>>> org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
>>>
>>>
>>> If you compare line numbers with the code you'll notice that this
>>> request is happening outside of a transaction. Line 1287 in
>>> AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
>>> ends up happening is all the classes assume someone else will end up
>>> closing the connection, and it never gets closed. If I set the max
>>> pool size to 1 GetMethod never returns.
>>>
>>> I'll look into why SlideToken doesn't think it's part of a
>>> transaction, but I figured you might be able to find it faster :).
>>>
>>> -James
>>>
>>> Oliver Zeigermann wrote:
>>>
>>>> James Mason wrote:
>>>>
>>>>> More below.
>>>>>
>>>>> Oliver Zeigermann wrote:
>>>>>
>>>>>> Another thing you may want to try. In
>>>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is
>>>>>> a read only request outside of a transaction a new connection is
>>>>>> retrieved from the pool, the request is done and the connection
>>>>>> closed. Maybe MySQL needs a commit or rollback before the
>>>>>> connection close? Maybe it starts a transaction even with a read
>>>>>> request? I do not know. To be on the save side try to add it and
>>>>>> do it quick replace:
>>>>>>
>>>>>>> connection.close();
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> with
>>>>>>
>>>>>>> connection.commit();
>>>>>>> connection.close();
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> But leave sequence methods as they are.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I modified retrieveObject() in AbstractRDBMSStore. Is this the
>>>>> method you meant? I haven't noticed any change in behavior.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Try this modification with *all* connection.close() statements in
>>>> AbstractRDBMSStore, except the ones for sequences.
>>>>
>>>>>>
>>>>>> Oliver
>>>>>>
>>>>>> Oliver Zeigermann wrote:
>>>>>>
>>>>>>> All this is in one transaction? Only a single client? Then a
>>>>>>> *real* deadlock is indeed unlikely.
>>>>>>>
>>>>>>> There is a pre-check outside of the real transaction to see if
>>>>>>> the resource you request is a collection. If so a HTML page will
>>>>>>> be generated. Maybe this is the source of the problem. To check,
>>>>>>> please comment out that check and see if it works. The check is
>>>>>>> done in org.apache.slide.webdav.WebdavServlet in line 153 with
>>>>>>> isCollection(req). Try replacing it with false for the test.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Tried this too. Didn't help.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Do not mean to let you down, but this was my last idea. Isn't there
>>>> anyone out there experiencing the same? You can't be the only one
>>>> using MySQL...
>>>>
>>>>> I'm going to try to figure out when/where the second connection is
>>>>> getting opened. Let me know if you think of anything else I could try.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Please tell if there is anything I can do to help you with this.
>>>>
>>>> Oliver
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by James Mason <ma...@apache.org>.
Oliver Zeigermann wrote:
> Hmm, first of all I notice our line number do not match. I suppose this
> is because of the removal of author tags and not relevant to our
> problem, do you agree?
Agreed. I've been adding debugging statements and I may have forgetten
to remove a few.
>
> I see what you mean and I really wonder why the connection never is
> closed. Are you using compression mode?
I'm not using compression at the moment. I don't suppose the fact that
there's no content could be affecting the way the input stream is
handled? I looked, and it doesn't seem like it. The reason I mention
this is WebFolders seems to do one of two things when uploading a file.
It either executes a HEAD then PUT, or it executes PUT, PROPFIND, GET
then PUT with the first PUT having no body.
-James
> When looking at method
> retrieveRevisionContent in StandardRDBMSAdapter (the one MySQLAdapter
> uses as well) the connection gets closed by
>
>> if (temporaryConnection) {
>> // XXX is needed, as calling store does not
>> know if connection should be closed now
>> connection.close();
>> }
>
>
> If you do not use compression the connection will be closed inside of
> JDBCAwareInputStream which will be copied in GetMethod.copy
>
> So, this seems to be alright (at least for me).
>
> I, however, notice the original input stream from the BLOB never gets
> closed in compression mode. In MS SQLServer (my default DB) this seems
> to be done when the statement or resultset gets closed, but maybe not
> with MySQL. Maybe MySQL can not free the connection as long as the input
> stream has not been closed? Simply do not know. Please test by
> switchting over to non compressed mode (if you use it in the first place).
>
> Hope this helps and we get closed to get this solved...
>
> Oliver
>
>
> James Mason wrote:
>
>> Well, I think I know what's happening now. Here's a stack trace from a
>> failed GET:
>>
>> java.lang.Throwable
>> at
>> org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
>>
>> at
>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
>>
>> at
>> org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
>>
>> at
>> org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
>>
>> at
>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
>> at
>> org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
>> at
>> org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
>>
>>
>> If you compare line numbers with the code you'll notice that this
>> request is happening outside of a transaction. Line 1287 in
>> AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
>> ends up happening is all the classes assume someone else will end up
>> closing the connection, and it never gets closed. If I set the max
>> pool size to 1 GetMethod never returns.
>>
>> I'll look into why SlideToken doesn't think it's part of a
>> transaction, but I figured you might be able to find it faster :).
>>
>> -James
>>
>> Oliver Zeigermann wrote:
>>
>>> James Mason wrote:
>>>
>>>> More below.
>>>>
>>>> Oliver Zeigermann wrote:
>>>>
>>>>> Another thing you may want to try. In
>>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is
>>>>> a read only request outside of a transaction a new connection is
>>>>> retrieved from the pool, the request is done and the connection
>>>>> closed. Maybe MySQL needs a commit or rollback before the
>>>>> connection close? Maybe it starts a transaction even with a read
>>>>> request? I do not know. To be on the save side try to add it and do
>>>>> it quick replace:
>>>>>
>>>>>> connection.close();
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> with
>>>>>
>>>>>> connection.commit();
>>>>>> connection.close();
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> But leave sequence methods as they are.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> I modified retrieveObject() in AbstractRDBMSStore. Is this the
>>>> method you meant? I haven't noticed any change in behavior.
>>>
>>>
>>>
>>>
>>> Try this modification with *all* connection.close() statements in
>>> AbstractRDBMSStore, except the ones for sequences.
>>>
>>>>>
>>>>> Oliver
>>>>>
>>>>> Oliver Zeigermann wrote:
>>>>>
>>>>>> All this is in one transaction? Only a single client? Then a
>>>>>> *real* deadlock is indeed unlikely.
>>>>>>
>>>>>> There is a pre-check outside of the real transaction to see if the
>>>>>> resource you request is a collection. If so a HTML page will be
>>>>>> generated. Maybe this is the source of the problem. To check,
>>>>>> please comment out that check and see if it works. The check is
>>>>>> done in org.apache.slide.webdav.WebdavServlet in line 153 with
>>>>>> isCollection(req). Try replacing it with false for the test.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Tried this too. Didn't help.
>>>
>>>
>>>
>>>
>>> Do not mean to let you down, but this was my last idea. Isn't there
>>> anyone out there experiencing the same? You can't be the only one
>>> using MySQL...
>>>
>>>> I'm going to try to figure out when/where the second connection is
>>>> getting opened. Let me know if you think of anything else I could try.
>>>
>>>
>>>
>>>
>>> Please tell if there is anything I can do to help you with this.
>>>
>>> Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
Hmm, first of all I notice our line number do not match. I suppose this
is because of the removal of author tags and not relevant to our
problem, do you agree?
I see what you mean and I really wonder why the connection never is
closed. Are you using compression mode? When looking at method
retrieveRevisionContent in StandardRDBMSAdapter (the one MySQLAdapter
uses as well) the connection gets closed by
> if (temporaryConnection) {
> // XXX is needed, as calling store does not know if connection should be closed now
> connection.close();
> }
If you do not use compression the connection will be closed inside of
JDBCAwareInputStream which will be copied in GetMethod.copy
So, this seems to be alright (at least for me).
I, however, notice the original input stream from the BLOB never gets
closed in compression mode. In MS SQLServer (my default DB) this seems
to be done when the statement or resultset gets closed, but maybe not
with MySQL. Maybe MySQL can not free the connection as long as the input
stream has not been closed? Simply do not know. Please test by
switchting over to non compressed mode (if you use it in the first place).
Hope this helps and we get closed to get this solved...
Oliver
James Mason wrote:
> Well, I think I know what's happening now. Here's a stack trace from a
> failed GET:
>
> java.lang.Throwable
> at
> org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
>
> at
> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
>
> at
> org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
>
> at
> org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
>
> at org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
> at org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
> at
> org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
>
> If you compare line numbers with the code you'll notice that this
> request is happening outside of a transaction. Line 1287 in
> AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
> ends up happening is all the classes assume someone else will end up
> closing the connection, and it never gets closed. If I set the max pool
> size to 1 GetMethod never returns.
>
> I'll look into why SlideToken doesn't think it's part of a transaction,
> but I figured you might be able to find it faster :).
>
> -James
>
> Oliver Zeigermann wrote:
>
>> James Mason wrote:
>>
>>> More below.
>>>
>>> Oliver Zeigermann wrote:
>>>
>>>> Another thing you may want to try. In
>>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is a
>>>> read only request outside of a transaction a new connection is
>>>> retrieved from the pool, the request is done and the connection
>>>> closed. Maybe MySQL needs a commit or rollback before the connection
>>>> close? Maybe it starts a transaction even with a read request? I do
>>>> not know. To be on the save side try to add it and do it quick replace:
>>>>
>>>>> connection.close();
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> with
>>>>
>>>>> connection.commit();
>>>>> connection.close();
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> But leave sequence methods as they are.
>>>
>>>
>>>
>>>
>>> I modified retrieveObject() in AbstractRDBMSStore. Is this the method
>>> you meant? I haven't noticed any change in behavior.
>>
>>
>>
>> Try this modification with *all* connection.close() statements in
>> AbstractRDBMSStore, except the ones for sequences.
>>
>>>>
>>>> Oliver
>>>>
>>>> Oliver Zeigermann wrote:
>>>>
>>>>> All this is in one transaction? Only a single client? Then a *real*
>>>>> deadlock is indeed unlikely.
>>>>>
>>>>> There is a pre-check outside of the real transaction to see if the
>>>>> resource you request is a collection. If so a HTML page will be
>>>>> generated. Maybe this is the source of the problem. To check,
>>>>> please comment out that check and see if it works. The check is
>>>>> done in org.apache.slide.webdav.WebdavServlet in line 153 with
>>>>> isCollection(req). Try replacing it with false for the test.
>>>
>>>
>>>
>>>
>>> Tried this too. Didn't help.
>>
>>
>>
>> Do not mean to let you down, but this was my last idea. Isn't there
>> anyone out there experiencing the same? You can't be the only one
>> using MySQL...
>>
>>> I'm going to try to figure out when/where the second connection is
>>> getting opened. Let me know if you think of anything else I could try.
>>
>>
>>
>> Please tell if there is anything I can do to help you with this.
>>
>> Oliver
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by James Mason <ma...@apache.org>.
Well, I think I know what's happening now. Here's a stack trace from a
failed GET:
java.lang.Throwable
at
org.apache.slide.store.impl.rdbms.StandardRDBMSAdapter.retrieveRevisionContent(StandardRDBMSAdapter.java:726)
at
org.apache.slide.store.impl.rdbms.AbstractRDBMSStore.retrieveRevisionContent(AbstractRDBMSStore.java:734)
at
org.apache.slide.store.AbstractStore.retrieveRevisionContent(AbstractStore.java:1314)
at
org.apache.slide.store.ExtendedStore.retrieveRevisionContent(ExtendedStore.java:492)
at org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:352)
at org.apache.slide.content.ContentImpl.retrieve(ContentImpl.java:322)
at
org.apache.slide.webdav.method.GetMethod.executeRequest(GetMethod.java:218)
If you compare line numbers with the code you'll notice that this
request is happening outside of a transaction. Line 1287 in
AbstractStore (isForceStoreEnlistment(uri)) is returning false. What
ends up happening is all the classes assume someone else will end up
closing the connection, and it never gets closed. If I set the max pool
size to 1 GetMethod never returns.
I'll look into why SlideToken doesn't think it's part of a transaction,
but I figured you might be able to find it faster :).
-James
Oliver Zeigermann wrote:
> James Mason wrote:
>
>> More below.
>>
>> Oliver Zeigermann wrote:
>>
>>> Another thing you may want to try. In
>>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is a
>>> read only request outside of a transaction a new connection is
>>> retrieved from the pool, the request is done and the connection
>>> closed. Maybe MySQL needs a commit or rollback before the connection
>>> close? Maybe it starts a transaction even with a read request? I do
>>> not know. To be on the save side try to add it and do it quick replace:
>>>
>>>> connection.close();
>>>
>>>
>>>
>>>
>>> with
>>>
>>>> connection.commit();
>>>> connection.close();
>>>
>>>
>>>
>>>
>>> But leave sequence methods as they are.
>>
>>
>>
>> I modified retrieveObject() in AbstractRDBMSStore. Is this the method
>> you meant? I haven't noticed any change in behavior.
>
>
> Try this modification with *all* connection.close() statements in
> AbstractRDBMSStore, except the ones for sequences.
>
>>>
>>> Oliver
>>>
>>> Oliver Zeigermann wrote:
>>>
>>>> All this is in one transaction? Only a single client? Then a *real*
>>>> deadlock is indeed unlikely.
>>>>
>>>> There is a pre-check outside of the real transaction to see if the
>>>> resource you request is a collection. If so a HTML page will be
>>>> generated. Maybe this is the source of the problem. To check, please
>>>> comment out that check and see if it works. The check is done in
>>>> org.apache.slide.webdav.WebdavServlet in line 153 with
>>>> isCollection(req). Try replacing it with false for the test.
>>
>>
>>
>> Tried this too. Didn't help.
>
>
> Do not mean to let you down, but this was my last idea. Isn't there
> anyone out there experiencing the same? You can't be the only one using
> MySQL...
>
>> I'm going to try to figure out when/where the second connection is
>> getting opened. Let me know if you think of anything else I could try.
>
>
> Please tell if there is anything I can do to help you with this.
>
> Oliver
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
James Mason wrote:
> More below.
>
> Oliver Zeigermann wrote:
>
>> Another thing you may want to try. In
>> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is a
>> read only request outside of a transaction a new connection is
>> retrieved from the pool, the request is done and the connection
>> closed. Maybe MySQL needs a commit or rollback before the connection
>> close? Maybe it starts a transaction even with a read request? I do
>> not know. To be on the save side try to add it and do it quick replace:
>>
>>> connection.close();
>>
>>
>>
>> with
>>
>>> connection.commit();
>>> connection.close();
>>
>>
>>
>> But leave sequence methods as they are.
>
>
> I modified retrieveObject() in AbstractRDBMSStore. Is this the method
> you meant? I haven't noticed any change in behavior.
Try this modification with *all* connection.close() statements in
AbstractRDBMSStore, except the ones for sequences.
>>
>> Oliver
>>
>> Oliver Zeigermann wrote:
>>
>>> All this is in one transaction? Only a single client? Then a *real*
>>> deadlock is indeed unlikely.
>>>
>>> There is a pre-check outside of the real transaction to see if the
>>> resource you request is a collection. If so a HTML page will be
>>> generated. Maybe this is the source of the problem. To check, please
>>> comment out that check and see if it works. The check is done in
>>> org.apache.slide.webdav.WebdavServlet in line 153 with
>>> isCollection(req). Try replacing it with false for the test.
>
>
> Tried this too. Didn't help.
Do not mean to let you down, but this was my last idea. Isn't there
anyone out there experiencing the same? You can't be the only one using
MySQL...
> I'm going to try to figure out when/where the second connection is
> getting opened. Let me know if you think of anything else I could try.
Please tell if there is anything I can do to help you with this.
Oliver
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by James Mason <ma...@apache.org>.
More below.
Oliver Zeigermann wrote:
> Another thing you may want to try. In
> org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is a
> read only request outside of a transaction a new connection is retrieved
> from the pool, the request is done and the connection closed. Maybe
> MySQL needs a commit or rollback before the connection close? Maybe it
> starts a transaction even with a read request? I do not know. To be on
> the save side try to add it and do it quick replace:
>
>> connection.close();
>
>
> with
>
>> connection.commit();
>> connection.close();
>
>
> But leave sequence methods as they are.
I modified retrieveObject() in AbstractRDBMSStore. Is this the method
you meant? I haven't noticed any change in behavior.
>
> Oliver
>
> Oliver Zeigermann wrote:
>
>> All this is in one transaction? Only a single client? Then a *real*
>> deadlock is indeed unlikely.
>>
>> There is a pre-check outside of the real transaction to see if the
>> resource you request is a collection. If so a HTML page will be
>> generated. Maybe this is the source of the problem. To check, please
>> comment out that check and see if it works. The check is done in
>> org.apache.slide.webdav.WebdavServlet in line 153 with
>> isCollection(req). Try replacing it with false for the test.
Tried this too. Didn't help.
I'm going to try to figure out when/where the second connection is
getting opened. Let me know if you think of anything else I could try.
Thanks,
James
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
Another thing you may want to try. In
org.apache.slide.store.impl.rdbms.AbstractRDBMSStore when there is a
read only request outside of a transaction a new connection is retrieved
from the pool, the request is done and the connection closed. Maybe
MySQL needs a commit or rollback before the connection close? Maybe it
starts a transaction even with a read request? I do not know. To be on
the save side try to add it and do it quick replace:
> connection.close();
with
> connection.commit();
> connection.close();
But leave sequence methods as they are.
Oliver
Oliver Zeigermann wrote:
> All this is in one transaction? Only a single client? Then a *real*
> deadlock is indeed unlikely.
>
> There is a pre-check outside of the real transaction to see if the
> resource you request is a collection. If so a HTML page will be
> generated. Maybe this is the source of the problem. To check, please
> comment out that check and see if it works. The check is done in
> org.apache.slide.webdav.WebdavServlet in line 153 with
> isCollection(req). Try replacing it with false for the test.
>
> Oliver
>
> James Mason wrote:
>
>> Bad news :(.
>>
>> 1) With the isolation level set to serializable the database returns
>> an error after a certain timeout period. I tried setting the isolation
>> level to read_committed, but the database never returned anything and
>> slide appeared to freeze. I'll try again in the morning with logging
>> turned on to see what's actually happening.
>>
>> 2) I have and it is. The logs I posted are with sequential-mode set to
>> "full". I can try setting it to "write", but from your earlier post I
>> got the impression full had a higher level of isolation.
>>
>> Would connection 3 keep trying to commit and succeed after connection
>> 4 gives up? I notice that connection 4 does rollback and set
>> autocommit (although doesn't issue a commit), but connection 3 never
>> finishes.
>>
>> Also, I'm using WebFolders as the client here. It issues two requests
>> when uploading a file, a HEAD and then a PUT. For some reason Slide is
>> trying to retrieve the contents from the database in response to the
>> HEAD request, even though the file doesn't exist yet.
>>
>> I'm also curious what could be causing Slide to open a second
>> connection to attempt the write to the database. Only one client is
>> talking to Slide, so I would think all the requests would be part of
>> the same transaction/connection.
>>
>> -James
>>
>> Oliver Zeigermann wrote:
>>
>>> What you describe and what the trace shows looks like the classical
>>> deadlock scenario to me.
>>>
>>> You observed Connection 3 can not commit, which is because it is dead
>>> locked by Connection 4. And the other way round.
>>>
>>> If I am correct with the above analysis there are two ways to get
>>> this fixed:
>>>
>>> (1) Set isolation level to "READ_COMMITTED" or
>>> (2) Update to the latest version from CVS and take over the setting
>>> for <parameter name="sequential-mode">full</parameter>
>>> from Domain.xml
>>>
>>> With MySQL I would recommend (1) as (2) is the "if everything else
>>> fails" solution.
>>>
>>> Oliver
>>>
>>> James Mason wrote:
>>>
>>>> A description of the behavior I'm seeing can be found at
>>>> http://www.mail-archive.com/slide-dev@jakarta.apache.org/msg11007.html.
>>>> I haven't tracked down the source of this problem yet, but I have
>>>> some more information and I'm hoping something will jump out at
>>>> someone more familiar with the code.
>>>>
>>>> Here are two snips from MySQL's general log file. The first is a
>>>> failed PUT of a single file using a CVS checkout from the beginning
>>>> of the week. The second is a successful PUT of the same file using
>>>> 2.1M1. I don't think the versions are significant, I just couldn't
>>>> get my checkout install to start working again after I broke it ;).
>>>>
>>>> ---------------- failed -------------------
>>>> 3 Query SELECT 1 FROM VERSION_HISTORY vh, URI u WHERE
>>>> vh.URI_ID = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and
>>>> REVISION_NO = '1.0'
>>>> 3 Query commit
>>>> 3 Query rollback
>>>> 3 Query SET autocommit=1
>>>> 3 Query SET autocommit=0
>>>> 3 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
>>>> 3 Query select vc.CONTENT from VERSION_CONTENT vc,
>>>> VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
>>>> vh.URI_ID = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and
>>>> vh.REVISION_NO = '1.0'
>>>> 4 Connect slide@localhost on slide
>>>> 4 Init DB slide
>>>> 4 Query select round('inf'), round('-inf'), round('nan')
>>>> 4 Query SHOW VARIABLES
>>>> 4 Query SET autocommit=1
>>>> 4 Query SET autocommit=0
>>>> 4 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
>>>> 4 Query select 1 from VERSION_CONTENT vc, VERSION_HISTORY vh,
>>>> URI u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID
>>>> and u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
>>>> 4 Query delete VERSION_CONTENT from VERSION_CONTENT vc,
>>>> VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
>>>> vh.REVISION_NO = '1.0' and vh.URI_ID=u.URI_ID AND
>>>> u.URI_STRING='/files/2nd Floor.jpg'
>>>> 4 Query rollback
>>>> 4 Query rollback
>>>> 4 Query SET autocommit=1
>>>>
>>>> ---------------- successful -------------------
>>>> 3 Query SELECT 1 FROM VERSION_HISTORY vh, URI u WHERE
>>>> vh.URI_ID = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and
>>>> REVISION_NO = '1.0'
>>>> 3 Query commit
>>>> 3 Query rollback
>>>> 3 Query SET autocommit=1
>>>> 3 Query SET autocommit=0
>>>> 3 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
>>>> 3 Query select 1 from VERSION_CONTENT vc, VERSION_HISTORY vh,
>>>> URI u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID
>>>> and u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
>>>> 3 Query delete VERSION_CONTENT from VERSION_CONTENT vc,
>>>> VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
>>>> vh.REVISION_NO = '1.0' and vh.URI_ID=u.URI_ID AND
>>>> u.URI_STRING='/files/2nd Floor.jpg'
>>>> 3 Query select 1 from VERSION v, URI u where v.URI_ID =
>>>> u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg'
>>>> 3 Query select 1 from VERSION_HISTORY vh, URI u where
>>>> vh.URI_ID = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and
>>>> REVISION_NO = '1.0'
>>>> 3 Query insert into VERSION_CONTENT (VERSION_ID, CONTENT)
>>>> select vh.VERSION_ID, '[... some binary junk ...]’
>>>>
>>>> ----------- end logs --------------
>>>>
>>>> Right off I'm seeing that the failed operation opens a second
>>>> connection (the first column is the connection number) to the
>>>> database to store the content *without* committing the first
>>>> transaction. I'm pretty sure this is the cause of the deadlock.
>>>>
>>>> The other odd behavior is that the failed PUT calls
>>>> retrieveRevisionContent just before it opens the second connection
>>>> to store the content. This is strange, because the file being
>>>> uploaded doesn't exist, so there's nothing in the database to
>>>> retrieve. I've also noticed in Slide's logs that on PUTs where a
>>>> deadlock occurs GetMethod throws an IllegalStateException while
>>>> trying to set the size of the response buffer (see the linked to
>>>> message above).
>>>>
>>>> -James
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
All this is in one transaction? Only a single client? Then a *real*
deadlock is indeed unlikely.
There is a pre-check outside of the real transaction to see if the
resource you request is a collection. If so a HTML page will be
generated. Maybe this is the source of the problem. To check, please
comment out that check and see if it works. The check is done in
org.apache.slide.webdav.WebdavServlet in line 153 with
isCollection(req). Try replacing it with false for the test.
Oliver
James Mason wrote:
> Bad news :(.
>
> 1) With the isolation level set to serializable the database returns an
> error after a certain timeout period. I tried setting the isolation
> level to read_committed, but the database never returned anything and
> slide appeared to freeze. I'll try again in the morning with logging
> turned on to see what's actually happening.
>
> 2) I have and it is. The logs I posted are with sequential-mode set to
> "full". I can try setting it to "write", but from your earlier post I
> got the impression full had a higher level of isolation.
>
> Would connection 3 keep trying to commit and succeed after connection 4
> gives up? I notice that connection 4 does rollback and set autocommit
> (although doesn't issue a commit), but connection 3 never finishes.
>
> Also, I'm using WebFolders as the client here. It issues two requests
> when uploading a file, a HEAD and then a PUT. For some reason Slide is
> trying to retrieve the contents from the database in response to the
> HEAD request, even though the file doesn't exist yet.
>
> I'm also curious what could be causing Slide to open a second connection
> to attempt the write to the database. Only one client is talking to
> Slide, so I would think all the requests would be part of the same
> transaction/connection.
>
> -James
>
> Oliver Zeigermann wrote:
>
>> What you describe and what the trace shows looks like the classical
>> deadlock scenario to me.
>>
>> You observed Connection 3 can not commit, which is because it is dead
>> locked by Connection 4. And the other way round.
>>
>> If I am correct with the above analysis there are two ways to get this
>> fixed:
>>
>> (1) Set isolation level to "READ_COMMITTED" or
>> (2) Update to the latest version from CVS and take over the setting
>> for <parameter name="sequential-mode">full</parameter>
>> from Domain.xml
>>
>> With MySQL I would recommend (1) as (2) is the "if everything else
>> fails" solution.
>>
>> Oliver
>>
>> James Mason wrote:
>>
>>> A description of the behavior I'm seeing can be found at
>>> http://www.mail-archive.com/slide-dev@jakarta.apache.org/msg11007.html.
>>> I haven't tracked down the source of this problem yet, but I have
>>> some more information and I'm hoping something will jump out at
>>> someone more familiar with the code.
>>>
>>> Here are two snips from MySQL's general log file. The first is a
>>> failed PUT of a single file using a CVS checkout from the beginning
>>> of the week. The second is a successful PUT of the same file using
>>> 2.1M1. I don't think the versions are significant, I just couldn't
>>> get my checkout install to start working again after I broke it ;).
>>>
>>> ---------------- failed -------------------
>>> 3 Query SELECT 1 FROM VERSION_HISTORY vh, URI u WHERE vh.URI_ID
>>> = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO
>>> = '1.0'
>>> 3 Query commit
>>> 3 Query rollback
>>> 3 Query SET autocommit=1
>>> 3 Query SET autocommit=0
>>> 3 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
>>> 3 Query select vc.CONTENT from VERSION_CONTENT vc,
>>> VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
>>> vh.URI_ID = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and
>>> vh.REVISION_NO = '1.0'
>>> 4 Connect slide@localhost on slide
>>> 4 Init DB slide
>>> 4 Query select round('inf'), round('-inf'), round('nan')
>>> 4 Query SHOW VARIABLES
>>> 4 Query SET autocommit=1
>>> 4 Query SET autocommit=0
>>> 4 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
>>> 4 Query select 1 from VERSION_CONTENT vc, VERSION_HISTORY vh,
>>> URI u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID
>>> and u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
>>> 4 Query delete VERSION_CONTENT from VERSION_CONTENT vc,
>>> VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
>>> vh.REVISION_NO = '1.0' and vh.URI_ID=u.URI_ID AND
>>> u.URI_STRING='/files/2nd Floor.jpg'
>>> 4 Query rollback
>>> 4 Query rollback
>>> 4 Query SET autocommit=1
>>>
>>> ---------------- successful -------------------
>>> 3 Query SELECT 1 FROM VERSION_HISTORY vh, URI u WHERE vh.URI_ID
>>> = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO
>>> = '1.0'
>>> 3 Query commit
>>> 3 Query rollback
>>> 3 Query SET autocommit=1
>>> 3 Query SET autocommit=0
>>> 3 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
>>> 3 Query select 1 from VERSION_CONTENT vc, VERSION_HISTORY vh,
>>> URI u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID
>>> and u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
>>> 3 Query delete VERSION_CONTENT from VERSION_CONTENT vc,
>>> VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
>>> vh.REVISION_NO = '1.0' and vh.URI_ID=u.URI_ID AND
>>> u.URI_STRING='/files/2nd Floor.jpg'
>>> 3 Query select 1 from VERSION v, URI u where v.URI_ID =
>>> u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg'
>>> 3 Query select 1 from VERSION_HISTORY vh, URI u where vh.URI_ID
>>> = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO
>>> = '1.0'
>>> 3 Query insert into VERSION_CONTENT (VERSION_ID, CONTENT)
>>> select vh.VERSION_ID, '[... some binary junk ...]’
>>>
>>> ----------- end logs --------------
>>>
>>> Right off I'm seeing that the failed operation opens a second
>>> connection (the first column is the connection number) to the
>>> database to store the content *without* committing the first
>>> transaction. I'm pretty sure this is the cause of the deadlock.
>>>
>>> The other odd behavior is that the failed PUT calls
>>> retrieveRevisionContent just before it opens the second connection to
>>> store the content. This is strange, because the file being uploaded
>>> doesn't exist, so there's nothing in the database to retrieve. I've
>>> also noticed in Slide's logs that on PUTs where a deadlock occurs
>>> GetMethod throws an IllegalStateException while trying to set the
>>> size of the response buffer (see the linked to message above).
>>>
>>> -James
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by James Mason <ma...@apache.org>.
Bad news :(.
1) With the isolation level set to serializable the database returns an
error after a certain timeout period. I tried setting the isolation
level to read_committed, but the database never returned anything and
slide appeared to freeze. I'll try again in the morning with logging
turned on to see what's actually happening.
2) I have and it is. The logs I posted are with sequential-mode set to
"full". I can try setting it to "write", but from your earlier post I
got the impression full had a higher level of isolation.
Would connection 3 keep trying to commit and succeed after connection 4
gives up? I notice that connection 4 does rollback and set autocommit
(although doesn't issue a commit), but connection 3 never finishes.
Also, I'm using WebFolders as the client here. It issues two requests
when uploading a file, a HEAD and then a PUT. For some reason Slide is
trying to retrieve the contents from the database in response to the
HEAD request, even though the file doesn't exist yet.
I'm also curious what could be causing Slide to open a second connection
to attempt the write to the database. Only one client is talking to
Slide, so I would think all the requests would be part of the same
transaction/connection.
-James
Oliver Zeigermann wrote:
> What you describe and what the trace shows looks like the classical
> deadlock scenario to me.
>
> You observed Connection 3 can not commit, which is because it is dead
> locked by Connection 4. And the other way round.
>
> If I am correct with the above analysis there are two ways to get this
> fixed:
>
> (1) Set isolation level to "READ_COMMITTED" or
> (2) Update to the latest version from CVS and take over the setting for
> <parameter name="sequential-mode">full</parameter> from
> Domain.xml
>
> With MySQL I would recommend (1) as (2) is the "if everything else
> fails" solution.
>
> Oliver
>
> James Mason wrote:
>
>> A description of the behavior I'm seeing can be found at
>> http://www.mail-archive.com/slide-dev@jakarta.apache.org/msg11007.html.
>> I haven't tracked down the source of this problem yet, but I have some
>> more information and I'm hoping something will jump out at someone
>> more familiar with the code.
>>
>> Here are two snips from MySQL's general log file. The first is a
>> failed PUT of a single file using a CVS checkout from the beginning of
>> the week. The second is a successful PUT of the same file using 2.1M1.
>> I don't think the versions are significant, I just couldn't get my
>> checkout install to start working again after I broke it ;).
>>
>> ---------------- failed -------------------
>> 3 Query SELECT 1 FROM VERSION_HISTORY vh, URI u WHERE vh.URI_ID
>> = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO =
>> '1.0'
>> 3 Query commit
>> 3 Query rollback
>> 3 Query SET autocommit=1
>> 3 Query SET autocommit=0
>> 3 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
>> 3 Query select vc.CONTENT from VERSION_CONTENT vc,
>> VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
>> vh.URI_ID = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and
>> vh.REVISION_NO = '1.0'
>> 4 Connect slide@localhost on slide
>> 4 Init DB slide
>> 4 Query select round('inf'), round('-inf'), round('nan')
>> 4 Query SHOW VARIABLES
>> 4 Query SET autocommit=1
>> 4 Query SET autocommit=0
>> 4 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
>> 4 Query select 1 from VERSION_CONTENT vc, VERSION_HISTORY vh,
>> URI u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID and
>> u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
>> 4 Query delete VERSION_CONTENT from VERSION_CONTENT vc,
>> VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
>> vh.REVISION_NO = '1.0' and vh.URI_ID=u.URI_ID AND
>> u.URI_STRING='/files/2nd Floor.jpg'
>> 4 Query rollback
>> 4 Query rollback
>> 4 Query SET autocommit=1
>>
>> ---------------- successful -------------------
>> 3 Query SELECT 1 FROM VERSION_HISTORY vh, URI u WHERE vh.URI_ID
>> = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO =
>> '1.0'
>> 3 Query commit
>> 3 Query rollback
>> 3 Query SET autocommit=1
>> 3 Query SET autocommit=0
>> 3 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
>> 3 Query select 1 from VERSION_CONTENT vc, VERSION_HISTORY vh,
>> URI u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID and
>> u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
>> 3 Query delete VERSION_CONTENT from VERSION_CONTENT vc,
>> VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
>> vh.REVISION_NO = '1.0' and vh.URI_ID=u.URI_ID AND
>> u.URI_STRING='/files/2nd Floor.jpg'
>> 3 Query select 1 from VERSION v, URI u where v.URI_ID = u.URI_ID
>> and u.URI_STRING = '/files/2nd Floor.jpg'
>> 3 Query select 1 from VERSION_HISTORY vh, URI u where vh.URI_ID
>> = u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO =
>> '1.0'
>> 3 Query insert into VERSION_CONTENT (VERSION_ID, CONTENT) select
>> vh.VERSION_ID, '[... some binary junk ...]’
>>
>> ----------- end logs --------------
>>
>> Right off I'm seeing that the failed operation opens a second
>> connection (the first column is the connection number) to the database
>> to store the content *without* committing the first transaction. I'm
>> pretty sure this is the cause of the deadlock.
>>
>> The other odd behavior is that the failed PUT calls
>> retrieveRevisionContent just before it opens the second connection to
>> store the content. This is strange, because the file being uploaded
>> doesn't exist, so there's nothing in the database to retrieve. I've
>> also noticed in Slide's logs that on PUTs where a deadlock occurs
>> GetMethod throws an IllegalStateException while trying to set the size
>> of the response buffer (see the linked to message above).
>>
>> -James
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org
Re: More on deadlocks
Posted by Oliver Zeigermann <ol...@zeigermann.de>.
What you describe and what the trace shows looks like the classical
deadlock scenario to me.
You observed Connection 3 can not commit, which is because it is dead
locked by Connection 4. And the other way round.
If I am correct with the above analysis there are two ways to get this
fixed:
(1) Set isolation level to "READ_COMMITTED" or
(2) Update to the latest version from CVS and take over the setting for
<parameter name="sequential-mode">full</parameter> from
Domain.xml
With MySQL I would recommend (1) as (2) is the "if everything else
fails" solution.
Oliver
James Mason wrote:
> A description of the behavior I'm seeing can be found at
> http://www.mail-archive.com/slide-dev@jakarta.apache.org/msg11007.html.
> I haven't tracked down the source of this problem yet, but I have some
> more information and I'm hoping something will jump out at someone more
> familiar with the code.
>
> Here are two snips from MySQL's general log file. The first is a failed
> PUT of a single file using a CVS checkout from the beginning of the
> week. The second is a successful PUT of the same file using 2.1M1. I
> don't think the versions are significant, I just couldn't get my
> checkout install to start working again after I broke it ;).
>
> ---------------- failed -------------------
> 3 Query SELECT 1 FROM VERSION_HISTORY vh, URI u WHERE vh.URI_ID =
> u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO = '1.0'
> 3 Query commit
> 3 Query rollback
> 3 Query SET autocommit=1
> 3 Query SET autocommit=0
> 3 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
> 3 Query select vc.CONTENT from VERSION_CONTENT vc, VERSION_HISTORY
> vh, URI u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID
> and u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
> 4 Connect slide@localhost on slide
> 4 Init DB slide
> 4 Query select round('inf'), round('-inf'), round('nan')
> 4 Query SHOW VARIABLES
> 4 Query SET autocommit=1
> 4 Query SET autocommit=0
> 4 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
> 4 Query select 1 from VERSION_CONTENT vc, VERSION_HISTORY vh, URI
> u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID and
> u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
> 4 Query delete VERSION_CONTENT from VERSION_CONTENT vc,
> VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
> vh.REVISION_NO = '1.0' and vh.URI_ID=u.URI_ID AND
> u.URI_STRING='/files/2nd Floor.jpg'
> 4 Query rollback
> 4 Query rollback
> 4 Query SET autocommit=1
>
> ---------------- successful -------------------
> 3 Query SELECT 1 FROM VERSION_HISTORY vh, URI u WHERE vh.URI_ID =
> u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO = '1.0'
> 3 Query commit
> 3 Query rollback
> 3 Query SET autocommit=1
> 3 Query SET autocommit=0
> 3 Query SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE
> 3 Query select 1 from VERSION_CONTENT vc, VERSION_HISTORY vh, URI
> u where vc.VERSION_ID = vh.VERSION_ID and vh.URI_ID = u.URI_ID and
> u.URI_STRING = '/files/2nd Floor.jpg' and vh.REVISION_NO = '1.0'
> 3 Query delete VERSION_CONTENT from VERSION_CONTENT vc,
> VERSION_HISTORY vh, URI u where vc.VERSION_ID = vh.VERSION_ID and
> vh.REVISION_NO = '1.0' and vh.URI_ID=u.URI_ID AND
> u.URI_STRING='/files/2nd Floor.jpg'
> 3 Query select 1 from VERSION v, URI u where v.URI_ID = u.URI_ID
> and u.URI_STRING = '/files/2nd Floor.jpg'
> 3 Query select 1 from VERSION_HISTORY vh, URI u where vh.URI_ID =
> u.URI_ID and u.URI_STRING = '/files/2nd Floor.jpg' and REVISION_NO = '1.0'
> 3 Query insert into VERSION_CONTENT (VERSION_ID, CONTENT) select
> vh.VERSION_ID, '[... some binary junk ...]’
>
> ----------- end logs --------------
>
> Right off I'm seeing that the failed operation opens a second connection
> (the first column is the connection number) to the database to store the
> content *without* committing the first transaction. I'm pretty sure this
> is the cause of the deadlock.
>
> The other odd behavior is that the failed PUT calls
> retrieveRevisionContent just before it opens the second connection to
> store the content. This is strange, because the file being uploaded
> doesn't exist, so there's nothing in the database to retrieve. I've also
> noticed in Slide's logs that on PUTs where a deadlock occurs GetMethod
> throws an IllegalStateException while trying to set the size of the
> response buffer (see the linked to message above).
>
> -James
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: slide-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: slide-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: slide-dev-help@jakarta.apache.org