TIMELY: Mysterious "database not found" error

[David Van Couvering <da...@vancouvering.com>]

Hi, guys.  I keep getting this random exception on a database that *does*
exist and just a moment before my process was happily accessing.

Caused by: java.sql.SQLException: Database
'C:\Vontu\Protect\scan\incremental_index\MONITOR\TARGET_1' not found.
	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown
Source)
	at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedConnection.handleDBNotFound(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
	at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
	at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
	at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
	at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
	at java.sql.DriverManager.getConnection(DriverManager.java:582)
	at java.sql.DriverManager.getConnection(DriverManager.java:185)


This appears to be closely related to another thread shutting down the
database.

I have explicit locking in place, which I have validated is working, that
prevents thread A from opening a connection to a database if thread B is
currently in the process of shutting it down.

What I do see, however, is that this error appears to occur if thread A
attempts to open a connection to the database *immediately after* thread B
finishes shutting it down and releases the lock.

So, I made a change where I hold the lock for 20 seconds after the shutdown
of the database completes.  However, the problem is still occurring.

I am really stumped and am not sure how to deal with this issue.

I suppose I could modify my code to retry if I get this error, but it's
really not a comfy feeling to do that.

Does anyone have an idea why this might be happening?  I tried reading
through your bootup code but it's really almost completely impossible to
understand, what with the levels of indirection and monitors and multiple
conditional branches inside one very very long method.

Your help much appreciated.  This issue is the top issue for us, and could
potentially hold us off from releasing the product.

I tried reproducing this with a multithreaded test outside of our product,
so I could give it to you for testing, but I was unsuccessful...

Many thanks,

David




-- 
David W. Van Couvering

http://www.linkedin.com/in/davidvc
http://davidvancouvering.blogspot.com
http://twitter.com/dcouvering