[jira] Updated: (DERBY-4733) Get error "Failed to create database - directory already exists" when attempting to connect to a database that failed during creation

["Mike Matrigali (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DERBY-4733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mike Matrigali updated DERBY-4733:
----------------------------------

    Summary: Get error "Failed to create database - directory already exists" when attempting to connect to a database that failed during creation  (was: Get error "Failed to create database - directory already exists" when shutting down a process that is creating a table)

I've changed the bug description to indicate the problem is about attempting to connect to a database which previously failed during creation.  If anyone has better wording, feel free to improve it.  I believe that the create table has nothing to do with the problem.

This problem sounds medium hard to fix.  Basically until the database is actually successfully created there is no existing mechanism to recover from errors while creating the database - especially if you are killing the process that is doing the database creation.  There is a bootstrap phase where we are getting the database to the point that it can handle all errors, but until then we basically rely on not enabling access to the db until
it is finished.  The .zip you posted is missing the necessary bootstrap files that derby
uses to open a database, and we don't lay down those files until we have finished 
creating the database.  Also as a performance enhancement I believe that we 
optimize database creation by doing a lot of operations unlogged, as it is assumed
no recovery is necessary for failures during middle of create database.

As has been commented I can see severe problems with Derby going ahead and deleting
the existing directory that it does not recognize as a database to make the creation 
succeed.  It is too easy for the case to be a user error, rather than an attempt to connect
to previously failed created database.  With derby I think many people just leave
the create=true flag on their connect line, so even though the database may think
it is doing a create, it might be surprising to users.  Probably what is causing this db
to not boot is the missing bootstrap file service.properties, which I think is instantiated
after we have forced the successful db creation to disk.  Unfortunately I have seen
too many users muck around with the internal files and trusting this file to not exist to
mean that we can go ahead and delete existing seg0 and log seems like a way to 
cause trouble.

It would seem useful if anyone is interested to provide better error messages.  Step one could be just to change the text of this one to mention that it might be possible that
a previous create database attempt failed and that user needs to clean up the existing
directory.  Step two could be to actually look for database artifacts that should get 
cleaned up, like log, seg0, tmp, db.lck.

There may be smaller fixes that could be made if whatever if causing the db to not
be created is something other than the process going away.  I don't know if existing
boot error handling is smart enough to know that it is failing during create database
and if that context knows what objects it could delete as part of the error recovery.

> Get error "Failed to create database - directory already exists" when attempting to connect to a database that failed during creation
> -------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: DERBY-4733
>                 URL: https://issues.apache.org/jira/browse/DERBY-4733
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.6.1.0
>         Environment: Windows XP
>            Reporter: David Van Couvering
>         Attachments: corruptme.zip, CorruptMePlease.java
>
>
> If you exit a process that is in the middle of booting a database or creating a table, then when you try to access the same database again, you get the error "Failed to create database 'foo', see next exception" followed by "Directory xxx already exists".
> A stack trace is below.  I am attaching a simple program that reproduces.  Just try running it multiple times.  I hit this after the second try.
> Opening database
> java.sql.SQLException: Failed to create database 'corruptme', see the next exception for details.
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
> 	at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
> 	at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedConnection.createDatabase(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
> 	at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
> 	at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> 	at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
> 	at java.sql.DriverManager.getConnection(Unknown Source)
> 	at java.sql.DriverManager.getConnection(Unknown Source)
> 	at CorruptMePlease$CreateTableTask.run(CorruptMePlease.java:34)
> 	at java.lang.Thread.run(Unknown Source)
> Caused by: java.sql.SQLException: Failed to create database 'corruptme', see the next exception for details.
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
> 	... 14 more
> Caused by: java.sql.SQLException: Directory C:\vontu\depot\sandbox_incremental\DerbyCorruption\corruptme already exists.
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
> 	at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
> 	at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
> 	... 11 more
> Caused by: ERROR XBM0J: Directory C:\vontu\depot\sandbox_incremental\DerbyCorruption\corruptme already exists.
> 	at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.StorageFactoryService$9.run(Unknown Source)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at org.apache.derby.impl.services.monitor.StorageFactoryService.createServiceRoot(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
> 	at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(Unknown Source)
> 	at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Unknown Source)
> 	... 11 more

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.