You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-user@db.apache.org by Pavel Bortnovskiy <pb...@jefferies.com> on 2012/01/13 00:01:27 UTC
Shutting down in-memory Derby gracefully
Hello, all:
While our application is shutting down, it is possible that some of its components may continue execution or even begin execution of the queries on the in-memory Derby DB. In such situation, Derby starts throwing all kinds of exceptions, including a Null Pointer Exception (pls see excerpt from our log below). Is it possible to have Derby shut down more gracefully, so that the application doesn't pester our error log files with these exceptions?
Of course, I may be able to work around by implementing an atomic reference counter, which gets incremented before a query is executed and decremented when it's done. And then, upon a request to shut down Derby, wait until that counter gets to 0 before sending the shutdown URL. But it seems kludgy to me and I would wish for a more elegant way for Derby itself to handle this, as it has the intimate knowledge of what's being executed and what's not.
Or, perhaps, you could recommend a better approach?
Thank you.
Pavel.
2012-01-12 16:15:23,224 [main] INFO Executing Derby Shutdown Command: jdbc:derby:memory:geni;drop=true
2012-01-12 16:15:23,230 SQL Exception due to executing statement
java.sql.SQLException: Java exception: ': java.lang.NullPointerException'.
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeQuery(Unknown Source)
...
Caused by: java.sql.SQLException: Java exception: ': java.lang.NullPointerException'.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
... 13 more
Caused by: java.lang.NullPointerException
at java.util.LinkedList.remove(LinkedList.java:225)
at org.apache.derby.impl.services.locks.LockControl.removeWaiter(Unknown Source)
at org.apache.derby.impl.services.locks.LockControl.getNextWaiter(Unknown Source)
at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source)
at org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(Unknown Source)
at org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(Unknown Source)
at org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(Unknown Source)
at org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(Unknown Source)
at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.lockPositionForRead(Unknown Source)
at org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(Unknown Source)
at org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchSet(Unknown Source)
at org.apache.derby.impl.store.access.BackingStoreHashTableFromScan.<init>(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.createBackingStoreHashtableFromScan(Unknown Source)
at org.apache.derby.impl.sql.execute.HashScanResultSet.openCore(Unknown Source)
at org.apache.derby.impl.sql.execute.JoinResultSet.openRight(Unknown Source)
at org.apache.derby.impl.sql.execute.JoinResultSet.openCore(Unknown Source)
at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.openCore(Unknown Source)
at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.openCore(Unknown Source)
at org.apache.derby.impl.sql.execute.GroupedAggregateResultSet.openCore(Unknown Source)
at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.openCore(Unknown Source)
at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.openCore(Unknown Source)
at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.open(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
... 6 more
2012-01-12 16:15:23,231 SQL Exception due to executing statement
java.sql.SQLNonTransientConnectionException: Database 'memory:geni' shutdown.
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeQuery(Unknown Source)
...
Caused by: java.sql.SQLException: Database 'memory:geni' shutdown.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
... 12 more
Caused by: ERROR 08006: Database 'memory:geni' shutdown.
at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.shutdownDatabaseException(Unknown Source)
... 9 more
Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority.
Re: Shutting down in-memory Derby gracefully
Posted by Katherine Marsden <km...@sbcglobal.net>.
On 1/13/2012 7:01 AM, Pavel Bortnovskiy wrote:
> Hello, Bryan:
>
> Thank you for your prompt response. I will report it as per guidelines.
>
> Would you be able to suggest anything that I could do in the meantime?
Hi Pavel, I don't have much experience with in-memory databases. Does it
help to connect with shutdown=true first or can one specify both
shutdown=true and drop=true? Unfortunately, while it may alleviate
symptoms, I think that the shutdown connection returns before it is
completely shutdown. I usually encourage people to put in a wait but
that might allow new connections to come in during the wait, but you
could have some simple synchronization code to disallow new connections
in your application if the shutdown is in process.
Re: Shutting down in-memory Derby gracefully
Posted by Kristian Waagan <kr...@oracle.com>.
On 17.01.12 03:14, Pavel Bortnovskiy wrote:
> Hello, all
>
> can someone please also shed light on the following:
>
> In addition to the problems described in the previous email (below), I also get intermittent " Database 'memory:test' not found" errors while shutting down Derby:
Hi Pavel,
I don't know what's causing this intermittent error. The first thing
that comes to mind is concurrent attempts to boot and/or drop the
database. I don't know if this could be happening in you application?
Regarding the errors you are seeing when dropping the database, I think
that's what you have to expect from Derby at this time when you have
active connections at the time of the drop/shutdown. I don't think this
is limited to the in-memory back end.
The only way to avoid these errors popping up is to coordinate the
shutdown/drop from the application layer.
As a side note, I did some work on a mechanism that you may find useful.
You can read the description of it under DERBY-4447 [1].
Regards,
--
Kristian
[1] https://issues.apache.org/jira/browse/DERBY-4447
>
> Error while shutting down:
> java.sql.SQLException: Database 'memory:test' not found.
> at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
> at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
> at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedConnection.handleDBNotFound(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
> at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
> at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
> at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
> at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
> at java.sql.DriverManager.getConnection(DriverManager.java:582)
> at java.sql.DriverManager.getConnection(DriverManager.java:207)
> Caused by: java.sql.SQLException: Database 'memory:test' not found.
> at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
> at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
> ... 21 more
>
> This error doesn't happen all the time, it's intermittent and I suspect that it might have be somehow connected to other problems I've been having trying to shutdown in-memory Derby DB gracefully.
>
> Can someone please help me understand what's causing this particular problem?
>
> My code that's shutting down Derby is quite simple, and oddly it works most of the times, but sometimes it throws this " Database 'memory:test' not found " error!
>
> Here is the excerpt of my shutdown code:
>
> private static final String DB_DROP_URL = "jdbc:derby:memory:%DB_NAME%;drop=true";
> private static final String DB_SHUTDOWN_URL = "jdbc:derby:;shutdown=true";
>
> private static class ShutdownAction {
> final private String m_url;
> final private int m_errorCode;
> final private String m_sqlState;
>
> private ShutdownAction(final String url, final int errorCode, final String sqlState) {
> m_url = url;
> m_errorCode = errorCode;
> m_sqlState = sqlState;
> }
>
> private void doAction(final String dbName) {
> final String url = m_url.replace("%DB_NAME%", dbName);
> logger.info("Executing Derby Shutdown Command: " + url);
> try {
> DriverManager.getConnection(url);
> } catch (SQLException e) {
> if ((e.getErrorCode() != m_errorCode) || (!m_sqlState.equals(e.getSQLState()))) {
> logger.error("Error while shutting down:", e);
> }
> }
> }
> }
>
> private static final ShutdownAction[] ACTIONS = {
> new ShutdownAction(DB_DROP_URL, 45000, "08006"),
> new ShutdownAction(DB_SHUTDOWN_URL, 50000, "XJ015")
> };
>
> private String m_dbName;
> private String m_dbCreateURL;
>
> private NetworkServerControlPinger m_derbyServerControl;
>
> @Override
> final public void shutdown() {
> for (final ShutdownAction action : ACTIONS) {
> action.doAction(m_dbName);
> }
>
> m_derbyServerControl.shutdown();
> }
>
>
>
> -----Original Message-----
> From: Pavel Bortnovskiy [mailto:pbortnovskiy@jefferies.com]
> Sent: Friday, January 13, 2012 10:01 AM
> To: Derby Discussion
> Subject: RE: Shutting down in-memory Derby gracefully
>
> Hello, Bryan:
>
> Thank you for your prompt response. I will report it as per guidelines.
>
> Would you be able to suggest anything that I could do in the meantime?
> We have watchdog processes monitoring error files and when the error files get populated with these messages, the watchdog raises all kinds of alarms.
> This is more than just a nuisance, as it's causing havoc in some cases.
>
> Thank you,
> Pavel.
>
> -----Original Message-----
> From: Bryan Pendleton [mailto:bpendleton.derby@gmail.com]
> Sent: Friday, January 13, 2012 9:58 AM
> To: Derby Discussion
> Subject: Re: Shutting down in-memory Derby gracefully
>
>> While our application is shutting down, it is possible that some of
>> its components may continue execution or even begin execution of the
>> queries on the in-memory Derby DB. In such situation, Derby starts throwing all kinds of exceptions, including a Null Pointer Exception (pls see excerpt from our log below).
>
> That looks like a bug to me, and worthy of reporting to the Derby developers.
>
> http://db.apache.org/derby/DerbyBugGuidelines.html
>
> thanks,
>
> bryan
>
>
> Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority.
> Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority.
RE: Shutting down in-memory Derby gracefully
Posted by Pavel Bortnovskiy <pb...@jefferies.com>.
Hello, all
can someone please also shed light on the following:
In addition to the problems described in the previous email (below), I also get intermittent " Database 'memory:test' not found" errors while shutting down Derby:
Error while shutting down:
java.sql.SQLException: Database 'memory:test' not found.
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleDBNotFound(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:207)
Caused by: java.sql.SQLException: Database 'memory:test' not found.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
... 21 more
This error doesn't happen all the time, it's intermittent and I suspect that it might have be somehow connected to other problems I've been having trying to shutdown in-memory Derby DB gracefully.
Can someone please help me understand what's causing this particular problem?
My code that's shutting down Derby is quite simple, and oddly it works most of the times, but sometimes it throws this " Database 'memory:test' not found " error!
Here is the excerpt of my shutdown code:
private static final String DB_DROP_URL = "jdbc:derby:memory:%DB_NAME%;drop=true";
private static final String DB_SHUTDOWN_URL = "jdbc:derby:;shutdown=true";
private static class ShutdownAction {
final private String m_url;
final private int m_errorCode;
final private String m_sqlState;
private ShutdownAction(final String url, final int errorCode, final String sqlState) {
m_url = url;
m_errorCode = errorCode;
m_sqlState = sqlState;
}
private void doAction(final String dbName) {
final String url = m_url.replace("%DB_NAME%", dbName);
logger.info("Executing Derby Shutdown Command: " + url);
try {
DriverManager.getConnection(url);
} catch (SQLException e) {
if ((e.getErrorCode() != m_errorCode) || (!m_sqlState.equals(e.getSQLState()))) {
logger.error("Error while shutting down:", e);
}
}
}
}
private static final ShutdownAction[] ACTIONS = {
new ShutdownAction(DB_DROP_URL, 45000, "08006"),
new ShutdownAction(DB_SHUTDOWN_URL, 50000, "XJ015")
};
private String m_dbName;
private String m_dbCreateURL;
private NetworkServerControlPinger m_derbyServerControl;
@Override
final public void shutdown() {
for (final ShutdownAction action : ACTIONS) {
action.doAction(m_dbName);
}
m_derbyServerControl.shutdown();
}
-----Original Message-----
From: Pavel Bortnovskiy [mailto:pbortnovskiy@jefferies.com]
Sent: Friday, January 13, 2012 10:01 AM
To: Derby Discussion
Subject: RE: Shutting down in-memory Derby gracefully
Hello, Bryan:
Thank you for your prompt response. I will report it as per guidelines.
Would you be able to suggest anything that I could do in the meantime?
We have watchdog processes monitoring error files and when the error files get populated with these messages, the watchdog raises all kinds of alarms.
This is more than just a nuisance, as it's causing havoc in some cases.
Thank you,
Pavel.
-----Original Message-----
From: Bryan Pendleton [mailto:bpendleton.derby@gmail.com]
Sent: Friday, January 13, 2012 9:58 AM
To: Derby Discussion
Subject: Re: Shutting down in-memory Derby gracefully
> While our application is shutting down, it is possible that some of
> its components may continue execution or even begin execution of the
> queries on the in-memory Derby DB. In such situation, Derby starts throwing all kinds of exceptions, including a Null Pointer Exception (pls see excerpt from our log below).
That looks like a bug to me, and worthy of reporting to the Derby developers.
http://db.apache.org/derby/DerbyBugGuidelines.html
thanks,
bryan
Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority.
Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority.
RE: Shutting down in-memory Derby gracefully
Posted by Pavel Bortnovskiy <pb...@jefferies.com>.
Hello, Bryan:
Thank you for your prompt response. I will report it as per guidelines.
Would you be able to suggest anything that I could do in the meantime?
We have watchdog processes monitoring error files and when the error files get populated with these messages, the watchdog raises all kinds of alarms.
This is more than just a nuisance, as it's causing havoc in some cases.
Thank you,
Pavel.
-----Original Message-----
From: Bryan Pendleton [mailto:bpendleton.derby@gmail.com]
Sent: Friday, January 13, 2012 9:58 AM
To: Derby Discussion
Subject: Re: Shutting down in-memory Derby gracefully
> While our application is shutting down, it is possible that some of
> its components may continue execution or even begin execution of the
> queries on the in-memory Derby DB. In such situation, Derby starts throwing all kinds of exceptions, including a Null Pointer Exception (pls see excerpt from our log below).
That looks like a bug to me, and worthy of reporting to the Derby developers.
http://db.apache.org/derby/DerbyBugGuidelines.html
thanks,
bryan
Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email, including any attachments, are confidential to the ordinary user of the email address to which it was addressed. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced at the request of regulators or in connection with civil litigation. Jefferies accepts no liability for any errors or omissions arising as a result of transmission. Use by other than intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the Financial Services Authority.
Re: Shutting down in-memory Derby gracefully
Posted by Bryan Pendleton <bp...@gmail.com>.
> While our application is shutting down, it is possible that some of its components may continue execution or even begin
> execution of the queries on the in-memory Derby DB. In such situation, Derby starts throwing all kinds of exceptions, including
> a Null Pointer Exception (pls see excerpt from our log below).
That looks like a bug to me, and worthy of reporting to the Derby developers.
http://db.apache.org/derby/DerbyBugGuidelines.html
thanks,
bryan