You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Lorenz Quack (JIRA)" <ji...@apache.org> on 2015/08/24 17:37:46 UTC
[jira] [Created] (QPID-6710) NPE masks IOException on running out
of disk space
Lorenz Quack created QPID-6710:
----------------------------------
Summary: NPE masks IOException on running out of disk space
Key: QPID-6710
URL: https://issues.apache.org/jira/browse/QPID-6710
Project: Qpid
Issue Type: Bug
Components: Java Broker
Reporter: Lorenz Quack
When the broker is running out of disk space the following issue can occur.
{noformat}
########################################################################
#
# Unhandled Exception org.apache.qpid.server.store.StoreException: Unexpected exception occurred on store operation in Thread virtualhost-default-iopool-55
#
# Exiting
#
########################################################################
org.apache.qpid.server.store.StoreException: Unexpected exception occurred on store operation
at org.apache.qpid.server.store.berkeleydb.StandardEnvironmentFacade.handleDatabaseException(StandardEnvironmentFacade.java:287)
at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore.removeMessage(AbstractBDBMessageStore.java:317)
at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.remove(AbstractBDBMessageStore.java:1234)
at org.apache.qpid.server.message.AbstractServerMessageImpl.decrementReference(AbstractServerMessageImpl.java:101)
at org.apache.qpid.server.message.AbstractServerMessageImpl.access$500(AbstractServerMessageImpl.java:37)
at org.apache.qpid.server.message.AbstractServerMessageImpl$Reference.release(AbstractServerMessageImpl.java:275)
at org.apache.qpid.server.protocol.v0_8.AMQChannel.deliverCurrentMessageIfComplete(AMQChannel.java:526)
at org.apache.qpid.server.protocol.v0_8.AMQChannel.publishContentBody(AMQChannel.java:655)
at org.apache.qpid.server.protocol.v0_8.AMQChannel.receiveMessageContent(AMQChannel.java:2519)
at org.apache.qpid.framing.ContentBody.process(ContentBody.java:105)
at org.apache.qpid.codec.AMQDecoder.processFrame(AMQDecoder.java:394)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.doProcessFrame(BrokerDecoder.java:114)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.access$000(BrokerDecoder.java:37)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder$1.run(BrokerDecoder.java:80)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder$1.run(BrokerDecoder.java:76)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.processFrame(BrokerDecoder.java:75)
at org.apache.qpid.codec.AMQDecoder.processInput(AMQDecoder.java:370)
at org.apache.qpid.codec.AMQDecoder.decodeBuffer(AMQDecoder.java:259)
at org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$2.run(AMQPConnection_0_8.java:319)
at org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$2.run(AMQPConnection_0_8.java:299)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:356)
at org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8.received(AMQPConnection_0_8.java:298)
at org.apache.qpid.server.transport.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:138)
at org.apache.qpid.server.transport.NonBlockingConnection.processAmqpData(NonBlockingConnection.java:465)
at org.apache.qpid.server.transport.NonBlockingConnectionPlainDelegate.processData(NonBlockingConnectionPlainDelegate.java:45)
at org.apache.qpid.server.transport.NonBlockingConnection.processData(NonBlockingConnection.java:398)
at org.apache.qpid.server.transport.NonBlockingConnection.readAndProcessData(NonBlockingConnection.java:349)
at org.apache.qpid.server.transport.NonBlockingConnectionPlainDelegate.doRead(NonBlockingConnectionPlainDelegate.java:39)
at org.apache.qpid.server.transport.NonBlockingConnection.doRead(NonBlockingConnection.java:337)
at org.apache.qpid.server.transport.NonBlockingConnection.doWork(NonBlockingConnection.java:229)
at org.apache.qpid.server.transport.NetworkConnectionScheduler.processConnection(NetworkConnectionScheduler.java:119)
at org.apache.qpid.server.transport.NetworkConnectionScheduler.access$000(NetworkConnectionScheduler.java:37)
at org.apache.qpid.server.transport.NetworkConnectionScheduler$2.run(NetworkConnectionScheduler.java:100)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore.removeMessage(AbstractBDBMessageStore.java:227)
... 37 more
{noformat}
Broker log file might have the issue reported in some cases as:
{noformat}
2015-07-23 05:19:30,460 INFO [default:FileSystemSpaceChecker] (q.m.v.filesystem_full) - [default:FileSystemSpaceChecker] [vh(/default)/ms(BDBMessageStore)] VHT-1006 : Filesystem is over 90 per cent full, enforcing flow control.
2015-07-23 05:19:46,668 ERROR [IO-/169.124.113.161:58633] (o.a.q.s.u.ServerScopedRuntimeException) - Unexpected exception occurred on store operation
com.sleepycat.je.LogWriteException: (JE 5.0.104) Environment must be closed, caused by: com.sleepycat.je.LogWriteException: Environment invalid because of previous exception: (JE 5.0.104) /apps/qpid/destruction-tests/qpid-broker/6.0.0-SNAPSHOT/work/default/messages java.io.IOException: No space left on device LOG_WRITE: IOExcep
{noformat}
However, in some other cases exception is not get stored in the logs leaving only NPE (reported into system output) to be the only evidence of an issue which might confuse the user about the nature of a problem
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org