You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2015/08/25 21:48:46 UTC
[jira] [Commented] (QPID-6710) NPE masks IOException on running out
of disk space
[ https://issues.apache.org/jira/browse/QPID-6710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14711848#comment-14711848 ]
ASF subversion and git services commented on QPID-6710:
-------------------------------------------------------
Commit 1697761 from orudyy@apache.org in branch 'java/trunk'
[ https://svn.apache.org/r1697761 ]
QPID-6710: [Java Broker] Propagate coalescing committer exceptions occuring on flushing data to disk into committing threads
Refactor EnvironmentFacade implementations to encapsulate underlying Environment
(work by Lorenz Quack <qu...@gmail.com> and Alex Rudyy <or...@apache.org>)
> NPE masks IOException on running out of disk space
> --------------------------------------------------
>
> Key: QPID-6710
> URL: https://issues.apache.org/jira/browse/QPID-6710
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
> Reporter: Lorenz Quack
> Attachments: 0001-QPID-6710-Java-Broker-NPE-masks-IOException-on-runni.patch
>
>
> When the broker is running out of disk space the following issue can occur.
> {noformat}
> ########################################################################
> #
> # Unhandled Exception org.apache.qpid.server.store.StoreException: Unexpected exception occurred on store operation in Thread virtualhost-default-iopool-55
> #
> # Exiting
> #
> ########################################################################
> org.apache.qpid.server.store.StoreException: Unexpected exception occurred on store operation
> at org.apache.qpid.server.store.berkeleydb.StandardEnvironmentFacade.handleDatabaseException(StandardEnvironmentFacade.java:287)
> at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore.removeMessage(AbstractBDBMessageStore.java:317)
> at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage.remove(AbstractBDBMessageStore.java:1234)
> at org.apache.qpid.server.message.AbstractServerMessageImpl.decrementReference(AbstractServerMessageImpl.java:101)
> at org.apache.qpid.server.message.AbstractServerMessageImpl.access$500(AbstractServerMessageImpl.java:37)
> at org.apache.qpid.server.message.AbstractServerMessageImpl$Reference.release(AbstractServerMessageImpl.java:275)
> at org.apache.qpid.server.protocol.v0_8.AMQChannel.deliverCurrentMessageIfComplete(AMQChannel.java:526)
> at org.apache.qpid.server.protocol.v0_8.AMQChannel.publishContentBody(AMQChannel.java:655)
> at org.apache.qpid.server.protocol.v0_8.AMQChannel.receiveMessageContent(AMQChannel.java:2519)
> at org.apache.qpid.framing.ContentBody.process(ContentBody.java:105)
> at org.apache.qpid.codec.AMQDecoder.processFrame(AMQDecoder.java:394)
> at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.doProcessFrame(BrokerDecoder.java:114)
> at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.access$000(BrokerDecoder.java:37)
> at org.apache.qpid.server.protocol.v0_8.BrokerDecoder$1.run(BrokerDecoder.java:80)
> at org.apache.qpid.server.protocol.v0_8.BrokerDecoder$1.run(BrokerDecoder.java:76)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.processFrame(BrokerDecoder.java:75)
> at org.apache.qpid.codec.AMQDecoder.processInput(AMQDecoder.java:370)
> at org.apache.qpid.codec.AMQDecoder.decodeBuffer(AMQDecoder.java:259)
> at org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$2.run(AMQPConnection_0_8.java:319)
> at org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8$2.run(AMQPConnection_0_8.java:299)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:356)
> at org.apache.qpid.server.protocol.v0_8.AMQPConnection_0_8.received(AMQPConnection_0_8.java:298)
> at org.apache.qpid.server.transport.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:138)
> at org.apache.qpid.server.transport.NonBlockingConnection.processAmqpData(NonBlockingConnection.java:465)
> at org.apache.qpid.server.transport.NonBlockingConnectionPlainDelegate.processData(NonBlockingConnectionPlainDelegate.java:45)
> at org.apache.qpid.server.transport.NonBlockingConnection.processData(NonBlockingConnection.java:398)
> at org.apache.qpid.server.transport.NonBlockingConnection.readAndProcessData(NonBlockingConnection.java:349)
> at org.apache.qpid.server.transport.NonBlockingConnectionPlainDelegate.doRead(NonBlockingConnectionPlainDelegate.java:39)
> at org.apache.qpid.server.transport.NonBlockingConnection.doRead(NonBlockingConnection.java:337)
> at org.apache.qpid.server.transport.NonBlockingConnection.doWork(NonBlockingConnection.java:229)
> at org.apache.qpid.server.transport.NetworkConnectionScheduler.processConnection(NetworkConnectionScheduler.java:119)
> at org.apache.qpid.server.transport.NetworkConnectionScheduler.access$000(NetworkConnectionScheduler.java:37)
> at org.apache.qpid.server.transport.NetworkConnectionScheduler$2.run(NetworkConnectionScheduler.java:100)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore.removeMessage(AbstractBDBMessageStore.java:227)
> ... 37 more
> {noformat}
> Broker log file might have the issue reported in some cases as:
> {noformat}
> 2015-07-23 05:19:30,460 INFO [default:FileSystemSpaceChecker] (q.m.v.filesystem_full) - [default:FileSystemSpaceChecker] [vh(/default)/ms(BDBMessageStore)] VHT-1006 : Filesystem is over 90 per cent full, enforcing flow control.
> 2015-07-23 05:19:46,668 ERROR [IO-/169.124.113.161:58633] (o.a.q.s.u.ServerScopedRuntimeException) - Unexpected exception occurred on store operation
> com.sleepycat.je.LogWriteException: (JE 5.0.104) Environment must be closed, caused by: com.sleepycat.je.LogWriteException: Environment invalid because of previous exception: (JE 5.0.104) /apps/qpid/destruction-tests/qpid-broker/6.0.0-SNAPSHOT/work/default/messages java.io.IOException: No space left on device LOG_WRITE: IOExcep
> {noformat}
> However, in some other cases exception is not get stored in the logs leaving only NPE (reported into system output) to be the only evidence of an issue which might confuse the user about the nature of a problem
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org