You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org> on 2012/03/02 03:14:59 UTC
[jira] [Commented] (QPID-3877) Modifying a message's headers may
cause a broker crash.
[ https://issues.apache.org/jira/browse/QPID-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220612#comment-13220612 ]
jiraposter@reviews.apache.org commented on QPID-3877:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4150/
-----------------------------------------------------------
Review request for qpid, Gordon Sim and Kim van der Riet.
Summary
-------
While running the broker with an async store, it is possible to crash the broker using perftest. This is due to a race condition where the broker is updating the message's headers (ttl) while the store is encoding the same.
To reproduce:
1) run broker with async store:
qpidd --load-module /home/kgiusti/store/cpp/lib/.libs/msgstore.so --auth no --port 8888 --num-jfile 16 --jfile-size-pgs 128 --data-dir /tmp/tmp.364FDjf0YW
2) run qpid-perftest in a loop against the broker:
while true; do qpid-perftest --port 8888 --mode fanout --count 25000 --size 256 --durable yes --nsubs 4 ; done
This addresses bug qpid-3877.
https://issues.apache.org/jira/browse/qpid-3877
Diffs
-----
/trunk/qpid/cpp/src/qpid/broker/Message.cpp 1295759
Diff: https://reviews.apache.org/r/4150/diff
Testing
-------
Ran the above tests without crash.
+make check.
Thanks,
Kenneth
> Modifying a message's headers may cause a broker crash.
> -------------------------------------------------------
>
> Key: QPID-3877
> URL: https://issues.apache.org/jira/browse/QPID-3877
> Project: Qpid
> Issue Type: Bug
> Components: C++ Broker
> Affects Versions: 0.16
> Reporter: Ken Giusti
> Assignee: Ken Giusti
> Fix For: 0.16
>
>
> Adjusting the message's TTL header can cause a crash if another thread is attempting to encode the same message. The below trace is from valgrind while the store is in use:
> ==15071== Thread 4:
> ==15071== Invalid read of size 8
> ==15071== at 0x52661F0: qpid::framing::DeliveryProperties::bodySize() const (DeliveryProperties.cpp:198)
> ==15071== by 0x5266248: qpid::framing::DeliveryProperties::encodedSize() const (DeliveryProperties.cpp:209)
> ==15071== by 0x52CC3D4: qpid::framing::AMQHeaderBody::encodedSize() const (AMQHeaderBody.h:45)
> ==15071== by 0x52CAFEE: qpid::framing::AMQFrame::encodedSize() const (AMQFrame.cpp:46)
> ==15071== by 0x4DB3B55: qpid::broker::Message::encodedHeaderSize() const (frame_functors.h:39)
> ==15071== by 0x5C1FBFC: mrg::msgstore::MessageStoreImpl::msgEncode(std::vector<char, std::allocator<char> >&, boost::intrusive_ptr<qpid::broker::PersistableMessage> const&) (Messag\
> eStoreImpl.cpp:1311)
> ==15071== by 0x5C31E8D: mrg::msgstore::MessageStoreImpl::store(qpid::broker::PersistableQueue const*, mrg::msgstore::TxnCtxt*, boost::intrusive_ptr<qpid::broker::PersistableMessage\
> > const&, bool) (MessageStoreImpl.cpp:1331)
> ==15071== by 0x5C32A0B: mrg::msgstore::MessageStoreImpl::enqueue(qpid::broker::TransactionContext*, boost::intrusive_ptr<qpid::broker::PersistableMessage> const&, qpid::broker::Per\
> sistableQueue const&) (MessageStoreImpl.cpp:1303)
> ==15071== by 0x4DBE65F: qpid::broker::MessageStoreModule::enqueue(qpid::broker::TransactionContext*, boost::intrusive_ptr<qpid::broker::PersistableMessage> const&, qpid::broker::Pe\
> rsistableQueue const&) (MessageStoreModule.cpp:125)
> ==15071== by 0x4DCFF31: qpid::broker::Queue::enqueue(qpid::broker::TransactionContext*, boost::intrusive_ptr<qpid::broker::Message>&, bool) (Queue.cpp:811)
> ==15071== by 0x4DD1951: qpid::broker::Queue::deliver(boost::intrusive_ptr<qpid::broker::Message>) (Queue.cpp:171)
> ==15071== by 0x4D798DE: qpid::broker::DeliverableMessage::deliverTo(boost::shared_ptr<qpid::broker::Queue> const&) (DeliverableMessage.cpp:33)
> ==15071== Address 0xca1d548 is 56 bytes inside a block of size 248 free'd
> ==15071== at 0x4A0545F: operator delete(void*) (vg_replace_malloc.c:387)
> ==15071== by 0x52CB195: qpid::framing::AMQFrame::cloneBody() (RefCounted.h:42)
> ==15071== by 0x4DB37DE: qpid::broker::Message::getHeaderBody() (Message.cpp:351)
> ==15071== by 0x4DB7471: qpid::framing::DeliveryProperties* qpid::broker::Message::getModifiableProperties<qpid::framing::DeliveryProperties>() (Message.h:208)
> ==15071== by 0x4DB65E7: qpid::broker::Message::adjustTtl() (Message.cpp:416)
> ==15071== by 0x4D7BB60: qpid::broker::DeliveryRecord::deliver(qpid::framing::Handler<qpid::framing::AMQFrame&>&, qpid::framing::SequenceNumber, unsigned short) (DeliveryRecord.cpp:\
> 80)
> ==15071== by 0x4E1A9B9: qpid::broker::SessionState::deliver(qpid::broker::DeliveryRecord&, bool) (SessionState.cpp:380)
> ==15071== by 0x4DFD345: qpid::broker::SemanticState::ConsumerImpl::deliver(qpid::broker::QueuedMessage&) (SemanticState.cpp:342)
> ==15071== by 0x4DD519C: qpid::broker::Queue::dispatch(boost::shared_ptr<qpid::broker::Consumer>) (Queue.cpp:393)
> ==15071== by 0x4E00759: qpid::broker::SemanticState::ConsumerImpl::doOutput() (SemanticState.cpp:741)
> ==15071== by 0x52F174C: qpid::sys::AggregateOutput::doOutput() (AggregateOutput.cpp:59)
> ==15071== by 0x4D6FC18: qpid::broker::Connection::doOutput() (Connection.cpp:354)
> ==15071==
> Gordon correctly points out that the message lock must be held while the headers are being encoded.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org