You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Kim van der Riet <ki...@redhat.com> on 2012/08/16 16:14:18 UTC
Review Request: Async Store Interface - test implementation on a performance
framework
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6642/
-----------------------------------------------------------
Review request for qpid and Andrew Stitcher.
Description
-------
Test implementation of the proposed async store interface. The implementation uses the interface to connect a performance test tool to a store (which is not implemented at this point) through the async interface at broker level. In order to do this, the broker must supply the interface and the supporting classes such as the response queue and context classes. The async store module is also present (asyncStore.so), and supplies the supporting classes on the store side such as the operation queue and the various handles needed by the store. Currently the async store module implements the async interface only, but does not actually store anything to disk. All the operations that should result in persistent records are null.
The module is built through cmake. A simple smoke test is also included, which can be run using "make test". No autotools integration has been done yet. The directory structure of these files is as follows:
qpid/asyncStore: Async store module, a broker plugin which implements the store side of the async store interface.
qpid/broker: Existing broker dir, but the classes necessary to support the async store interface from the broker side have been added.
tests/storePerftools: Performance measurement framework for the async store.
This review is intended to show the current structure and design philosophy. The next phase will be to start to tie this interface into the current broker code and to add actual persistence.
I will post a more detailed design overview on the Qpid dev list soon.
General JIRA: QPID-3858 "Develop asyncronous store interface for qpid" (https://issues.apache.org/jira/browse/QPID-3858)
Diffs
-----
./trunk/qpid/cpp/src/CMakeLists.txt 1368650
./trunk/qpid/cpp/src/Makefile.am 1368650
./trunk/qpid/cpp/src/asyncstore.cmake PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/AsyncOperation.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/AsyncOperation.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/AsyncStoreHandle.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/AsyncStoreImpl.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/AsyncStoreImpl.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/AsyncStoreOptions.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/AsyncStoreOptions.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/AtomicCounter.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/ConfigHandleImpl.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/ConfigHandleImpl.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/EnqueueHandleImpl.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/EnqueueHandleImpl.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/EventHandleImpl.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/EventHandleImpl.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/MessageHandleImpl.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/MessageHandleImpl.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/OperationQueue.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/OperationQueue.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/Plugin.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/Plugin.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/QueueHandleImpl.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/QueueHandleImpl.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/README PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/RunState.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/RunState.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/TxnHandleImpl.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/TxnHandleImpl.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/AioCallback.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/AsyncJournal.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/AsyncJournal.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/AtomicCounter.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/Configuration.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/DataOpState.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/DataOpState.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/DataToken.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/DataToken.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/DataWrComplState.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/DataWrComplState.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/DequeueHeader.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/DequeueHeader.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/EnqueueHeader.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/EnqueueHeader.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/EventHeader.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/EventHeader.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/FileHeader.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/FileHeader.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/JournalDirectory.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/JournalDirectory.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/JournalError.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/JournalError.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/JournalParameters.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/JournalParameters.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/JournalRunState.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/JournalRunState.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/RecordHeader.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/RecordHeader.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/RecordIdCounter.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/RecordTail.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/RecordTail.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/ScopedLock.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/ScopedLock.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/State.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/Streamable.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/Streamable.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/TransactionHeader.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/asyncStore/jrnl2/TransactionHeader.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/AsyncResultHandle.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/AsyncResultHandle.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/AsyncResultHandleImpl.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/AsyncResultHandleImpl.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/AsyncResultQueueImpl.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/AsyncResultQueueImpl.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/AsyncStore.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/Broker.h 1368650
./trunk/qpid/cpp/src/qpid/broker/Broker.cpp 1368650
./trunk/qpid/cpp/src/qpid/broker/ConfigHandle.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/ConfigHandle.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/EnqueueHandle.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/EnqueueHandle.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/EventHandle.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/EventHandle.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/Handle.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/MessageHandle.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/MessageHandle.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/PersistableMessage.h 1368650
./trunk/qpid/cpp/src/qpid/broker/PersistableMessage.cpp 1368650
./trunk/qpid/cpp/src/qpid/broker/PrivateImplRef.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/QueueAsyncContext.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/QueueAsyncContext.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/QueueEvents.h 1368650
./trunk/qpid/cpp/src/qpid/broker/QueueEvents.cpp 1368650
./trunk/qpid/cpp/src/qpid/broker/QueueHandle.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/QueueHandle.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleConsumer.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleDeliverable.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleDeliverable.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleDeliveryRecord.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleDeliveryRecord.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleMessage.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleMessage.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleMessageAsyncContext.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleMessageAsyncContext.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleMessageDeque.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleMessageDeque.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleMessages.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleQueue.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleQueue.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleQueuedMessage.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleQueuedMessage.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleTxnAccept.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleTxnAccept.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleTxnBuffer.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleTxnBuffer.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleTxnOp.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleTxnPublish.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/SimpleTxnPublish.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/TxnAsyncContext.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/TxnAsyncContext.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/TxnHandle.h PRE-CREATION
./trunk/qpid/cpp/src/qpid/broker/TxnHandle.cpp PRE-CREATION
./trunk/qpid/cpp/src/qpid/messaging/PrivateImplRef.h 1368650
./trunk/qpid/cpp/src/tests/CMakeLists.txt 1368650
./trunk/qpid/cpp/src/tests/QueueTest.cpp 1368650
./trunk/qpid/cpp/src/tests/asyncstore.cmake PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/asyncPerf/MessageConsumer.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/asyncPerf/MessageConsumer.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/asyncPerf/MessageProducer.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/asyncPerf/MessageProducer.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/asyncPerf/PerfTest.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/asyncPerf/PerfTest.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/asyncPerf/TestOptions.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/asyncPerf/TestOptions.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/asyncPerf/TestResult.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/asyncPerf/TestResult.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/asyncStorePerf_smoke_test.sh PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/Parameters.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/PerftoolError.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/PerftoolError.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/ScopedTimable.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/ScopedTimable.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/ScopedTimer.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/ScopedTimer.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/Streamable.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/Streamable.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/TestOptions.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/TestOptions.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/TestParameters.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/TestParameters.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/TestResult.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/Thread.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/common/Thread.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/jrnl2Perf_smoke_test.sh PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/jrnlPerf/Journal.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/jrnlPerf/Journal.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/jrnlPerf/JournalParameters.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/jrnlPerf/JournalParameters.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/jrnlPerf/PerfTest.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/jrnlPerf/PerfTest.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/jrnlPerf/TestResult.h PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/jrnlPerf/TestResult.cpp PRE-CREATION
./trunk/qpid/cpp/src/tests/storePerftools/version.h PRE-CREATION
./trunk/qpid/cpp/src/versions.cmake 1368650
Diff: https://reviews.apache.org/r/6642/diff/
Testing
-------
Functional testing using transactions of various sizes and a variable number of queues and producers/queue and consumers/queue have been completed on my workstation. No testing yet of messages which are sent to more than one queue has been performed. As with most async designs, a good deal of testing on different hardware would help expose race conditions and other async problems.
Thanks,
Kim van der Riet