You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Kim van der Riet (JIRA)" <ji...@apache.org> on 2013/11/26 10:14:35 UTC
[jira] [Commented] (QPID-5358) Linearstore: Checksums not
implemented in record tail
[ https://issues.apache.org/jira/browse/QPID-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13832427#comment-13832427 ]
Kim van der Riet commented on QPID-5358:
----------------------------------------
In the legacy store (which uses a circular file journal), there is a possible scenario in which bad data could be recovered without being detected. In this scenario, a single record must span 3 or more journal files. If at the time of broker failure the files containing the start and end (tail) of the enqueue record have been written to disk, but some or all of the intermediate file(s) remain unwritten (awaiting async i/o), then upon recovery, the intermediate file data will be read as a part of the record without the write failure being detected.
This scenario has never been observed in practice, as most modern disk controllers tend to result in ordered async i/o. However, the probability of such a failure remains, and is not easily quantifiable.
In the case of linearstore (assuming the same scenario), it may be possible for files to be placed into the linearstore file sequence from the Empty File Pool, and yet remain unwritten because the write sequence of buffering async i/o controllers is not guaranteed.
The linearstore has added a checksum field to the record tail for enqueued records. A checksum algorithm which is inexpensive and yet sufficiently effective at detecting this scenario is required. The checksum will include the xid and data sections of the enqueue record, and the xid of dequeue and transaction records. Upon recovery, the checksum of the recovered data and xid is calculated and compared with the checksum recorded in the record tail. This strategy ought to detect if any of the intermediate files in a large enqueue record remain unwritten at broker failure.
> Linearstore: Checksums not implemented in record tail
> -----------------------------------------------------
>
> Key: QPID-5358
> URL: https://issues.apache.org/jira/browse/QPID-5358
> Project: Qpid
> Issue Type: Bug
> Components: C++ Broker
> Reporter: Kim van der Riet
> Assignee: Kim van der Riet
> Labels: linearstore
>
> The linearstore now implements a 32-bit checksum in the record tail to check data integrity of the xid and data sections of the record. These have not been implemented (the value 0x0 is hardwired currently) and need some developement effort.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org