You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Siyao Meng (Jira)" <ji...@apache.org> on 2023/02/21 16:41:00 UTC
[jira] [Resolved] (HDDS-7906) Wait for checkpoint creation if snapshot in cache and not committed to DB.
[ https://issues.apache.org/jira/browse/HDDS-7906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siyao Meng resolved HDDS-7906.
------------------------------
Fix Version/s: 1.4.0
Resolution: Fixed
> Wait for checkpoint creation if snapshot in cache and not committed to DB.
> --------------------------------------------------------------------------
>
> Key: HDDS-7906
> URL: https://issues.apache.org/jira/browse/HDDS-7906
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Sadanand Shenoy
> Assignee: Sadanand Shenoy
> Priority: Major
> Labels: ozone-snapshot, pull-request-available
> Fix For: 1.4.0
>
>
> For most OM requests like create, delete , rename etc, the flow in HA is as follows
> -> Create OmRequest -> PreExecute -> validateAndUpdateCache -> finally update the OM Rocksdb.
> validateAndUpdateCache basically takes a bucket lock for write operations and updates the Table cache for the respective table on which the op is run on.
> Apart from updating the cache it also adds the OMResponse to the DoubleBuffer which is operated on by an always running daemon thread that calls OzoneManagerDoubleBuffer#flush every time a new entry is added to the buffer. However OM doesn't wait for the op to be flushed to return the response to the client instead it sends the response as soon as the cache And Double buffer is updated, and bucket lock is not taken during actual put to DB.
> This addition to cache is useful as it protects subsequent reads from accessing the DB where the flush might not have happened.
> Eg
> *Rename Key*
> t1-> Update renamed Key in Cache and add response to DoubleBuffer and response is returned and bucket lock released but RocksDB is not yet updated with renamed value.
> t2 -> Client issues read on the renamed path, During read it first check cache and sees renamed value.
> t3-> Updates the renamed value to actual DB and cleans up cache.
>
> However for *CreateSnapshot*
> t1-> Update snapshot info in Cache and return response to the client after bucket lock released.
> t2-> Client issues read on the snapshot path, During read it finds the Snapshot info object and the checkpoint dir location from the Snapshot Table Cache, however the checkpoint is actually created when actual RocksDB is updated in OMCreateSnapshotResponse#addToDBBatch and the read fails
> t3-> Add snapshotInfo to DB and create checkpoint.
>
> One of the fixes here I can think of is to wait for the checkpoint dir creation during read if the snapshot info is in cache.
> Another solution would also be to create checkpoint in validateAndUpdateCache itself
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org