You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Hemant Kumar (Jira)" <ji...@apache.org> on 2023/02/01 23:38:00 UTC
[jira] [Commented] (HDDS-7871) Improve key prefix verification in createFakeDirIfShould
[ https://issues.apache.org/jira/browse/HDDS-7871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17683175#comment-17683175 ]
Hemant Kumar commented on HDDS-7871:
------------------------------------
Snapshot restore tests were added as part of [PR|https://github.com/apache/ozone/pull/4148/files#diff-2186b1fe62df6e3d2018bb4a9491761f79cff48e1ea81c3d50f85cd5f5bda7c7R256)] and test to restore snapshot to another bucket fails sometime for *_Legacy bucket_* layout with exception *_File Exists._* In the deep dive, it was found that it is because of [PR|https://github.com/apache/ozone/pull/4038]. In the PR, [seek operation was used on RocksDB table|https://github.com/apache/ozone/pull/4038/files#diff-bde0dade7dd5ddda419499f4f999d25d40fcec1412e0ce809c36ffd1be473f22R1302-R1304] which sometime returns non-null response and causes fake dir creation. It is possible that RocksDB's seek operation uses bloom filter which doesn't guarantee that key exists (false positive).
To fix this, we should add volume and bucket check [here|https://github.com/apache/ozone/pull/4038/files#diff-bde0dade7dd5ddda419499f4f999d25d40fcec1412e0ce809c36ffd1be473f22R1306]. We may or may not need to iterate over the iterator.
> Improve key prefix verification in createFakeDirIfShould
> --------------------------------------------------------
>
> Key: HDDS-7871
> URL: https://issues.apache.org/jira/browse/HDDS-7871
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Siyao Meng
> Priority: Critical
>
> h2. Background
> {{KeyManagerImpl#createFakeDirIfShould}} is intended to return a "fake" OmKeyInfo entry for a directory to the client in the case of OBS or LEGACY bucket. It is called from {{getOzoneFileStatus}} from {{KeyManagerImpl#getFileStatus}}. This was added in HDDS-7253.
> h2. Discovery
> However, due to RockDB [prefix seek using a bloom filter|https://github.com/facebook/rocksdb/wiki/Prefix-Seek], leading to {{keyValue}} [returning non-null|https://github.com/apache/ozone/blob/106b193c2ae9e111ba273b1f9dcb1c1a6e9057cb/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java#L1184-L1188] even if such volume+bucket+key prefix doesn't exist at all at times, it appears that the current prefix check logic in {{createFakeDirIfShould}} is not sufficient in verification.
> This is discovered by [~hemantk] while debugging HDDS-7861, a snapshot test, while the issue itself does not appear to be related to snapshot code additions.
> h2. Potential Solution
> Check volume and bucket names as well inside {{if (keyValue != null)}}
> cc [~hemantk]
> cc [~XiChen]] [~ckj] [~duongnguyen] that authored/reviewed patch HDDS-7253
> cc [~erose] the OBS/FSO bucket expert
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org