You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Siyao Meng (Jira)" <ji...@apache.org> on 2023/02/01 23:39:00 UTC
[jira] [Updated] (HDDS-7871) Improve key prefix verification in createFakeDirIfShould
[ https://issues.apache.org/jira/browse/HDDS-7871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siyao Meng updated HDDS-7871:
-----------------------------
Description:
h2. Background
{{KeyManagerImpl#createFakeDirIfShould}} is intended to return a "fake" OmKeyInfo entry for a directory to the client in the case of OBS or LEGACY bucket. It is called from {{getOzoneFileStatus}} from {{KeyManagerImpl#getFileStatus}}. This was added in HDDS-7253.
h2. Discovery
However, due to RockDB [prefix seek using a bloom filter|https://github.com/facebook/rocksdb/wiki/Prefix-Seek], leading to {{keyValue}} [returning non-null|https://github.com/apache/ozone/blob/106b193c2ae9e111ba273b1f9dcb1c1a6e9057cb/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java#L1184-L1188] even if such volume+bucket+key prefix doesn't exist at all *at times*, it appears that the current prefix check logic in {{createFakeDirIfShould}} is not sufficient in verification.
This is discovered by [~hemantk] while debugging HDDS-7861, a snapshot test, while the issue itself does not appear to be related to snapshot code additions.
h2. Potential Solution
Check volume and bucket names as well inside {{if (keyValue != null)}}
cc [~hemantk]
cc [~XiChen] [~ckj] [~duongnguyen] that authored/reviewed patch HDDS-7253
cc [~erose] the OBS/FSO bucket expert
was:
h2. Background
{{KeyManagerImpl#createFakeDirIfShould}} is intended to return a "fake" OmKeyInfo entry for a directory to the client in the case of OBS or LEGACY bucket. It is called from {{getOzoneFileStatus}} from {{KeyManagerImpl#getFileStatus}}. This was added in HDDS-7253.
h2. Discovery
However, due to RockDB [prefix seek using a bloom filter|https://github.com/facebook/rocksdb/wiki/Prefix-Seek], leading to {{keyValue}} [returning non-null|https://github.com/apache/ozone/blob/106b193c2ae9e111ba273b1f9dcb1c1a6e9057cb/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java#L1184-L1188] even if such volume+bucket+key prefix doesn't exist at all at times, it appears that the current prefix check logic in {{createFakeDirIfShould}} is not sufficient in verification.
This is discovered by [~hemantk] while debugging HDDS-7861, a snapshot test, while the issue itself does not appear to be related to snapshot code additions.
h2. Potential Solution
Check volume and bucket names as well inside {{if (keyValue != null)}}
cc [~hemantk]
cc [~XiChen]] [~ckj] [~duongnguyen] that authored/reviewed patch HDDS-7253
cc [~erose] the OBS/FSO bucket expert
> Improve key prefix verification in createFakeDirIfShould
> --------------------------------------------------------
>
> Key: HDDS-7871
> URL: https://issues.apache.org/jira/browse/HDDS-7871
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Siyao Meng
> Priority: Critical
>
> h2. Background
> {{KeyManagerImpl#createFakeDirIfShould}} is intended to return a "fake" OmKeyInfo entry for a directory to the client in the case of OBS or LEGACY bucket. It is called from {{getOzoneFileStatus}} from {{KeyManagerImpl#getFileStatus}}. This was added in HDDS-7253.
> h2. Discovery
> However, due to RockDB [prefix seek using a bloom filter|https://github.com/facebook/rocksdb/wiki/Prefix-Seek], leading to {{keyValue}} [returning non-null|https://github.com/apache/ozone/blob/106b193c2ae9e111ba273b1f9dcb1c1a6e9057cb/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java#L1184-L1188] even if such volume+bucket+key prefix doesn't exist at all *at times*, it appears that the current prefix check logic in {{createFakeDirIfShould}} is not sufficient in verification.
> This is discovered by [~hemantk] while debugging HDDS-7861, a snapshot test, while the issue itself does not appear to be related to snapshot code additions.
> h2. Potential Solution
> Check volume and bucket names as well inside {{if (keyValue != null)}}
> cc [~hemantk]
> cc [~XiChen] [~ckj] [~duongnguyen] that authored/reviewed patch HDDS-7253
> cc [~erose] the OBS/FSO bucket expert
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org