You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Hanisha Koneru (Jira)" <ji...@apache.org> on 2020/10/01 21:12:00 UTC
[jira] [Resolved] (HDDS-3869) Use different column families for
datanode block and metadata
[ https://issues.apache.org/jira/browse/HDDS-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hanisha Koneru resolved HDDS-3869.
----------------------------------
Fix Version/s: 1.1.0
Resolution: Fixed
> Use different column families for datanode block and metadata
> -------------------------------------------------------------
>
> Key: HDDS-3869
> URL: https://issues.apache.org/jira/browse/HDDS-3869
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Datanode
> Reporter: Ethan Rose
> Assignee: Ethan Rose
> Priority: Critical
> Labels: pull-request-available, upgrade
> Fix For: 1.1.0
>
>
> Currently datanodes place all of their data under the default column family in RocksDB. This differs from OM and SCM which organize their data into different column families based on its type. This feature will first move the datanode code off of the database utilities in the hadoop.hdds.utils package (which has no column family support), and move them to the newer utilities used by OM and SCM in the hadoop.hdds.utils.db package (which has column family support). The datanode will divide its data into three column families:
> # block_data: String keys (block id with optional prefix) map to BlockData objects
> # metadata: String keys (name of metadata field) map to Long objects.
> # deleted_blocks: String keys (block id with optional prefix) map to the ChunkInfo lists (lists of chunks corresponding to the block that was deleted).
> A new field, called 'schemaVersion' will be added to container files to indicate whether the container was created using the original schema version 1, where everything was in the default column family, or this new schema version 2. Code should be able to process older schema versions for backwards compatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org