You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Anu Engineer (JIRA)" <ji...@apache.org> on 2017/06/06 23:55:18 UTC
[jira] [Comment Edited] (HDFS-11918) Ozone: Encapsulate KSM
metadata key into protobuf messages for better (de)serialization
[ https://issues.apache.org/jira/browse/HDFS-11918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16039668#comment-16039668 ]
Anu Engineer edited comment on HDFS-11918 at 6/6/17 11:55 PM:
--------------------------------------------------------------
[~cheersyang] At high level it seems like an interesting idea. However, the Rest API needs to parse these keys as they come in and route the calls appropriately. So I am not sure if we can avoid the heavy parsing part. Perhaps in the database part this has some advantage. Now that we have KSM almost build out, if we you can quickly review some of the places where the parsing is used and compare and contrast this approach, it will be helpful in deciding how much cleaner this approach is.
From reading this patch, it looks like a clean approach, but plain strings also have some advantages, including not needing a tool to understand what the value of a key is.
was (Author: anu):
[~cheersyang] At high level it seems like an interesting idea. However, the Rest API needs to parse these keys as they come in and route the calls appropriately. Do I am not sure if we can avoid the heavy parsing part. Perhaps in the database part this has some advantage. Now that we have KSM almost build out, if we you can quickly review some of the places where the parsing is used and compare and contrast this approach, it will be helpful in deciding how much cleaner this approach is.
From reading this patch, it looks like a clean approach, but plain strings also have some advantages, including not needing a tool understand what the value of a key is.
> Ozone: Encapsulate KSM metadata key into protobuf messages for better (de)serialization
> ---------------------------------------------------------------------------------------
>
> Key: HDFS-11918
> URL: https://issues.apache.org/jira/browse/HDFS-11918
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Priority: Critical
> Attachments: HDFS-11918-HDFS-7240.001.patch
>
>
> There are multiple type of keys stored in KSM database
> # Volume Key
> # Bucket Key
> # Object Key
> # User Key
> Currently they are represented as plain string with some conventions, such as
> # /volume
> # /volume/bucket
> # /volume/bucket/key
> # $user
> this approach makes it so difficult to parse volume/bucket/keys from KSM database. Propose to encapsulate these types of keys into protobuf messages, and take advantage of protobuf to serialize(deserialize) classes to byte arrays (and vice versa).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org