[jira] [Commented] (HDFS-17463) Support the switch StringTable Split ID feature

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18023367#comment-18023367 ] 

ASF GitHub Bot commented on HDFS-17463:
---------------------------------------

github-actions[bot] commented on PR #6736:
URL: https://github.com/apache/hadoop/pull/6736#issuecomment-3342140007

   We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
   If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
   Thanks all for your contribution.




> Support the switch StringTable Split ID feature
> -----------------------------------------------
>
>                 Key: HDFS-17463
>                 URL: https://issues.apache.org/jira/browse/HDFS-17463
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 3.2.0, 3.3.5, 3.3.3, 3.3.4
>            Reporter: wangzhihui
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: Image_struct.png, error.png
>
>
> desc:
>  * 
> Hadoop 3.2 introduced optimization features for HDFS StringTable (b60ca37914b22550e3630fa02742d40697decb3), It resulted in lower versions of Hadoop upgraded to 3.2 and later versions not supporting downgrade operations. 
> !error.png!
>  * This issue has also been discussed in HDFS-14831, and it is recommended to revert the feature, but it cannot fundamentally solve the problem。
>  * 
> Therefore, we have added an optimization to support downgrading
>  
> Solution:
>  * First, we will add the "dfs. image. save. splitId. stringTable" conf switch "StringTable optimization feature" is enabled
>  * When the conf value is false, an Image file compatible with lower versions of HDFS is generated to support downgrading.
>  * 
> The difference in HDFS Image file format between Hadoop 3.1.1 and Hadoop 3.2 is shown in the following figure.
>  * With the sub-sections feature introduced in HDFS-14617, Protobuf can support compatible reading.
>  * 
> The data structure causing incompatible differences is mainly StringTable.
> !Image_struct.png|width=396,height=163!
>  * In "dfs.image.save.splitId.stringTable = false " the Id generation order of StringTable starts from 0 to Integer.Max. When true, the Id value range follows the latest rules.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org