You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Janus Chow (Jira)" <ji...@apache.org> on 2021/07/21 10:24:00 UTC
[jira] [Commented] (HDDS-5472) Old versions of location in
OmKeyLocationInfoGroup causes OOM of OM
[ https://issues.apache.org/jira/browse/HDDS-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17384816#comment-17384816 ]
Janus Chow commented on HDDS-5472:
----------------------------------
Found out I can not attach the file.
The file is as follows:
{code:java}
keyLocationVersions: [version:0 isMultipartKey:false107544261432440863 || , version:1 isMultipartKey:false107544261432440863 || 107544261432440864 || , version:2 isMultipartKey:false107544261432440863 || 107544261432440864 || 107544261432440865 || , version:3 isMultipartKey:false107544261432440863 || 107544261432440864 || 107544261432440865 || 107544261432440866 || , version:4 isMultipartKey:false107544261432440863 || 107544261432440864 || 107544261432440865 || 107544261432440866 || 107544261432440867 || , version:5 isMultipartKey:false107544261432440863 || 107544261432440864 || 107544261432440865 || 107544261432440866 || 107544261432440867 || 107544261432440868 || , version:6 isMultipartKey:false107544261432440863 || 107544261432440864 || 107544261432440865 || 107544261432440866 || 107544261432440867 || 107544261432440868 || 107544261432440869 || , version:7 isMultipartKey:false107544261432440863 || 107544261432440864 || 107544261432440865 || 107544261432440866 || 107544261432440867 || 107544261432440868 || 107544261432440869 || 107544261432440870 || , version:8 isMultipartKey:false107544261432440863 || 107544261432440864 || 107544261432440865 || 107544261432440866 || 107544261432440867 || 107544261432440868 || 107544261432440869 || 107544261432440870 || 107544261432440871 || , version:9 isMultipartKey:false107544261432440863 || 107544261432440864 || 107544261432440865 || 107544261432440866 || 107544261432440867 || 107544261432440868 || 107544261432440869 || 107544261432440870 || 107544261432440871 || 107544261432440872 || , version:10 isMultipartKey:false107544261432440863 || 107544261432440864 || 107544261432440865 || 107544261432440866 || 107544261432440867 || 107544261432440868 || 107544261432440869 || 107544261432440870 || 107544261432440871 || 107544261432440872 || 107544261432440874 || ,....
{code}
> Old versions of location in OmKeyLocationInfoGroup causes OOM of OM
> -------------------------------------------------------------------
>
> Key: HDDS-5472
> URL: https://issues.apache.org/jira/browse/HDDS-5472
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Janus Chow
> Assignee: Janus Chow
> Priority: Major
> Labels: pull-request-available
>
> Currently the keyLocation of different versions is stored redundantly in OmKeyInfo.
> * OmKeyInfo:
> ** keyLocationVersions: List<OmKeyLocationInfoGroup> stores different versions of information in a list.
> * OmKeyLocationInfoGroup:
> ** locationVersionMap: Map<Long, List<OmKeyLocationInfo>> stores different versions of location in a map.
> If the versions are large, the redundent location is causing the large GC overhead for OM, in our cluster, the OM even crashes because of OOM.
> This ticket is to remove the redundant location information to keep OM healthy.
> Attached the OmKeyInfo of a key with 549 versions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org