You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Attila Doroszlai (Jira)" <ji...@apache.org> on 2019/10/18 11:59:00 UTC
[jira] [Comment Edited] (HDDS-2331) Client OOME due to buffer
retention
[ https://issues.apache.org/jira/browse/HDDS-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954527#comment-16954527 ]
Attila Doroszlai edited comment on HDDS-2331 at 10/18/19 11:58 AM:
-------------------------------------------------------------------
Hi [~szetszwo], can you please check this issue? It seems to be caused by HDDS-2169, as reverting that [change|https://github.com/apache/hadoop-ozone/commit/e70ea7b6] allows Freon to continue creating keys.
CC [~msingh]
was (Author: adoroszlai):
Hi [~szetszwo], can you please check this issue? It seems to be caused by HDDS-2169, as reverting that [change|https://github.com/apache/hadoop-ozone/commit/e70ea7b6] allows Freon to continue creating keys.
> Client OOME due to buffer retention
> -----------------------------------
>
> Key: HDDS-2331
> URL: https://issues.apache.org/jira/browse/HDDS-2331
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Client
> Affects Versions: 0.5.0
> Reporter: Attila Doroszlai
> Priority: Critical
> Attachments: profiler.png
>
>
> Freon random key generator exhausts default heap after just few hundred 1MB keys. Heap dump on OOME reveals 150+ instances of {{ContainerCommandRequestMessage}}, each with 16MB {{byte[]}}.
> Steps to reproduce:
> # Start Ozone cluster with 1 datanode
> # Start Freon (5K keys of size 1MB)
> Result: OOME after a few hundred keys
> {noformat}
> $ cd hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/ozone
> $ docker-compose up -d
> $ docker-compose exec scm bash
> $ export HADOOP_OPTS='-XX:+HeapDumpOnOutOfMemoryError'
> $ ozone freon rk --numOfThreads 1 --numOfVolumes 1 --numOfBuckets 1 --replicationType RATIS --factor ONE --keySize 1048576 --numOfKeys 5120 --bufferSize 65536
> ...
> java.lang.OutOfMemoryError: Java heap space
> Dumping heap to java_pid289.hprof ...
> Heap dump file created [1456141975 bytes in 7.760 secs]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org