You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2019/10/18 11:30:00 UTC

[jira] [Updated] (HDDS-2330) Random key generator can get stuck

     [ https://issues.apache.org/jira/browse/HDDS-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HDDS-2330:
---------------------------------
    Labels: pull-request-available  (was: )

> Random key generator can get stuck
> ----------------------------------
>
>                 Key: HDDS-2330
>                 URL: https://issues.apache.org/jira/browse/HDDS-2330
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>          Components: freon
>            Reporter: Attila Doroszlai
>            Assignee: Attila Doroszlai
>            Priority: Major
>              Labels: pull-request-available
>
> Freon's random key generator can get stuck waiting for completion (without any hint to what's happening) if object creation encounters any non-IOException.
> Steps to reproduce:
> # Start Ozone cluster with 1 datanode
> # Start Freon (5K keys of size 1MB)
> Result: after a few hundred keys progress stops.
> {noformat}
> $ docker-compose exec scm ozone freon rk --numOfThreads 1 --numOfVolumes 1 --numOfBuckets 1 --replicationType RATIS --factor ONE --keySize $(echo '2^20' | bc -lq) --numOfKeys $(echo '5 * 2^10' | bc -lq) --bufferSize $(echo '2^16' | bc -lq)
> 2019-10-18 10:44:45,224 INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
> 2019-10-18 10:44:45,381 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
> 2019-10-18 10:44:45,381 INFO impl.MetricsSystemImpl: ozone-freon metrics system started
> 2019-10-18 10:44:47,140 [main] INFO       - Number of Threads: 1
> 2019-10-18 10:44:47,145 [main] INFO       - Number of Volumes: 1.
> 2019-10-18 10:44:47,146 [main] INFO       - Number of Buckets per Volume: 1.
> 2019-10-18 10:44:47,146 [main] INFO       - Number of Keys per Bucket: 5120.
> 2019-10-18 10:44:47,147 [main] INFO       - Key size: 1048576 bytes
> 2019-10-18 10:44:47,147 [main] INFO       - Buffer size: 65536 bytes
> 2019-10-18 10:44:47,147 [main] INFO       - validateWrites : false
> 2019-10-18 10:44:47,151 [main] INFO       - Starting progress bar Thread.
> ...
>  7.07% |????????                                                                                             |  362/5120 
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org