You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Franklinsam Paul (Jira)" <ji...@apache.org> on 2022/08/31 18:39:00 UTC
[jira] [Created] (HDDS-7196) Disk space used by failed job(teragen here) is not reclaimable
Franklinsam Paul created HDDS-7196:
--------------------------------------
Summary: Disk space used by failed job(teragen here) is not reclaimable
Key: HDDS-7196
URL: https://issues.apache.org/jira/browse/HDDS-7196
Project: Apache Ozone
Issue Type: Improvement
Components: Ozone Datanode
Environment: |Apache Ozone|1.0.0|
Reporter: Franklinsam Paul
On Fresh ozone cluster, ran a tergane job and killed it around 25% completion. this left ozone used about 74.4GB but none of the files written is listing.
Issue can be reproducible with below steps. ( snapshots from the recon UI will be attached for usage reference)
{code:java}
ozone sh volume create o3://ozonefrankserviceid/testvol/
ozone sh bucket create o3://ozonefrankserviceid/testvol/testbucketyarn jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen -Dmapreduce.job.maps=2 1000000000 ofs://ozonefrankserviceid/testvol/testbucket
ozone sh volume create o3://ozonefrankserviceid/testvol/
ozone sh bucket create o3://ozonefrankserviceid/testvol/testbucket
yarn jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen -Dmapreduce.job.maps=2 1000000000 ofs://ozonefrankserviceid/testvol/testbucket/teragentest1
ozone fs -ls ofs://ozonefrankserviceid/testvol/testbucket/teragentest1
ozone fs -ls ofs://ozonefrankserviceid/testvol/testbucket/teragentest1/_temporary
ozone fs -ls ofs://ozonefrankserviceid/testvol/testbucket/teragentest1/_temporary/1
ozone fs -ls ofs://ozonefrankserviceid/testvol/testbucket/teragentest1/_temporary/1/_temporary
ozone fs -du -s -h ofs://ozonefrankserviceid/testvol/testbucket/teragentest1/_temporary/1/_temporary
ozone fs -ls ofs://ozonefrankserviceid/testvol/testbucket/teragentest1/_temporary/1/_temporary/attempt_1661777485132_0001_m_000000_2 --> no files/bject
ozone fs -ls ofs://ozonefrankserviceid/testvol/testbucket/teragentest1/_temporary/1/_temporary/attempt_1661777485132_0001_m_000001_2 --> no files/object
Ozone usage is increased in the recon UI as 75GB
hdfs dfs -rm -r -skipTrash ofs://ozonefrankserviceid/testvol/testbucket/teragentest1
ozone sh bucket delete o3://ozonefrankserviceid/testvol/testbucket
[root@DNHOST1 ozone-conf]# grep -A1 'hdds.datanode.dir' ozone-site.xml
<name>hdds.datanode.dir</name>
<value>/var/lib/hadoop-ozone/datanode/data</value>
[root@DNHOST1 ozone-conf]#[root@DNHOST1 containerDir0]# du -sh /var/lib/hadoop-ozone/datanode/data/hdds/a9461a7f-ef81-4942-a278-15ff7602df14/current/containerDir0/
26G /var/lib/hadoop-ozone/datanode/data/hdds/a9461a7f-ef81-4942-a278-15ff7602df14/current/containerDir0/
[root@DNHOST1 containerDir0]#
[root@DNHOST1 chunks]# ozone sh volume list o3://ozonefrankserviceid/ -a |egrep 'name|usedNamespace'
"name" : "s3v",
"usedNamespace" : 0,
"name" : "om",
"name" : "testvol",
"usedNamespace" : 0,
"name" : "hive/HMSHOST.example.com@SUPPORT.COM",
"name" : "hive",
[root@DNHOST1 chunks]# {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org