[jira] [Created] (HDDS-4963) Container metadata not removed after deleting Ozone files

["Wei-Chiu Chuang (Jira)" <ji...@apache.org>]

Wei-Chiu Chuang created HDDS-4963:
-------------------------------------

             Summary: Container metadata not removed after deleting Ozone files
                 Key: HDDS-4963
                 URL: https://issues.apache.org/jira/browse/HDDS-4963
             Project: Apache Ozone
          Issue Type: Bug
    Affects Versions: 1.0.0
            Reporter: Wei-Chiu Chuang


Steps to repro
{code}
 ozone sh volume create o3://ozone1/vol1
 ozone sh bucket create o3://ozone1/vol1/bucket1

# run teragen to generate data
yarn jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen -Dmapreduce.job.maps=10 -DmDmapreduce.map.memory.mb=4096 -Dmapreduce.reduce.memory.mb=4096 1000000000 o3fs://bucket1.vol1.ozone1/teragen1

# delete, skip trash
hdfs dfs -rm -r -skipTrash o3fs://bucket1.vol1.ozone1/teragen1

# forcefully close containers
for i in {1..1000}; do sudo -u hdfs ozone admin container close ${i}; done
{code}

Files are deleted from namespace, but space not released.
All containers are CLOSED. OM's keyTable is empty. openKeyTable empty.

The metadata of each leftover container occupies ~140mb in space. Over time, a DN can accumulate tens or even hundreds of GB of wasted space.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org