You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Stanislav Matytsin (Jira)" <ji...@apache.org> on 2024/02/11 15:45:00 UTC
[jira] [Created] (HDFS-17379) setQuota/setSpaceQuota command doesn't fail if the directory would immediately exceed the new quota
Stanislav Matytsin created HDFS-17379:
-----------------------------------------
Summary: setQuota/setSpaceQuota command doesn't fail if the directory would immediately exceed the new quota
Key: HDFS-17379
URL: https://issues.apache.org/jira/browse/HDFS-17379
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs
Affects Versions: 3.1.3
Reporter: Stanislav Matytsin
With regards to documentation - [https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html#Space_Quotas]
* hdfs dfsadmin -setSpaceQuota <N> <directory>...<directory>
Set the space quota to be N bytes for each directory. This is a hard limit on total size of all the files under the directory tree. The space quota takes replication also into account, i.e. one GB of data with replication of 3 consumes 3GB of quota. N can also be specified with a binary prefix for convenience, for e.g. 50g for 50 gigabytes and 2t for 2 terabytes etc. Best effort for each directory, with faults reported if N is neither zero nor a positive integer, the directory does not exist or it is a file, or the directory would immediately exceed the new quota.
So, command to set Quota have to fail if quota already exceeded
Actual result – command finished with exit code 0
Steps to reproduce:
- put file 1GB into hdfs folder
- try to set quota of 512MB
hdfs dfs -ls /hdfs_dir
Exit code: 0
Found 1 items
- 3 user hadoop 1073741824 2024-01-17 18:57 /hdfs_dir/File
hdfs dfsadmin -setSpaceQuota 512m /hdfs_dir
result:
Exit code: 0
Quota report shows negative value for SPACE_REM_QUOTA
{code:java}
REM_SPACE_QUOTA -2.5GB
QUOTA REM_QUOTA SPACE_QUOTA REM_SPACE_QUOTA DIR_COUNT FILE_COUNT CONTENT_SIZE PATHNAME
none inf 512 M -2.5 G 1 1 1 G /hdfs_dir
{code}
The same situation with setQuota method, if we try to set name quota, for example, 3 for folder already contained 5 files – it doesn’t fail and shows negative value for REM_QUOTA
The reason for this behavior is absent quota check during SET operations
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org