You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Mohanad Elsafty (Jira)" <ji...@apache.org> on 2023/01/31 13:48:00 UTC
[jira] [Updated] (HDDS-7858) Report correct quota and used bytes for -df
[ https://issues.apache.org/jira/browse/HDDS-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mohanad Elsafty updated HDDS-7858:
----------------------------------
Description:
Alluxio/Hadoop reported wrong values when -df
hadoop fs -df ofs://.....
hadoop fs -df o3fs://....
ozone fs -df ofs://.....
ozone fs -df o3fs://....
All reported (Long.MAX_VALUE, 0, Long.MAX_VALUE)
What I was expecting is correct value (quota, used bytes, available bytes).
Turned out there is no implementation for both interfaces (ofs/o3fs) I made some updates and the values looks correct now.
{code:java}
$ ozone shell volume create /vol1
$ ozone shell bucket create /vol1/buck1 --quota=5GB
$ ls -lah ./Dockerfile
-rw-r--r-- 1 hadoop hadoop 877 Jan 31 11:48 ./Dockerfile
$ ozone shell key put /vol1/buck1/key1 ./Dockerfile -r ONE -t RATIS
$ ozone fs -df ofs://om/vol1/buck1/key1
Filesystem Size Used Available Use%
ofs://om 5368709120 877 5368708243 0%
$ hadoop fs -df ofs://om/vol1/buck1/key1
Filesystem Size Used Available Use%
ofs://om 5368709120 877 5368708243 0%
$ ozone fs -df o3fs://buck1.vol1.om/key1
Filesystem Size Used Available Use%
o3fs://buck1.vol1.om 5368709120 877 5368708243 0%
$ hadoop fs -df o3fs://buck1.vol1.om/key1
Filesystem Size Used Available Use%
o3fs://buck1.vol1.om 5368709120 877 5368708243 0% {code}
Note: usage is reported correctly with larger used bytes.
I made the implementation with earlier versions of mind so here is how it works:
It will check first for the bucket's quota if there is it will be reported, if not it will check for the volume's quota and report it if there is. other than this default to *Long.MAX_VALUE*
After HDDS-7751 creating a bucket without quota when parent volume has quota is not possible. in other words volume quota will not be reported in any case.
But this works for earlier versions before HDDS-7751 as well.
was:
Alluxio/Hadoop reported wrong values when -df
hadoop fs -df ofs://.....
hadoop fs -df o3fs://....
ozone fs -df ofs://.....
ozone fs -df o3fs://....
All reported (Long.MAX_VALUE, 0, Long.MAX_VALUE)
What I was expecting is correct value (quota, used bytes, available bytes).
Turned out there is no implementation for both interfaces (ofs/o3fs) I made some updates and the values looks correct now.
{code:java}
$ ozone shell volume create /vol1
$ ozone shell bucket create /vol1/buck1 --quota=5GB
$ ls -lah ./Dockerfile
-rw-r--r-- 1 hadoop hadoop 877 Jan 31 11:48 ./Dockerfile
$ ozone shell key put /vol1/buck1/key1 ./Dockerfile -r ONE -t RATIS
$ ozone fs -df ofs://om/vol1/buck1/key1
Filesystem Size Used Available Use%
ofs://om 5368709120 877 5368708243 0%
$ hadoop fs -df ofs://om/vol1/buck1/key1
Filesystem Size Used Available Use%
ofs://om 5368709120 877 5368708243 0%
$ ozone fs -df o3fs://buck1.vol1.om/key1
Filesystem Size Used Available Use%
o3fs://buck1.vol1.om 5368709120 877 5368708243 0%
$ hadoop fs -df o3fs://buck1.vol1.om/key1
Filesystem Size Used Available Use% o3fs://buck1.vol1.om 5368709120 877 5368708243 0% {code}
Note: usage is reported correctly with larger used bytes.
I made the implementation with earlier versions of mind so here is how it works:
It will check first for the bucket's quota if there is it will be reported, if not it will check for the volume's quota and report it if there is. other than this default to *Long.MAX_VALUE*
After HDDS-7751 creating a bucket without quota when parent volume has quota is not possible. in other words volume quota will not be reported in any case.
But this works for earlier versions before HDDS-7751 as well.
> Report correct quota and used bytes for -df
> -------------------------------------------
>
> Key: HDDS-7858
> URL: https://issues.apache.org/jira/browse/HDDS-7858
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Mohanad Elsafty
> Assignee: Mohanad Elsafty
> Priority: Major
>
> Alluxio/Hadoop reported wrong values when -df
> hadoop fs -df ofs://.....
> hadoop fs -df o3fs://....
> ozone fs -df ofs://.....
> ozone fs -df o3fs://....
> All reported (Long.MAX_VALUE, 0, Long.MAX_VALUE)
> What I was expecting is correct value (quota, used bytes, available bytes).
>
> Turned out there is no implementation for both interfaces (ofs/o3fs) I made some updates and the values looks correct now.
>
>
> {code:java}
> $ ozone shell volume create /vol1
> $ ozone shell bucket create /vol1/buck1 --quota=5GB
> $ ls -lah ./Dockerfile
> -rw-r--r-- 1 hadoop hadoop 877 Jan 31 11:48 ./Dockerfile
> $ ozone shell key put /vol1/buck1/key1 ./Dockerfile -r ONE -t RATIS
> $ ozone fs -df ofs://om/vol1/buck1/key1
> Filesystem Size Used Available Use%
> ofs://om 5368709120 877 5368708243 0%
> $ hadoop fs -df ofs://om/vol1/buck1/key1
> Filesystem Size Used Available Use%
> ofs://om 5368709120 877 5368708243 0%
> $ ozone fs -df o3fs://buck1.vol1.om/key1
> Filesystem Size Used Available Use%
> o3fs://buck1.vol1.om 5368709120 877 5368708243 0%
> $ hadoop fs -df o3fs://buck1.vol1.om/key1
> Filesystem Size Used Available Use%
> o3fs://buck1.vol1.om 5368709120 877 5368708243 0% {code}
>
> Note: usage is reported correctly with larger used bytes.
>
> I made the implementation with earlier versions of mind so here is how it works:
> It will check first for the bucket's quota if there is it will be reported, if not it will check for the volume's quota and report it if there is. other than this default to *Long.MAX_VALUE*
>
>
> After HDDS-7751 creating a bucket without quota when parent volume has quota is not possible. in other words volume quota will not be reported in any case.
> But this works for earlier versions before HDDS-7751 as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org