You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/12/24 23:31:38 UTC
[GitHub] [ozone] amaliujia opened a new pull request #1736: HDDS-4594. Add namespace quota to documentation
amaliujia opened a new pull request #1736:
URL: https://github.com/apache/ozone/pull/1736
## What changes were proposed in this pull request?
Add namespace quota to documentation
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4594
## How was this patch tested?
UT
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia commented on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-751130128
R: @captainzmc
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc commented on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
captainzmc commented on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-751562736
Thanks for @amaliujia's update. I think there are some case we need to confirm.
Whether 'f' in Storage Space applies equally to Namespace quota? If so, we need to list. This part of namespace has been modified in #1723 (reviewing).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia edited a comment on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
amaliujia edited a comment on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-751414009
@linyiqun
I think for namespace, it will be different than space. Namespace quota is independent on each level (volume, bucket, etc.)
Namespace quota on volume only controls number of bucket names, and namespace on bucket only controls keys in that bucket. This is different from space quota in which volume quota has a relationship with its buckets quotas.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] linyiqun commented on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
linyiqun commented on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-752021501
Also +1 from me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia commented on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-751186941
@captainzmc
comments addressed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia edited a comment on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
amaliujia edited a comment on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-751414009
@linyiqun
I think for namespace, it will be different than space. Namespace quota is independent on each level (volume, bucket, etc.)
Namespace quota on volume only controls number of bucket names, and namespace on bucket only controls keys in that bucket. This is different from space quota in which volume quota has a relationship with its buckets quotas.
What currently is missing is the ability to control all keys within a volume. If there is use case, we could add support for it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia commented on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-751574312
@captainzmc
f. does not apply to namespace in my opinion. Note that the difference so far for namespace quota is that it is set independently on volume and bucket (not like space quota).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc commented on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
captainzmc commented on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-751598469
> @captainzmc
> f. does not apply to namespace in my opinion. Note that the difference so far for namespace quota is that it is set independently on volume and bucket (not like space quota).
Agreed, I'll remove the namespace restriction in #1723.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc commented on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
captainzmc commented on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-752068310
Merged this PR. Thanks for @amaliujia's patch and @linyiqun‘s review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc commented on a change in pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
captainzmc commented on a change in pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#discussion_r549204685
##########
File path: hadoop-hdds/docs/content/feature/Quota.md
##########
@@ -73,10 +87,35 @@ This behavior changes the quota for Bucket1 to 10GB
Total bucket quota should not be greater than its Volume quota. If we have a 10MB Volume, The sum of the sizes of all buckets under this volume cannot exceed 10MB, otherwise the bucket set quota fails.
+### Namespace quota
+Namespace quota is a number that represents how many unique names can be used. This number cannot be greater than LONG.MAX_VALUE in Java.
+
+#### Volume Namespace quota
+```shell
+bin/ozone sh volume create --namespace-quota 100 /volume1
+```
+This means setting the namespace quota of Volume1 to 100.
+
+```shell
+bin/ozone sh volume setquota --namespace-quota 1000 /volume1
+```
+This behavior changes the namespace quota of Volume1 to 1000.
+
+#### Bucket Namespace quota
+```shell
+bin/ozone sh bucket create --namespace-quota 100 /volume1/bucket1
+```
+That means bucket1 allows us to use 100 of namespace.
+
+```shell
+bin/ozone sh bucket setquota --namespace-quota 1000 /volume1/bucket1
+```
+This behavior changes the quota for Bucket1 to 1000.
+
#### Clear the quota for volume and bucket
```shell
-bin/ozone sh volume clrquota --space-quota /volume1
-bin/ozone sh bucket clrquota --space-quota /volume1/bucket1
+bin/ozone sh volume clrquota --space-quota --namespace-quota /volume1
+bin/ozone sh bucket clrquota --space-quota --namespace-quota /volume1/bucket1
Review comment:
**There's something wrong with the directory structure. At present is:**
#space quota
……
#namespace quota
……
## clear quota
## check quota and usage
**We can change it to:**
#space quota
……
## clear quota
## check quota and usage
#namespace quota
……
## clear quota
## check quota and usage
**or**
#space quota and namespace quota
……
## clear quota
## check quota and usage
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia commented on a change in pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#discussion_r548782469
##########
File path: hadoop-hdds/docs/content/feature/Quota.md
##########
@@ -45,6 +45,18 @@ e. If the cluster is upgraded from old version less than 1.1.0, use of quota on
f. If volume's quota is enabled then bucket's quota cannot be cleared.
+2. Namespace quota
+
+Administrators should be able to define how many namespace a Volume or Bucket can use. The following settings for namespace quota are supported:
+
+a. By default, the namespace quota for volume and bucket is not enabled.
+
+b. When volume namespace quota is enabled, the total number of buckets under the volume, cannot exceed the volume namespace quota.
+
+c. When bucket namespace quota is enabled, the totoal number of keys under the bucket, cannot exceed the bucket namespace quota.
+
+d. Linked buckets do not consume namespace quota.
+
Review comment:
Do we also need example client usage for namespace quota in the following section?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia commented on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-751414009
@linyiqun
I think for namespace, it will be different than space. Namespace quota is independent on each level (volume, bucket, etc.)
Namespace quota on volume only controls number of bucket names, and namespace on bucket only controls keys in that bucket. This is different from space quota in which volume quota is a relationship with its buckets quotas.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc merged pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
captainzmc merged pull request #1736:
URL: https://github.com/apache/ozone/pull/1736
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc edited a comment on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
captainzmc edited a comment on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-751598469
> @captainzmc
> f. does not apply to namespace in my opinion. Note that the difference so far for namespace quota is that it is set independently on volume and bucket (not like space quota).
Agree, I'll remove the namespace restriction in #1723.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc commented on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
captainzmc commented on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-751152776
@amaliujia Thanks for your patch. Could you also update the Client Usage about Namespace quota?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc commented on a change in pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
captainzmc commented on a change in pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#discussion_r549204685
##########
File path: hadoop-hdds/docs/content/feature/Quota.md
##########
@@ -73,10 +87,35 @@ This behavior changes the quota for Bucket1 to 10GB
Total bucket quota should not be greater than its Volume quota. If we have a 10MB Volume, The sum of the sizes of all buckets under this volume cannot exceed 10MB, otherwise the bucket set quota fails.
+### Namespace quota
+Namespace quota is a number that represents how many unique names can be used. This number cannot be greater than LONG.MAX_VALUE in Java.
+
+#### Volume Namespace quota
+```shell
+bin/ozone sh volume create --namespace-quota 100 /volume1
+```
+This means setting the namespace quota of Volume1 to 100.
+
+```shell
+bin/ozone sh volume setquota --namespace-quota 1000 /volume1
+```
+This behavior changes the namespace quota of Volume1 to 1000.
+
+#### Bucket Namespace quota
+```shell
+bin/ozone sh bucket create --namespace-quota 100 /volume1/bucket1
+```
+That means bucket1 allows us to use 100 of namespace.
+
+```shell
+bin/ozone sh bucket setquota --namespace-quota 1000 /volume1/bucket1
+```
+This behavior changes the quota for Bucket1 to 1000.
+
#### Clear the quota for volume and bucket
```shell
-bin/ozone sh volume clrquota --space-quota /volume1
-bin/ozone sh bucket clrquota --space-quota /volume1/bucket1
+bin/ozone sh volume clrquota --space-quota --namespace-quota /volume1
+bin/ozone sh bucket clrquota --space-quota --namespace-quota /volume1/bucket1
Review comment:
There's something wrong with the directory structure. At present is:
#space quota
……
#namespace quota
……
## clear quota
## check quota and usage
We can change it to:
#space quota
……
## clear quota
## check quota and usage
#namespace quota
……
## clear quota
## check quota and usage
or
#space quota and namespace quota
……
## clear quota
## check quota and usage
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc commented on a change in pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
captainzmc commented on a change in pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#discussion_r548782687
##########
File path: hadoop-hdds/docs/content/feature/Quota.md
##########
@@ -45,6 +45,18 @@ e. If the cluster is upgraded from old version less than 1.1.0, use of quota on
f. If volume's quota is enabled then bucket's quota cannot be cleared.
+2. Namespace quota
+
+Administrators should be able to define how many namespace a Volume or Bucket can use. The following settings for namespace quota are supported:
+
+a. By default, the namespace quota for volume and bucket is not enabled.
+
+b. When volume namespace quota is enabled, the total number of buckets under the volume, cannot exceed the volume namespace quota.
+
+c. When bucket namespace quota is enabled, the totoal number of keys under the bucket, cannot exceed the bucket namespace quota.
+
+d. Linked buckets do not consume namespace quota.
Review comment:
Whether 'e' in Storage Space applies equally to Namespace quota. If so, we need to list.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] amaliujia commented on pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#issuecomment-751879143
@captainzmc
finally I was able to re-trigger the CI (seemed that CI didn't work for a while). Any other comments on this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] captainzmc commented on a change in pull request #1736: HDDS-4594. Add namespace quota to documentation
Posted by GitBox <gi...@apache.org>.
captainzmc commented on a change in pull request #1736:
URL: https://github.com/apache/ozone/pull/1736#discussion_r549205140
##########
File path: hadoop-hdds/docs/content/feature/Quota.zh.md
##########
@@ -70,10 +84,35 @@ bin/ozone sh bucket setquota --space-quota 10GB /volume1/bucket1
bucket的总配额 不应大于其Volume的配额。让我们看一个例子，如果我们有一个10MB的Volume，该volume下所有bucket的大小之和不能超过10MB，否则设置bucket quota将失败。
+### Namespace quota
+命名空间配额是一个数字，其代表由多少个名字能够使用。这个数字不能超过Java long数据类型的最大值。
+
+#### Volume Namespace quota
+```shell
+bin/ozone sh volume create --namespace-quota 100 /volume1
+```
+这意味着将volume1的命名空间配额设置为100。
+
+```shell
+bin/ozone sh volume setquota --namespace-quota 1000 /volume1
+```
+此行为将volume1的命名空间配额更改为1000。
+
+#### Bucket Namespace quota
+```shell
+bin/ozone sh bucket create --namespace-quota 100 /volume1/bucket1
+```
+这意味着bucket1允许我们使用100的命名空间。
+
+```shell
+bin/ozone sh bucket setquota --namespace-quota 1000 /volume1/bucket1
+```
+该行为将bucket1的命名空间配额更改为1000。
+
#### 清除volume和bucket的配额
```shell
-bin/ozone sh volume clrquota --space-quota /volume1
-bin/ozone sh bucket clrquota --space-quota /volume1/bucket1
+bin/ozone sh volume clrquota --space-quota --namespace-quota /volume1
+bin/ozone sh bucket clrquota --space-quota --namespace-quota /volume1/bucket1
Review comment:
There's something wrong with the directory structure. At present is:
`#space quota`
……
`#namespace quota`
……
`## clear quota`
`## check quota and usage`
We can change it to:
`#space quota`
……
`## clear quota`
`## check quota and usage`
`#namespace quota`
……
`## clear quota`
`## check quota and usage`
or
`#space quota and namespace quota`
……
`## clear quota`
`## check quota and usage`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org