You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Josh Elser (JIRA)" <ji...@apache.org> on 2017/03/24 21:26:41 UTC

[jira] [Comment Edited] (HBASE-16961) FileSystem Quotas

    [ https://issues.apache.org/jira/browse/HBASE-16961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15941163#comment-15941163 ] 

Josh Elser edited comment on HBASE-16961 at 3/24/17 9:26 PM:
-------------------------------------------------------------

[~apurtell] and others, here's a [little script|https://issues.apache.org/jira/secure/attachment/12860455/hbase-quota-test.sh] you can walk through for a general purpose scenario of setting a space quota on a namespace.

Setup-wise:

# Build the HEAD of HBASE-16961
# Set {{hbase.quota.enabled=true}} in hbase-site.xml

Optionally: you can also tweak some of the chore properties to see more responsiveness:

{noformat}
  <property>
    <name>hbase.regionserver.quotas.fs.utilization.chore.period</name>
    <value>60000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.fs.utilization.chore.delay</name>
    <value>1000</value>
  </property>
  <property>
    <name>hbase.master.quotas.observer.chore.period</name>
    <value>30000</value>
  </property>
  <property>
    <name>hbase.master.quotas.observer.chore.delay</name>
    <value>1000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.policy.refresher.chore.period</name>
    <value>30000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.policy.refresher.chore.delay</name>
    <value>1000</value>
  </property>
{noformat}


was (Author: elserj):
[~apurtell] and others, here's a little script you can walk through for a general purpose scenario of setting a space quota on a namespace.

Setup-wise:

# Build the HEAD of HBASE-16961
# Set {{hbase.quota.enabled=true}} in hbase-site.xml

Optionally: you can also tweak some of the chore properties to see more responsiveness:

{noformat}
  <property>
    <name>hbase.regionserver.quotas.fs.utilization.chore.period</name>
    <value>60000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.fs.utilization.chore.delay</name>
    <value>1000</value>
  </property>
  <property>
    <name>hbase.master.quotas.observer.chore.period</name>
    <value>30000</value>
  </property>
  <property>
    <name>hbase.master.quotas.observer.chore.delay</name>
    <value>1000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.policy.refresher.chore.period</name>
    <value>30000</value>
  </property>
  <property>
    <name>hbase.regionserver.quotas.policy.refresher.chore.delay</name>
    <value>1000</value>
  </property>
{noformat}

> FileSystem Quotas
> -----------------
>
>                 Key: HBASE-16961
>                 URL: https://issues.apache.org/jira/browse/HBASE-16961
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>         Attachments: hbase-quota-test.sh
>
>
> Umbrella issue for tracking the filesystem utilization of HBase data, defining quotas on that utilization, and enforcement when utilization exceeds the limits of the quota.
> At a high level: we can define quotas on tables and namespaces. Region size is computed by RegionServers and sent to the Master. The Master inspects the sizes of Regions, rolling up to table and namespace sizes. Defined quotas in the quota table are evaluated given the computed sizes, and, for those tables/namespaces violating the quota, RegionServers are informed to take some action to limit any further filesystem growth by that table/namespace.
> Discuss: https://lists.apache.org/thread.html/66a4b0c3725b5cbdd61dd6111c43847adaeef7b7da5f4cd045df30ef@%3Cdev.hbase.apache.org%3E
> Design Doc: http://home.apache.org/~elserj/hbase/FileSystemQuotasforApacheHBase.pdf or https://docs.google.com/document/d/1VtLWDkB2tpwc_zgCNPE1ulZOeecF-YA2FYSK3TSs_bw/edit?usp=sharing



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)