You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Chance Li (JIRA)" <ji...@apache.org> on 2018/03/07 03:46:00 UTC

[jira] [Issue Comment Deleted] (HBASE-19389) Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted

     [ https://issues.apache.org/jira/browse/HBASE-19389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chance Li updated HBASE-19389:
------------------------------
    Comment: was deleted

(was: [~stack]

thanks, sir.

{quote} less throughput, but way less CPU used (better 95th percentiles, etc), and handlers free for other tasks

Yes sir, this is our purpose.

{quote}Would it make sense to put this limit behind a configuration gate?

I think we need it. It is mainly for to enable us to 'remove' it , even though we can't totally remove it  because of the added counter on the path.

{quote}Also of note is how your compares show the asyncwal being slightly better than old wal.

Let me check again.  And this async_wal is one of Durablilty types. it's not WAL.

[~carp84] ok, I will work on it.)

> Limit concurrency of put with dense (hundreds) columns to prevent write handler exhausted
> -----------------------------------------------------------------------------------------
>
>                 Key: HBASE-19389
>                 URL: https://issues.apache.org/jira/browse/HBASE-19389
>             Project: HBase
>          Issue Type: Improvement
>          Components: Performance
>    Affects Versions: 2.0.0
>         Environment: 2000+ Region Servers
> PCI-E ssd
>            Reporter: Chance Li
>            Assignee: Chance Li
>            Priority: Critical
>             Fix For: 2.0.0
>
>         Attachments: CSLM-concurrent-write.png, HBASE-19389-branch-2-V2.patch, HBASE-19389-branch-2-V3.patch, HBASE-19389-branch-2-V4.patch, HBASE-19389-branch-2-V5.patch, HBASE-19389-branch-2-V6.patch, HBASE-19389-branch-2-V7.patch, HBASE-19389-branch-2-V8.patch, HBASE-19389-branch-2.patch, metrics-1.png, ycsb-result.png
>
>
> In a large cluster, with a large number of clients, we found the RS's handlers are all busy sometimes. And after investigation we found the root cause is about CSLM, such as compare function heavy load. We reviewed the related WALs, and found that there were many columns (more than 1000 columns) were writing at that time.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)