You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2016/10/22 00:02:14 UTC

[jira] [Updated] (HBASE-15709) Handle large edits for asynchronous WAL

     [ https://issues.apache.org/jira/browse/HBASE-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack updated HBASE-15709:
--------------------------
    Priority: Critical  (was: Major)

How we going to reject big Cells?

Would it be crazy running two WALs per RegionServer? One the new async and the other the old version so we could still do big types?  (Yes, it would).

Is this a limitation of fan-out only? Why so? When a bigger than 16M patch comes in on current WAL, how is it handled? Its cut up into checksum sized chunks and sent down the pipeline. Is it because of the fanout?

> Handle large edits for asynchronous WAL
> ---------------------------------------
>
>                 Key: HBASE-15709
>                 URL: https://issues.apache.org/jira/browse/HBASE-15709
>             Project: HBase
>          Issue Type: Sub-task
>          Components: io, wal
>            Reporter: Duo Zhang
>            Priority: Critical
>
> First, FanOutOneBlockAsyncDFSOutput can not work if the buffered data is larger than PacketReceiver.MAX_PACKET_SIZE(16MB).
> Second, since we only allow one block here, we need to make sure we do not exceed the block size after writing a large chunk of data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)