[jira] [Commented] (HBASE-19486) Periodically ensure records are not buffered too long by BufferedMutator

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16302450#comment-16302450 ] 

Hadoop QA commented on HBASE-19486:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  4s{color} | {color:red} HBASE-19486 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.6.0/precommit-patchnames for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-19486 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12903525/HBASE-19486-20171223-1438-trigger-qa-run.patch |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/10656/console |
| Powered by | Apache Yetus 0.6.0   http://yetus.apache.org |


This message was automatically generated.



>  Periodically ensure records are not buffered too long by BufferedMutator
> -------------------------------------------------------------------------
>
>                 Key: HBASE-19486
>                 URL: https://issues.apache.org/jira/browse/HBASE-19486
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client
>            Reporter: Niels Basjes
>            Assignee: Niels Basjes
>         Attachments: HBASE-19486-20171212-2117.patch, HBASE-19486-20171218-1229.patch, HBASE-19486-20171218-1300.patch, HBASE-19486-20171219-0933.patch, HBASE-19486-20171219-1026.patch, HBASE-19486-20171219-1122-trigger-qa-run.patch, HBASE-19486-20171220-1612-trigger-qa-run.patch, HBASE-19486-20171220-2228-trigger-qa-run.patch, HBASE-19486-20171223-1438-trigger-qa-run.patch
>
>
> I'm working on several projects where we are doing stream / event type processing instead of batch type processing. We mostly use Apache Flink and Apache Beam for these projects.
> When we ingest a continuous stream of events and feed that into HBase via a BufferedMutator this all works fine. The buffer fills up at a predictable rate and we can make sure it flushes several times per second into HBase by tuning the buffer size.
> We also have situations where the event rate is unpredictable. Some times because the source is in reality a batch job that puts records into Kafka, sometimes because it is the "predictable in production" application in our testing environment (where only the dev triggers a handful of events).
> For these kinds of use cases we need a way to 'force' the BufferedMutator to automatically flush any records in the buffer even if the buffer is not full.
> I'll put up a pull request with a proposed implementation for review against the master (i.e. 3.0.0).
> When approved I would like to backport this to the 1.x and 2.x versions of the client in the same (as close as possible) way.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)