You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2018/01/02 09:11:00 UTC
[jira] [Commented] (HBASE-19486) Periodically ensure records are
not buffered too long by BufferedMutator
[ https://issues.apache.org/jira/browse/HBASE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307787#comment-16307787 ]
Hadoop QA commented on HBASE-19486:
-----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 53s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m 5s{color} | {color:green} branch has no errors when building our shaded downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m 34s{color} | {color:green} patch has no errors when building our shaded downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 19m 8s{color} | {color:green} Patch does not cause any errors with Hadoop 2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 41s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 9s{color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 58s{color} | {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19486 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12904169/HBASE-19486.20180102-081903-addendum.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fcdbc7a21572 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 12:48:20 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh |
| git revision | master / 6708d54478 |
| maven | version: Apache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d; 2017-10-18T07:58:13Z) |
| Default Java | 1.8.0_151 |
| Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/10841/testReport/ |
| modules | C: hbase-client U: hbase-client |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/10841/console |
| Powered by | Apache Yetus 0.6.0 http://yetus.apache.org |
This message was automatically generated.
> Periodically ensure records are not buffered too long by BufferedMutator
> -------------------------------------------------------------------------
>
> Key: HBASE-19486
> URL: https://issues.apache.org/jira/browse/HBASE-19486
> Project: HBase
> Issue Type: Improvement
> Components: Client
> Reporter: Niels Basjes
> Assignee: Niels Basjes
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19486-20171212-2117.patch, HBASE-19486-20171218-1229.patch, HBASE-19486-20171218-1300.patch, HBASE-19486-20171219-0933.patch, HBASE-19486-20171219-1026.patch, HBASE-19486-20171219-1122-trigger-qa-run.patch, HBASE-19486-20171220-1612-trigger-qa-run.patch, HBASE-19486-20171220-2228-trigger-qa-run.patch, HBASE-19486-20171223-1438-trigger-qa-run.patch, HBASE-19486-20171223-1728-trigger-qa-run.patch, HBASE-19486-20171223-2222-trigger-qa-run.patch, HBASE-19486-20171224-1101-trigger-qa-run.patch, HBASE-19486-20171224-1602.patch, HBASE-19486-branch-1.v0.patch, HBASE-19486-branch-1.v1.patch, HBASE-19486.20171231-105839-addendum.patch, HBASE-19486.20180102-081903-addendum.patch, HBASE-19486.v0.patch
>
>
> I'm working on several projects where we are doing stream / event type processing instead of batch type processing. We mostly use Apache Flink and Apache Beam for these projects.
> When we ingest a continuous stream of events and feed that into HBase via a BufferedMutator this all works fine. The buffer fills up at a predictable rate and we can make sure it flushes several times per second into HBase by tuning the buffer size.
> We also have situations where the event rate is unpredictable. Some times because the source is in reality a batch job that puts records into Kafka, sometimes because it is the "predictable in production" application in our testing environment (where only the dev triggers a handful of events).
> For these kinds of use cases we need a way to 'force' the BufferedMutator to automatically flush any records in the buffer even if the buffer is not full.
> I'll put up a pull request with a proposed implementation for review against the master (i.e. 3.0.0).
> When approved I would like to backport this to the 1.x and 2.x versions of the client in the same (as close as possible) way.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)