You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Mike Percy <mp...@apache.org> on 2014/03/02 03:47:03 UTC
Review Request 18668: Support coalescing increments in HBaseSink
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18668/
-----------------------------------------------------------
Review request for Flume.
Bugs: FLUME-2338
https://issues.apache.org/jira/browse/FLUME-2338
Repository: flume-git
Description
-------
Coalesce support for HBaseSink. Also includes a BatchAware interface, allowing serializers that implement it to be aware of batching so that they can choose keys to optimize the effect of coalescing, if desired.
Diffs
-----
flume-ng-doc/sphinx/FlumeUserGuide.rst 96bf73e
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/BatchAware.java PRE-CREATION
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java c4a666c
flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/IncrementHBaseSerializer.java PRE-CREATION
flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java d1b0182
Diff: https://reviews.apache.org/r/18668/diff/
Testing
-------
Added unit tests. Tested a version of this patch (prior to a rebase for trunk) under high load and the approach is very effective for certain high-increment workloads that end up with a lot of contention on keys in HBase.
Thanks,
Mike Percy
Re: Review Request 18668: Support coalescing increments in HBaseSink
Posted by Mike Percy <mp...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18668/
-----------------------------------------------------------
(Updated March 3, 2014, 5:09 a.m.)
Review request for Flume.
Changes
-------
Fixed more HBase 0.96 issues
Bugs: FLUME-2338
https://issues.apache.org/jira/browse/FLUME-2338
Repository: flume-git
Description
-------
Coalesce support for HBaseSink. Also includes a BatchAware interface, allowing serializers that implement it to be aware of batching so that they can choose keys to optimize the effect of coalescing, if desired.
Diffs (updated)
-----
flume-ng-doc/sphinx/FlumeUserGuide.rst 96bf73e
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/BatchAware.java PRE-CREATION
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java c4a666c
flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/IncrementHBaseSerializer.java PRE-CREATION
flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java d1b0182
Diff: https://reviews.apache.org/r/18668/diff/
Testing
-------
Added unit tests. Tested a version of this patch (prior to a rebase for trunk) under high load and the approach is very effective for certain high-increment workloads that end up with a lot of contention on keys in HBase.
Thanks,
Mike Percy
Re: Review Request 18668: Support coalescing increments in HBaseSink
Posted by Mike Percy <mp...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18668/
-----------------------------------------------------------
(Updated March 2, 2014, 1:09 p.m.)
Review request for Flume.
Changes
-------
Added tentative HBase 0.96 support via reflection. Tested only against HBase 0.92
Bugs: FLUME-2338
https://issues.apache.org/jira/browse/FLUME-2338
Repository: flume-git
Description
-------
Coalesce support for HBaseSink. Also includes a BatchAware interface, allowing serializers that implement it to be aware of batching so that they can choose keys to optimize the effect of coalescing, if desired.
Diffs (updated)
-----
flume-ng-doc/sphinx/FlumeUserGuide.rst 96bf73e
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/BatchAware.java PRE-CREATION
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSink.java c4a666c
flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/IncrementHBaseSerializer.java PRE-CREATION
flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestHBaseSink.java d1b0182
Diff: https://reviews.apache.org/r/18668/diff/
Testing
-------
Added unit tests. Tested a version of this patch (prior to a rebase for trunk) under high load and the approach is very effective for certain high-increment workloads that end up with a lot of contention on keys in HBase.
Thanks,
Mike Percy