You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Hari Shreedharan <hs...@cloudera.com> on 2012/07/27 07:34:59 UTC
Review Request: FLUME-1398: Concurrency improvements to Async hbase sink
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6170/
-----------------------------------------------------------
Review request for Flume.
Description
-------
Made some concurrency modifications to Async hbase sink. Also, now backs off only if no events were fetched from channel - like Avro sink.
This addresses bug FLUME-1398.
https://issues.apache.org/jira/browse/FLUME-1398
Diffs
-----
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java 078c5a3
Diff: https://reviews.apache.org/r/6170/diff/
Testing
-------
Exising unit tests pass. Functional testing done.
Thanks,
Hari Shreedharan
Re: Review Request: FLUME-1398: Concurrency improvements to Async hbase sink
Posted by Hari Shreedharan <hs...@cloudera.com>.
> On July 27, 2012, 3:09 p.m., Jarek Cecho wrote:
> > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java, lines 151-165
> > <https://reviews.apache.org/r/6170/diff/2/?file=130109#file130109line151>
> >
> > Just a question here - do we really need to create new callback objects here each time? Wouldn't be possible to reuse them and spare the time on creating objects?
Yes, you are right. Updating patch.
> On July 27, 2012, 3:09 p.m., Jarek Cecho wrote:
> > flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java, lines 134-135
> > <https://reviews.apache.org/r/6170/diff/2/?file=130109#file130109line134>
> >
> > I believe that assigning zero to variable "i" might be done here only once.
fixing.
- Hari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6170/#review9534
-----------------------------------------------------------
On July 27, 2012, 7:29 a.m., Hari Shreedharan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6170/
> -----------------------------------------------------------
>
> (Updated July 27, 2012, 7:29 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Made some concurrency modifications to Async hbase sink. Also, now backs off only if no events were fetched from channel - like Avro sink.
>
>
> This addresses bug FLUME-1398.
> https://issues.apache.org/jira/browse/FLUME-1398
>
>
> Diffs
> -----
>
> flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java 078c5a3
>
> Diff: https://reviews.apache.org/r/6170/diff/
>
>
> Testing
> -------
>
> Exising unit tests pass. Functional testing done.
>
>
> Thanks,
>
> Hari Shreedharan
>
>
Re: Review Request: FLUME-1398: Concurrency improvements to Async hbase sink
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6170/#review9534
-----------------------------------------------------------
Hi Hari,
thanks you for your patch, I just do have one question and one small suggestion:
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java
<https://reviews.apache.org/r/6170/#comment20373>
I believe that assigning zero to variable "i" might be done here only once.
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java
<https://reviews.apache.org/r/6170/#comment20374>
Just a question here - do we really need to create new callback objects here each time? Wouldn't be possible to reuse them and spare the time on creating objects?
Jarcec
- Jarek Cecho
On July 27, 2012, 7:29 a.m., Hari Shreedharan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6170/
> -----------------------------------------------------------
>
> (Updated July 27, 2012, 7:29 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Made some concurrency modifications to Async hbase sink. Also, now backs off only if no events were fetched from channel - like Avro sink.
>
>
> This addresses bug FLUME-1398.
> https://issues.apache.org/jira/browse/FLUME-1398
>
>
> Diffs
> -----
>
> flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java 078c5a3
>
> Diff: https://reviews.apache.org/r/6170/diff/
>
>
> Testing
> -------
>
> Exising unit tests pass. Functional testing done.
>
>
> Thanks,
>
> Hari Shreedharan
>
>
Re: Review Request: FLUME-1398: Concurrency improvements to Async hbase sink
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6170/#review9542
-----------------------------------------------------------
Ship it!
Looks good to me. I'll commit it shortly.
Jarcec
- Jarek Cecho
On July 27, 2012, 4:38 p.m., Hari Shreedharan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6170/
> -----------------------------------------------------------
>
> (Updated July 27, 2012, 4:38 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Made some concurrency modifications to Async hbase sink. Also, now backs off only if no events were fetched from channel - like Avro sink.
>
>
> This addresses bug FLUME-1398.
> https://issues.apache.org/jira/browse/FLUME-1398
>
>
> Diffs
> -----
>
> flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java 078c5a3
>
> Diff: https://reviews.apache.org/r/6170/diff/
>
>
> Testing
> -------
>
> Exising unit tests pass. Functional testing done.
>
>
> Thanks,
>
> Hari Shreedharan
>
>
Re: Review Request: FLUME-1398: Concurrency improvements to Async hbase sink
Posted by Hari Shreedharan <hs...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6170/
-----------------------------------------------------------
(Updated July 27, 2012, 4:38 p.m.)
Review request for Flume.
Changes
-------
Updating per Jarcec's callback.
Description
-------
Made some concurrency modifications to Async hbase sink. Also, now backs off only if no events were fetched from channel - like Avro sink.
This addresses bug FLUME-1398.
https://issues.apache.org/jira/browse/FLUME-1398
Diffs (updated)
-----
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java 078c5a3
Diff: https://reviews.apache.org/r/6170/diff/
Testing
-------
Exising unit tests pass. Functional testing done.
Thanks,
Hari Shreedharan
Re: Review Request: FLUME-1398: Concurrency improvements to Async hbase sink
Posted by Hari Shreedharan <hs...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6170/
-----------------------------------------------------------
(Updated July 27, 2012, 7:29 a.m.)
Review request for Flume.
Changes
-------
Added metrics to the sink.
Description
-------
Made some concurrency modifications to Async hbase sink. Also, now backs off only if no events were fetched from channel - like Avro sink.
This addresses bug FLUME-1398.
https://issues.apache.org/jira/browse/FLUME-1398
Diffs (updated)
-----
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java 078c5a3
Diff: https://reviews.apache.org/r/6170/diff/
Testing
-------
Exising unit tests pass. Functional testing done.
Thanks,
Hari Shreedharan