You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Mike Percy <mp...@gmail.com> on 2012/06/28 01:59:26 UTC
Review Request: FLUME-1330: Avro Source should not use Fixed thread pool for
boss threads when pool size is specified
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5622/
-----------------------------------------------------------
Review request for Flume.
Description
-------
This patch uses a cached thread pool for boss threads in Netty instead of a fixed size pool of maxThreads size. It also improves some error messages in the AvroSource.
Additionally, I improved the error messages in ChannelProcessor (now includes channel name in error messages) and made sure that it catches Throwable in the non-batch case.
This addresses bug FLUME-1330.
https://issues.apache.org/jira/browse/FLUME-1330
Diffs
-----
flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java 07b04e2
flume-ng-core/src/main/java/org/apache/flume/sink/DefaultSinkFactory.java c89eeb6
flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 1d68d87
Diff: https://reviews.apache.org/r/5622/diff/
Testing
-------
Unit tests pass.
Thanks,
Mike Percy
Re: Review Request: FLUME-1330: Avro Source should not use Fixed thread pool
for boss threads when pool size is specified
Posted by Hari Shreedharan <hs...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5622/#review8684
-----------------------------------------------------------
Ship it!
Ship It!
- Hari Shreedharan
On June 27, 2012, 11:59 p.m., Mike Percy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5622/
> -----------------------------------------------------------
>
> (Updated June 27, 2012, 11:59 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> This patch uses a cached thread pool for boss threads in Netty instead of a fixed size pool of maxThreads size. It also improves some error messages in the AvroSource.
>
> Additionally, I improved the error messages in ChannelProcessor (now includes channel name in error messages) and made sure that it catches Throwable in the non-batch case.
>
>
> This addresses bug FLUME-1330.
> https://issues.apache.org/jira/browse/FLUME-1330
>
>
> Diffs
> -----
>
> flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java 07b04e2
> flume-ng-core/src/main/java/org/apache/flume/sink/DefaultSinkFactory.java c89eeb6
> flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 1d68d87
>
> Diff: https://reviews.apache.org/r/5622/diff/
>
>
> Testing
> -------
>
> Unit tests pass.
>
>
> Thanks,
>
> Mike Percy
>
>
Re: Review Request: FLUME-1330: Avro Source should not use Fixed thread pool
for boss threads when pool size is specified
Posted by Mubarak Seyed <se...@apple.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5622/#review8683
-----------------------------------------------------------
lgtm. Minor comments related to rpc failure counts.
flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java
<https://reviews.apache.org/r/5622/#comment18346>
counterGroup.incrementAndGet("rpc.failure");
flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java
<https://reviews.apache.org/r/5622/#comment18345>
I think we are not counting the RPC batch failure here
counterGroup.incrementAndGet("rpc.batch.failure");
- Mubarak Seyed
On June 27, 2012, 11:59 p.m., Mike Percy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5622/
> -----------------------------------------------------------
>
> (Updated June 27, 2012, 11:59 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> This patch uses a cached thread pool for boss threads in Netty instead of a fixed size pool of maxThreads size. It also improves some error messages in the AvroSource.
>
> Additionally, I improved the error messages in ChannelProcessor (now includes channel name in error messages) and made sure that it catches Throwable in the non-batch case.
>
>
> This addresses bug FLUME-1330.
> https://issues.apache.org/jira/browse/FLUME-1330
>
>
> Diffs
> -----
>
> flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java 07b04e2
> flume-ng-core/src/main/java/org/apache/flume/sink/DefaultSinkFactory.java c89eeb6
> flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 1d68d87
>
> Diff: https://reviews.apache.org/r/5622/diff/
>
>
> Testing
> -------
>
> Unit tests pass.
>
>
> Thanks,
>
> Mike Percy
>
>
Re: Review Request: FLUME-1330: Avro Source should not use Fixed thread pool
for boss threads when pool size is specified
Posted by Mubarak Seyed <se...@apple.com>.
> On June 28, 2012, 12:52 a.m., Mike Percy wrote:
> > Mubarak, thanks for looking. Since this has already been committed, please feel free to submit a patch with the additional metrics and I will commit it. Thanks!
Thanks Mike, will do.
- Mubarak
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5622/#review8687
-----------------------------------------------------------
On June 27, 2012, 11:59 p.m., Mike Percy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5622/
> -----------------------------------------------------------
>
> (Updated June 27, 2012, 11:59 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> This patch uses a cached thread pool for boss threads in Netty instead of a fixed size pool of maxThreads size. It also improves some error messages in the AvroSource.
>
> Additionally, I improved the error messages in ChannelProcessor (now includes channel name in error messages) and made sure that it catches Throwable in the non-batch case.
>
>
> This addresses bug FLUME-1330.
> https://issues.apache.org/jira/browse/FLUME-1330
>
>
> Diffs
> -----
>
> flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java 07b04e2
> flume-ng-core/src/main/java/org/apache/flume/sink/DefaultSinkFactory.java c89eeb6
> flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 1d68d87
>
> Diff: https://reviews.apache.org/r/5622/diff/
>
>
> Testing
> -------
>
> Unit tests pass.
>
>
> Thanks,
>
> Mike Percy
>
>
Re: Review Request: FLUME-1330: Avro Source should not use Fixed thread pool
for boss threads when pool size is specified
Posted by Mike Percy <mp...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/5622/#review8687
-----------------------------------------------------------
Mubarak, thanks for looking. Since this has already been committed, please feel free to submit a patch with the additional metrics and I will commit it. Thanks!
- Mike Percy
On June 27, 2012, 11:59 p.m., Mike Percy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/5622/
> -----------------------------------------------------------
>
> (Updated June 27, 2012, 11:59 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> This patch uses a cached thread pool for boss threads in Netty instead of a fixed size pool of maxThreads size. It also improves some error messages in the AvroSource.
>
> Additionally, I improved the error messages in ChannelProcessor (now includes channel name in error messages) and made sure that it catches Throwable in the non-batch case.
>
>
> This addresses bug FLUME-1330.
> https://issues.apache.org/jira/browse/FLUME-1330
>
>
> Diffs
> -----
>
> flume-ng-core/src/main/java/org/apache/flume/channel/ChannelProcessor.java 07b04e2
> flume-ng-core/src/main/java/org/apache/flume/sink/DefaultSinkFactory.java c89eeb6
> flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java 1d68d87
>
> Diff: https://reviews.apache.org/r/5622/diff/
>
>
> Testing
> -------
>
> Unit tests pass.
>
>
> Thanks,
>
> Mike Percy
>
>