You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Brock Noland <br...@cloudera.com> on 2012/02/20 02:35:24 UTC
Review Request: FLUME-979: ExecSource should optionally restart the command
when it exits
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3962/
-----------------------------------------------------------
Review request for Flume.
Summary
-------
1) Allows users to configure a restart flag and sleep interval for the ExecSource should the command exit.
2) Currently no one is reading standard error for the sub process. It is not ignored like the documentation suggests. A thread is added which by default reads and throws away stderr. If a flag is turned on it will logstderr. This kind of flag can be very useful when debugging why a command is not outputting the expected data.
This addresses bug FLUME-979.
https://issues.apache.org/jira/browse/FLUME-979
Diffs
-----
flume-ng-core/pom.xml d753fa1
flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java a96016c
flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java 3014368
Diff: https://reviews.apache.org/r/3962/diff
Testing
-------
A test for the restart flag and restartThrottle flag were added. All unit tests pass.
Thanks,
Brock
Re: Review Request: FLUME-979: ExecSource should optionally restart the
command when it exits
Posted by Arvind Prabhakar <ar...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3962/#review6184
-----------------------------------------------------------
Ship it!
+1. Changes look good Brock. Please update the patch by removing the *.orig files from it. Also, it will be great if you could wrap the source lines that exceed 80 char limit.
No need for follow-up review, just attaching the patch to the Jira will be great. Thanks for all the help with this.
- Arvind
On 2012-03-21 05:13:45, Brock Noland wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3962/
> -----------------------------------------------------------
>
> (Updated 2012-03-21 05:13:45)
>
>
> Review request for Flume.
>
>
> Summary
> -------
>
> 1) Allows users to configure a restart flag and sleep interval for the ExecSource should the command exit.
> 2) Currently no one is reading standard error for the sub process. It is not ignored like the documentation suggests. If the command generates enough stderr to fill it's buffer, it will be blocked until someone reads that buffer. A thread is added which by default reads and throws away stderr. If a flag is turned on it will logstderr. This kind of flag can be very useful when debugging why a command is not outputting the expected data.
>
>
> This addresses bug FLUME-979.
> https://issues.apache.org/jira/browse/FLUME-979
>
>
> Diffs
> -----
>
> flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java.orig PRE-CREATION
> flume-ng-core/src/main/java/org/apache/flume/source/ExecSourceConfigurationConstants.java PRE-CREATION
> flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java 6035270
> flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java.orig PRE-CREATION
> flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java b6b1181
>
> Diff: https://reviews.apache.org/r/3962/diff
>
>
> Testing
> -------
>
> A test for the restart flag and restartThrottle flag were added. All unit tests pass.
>
>
> Thanks,
>
> Brock
>
>
Re: Review Request: FLUME-979: ExecSource should optionally restart the
command when it exits
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3962/
-----------------------------------------------------------
(Updated 2012-03-21 05:13:45.221516)
Review request for Flume.
Changes
-------
For config class. Updated.
Summary
-------
1) Allows users to configure a restart flag and sleep interval for the ExecSource should the command exit.
2) Currently no one is reading standard error for the sub process. It is not ignored like the documentation suggests. If the command generates enough stderr to fill it's buffer, it will be blocked until someone reads that buffer. A thread is added which by default reads and throws away stderr. If a flag is turned on it will logstderr. This kind of flag can be very useful when debugging why a command is not outputting the expected data.
This addresses bug FLUME-979.
https://issues.apache.org/jira/browse/FLUME-979
Diffs (updated)
-----
flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java.orig PRE-CREATION
flume-ng-core/src/main/java/org/apache/flume/source/ExecSourceConfigurationConstants.java PRE-CREATION
flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java 6035270
flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java.orig PRE-CREATION
flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java b6b1181
Diff: https://reviews.apache.org/r/3962/diff
Testing
-------
A test for the restart flag and restartThrottle flag were added. All unit tests pass.
Thanks,
Brock
Re: Review Request: FLUME-979: ExecSource should optionally restart the
command when it exits
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3962/
-----------------------------------------------------------
(Updated 2012-03-21 05:11:39.725708)
Review request for Flume.
Changes
-------
Updated patch based on comments and rebased to trunk.
Summary
-------
1) Allows users to configure a restart flag and sleep interval for the ExecSource should the command exit.
2) Currently no one is reading standard error for the sub process. It is not ignored like the documentation suggests. If the command generates enough stderr to fill it's buffer, it will be blocked until someone reads that buffer. A thread is added which by default reads and throws away stderr. If a flag is turned on it will logstderr. This kind of flag can be very useful when debugging why a command is not outputting the expected data.
This addresses bug FLUME-979.
https://issues.apache.org/jira/browse/FLUME-979
Diffs (updated)
-----
flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java b6b1181
flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java 6035270
Diff: https://reviews.apache.org/r/3962/diff
Testing
-------
A test for the restart flag and restartThrottle flag were added. All unit tests pass.
Thanks,
Brock
Re: Review Request: FLUME-979: ExecSource should optionally restart the
command when it exits
Posted by Brock Noland <br...@cloudera.com>.
> On 2012-03-21 03:09:20, Arvind Prabhakar wrote:
> > Changes look good Brock. Some minor feedback:
> > * Please remove the extra whitespaces/tabs that are highlighted as red on the review
> >
done
> On 2012-03-21 03:09:20, Arvind Prabhakar wrote:
> > flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java, line 211
> > <https://reviews.apache.org/r/3962/diff/3/?file=75933#file75933line211>
> >
> > Please create a ExecSourceConfigurationConstants class and declare these strings as public static final String CONFIG_RESTART_THROTTLE etc.
done
On 2012-03-21 03:09:20, Brock Noland wrote:
> > The patch will also need to be rebased on the trunk as the sources have changed a bit since the last time.
done
- Brock
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3962/#review6159
-----------------------------------------------------------
On 2012-02-20 14:21:02, Brock Noland wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3962/
> -----------------------------------------------------------
>
> (Updated 2012-02-20 14:21:02)
>
>
> Review request for Flume.
>
>
> Summary
> -------
>
> 1) Allows users to configure a restart flag and sleep interval for the ExecSource should the command exit.
> 2) Currently no one is reading standard error for the sub process. It is not ignored like the documentation suggests. If the command generates enough stderr to fill it's buffer, it will be blocked until someone reads that buffer. A thread is added which by default reads and throws away stderr. If a flag is turned on it will logstderr. This kind of flag can be very useful when debugging why a command is not outputting the expected data.
>
>
> This addresses bug FLUME-979.
> https://issues.apache.org/jira/browse/FLUME-979
>
>
> Diffs
> -----
>
> flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java 3014368
> flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java a96016c
>
> Diff: https://reviews.apache.org/r/3962/diff
>
>
> Testing
> -------
>
> A test for the restart flag and restartThrottle flag were added. All unit tests pass.
>
>
> Thanks,
>
> Brock
>
>
Re: Review Request: FLUME-979: ExecSource should optionally restart the
command when it exits
Posted by Arvind Prabhakar <ar...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3962/#review6159
-----------------------------------------------------------
Changes look good Brock. Some minor feedback:
* Please remove the extra whitespaces/tabs that are highlighted as red on the review
flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java
<https://reviews.apache.org/r/3962/#comment13244>
Please create a ExecSourceConfigurationConstants class and declare these strings as public static final String CONFIG_RESTART_THROTTLE etc.
The patch will also need to be rebased on the trunk as the sources have changed a bit since the last time.
- Arvind
On 2012-02-20 14:21:02, Brock Noland wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3962/
> -----------------------------------------------------------
>
> (Updated 2012-02-20 14:21:02)
>
>
> Review request for Flume.
>
>
> Summary
> -------
>
> 1) Allows users to configure a restart flag and sleep interval for the ExecSource should the command exit.
> 2) Currently no one is reading standard error for the sub process. It is not ignored like the documentation suggests. If the command generates enough stderr to fill it's buffer, it will be blocked until someone reads that buffer. A thread is added which by default reads and throws away stderr. If a flag is turned on it will logstderr. This kind of flag can be very useful when debugging why a command is not outputting the expected data.
>
>
> This addresses bug FLUME-979.
> https://issues.apache.org/jira/browse/FLUME-979
>
>
> Diffs
> -----
>
> flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java 3014368
> flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java a96016c
>
> Diff: https://reviews.apache.org/r/3962/diff
>
>
> Testing
> -------
>
> A test for the restart flag and restartThrottle flag were added. All unit tests pass.
>
>
> Thanks,
>
> Brock
>
>
Re: Review Request: FLUME-979: ExecSource should optionally restart the
command when it exits
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3962/
-----------------------------------------------------------
(Updated 2012-02-20 14:21:02.192435)
Review request for Flume.
Changes
-------
Added note about reading stderr of a sub-process is required.
Summary (updated)
-------
1) Allows users to configure a restart flag and sleep interval for the ExecSource should the command exit.
2) Currently no one is reading standard error for the sub process. It is not ignored like the documentation suggests. If the command generates enough stderr to fill it's buffer, it will be blocked until someone reads that buffer. A thread is added which by default reads and throws away stderr. If a flag is turned on it will logstderr. This kind of flag can be very useful when debugging why a command is not outputting the expected data.
This addresses bug FLUME-979.
https://issues.apache.org/jira/browse/FLUME-979
Diffs
-----
flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java 3014368
flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java a96016c
Diff: https://reviews.apache.org/r/3962/diff
Testing
-------
A test for the restart flag and restartThrottle flag were added. All unit tests pass.
Thanks,
Brock
Re: Review Request: FLUME-979: ExecSource should optionally restart the
command when it exits
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3962/
-----------------------------------------------------------
(Updated 2012-02-20 01:37:42.014731)
Review request for Flume.
Changes
-------
Removed unneeded mockito change.
Summary
-------
1) Allows users to configure a restart flag and sleep interval for the ExecSource should the command exit.
2) Currently no one is reading standard error for the sub process. It is not ignored like the documentation suggests. A thread is added which by default reads and throws away stderr. If a flag is turned on it will logstderr. This kind of flag can be very useful when debugging why a command is not outputting the expected data.
This addresses bug FLUME-979.
https://issues.apache.org/jira/browse/FLUME-979
Diffs (updated)
-----
flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java 3014368
flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java a96016c
Diff: https://reviews.apache.org/r/3962/diff
Testing
-------
A test for the restart flag and restartThrottle flag were added. All unit tests pass.
Thanks,
Brock