You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Israel Ekpo <is...@aicer.org> on 2013/04/28 07:24:12 UTC
Review Request: FLUME-2030 - Documentation of Configuration Changes
JMSSource, HBaseSink, AsyncHBaseSink and ElasticSearchSink
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10817/
-----------------------------------------------------------
Review request for Flume.
Description
-------
FLUME-2030 - Documentation of Configuration Changes JMSSource, HBaseSink, AsyncHBaseSink and ElasticSearchSink
- Updated user guide to illustrate the replacement of FQCNs with enum constants (lowercased) for built-in sources and sinks.
- Added System Requirements needed to run Flume.
- Added documentation to encourage users to migrate to 1.x from 0.9.x so as to take advantage of improvements available in Flume NG
- Added documentation to inform users that Apache Flume is not limited only to log data aggregation.
This addresses bugs FLUME-1886, FLUME-1889, FLUME-1994, and FLUME-2030.
https://issues.apache.org/jira/browse/FLUME-1886
https://issues.apache.org/jira/browse/FLUME-1889
https://issues.apache.org/jira/browse/FLUME-1994
https://issues.apache.org/jira/browse/FLUME-2030
Diffs
-----
flume-ng-doc/sphinx/FlumeUserGuide.rst 38f2205
Diff: https://reviews.apache.org/r/10817/diff/
Testing
-------
N/A
Thanks,
Israel Ekpo
Re: Review Request: FLUME-2030 - Documentation of Configuration Changes
JMSSource, HBaseSink, AsyncHBaseSink and ElasticSearchSink
Posted by Israel Ekpo <is...@aicer.org>.
> On April 29, 2013, 4:10 p.m., Edward Sargisson wrote:
> > flume-ng-doc/sphinx/FlumeUserGuide.rst, line 54
> > <https://reviews.apache.org/r/10817/diff/1/?file=285292#file285292line54>
> >
> > "Sufficient" is meaningless to Operations staff. The default service script uses 20MB - which works okay-ish unless you use the memory channel. (By okay-ish I mean that I've seen Flume stop with an OOM which was most likely a memory leak.)
I see what you mean. I will think more about this and come up with reasonable values that makes sense for typical use cases. Do you have any recommendations?
- Israel
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10817/#review19860
-----------------------------------------------------------
On April 28, 2013, 5:24 a.m., Israel Ekpo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10817/
> -----------------------------------------------------------
>
> (Updated April 28, 2013, 5:24 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> FLUME-2030 - Documentation of Configuration Changes JMSSource, HBaseSink, AsyncHBaseSink and ElasticSearchSink
>
> - Updated user guide to illustrate the replacement of FQCNs with enum constants (lowercased) for built-in sources and sinks.
> - Added System Requirements needed to run Flume.
> - Added documentation to encourage users to migrate to 1.x from 0.9.x so as to take advantage of improvements available in Flume NG
> - Added documentation to inform users that Apache Flume is not limited only to log data aggregation.
>
>
> This addresses bugs FLUME-1886, FLUME-1889, FLUME-1994, and FLUME-2030.
> https://issues.apache.org/jira/browse/FLUME-1886
> https://issues.apache.org/jira/browse/FLUME-1889
> https://issues.apache.org/jira/browse/FLUME-1994
> https://issues.apache.org/jira/browse/FLUME-2030
>
>
> Diffs
> -----
>
> flume-ng-doc/sphinx/FlumeUserGuide.rst 38f2205
>
> Diff: https://reviews.apache.org/r/10817/diff/
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Israel Ekpo
>
>
Re: Review Request: FLUME-2030 - Documentation of Configuration Changes
JMSSource, HBaseSink, AsyncHBaseSink and ElasticSearchSink
Posted by Israel Ekpo <is...@aicer.org>.
> On April 29, 2013, 4:10 p.m., Edward Sargisson wrote:
> > flume-ng-doc/sphinx/FlumeUserGuide.rst, line 31
> > <https://reviews.apache.org/r/10817/diff/1/?file=285292#file285292line31>
> >
> > "not only restricted" doesn't make sense in this context. Consider "is not restricted to".
Thanks I will fix this
> On April 29, 2013, 4:10 p.m., Edward Sargisson wrote:
> > flume-ng-doc/sphinx/FlumeUserGuide.rst, line 33
> > <https://reviews.apache.org/r/10817/diff/1/?file=285292#file285292line33>
> >
> > Commas need to be inserted for correct grammar. "including, but not limited to, network"
This will be addressed.
> On April 29, 2013, 4:10 p.m., Edward Sargisson wrote:
> > flume-ng-doc/sphinx/FlumeUserGuide.rst, line 34
> > <https://reviews.apache.org/r/10817/diff/1/?file=285292#file285292line34>
> >
> > Correct me if I'm wrong but Flume may have a problem if the size of each event gets too large? I'm not sure what might be too large but the software is designed with a log line in mind - and a log line might only get to a few KB.
Just like how SpoolDirectorySource is configured, limits for event size can be set so that occurrences that exceed that threshold will be truncated.
- Israel
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10817/#review19860
-----------------------------------------------------------
On April 28, 2013, 5:24 a.m., Israel Ekpo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10817/
> -----------------------------------------------------------
>
> (Updated April 28, 2013, 5:24 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> FLUME-2030 - Documentation of Configuration Changes JMSSource, HBaseSink, AsyncHBaseSink and ElasticSearchSink
>
> - Updated user guide to illustrate the replacement of FQCNs with enum constants (lowercased) for built-in sources and sinks.
> - Added System Requirements needed to run Flume.
> - Added documentation to encourage users to migrate to 1.x from 0.9.x so as to take advantage of improvements available in Flume NG
> - Added documentation to inform users that Apache Flume is not limited only to log data aggregation.
>
>
> This addresses bugs FLUME-1886, FLUME-1889, FLUME-1994, and FLUME-2030.
> https://issues.apache.org/jira/browse/FLUME-1886
> https://issues.apache.org/jira/browse/FLUME-1889
> https://issues.apache.org/jira/browse/FLUME-1994
> https://issues.apache.org/jira/browse/FLUME-2030
>
>
> Diffs
> -----
>
> flume-ng-doc/sphinx/FlumeUserGuide.rst 38f2205
>
> Diff: https://reviews.apache.org/r/10817/diff/
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Israel Ekpo
>
>
Re: Review Request: FLUME-2030 - Documentation of Configuration Changes
JMSSource, HBaseSink, AsyncHBaseSink and ElasticSearchSink
Posted by Edward Sargisson <es...@pobox.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10817/#review19860
-----------------------------------------------------------
flume-ng-doc/sphinx/FlumeUserGuide.rst
<https://reviews.apache.org/r/10817/#comment40957>
"not only restricted" doesn't make sense in this context. Consider "is not restricted to".
flume-ng-doc/sphinx/FlumeUserGuide.rst
<https://reviews.apache.org/r/10817/#comment40958>
Commas need to be inserted for correct grammar. "including, but not limited to, network"
flume-ng-doc/sphinx/FlumeUserGuide.rst
<https://reviews.apache.org/r/10817/#comment40959>
Correct me if I'm wrong but Flume may have a problem if the size of each event gets too large? I'm not sure what might be too large but the software is designed with a log line in mind - and a log line might only get to a few KB.
flume-ng-doc/sphinx/FlumeUserGuide.rst
<https://reviews.apache.org/r/10817/#comment40960>
"Sufficient" is meaningless to Operations staff. The default service script uses 20MB - which works okay-ish unless you use the memory channel. (By okay-ish I mean that I've seen Flume stop with an OOM which was most likely a memory leak.)
- Edward Sargisson
On April 28, 2013, 5:24 a.m., Israel Ekpo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10817/
> -----------------------------------------------------------
>
> (Updated April 28, 2013, 5:24 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> FLUME-2030 - Documentation of Configuration Changes JMSSource, HBaseSink, AsyncHBaseSink and ElasticSearchSink
>
> - Updated user guide to illustrate the replacement of FQCNs with enum constants (lowercased) for built-in sources and sinks.
> - Added System Requirements needed to run Flume.
> - Added documentation to encourage users to migrate to 1.x from 0.9.x so as to take advantage of improvements available in Flume NG
> - Added documentation to inform users that Apache Flume is not limited only to log data aggregation.
>
>
> This addresses bugs FLUME-1886, FLUME-1889, FLUME-1994, and FLUME-2030.
> https://issues.apache.org/jira/browse/FLUME-1886
> https://issues.apache.org/jira/browse/FLUME-1889
> https://issues.apache.org/jira/browse/FLUME-1994
> https://issues.apache.org/jira/browse/FLUME-2030
>
>
> Diffs
> -----
>
> flume-ng-doc/sphinx/FlumeUserGuide.rst 38f2205
>
> Diff: https://reviews.apache.org/r/10817/diff/
>
>
> Testing
> -------
>
> N/A
>
>
> Thanks,
>
> Israel Ekpo
>
>