You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Phil Scala <sc...@gmail.com> on 2013/07/01 17:53:40 UTC
Review Request 12211: FLUME-1899 - Make the spool directory source work with
sub directories
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12211/
-----------------------------------------------------------
Review request for Flume.
Bugs: https://issues.apache.org/jira/browse/FLUME-1899
https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/FLUME-1899
Repository: flume-git
Description
-------
Added recursive directory walking to the ReliableSpoolingFileEventReader. the same ignore pattern is used against the sub-directory names as well.
Diffs
-----
flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java f82fe1f
flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java 641b5c6
flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java f3cc703
flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java 652d2a2
flume-ng-doc/sphinx/FlumeUserGuide.rst 1b4d216
Diff: https://reviews.apache.org/r/12211/diff/
Testing
-------
Added new unit test, local testing on windows and CentOS. By default the setting is not enabled, need to update configuration i.e. spool-agent.sources.spooler.recursiveDirectorySearch = true
Thanks,
Phil Scala
Re: Review Request 12211: FLUME-1899 - Make the spool directory source work
with sub directories
Posted by Alexander Alten-Lorenz <al...@cloudera.com>.
> On July 17, 2013, 5:05 p.m., Alexander Alten-Lorenz wrote:
> > Can you please reformat to remove the leading spaces and empty lines? They're marked red when you view the diff.
> > Would you create a patch for our docs too?
>
> Phil Scala wrote:
> Hi Alexender, I can it will take me some time, as I am busy on a business trip and do not have the environment 100% setup with me. If you choose "hide whitespace changes" in the review board diff, then it looks pretty reasonable to me. Let me know if that's not sufficient.
>
> And, yes, the patch included the user guide changes.
Phil,
no worries, thanks for your response.
Yes, you're right, RB will remove the spaces / empty lines. But when a committer push the changes into trunk, the spaces will be carried too. So please remove them in the original patch file. Let me know when I can do this too, I'm happy to help here.
Sorry, I missed the patch against FlumeUserGuide.rst - so please ignore this.
Thanks
- Alexander
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12211/#review23295
-----------------------------------------------------------
On July 1, 2013, 3:53 p.m., Phil Scala wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12211/
> -----------------------------------------------------------
>
> (Updated July 1, 2013, 3:53 p.m.)
>
>
> Review request for Flume.
>
>
> Bugs: https://issues.apache.org/jira/browse/FLUME-1899
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/FLUME-1899
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> Added recursive directory walking to the ReliableSpoolingFileEventReader. the same ignore pattern is used against the sub-directory names as well.
>
>
> Diffs
> -----
>
> flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java f82fe1f
> flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java 641b5c6
> flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java f3cc703
> flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java 652d2a2
> flume-ng-doc/sphinx/FlumeUserGuide.rst 1b4d216
>
> Diff: https://reviews.apache.org/r/12211/diff/
>
>
> Testing
> -------
>
> Added new unit test, local testing on windows and CentOS. By default the setting is not enabled, need to update configuration i.e. spool-agent.sources.spooler.recursiveDirectorySearch = true
>
>
> Thanks,
>
> Phil Scala
>
>
Re: Review Request 12211: FLUME-1899 - Make the spool directory source work
with sub directories
Posted by Phil Scala <ph...@globalrelay.net>.
> On July 17, 2013, 5:05 p.m., Alexander Alten-Lorenz wrote:
> > Can you please reformat to remove the leading spaces and empty lines? They're marked red when you view the diff.
> > Would you create a patch for our docs too?
Hi Alexender, I can it will take me some time, as I am busy on a business trip and do not have the environment 100% setup with me. If you choose "hide whitespace changes" in the review board diff, then it looks pretty reasonable to me. Let me know if that's not sufficient.
And, yes, the patch included the user guide changes.
- Phil
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12211/#review23295
-----------------------------------------------------------
On July 1, 2013, 3:53 p.m., Phil Scala wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12211/
> -----------------------------------------------------------
>
> (Updated July 1, 2013, 3:53 p.m.)
>
>
> Review request for Flume.
>
>
> Bugs: https://issues.apache.org/jira/browse/FLUME-1899
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/FLUME-1899
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> Added recursive directory walking to the ReliableSpoolingFileEventReader. the same ignore pattern is used against the sub-directory names as well.
>
>
> Diffs
> -----
>
> flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java f82fe1f
> flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java 641b5c6
> flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java f3cc703
> flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java 652d2a2
> flume-ng-doc/sphinx/FlumeUserGuide.rst 1b4d216
>
> Diff: https://reviews.apache.org/r/12211/diff/
>
>
> Testing
> -------
>
> Added new unit test, local testing on windows and CentOS. By default the setting is not enabled, need to update configuration i.e. spool-agent.sources.spooler.recursiveDirectorySearch = true
>
>
> Thanks,
>
> Phil Scala
>
>
Re: Review Request 12211: FLUME-1899 - Make the spool directory source work
with sub directories
Posted by Alexander Alten-Lorenz <al...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12211/#review23295
-----------------------------------------------------------
Can you please reformat to remove the leading spaces and empty lines? They're marked red when you view the diff.
Would you create a patch for our docs too?
- Alexander Alten-Lorenz
On July 1, 2013, 3:53 p.m., Phil Scala wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12211/
> -----------------------------------------------------------
>
> (Updated July 1, 2013, 3:53 p.m.)
>
>
> Review request for Flume.
>
>
> Bugs: https://issues.apache.org/jira/browse/FLUME-1899
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/FLUME-1899
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> Added recursive directory walking to the ReliableSpoolingFileEventReader. the same ignore pattern is used against the sub-directory names as well.
>
>
> Diffs
> -----
>
> flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java f82fe1f
> flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java 641b5c6
> flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java f3cc703
> flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java 652d2a2
> flume-ng-doc/sphinx/FlumeUserGuide.rst 1b4d216
>
> Diff: https://reviews.apache.org/r/12211/diff/
>
>
> Testing
> -------
>
> Added new unit test, local testing on windows and CentOS. By default the setting is not enabled, need to update configuration i.e. spool-agent.sources.spooler.recursiveDirectorySearch = true
>
>
> Thanks,
>
> Phil Scala
>
>
Re: Review Request 12211: FLUME-1899 - Make the spool directory source work
with sub directories
Posted by Phil Scala <sc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12211/
-----------------------------------------------------------
(Updated July 25, 2013, 6:40 p.m.)
Review request for Flume.
Changes
-------
there were some whitespace changes that I missed, uploaded a new DIFF
Bugs: https://issues.apache.org/jira/browse/FLUME-1899
https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/FLUME-1899
Repository: flume-git
Description
-------
Added recursive directory walking to the ReliableSpoolingFileEventReader. the same ignore pattern is used against the sub-directory names as well.
Diffs (updated)
-----
flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java f82fe1f
flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java 7145580
flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java f3cc703
flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java 652d2a2
flume-ng-doc/sphinx/FlumeUserGuide.rst f84cda5
Diff: https://reviews.apache.org/r/12211/diff/
Testing
-------
Added new unit test, local testing on windows and CentOS. By default the setting is not enabled, need to update configuration i.e. spool-agent.sources.spooler.recursiveDirectorySearch = true
Thanks,
Phil Scala
Re: Review Request 12211: FLUME-1899 - Make the spool directory source work
with sub directories
Posted by Phil Scala <sc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12211/
-----------------------------------------------------------
(Updated July 25, 2013, 6:31 p.m.)
Review request for Flume.
Changes
-------
I created a new DIFF with hopefully fewer whitespace changes. No functional code changes were made.
Bugs: https://issues.apache.org/jira/browse/FLUME-1899
https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/FLUME-1899
Repository: flume-git
Description
-------
Added recursive directory walking to the ReliableSpoolingFileEventReader. the same ignore pattern is used against the sub-directory names as well.
Diffs (updated)
-----
flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java f82fe1f
flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java 7145580
flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java f3cc703
flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java 652d2a2
flume-ng-doc/sphinx/FlumeUserGuide.rst f84cda5
Diff: https://reviews.apache.org/r/12211/diff/
Testing
-------
Added new unit test, local testing on windows and CentOS. By default the setting is not enabled, need to update configuration i.e. spool-agent.sources.spooler.recursiveDirectorySearch = true
Thanks,
Phil Scala