You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by jm...@apache.org on 2011/09/30 17:09:20 UTC
Review Request: FLUME-734: escapedFormatDfs goes into a file creation frenzy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2129/
-----------------------------------------------------------
Review request for Flume, Prasad Mujumdar and Eric Sammer.
Summary
-------
Output formats are now changed into function expressions and a new output format is created for each customdfssink. (this happens when there are multiple files being written by a escapedFormatDfs, and there was a shared outputformat object).
commit 20a85af7d21a2a33c63903c794b4bd0d3dd2be02
Author: Jonathan Hsieh <jo...@cloudera.com>
Date: Wed Aug 10 09:18:02 2011 -0700
FLUME-734: escapedFormatDfs goes into a file creation frenzy
This addresses bug flume-734.
https://issues.apache.org/jira/browse/flume-734
Diffs
-----
flume-core/src/main/java/com/cloudera/flume/conf/FlumeConfiguration.java 8ecfed1
flume-core/src/main/java/com/cloudera/flume/conf/SinkBuilderUtil.java 2aaa566
flume-core/src/main/java/com/cloudera/flume/handlers/hdfs/EscapedCustomDfsSink.java 20ebdfd
flume-core/src/test/java/com/cloudera/flume/handlers/hdfs/TestEscapedCustomOutputDfs.java 7618acd
flume-core/src/test/java/com/cloudera/flume/handlers/rolling/TestRollSink.java 1fd788f
Diff: https://reviews.apache.org/r/2129/diff
Testing
-------
Tests pass.
>From comments on jira, an earlier version of this patch is working for a few folks already.
Thanks,
jmhsieh
Re: Review Request: FLUME-734: escapedFormatDfs goes into a file creation
frenzy
Posted by jm...@apache.org.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2129/#review2230
-----------------------------------------------------------
flume-core/src/test/java/com/cloudera/flume/handlers/rolling/TestRollSink.java
<https://reviews.apache.org/r/2129/#comment5178>
Added context here.
- jmhsieh
On 2011-09-30 20:47:52, jmhsieh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2129/
> -----------------------------------------------------------
>
> (Updated 2011-09-30 20:47:52)
>
>
> Review request for Flume, Prasad Mujumdar and Eric Sammer.
>
>
> Summary
> -------
>
> Output formats are now changed into function expressions and a new output format is created for each customdfssink. (this happens when there are multiple files being written by a escapedFormatDfs, and there was a shared outputformat object).
>
> commit 20a85af7d21a2a33c63903c794b4bd0d3dd2be02
> Author: Jonathan Hsieh <jo...@cloudera.com>
> Date: Wed Aug 10 09:18:02 2011 -0700
>
> FLUME-734: escapedFormatDfs goes into a file creation frenzy
>
>
> This addresses bug flume-734.
> https://issues.apache.org/jira/browse/flume-734
>
>
> Diffs
> -----
>
> flume-core/src/main/java/com/cloudera/flume/conf/SinkBuilderUtil.java 2aaa566
> flume-core/src/main/java/com/cloudera/flume/handlers/hdfs/EscapedCustomDfsSink.java 20ebdfd
> flume-core/src/test/java/com/cloudera/flume/handlers/hdfs/TestEscapedCustomOutputDfs.java 7618acd
> flume-core/src/test/java/com/cloudera/flume/handlers/rolling/TestRollSink.java 3096f59
> flume-core/src/main/java/com/cloudera/flume/conf/FlumeConfiguration.java 8ecfed1
>
> Diff: https://reviews.apache.org/r/2129/diff
>
>
> Testing
> -------
>
> Tests pass.
>
> From comments on jira, an earlier version of this patch is working for a few folks already.
>
>
> Thanks,
>
> jmhsieh
>
>
Re: Review Request: FLUME-734: escapedFormatDfs goes into a file creation
frenzy
Posted by jm...@apache.org.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2129/
-----------------------------------------------------------
(Updated 2011-09-30 20:47:52.242185)
Review request for Flume, Prasad Mujumdar and Eric Sammer.
Changes
-------
After FLUME-768, a minor tweak was required to make it compile and pass.
Summary
-------
Output formats are now changed into function expressions and a new output format is created for each customdfssink. (this happens when there are multiple files being written by a escapedFormatDfs, and there was a shared outputformat object).
commit 20a85af7d21a2a33c63903c794b4bd0d3dd2be02
Author: Jonathan Hsieh <jo...@cloudera.com>
Date: Wed Aug 10 09:18:02 2011 -0700
FLUME-734: escapedFormatDfs goes into a file creation frenzy
This addresses bug flume-734.
https://issues.apache.org/jira/browse/flume-734
Diffs (updated)
-----
flume-core/src/main/java/com/cloudera/flume/conf/SinkBuilderUtil.java 2aaa566
flume-core/src/main/java/com/cloudera/flume/handlers/hdfs/EscapedCustomDfsSink.java 20ebdfd
flume-core/src/test/java/com/cloudera/flume/handlers/hdfs/TestEscapedCustomOutputDfs.java 7618acd
flume-core/src/test/java/com/cloudera/flume/handlers/rolling/TestRollSink.java 3096f59
flume-core/src/main/java/com/cloudera/flume/conf/FlumeConfiguration.java 8ecfed1
Diff: https://reviews.apache.org/r/2129/diff
Testing
-------
Tests pass.
>From comments on jira, an earlier version of this patch is working for a few folks already.
Thanks,
jmhsieh
Re: Review Request: FLUME-734: escapedFormatDfs goes into a file creation
frenzy
Posted by Prasad Mujumdar <pr...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2129/#review2221
-----------------------------------------------------------
Ship it!
Looks fine to me.
Note that the TestRollSink.java has a new test case getting added as part of Flume-768. The EscapedCustomDfsSink is called from that new test. We'll need that minor change in the new test after both patches are committed.
thanx
Prasad
- Prasad
On 2011-09-30 15:09:20, jmhsieh wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/2129/
> -----------------------------------------------------------
>
> (Updated 2011-09-30 15:09:20)
>
>
> Review request for Flume, Prasad Mujumdar and Eric Sammer.
>
>
> Summary
> -------
>
> Output formats are now changed into function expressions and a new output format is created for each customdfssink. (this happens when there are multiple files being written by a escapedFormatDfs, and there was a shared outputformat object).
>
> commit 20a85af7d21a2a33c63903c794b4bd0d3dd2be02
> Author: Jonathan Hsieh <jo...@cloudera.com>
> Date: Wed Aug 10 09:18:02 2011 -0700
>
> FLUME-734: escapedFormatDfs goes into a file creation frenzy
>
>
> This addresses bug flume-734.
> https://issues.apache.org/jira/browse/flume-734
>
>
> Diffs
> -----
>
> flume-core/src/main/java/com/cloudera/flume/conf/FlumeConfiguration.java 8ecfed1
> flume-core/src/main/java/com/cloudera/flume/conf/SinkBuilderUtil.java 2aaa566
> flume-core/src/main/java/com/cloudera/flume/handlers/hdfs/EscapedCustomDfsSink.java 20ebdfd
> flume-core/src/test/java/com/cloudera/flume/handlers/hdfs/TestEscapedCustomOutputDfs.java 7618acd
> flume-core/src/test/java/com/cloudera/flume/handlers/rolling/TestRollSink.java 1fd788f
>
> Diff: https://reviews.apache.org/r/2129/diff
>
>
> Testing
> -------
>
> Tests pass.
>
> From comments on jira, an earlier version of this patch is working for a few folks already.
>
>
> Thanks,
>
> jmhsieh
>
>