You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Jarek Cecho <ja...@apache.org> on 2012/12/10 03:20:58 UTC
Review Request: FLUME-1702 HDFSEventSink should write to a hidden file as
opposed to a .tmp file
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8438/
-----------------------------------------------------------
Review request for Flume.
Description
-------
I've added two additional HDFS sink properties "inUsePrefix" and "inUseSuffix" allowing user to override both prefix and suffix of temporal files.
This addresses bug FLUME-1702.
https://issues.apache.org/jira/browse/FLUME-1702
Diffs
-----
flume-ng-doc/sphinx/FlumeUserGuide.rst 21ca5cc62450932f334d933e9ed50d224e10249a
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java d0ff6e34399886ee4a85d983243f76899a966b79
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 64ac2d73520be961617e3dd22b862305e4e00573
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java b191ef3fd36be6cc4d8f754bb2ad9898994df03f
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java d23f09dd9e8ef5c257bdb3b08bf8deec57641b0b
Diff: https://reviews.apache.org/r/8438/diff/
Testing
-------
Current unit tests seems to be passing. I've also added two additional tests to verify new functionality.
Thanks,
Jarek Cecho
Re: Review Request: FLUME-1702 HDFSEventSink should write to a hidden file
as opposed to a .tmp file
Posted by Brock Noland <br...@cloudera.com>.
> On Dec. 13, 2012, 7:03 p.m., Brock Noland wrote:
> > I am seeing a test failure with this patch:
> >
> >
> > Tests in error:
> > testIntervalRollerBug(org.apache.flume.sink.hdfs.TestBucketWriter): String index out of range: -20
> >
> >
Jarcec,
Otherwise the patch looks good! Thank you for your contribution!
Brock
- Brock
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8438/#review14454
-----------------------------------------------------------
On Dec. 10, 2012, 2:20 a.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8438/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2012, 2:20 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> I've added two additional HDFS sink properties "inUsePrefix" and "inUseSuffix" allowing user to override both prefix and suffix of temporal files.
>
>
> This addresses bug FLUME-1702.
> https://issues.apache.org/jira/browse/FLUME-1702
>
>
> Diffs
> -----
>
> flume-ng-doc/sphinx/FlumeUserGuide.rst 21ca5cc62450932f334d933e9ed50d224e10249a
> flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java d0ff6e34399886ee4a85d983243f76899a966b79
> flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 64ac2d73520be961617e3dd22b862305e4e00573
> flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java b191ef3fd36be6cc4d8f754bb2ad9898994df03f
> flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java d23f09dd9e8ef5c257bdb3b08bf8deec57641b0b
>
> Diff: https://reviews.apache.org/r/8438/diff/
>
>
> Testing
> -------
>
> Current unit tests seems to be passing. I've also added two additional tests to verify new functionality.
>
>
> Thanks,
>
> Jarek Cecho
>
>
Re: Review Request: FLUME-1702 HDFSEventSink should write to a hidden file
as opposed to a .tmp file
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8438/#review14454
-----------------------------------------------------------
I am seeing a test failure with this patch:
Tests in error:
testIntervalRollerBug(org.apache.flume.sink.hdfs.TestBucketWriter): String index out of range: -20
- Brock Noland
On Dec. 10, 2012, 2:20 a.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8438/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2012, 2:20 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> I've added two additional HDFS sink properties "inUsePrefix" and "inUseSuffix" allowing user to override both prefix and suffix of temporal files.
>
>
> This addresses bug FLUME-1702.
> https://issues.apache.org/jira/browse/FLUME-1702
>
>
> Diffs
> -----
>
> flume-ng-doc/sphinx/FlumeUserGuide.rst 21ca5cc62450932f334d933e9ed50d224e10249a
> flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java d0ff6e34399886ee4a85d983243f76899a966b79
> flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 64ac2d73520be961617e3dd22b862305e4e00573
> flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java b191ef3fd36be6cc4d8f754bb2ad9898994df03f
> flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java d23f09dd9e8ef5c257bdb3b08bf8deec57641b0b
>
> Diff: https://reviews.apache.org/r/8438/diff/
>
>
> Testing
> -------
>
> Current unit tests seems to be passing. I've also added two additional tests to verify new functionality.
>
>
> Thanks,
>
> Jarek Cecho
>
>
Re: Review Request: FLUME-1702 HDFSEventSink should write to a hidden file
as opposed to a .tmp file
Posted by Brock Noland <br...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8438/#review14673
-----------------------------------------------------------
Ship it!
Ship It!
- Brock Noland
On Dec. 16, 2012, 7:03 a.m., Jarek Cecho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8438/
> -----------------------------------------------------------
>
> (Updated Dec. 16, 2012, 7:03 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> I've added two additional HDFS sink properties "inUsePrefix" and "inUseSuffix" allowing user to override both prefix and suffix of temporal files.
>
>
> This addresses bug FLUME-1702.
> https://issues.apache.org/jira/browse/FLUME-1702
>
>
> Diffs
> -----
>
> flume-ng-doc/sphinx/FlumeUserGuide.rst 649071020b2281f9a1fb772c0672c8879655e8dd
> flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java d0ff6e34399886ee4a85d983243f76899a966b79
> flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 64ac2d73520be961617e3dd22b862305e4e00573
> flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java b191ef3fd36be6cc4d8f754bb2ad9898994df03f
> flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java d23f09dd9e8ef5c257bdb3b08bf8deec57641b0b
>
> Diff: https://reviews.apache.org/r/8438/diff/
>
>
> Testing
> -------
>
> Current unit tests seems to be passing. I've also added two additional tests to verify new functionality.
>
>
> Thanks,
>
> Jarek Cecho
>
>
Re: Review Request: FLUME-1702 HDFSEventSink should write to a hidden file
as opposed to a .tmp file
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8438/
-----------------------------------------------------------
(Updated Dec. 16, 2012, 7:03 a.m.)
Review request for Flume.
Changes
-------
Thanks Brock for finding the failing test! It was failing only on mac os x, so I did not notice it before. Please accept my apologies for that.
Description
-------
I've added two additional HDFS sink properties "inUsePrefix" and "inUseSuffix" allowing user to override both prefix and suffix of temporal files.
This addresses bug FLUME-1702.
https://issues.apache.org/jira/browse/FLUME-1702
Diffs (updated)
-----
flume-ng-doc/sphinx/FlumeUserGuide.rst 649071020b2281f9a1fb772c0672c8879655e8dd
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java d0ff6e34399886ee4a85d983243f76899a966b79
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java 64ac2d73520be961617e3dd22b862305e4e00573
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java b191ef3fd36be6cc4d8f754bb2ad9898994df03f
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java d23f09dd9e8ef5c257bdb3b08bf8deec57641b0b
Diff: https://reviews.apache.org/r/8438/diff/
Testing
-------
Current unit tests seems to be passing. I've also added two additional tests to verify new functionality.
Thanks,
Jarek Cecho