You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by Alexander Alten-Lorenz <al...@cloudera.com> on 2013/07/18 01:45:24 UTC
Re: Review Request 10834: FLUME-2027: Check for default replication fails on
federated cluster in hdfs sink
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10834/#review23343
-----------------------------------------------------------
Ship it!
looks good
- Alexander Alten-Lorenz
On May 1, 2013, 7:36 a.m., Mike Percy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10834/
> -----------------------------------------------------------
>
> (Updated May 1, 2013, 7:36 a.m.)
>
>
> Review request for Flume.
>
>
> Bugs: FLUME-2027
> https://issues.apache.org/jira/browse/FLUME-2027
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> Federated HDFS has a new API which has been ported to Hadoop 1.1.0 and Hadoop 2.0.0-alpha from HADOOP-8014 which takes a Path argument to the FileSystem.getDefaultReplication() call. This patch simply moves the existing implementation over to that new API.
>
>
> Diffs
> -----
>
> flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/AbstractHDFSWriter.java ff4f223
> flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java 0c618b5
> flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java c87fafe
> flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java 1a401d6
>
> Diff: https://reviews.apache.org/r/10834/diff/
>
>
> Testing
> -------
>
> Unit tests pass.
>
> Tested this by running the unit tests under both Hadoop 1.0.1 and Hadoop 1.1.2 (manually modified the pom files) and looking at the DEBUG output from the unit tests. The debug output indicated that HADOOP-8014 is used if it's available, otherwise the existing API is used. This should be sufficient for all cases.
>
>
> Thanks,
>
> Mike Percy
>
>