You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flume.apache.org by mp...@apache.org on 2016/07/10 22:38:31 UTC
flume git commit: FLUME-2950.
ReliableSpoolingFileEventReader.rollCurrentFile is broken
Repository: flume
Updated Branches:
refs/heads/trunk 209c1e254 -> ba64b1267
FLUME-2950. ReliableSpoolingFileEventReader.rollCurrentFile is broken
FLUME-2939 introduced a Java 7 API (nio.file) to
ReliableSpoolingFileEventReader. However, the committed patch breaks
rollCurrentFile() on Windows machines. File rename might be not atomic,
in such cases, and under the assumption that the rolled file and the
current file store the same data, Flume only logs a warning and does not
assume that the user broke the contract that the spooling directory
source demands.
In order to determine whether the files store the same data, we used the
com.google.common.io.Files.equal() method. However, FLUME-2939 uses
java.nio.file.Files.isSameFile() which has different semantics.
Consequently, this relaxation does not hold anymore. In addition,
TestSpoolingFileLineReader.testDestinationExistsAndSameFileWindows
fails. This patch brings back the old check using Files.equal().
(Lior Zeno via Mike Percy)
Project: http://git-wip-us.apache.org/repos/asf/flume/repo
Commit: http://git-wip-us.apache.org/repos/asf/flume/commit/ba64b126
Tree: http://git-wip-us.apache.org/repos/asf/flume/tree/ba64b126
Diff: http://git-wip-us.apache.org/repos/asf/flume/diff/ba64b126
Branch: refs/heads/trunk
Commit: ba64b12670f7b2c9ce079f8898f4601689a51def
Parents: 209c1e2
Author: Lior Zeno <li...@gmail.com>
Authored: Sun Jul 10 15:03:08 2016 -0700
Committer: Mike Percy <mp...@apache.org>
Committed: Sun Jul 10 15:17:45 2016 -0700
----------------------------------------------------------------------
.../apache/flume/client/avro/ReliableSpoolingFileEventReader.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flume/blob/ba64b126/flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java
----------------------------------------------------------------------
diff --git a/flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java b/flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java
index ca5308c..01381a5 100644
--- a/flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java
+++ b/flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java
@@ -418,7 +418,7 @@ public class ReliableSpoolingFileEventReader implements ReliableEventReader {
* file was already rolled but the rename was not atomic. If that seems
* likely, we let it pass with only a warning.
*/
- if (Files.isSameFile(currentFile.get().getFile().toPath(), dest.toPath())) {
+ if (com.google.common.io.Files.equal(currentFile.get().getFile(), dest)) {
logger.warn("Completed file " + dest +
" already exists, but files match, so continuing.");
boolean deleted = fileToRoll.delete();