[GitHub] justinleet commented on a change in pull request #1338: METRON-2005: Batch Writer writes 0-byte files to HDFS on rotation

[GitBox <gi...@apache.org>]

justinleet commented on a change in pull request #1338: METRON-2005: Batch Writer writes 0-byte files to HDFS on rotation
URL: https://github.com/apache/metron/pull/1338#discussion_r257330347
 
 

 ##########
 File path: metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/SourceHandler.java
 ##########
 @@ -128,28 +133,27 @@ public void run() {
     }
   }
 
+  // Closes the output file, but ensures any RotationActions are performed.
   protected void rotateOutputFile() throws IOException {
-    LOG.info("Rotating output file...");
+    LOG.debug("Rotating output file...");
     long start = System.currentTimeMillis();
     synchronized (this.writeLock) {
       closeOutputFile();
       // Want to use the callback to make sure we have an accurate count of open files.
       cleanupCallback();
-      this.rotation++;
 
-      Path newFile = createOutputFile();
-      LOG.info("Performing {} file rotation actions.", this.rotationActions.size());
+      LOG.debug("Performing {} file rotation actions.", this.rotationActions.size());
       for (RotationAction action : this.rotationActions) {
         action.execute(this.fs, this.currentFile);
       }
-      this.currentFile = newFile;
     }
     long time = System.currentTimeMillis() - start;
     LOG.info("File rotation took {} ms", time);
   }
 
   private Path createOutputFile() throws IOException {
-    Path path = new Path(this.fileNameFormat.getPath(), this.fileNameFormat.getName(this.rotation, System.currentTimeMillis()));
+    Path path = new Path(this.fileNameFormat.getPath(), this.fileNameFormat.getName(0, System.currentTimeMillis()));
 
 Review comment:
   Given that we don't rotate anymore, I just killed the variable and hardcoded it to 0.  There's never a rotation >0 that has data. I can change this (or comment) if it'd make it more clear.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services