You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by GitBox <gi...@apache.org> on 2020/08/18 00:21:08 UTC

[GitHub] [incubator-gobblin] ZihanLi58 commented on a change in pull request #3079: [GOBBLIN-1231]Make re-compaction be able to write to a new folder based on the executCount

ZihanLi58 commented on a change in pull request #3079:
URL: https://github.com/apache/incubator-gobblin/pull/3079#discussion_r471843780



##########
File path: gobblin-api/src/main/java/org/apache/gobblin/configuration/ConfigurationKeys.java
##########
@@ -993,6 +993,13 @@
   public static final String COMPACTION_PRIORITIZER_ALIAS = COMPACTION_PRIORITIZATION_PREFIX + "prioritizerAlias";
   public static final String COMPACTION_ESTIMATOR = COMPACTION_PRIORITIZATION_PREFIX + "estimator";
 
+  /***
+   * Configuration properties related to Re-compaction
+   */
+  public static String COMPACTION_DIRECTORY_FORMAT = "compaction_%s";

Review comment:
       Actually both recompaction and compaction will write to this directory if recompaction.write.to.new.folder is enabled. The daily foler will looks like /daily/2020/09/12/compaction_1 (output of first run of compaction) /daily/2020/09/12/compaction_2(output of re-compaction), so i think the COMPACTION_DIRECTORY_FORMAT will be accurate. What's your thoughts?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org