[GitHub] [incubator-gobblin] amarnathkarthik commented on a change in pull request #2633: GOBBLIN-759: Added feature to support DistCP to copy files that were …

[GitBox <gi...@apache.org>]

amarnathkarthik commented on a change in pull request #2633: GOBBLIN-759: Added feature to support DistCP to copy files that were …
URL: https://github.com/apache/incubator-gobblin/pull/2633#discussion_r379849244
 
 

 ##########
 File path: gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/TimestampBasedCopyableDataset.java
 ##########
 @@ -73,6 +71,7 @@
   private final VersionSelectionPolicy<TimestampedDatasetVersion> versionSelectionPolicy;
   private final ExecutorService executor;
   private final FileSystem srcFs;
+  private final CopyableFileFilter copyableFileFilter;
 
 Review comment:
   PathFilter interfaces do not support operation to merge the filter, also before filtering through the data range filter, hidden files are removed from the list by the existing control flow.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services