[GitHub] [hadoop] mukund-thakur commented on a change in pull request #3240: HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out.

[GitBox <gi...@apache.org>]

mukund-thakur commented on a change in pull request #3240:
URL: https://github.com/apache/hadoop/pull/3240#discussion_r678038991



##########
File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contract/AbstractContractDistCpTest.java
##########
@@ -659,6 +683,41 @@ private int getTotalFiles() {
     return totalFiles;
   }
 
+  /**
+   * Override point: should direct write always be used?
+   * false by default; enable for stores where rename is slow.
+   * @return true if direct write should be used in all tests.
+   */
+  protected boolean directWriteAlways() {

Review comment:
       nit: method name contains Always but default is false, a bit ambiguous no?

##########
File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contract/AbstractContractDistCpTest.java
##########
@@ -758,7 +828,7 @@ public void testDistCpWithUpdateExistFile() throws Exception {
     verifyPathExists(remoteFS, "", source);
     verifyPathExists(localFS, "", dest);
     DistCpTestUtils.assertRunDistCp(DistCpConstants.SUCCESS, source.toString(),
-        dest.toString(), "-delete -update", conf);
+        dest.toString(), "-delete -update" + getDefaultCLIOptions(), conf);

Review comment:
       nit : options can be refactored like it is done above.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org