[GitHub] [carbondata] marchpure commented on a change in pull request #3589: [CARBONDATA-3667] Insert stage recover processing of the partition ta…

[GitBox <gi...@apache.org>]

marchpure commented on a change in pull request #3589: [CARBONDATA-3667] Insert stage recover processing of the partition ta…
URL: https://github.com/apache/carbondata/pull/3589#discussion_r369993207
 
 

 ##########
 File path: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonInsertFromStageCommand.scala
 ##########
 @@ -431,8 +318,53 @@ case class CarbonInsertFromStageCommand(
     }.map { future =>
       future.get()
     }
+  }
+
+  /**
+   * Delete stage file and success file with retry
+   */
+  private def deleteStageFilesWithRetry(
+      executorService: ExecutorService,
+      _stageFiles: Array[(CarbonFile, CarbonFile)]): Unit = {
+    val startTime = System.currentTimeMillis()
+    var retryCount = DELETE_STAGEFILES_RETRY_TIMES
+    var stageFiles = _stageFiles
+    while (stageFiles.length > 0 && retryCount > 0) {
+      retryCount -= 1
+      // 1) delete stage files
+      deleteStageFiles(executorService, stageFiles)
+      // 2) ensure stage files are all deleted
+      stageFiles = ensureStageFilesAreAllDeleted(executorService, stageFiles)
 
 Review comment:
   solved

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services