You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2019/12/02 18:54:51 UTC

[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #4874: Pinot ingestion job - Standalone

mayankshriv commented on a change in pull request #4874: Pinot ingestion job - Standalone
URL: https://github.com/apache/incubator-pinot/pull/4874#discussion_r352755163
 
 

 ##########
 File path: pinot-ingestion-jobs/pinot-ingestion-common/src/main/java/org/apache/pinot/ingestion/common/DefaultControllerRestApi.java
 ##########
 @@ -121,7 +122,47 @@ public void pushSegments(FileSystem fileSystem, List<Path> tarFilePaths) {
             LOGGER.error("Caught exception while pushing segment: {} to location: {}, retry {}/{}", segmentName,
                 pushLocation, retry, _retry, e);
             if (retry == _retry) {
-              throw new RuntimeException(String.format("Failed to push segment %s to %s with %d retries",segmentName, pushLocation, retry), e);
+              throw new RuntimeException(
+                  String.format("Failed to push segment %s to %s with %d retries", segmentName, pushLocation, retry),
+                  e);
+            }
+            try {
+              // Exponential back-off, max sleep time is 64 seconds.
+              Thread.sleep(1000 * (int) Math.pow(2, Math.min(retry + 1, 6)));
+            } catch (InterruptedException ex) {
+              // Swallow
+            }
+          }
+        }
+      }
+    }
+  }
+
+  @Override
+  public void pushSegments(PinotFS fileSystem, List<String> tarFilePaths) {
 
 Review comment:
   Seems like there's a lot of duplicate code between this version and the previous version of this method. Would be good to refactor so that we avoid duplicates.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org