[GitHub] [accumulo] keith-turner commented on a change in pull request #1367: Retry new Bulk import on merge. Fixes #471

[GitBox <gi...@apache.org>]

keith-turner commented on a change in pull request #1367: Retry new Bulk import on merge. Fixes #471
URL: https://github.com/apache/accumulo/pull/1367#discussion_r332102159
 
 

 ##########
 File path: core/src/main/java/org/apache/accumulo/core/clientImpl/bulk/BulkImport.java
 ##########
 @@ -125,21 +126,47 @@ public void load()
     Path srcPath = checkPath(fs, dir);
 
     SortedMap<KeyExtent,Bulk.Files> mappings;
-    if (plan == null) {
-      mappings = computeMappingFromFiles(fs, tableId, srcPath);
-    } else {
-      mappings = computeMappingFromPlan(fs, tableId, srcPath);
-    }
+    TableOperationsImpl tableOps = new TableOperationsImpl(context);
+    // retry if a merge occurs
+    boolean retry = true;
+    while (retry) {
+      if (plan == null) {
+        mappings = computeMappingFromFiles(fs, tableId, srcPath);
+      } else {
+        mappings = computeMappingFromPlan(fs, tableId, srcPath);
+      }
 
-    if (mappings.isEmpty())
-      throw new IllegalArgumentException("Attempted to import zero files from " + srcPath);
+      if (mappings.isEmpty())
+        throw new IllegalArgumentException("Attempted to import zero files from " + srcPath);
 
-    BulkSerialize.writeLoadMapping(mappings, srcPath.toString(), fs::create);
+      BulkSerialize.writeLoadMapping(mappings, srcPath.toString(), fs::create);
+
+      List<ByteBuffer> args = Arrays.asList(ByteBuffer.wrap(tableId.canonical().getBytes(UTF_8)),
+          ByteBuffer.wrap(srcPath.toString().getBytes(UTF_8)),
+          ByteBuffer.wrap((setTime + "").getBytes(UTF_8)));
+      try {
+        tableOps.doBulkFateOperation(args, tableName);
+        retry = false;
+      } catch (AccumuloBulkMergeException ae) {
+        if (plan != null) {
+          checkPlanForSplits();
+        }
 
 Review comment:
   This will retry immediately.  Could use an Accumulo Retry object to sleep between retries.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services