[GitHub] [accumulo] keith-turner commented on issue #1327: No-chop merges

["keith-turner (via GitHub)" <gi...@apache.org>]

keith-turner commented on issue #1327:
URL: https://github.com/apache/accumulo/issues/1327#issuecomment-1445235972

   I like option number 2 that you mentioned.  Its hard to say for sure w/o trying it, but that one feels like it might be the best.  I looked around the code and [MultiIterator](https://github.com/apache/accumulo/blob/main/core/src/main/java/org/apache/accumulo/core/iteratorsImpl/system/MultiIterator.java) seems like it would be part of the solution.  Interestingly MultiIterator does the fencing and combines multiple iterators.  We may want to create a standalone fencing iterator for efficiency.
   
   Not sure where this would go in the code exactly, but maybe we could do something like the following for an  rfile iterator with a list of ranges.  This could be one way to implement option 2.
   
   ```java
     SortedKeyValueIterator<Key,Value> fenceIterator(SortedKeyValueIterator<Key,Value> iter, List<Ranges> ranges) {
       if(ranges.size() == 0){
         return iter;
       } else if (ranges.size() == 1) {
         return new FencedIterator(iter, ranges.get(0));
       } else {
         var fencedIters = ranges.stream().map(range -> new FencedIterator(iter.deepCopy(), range)).collect(Collectors.toList());
         return new MultiIterator(fencedIters, false);
       }
     }
   ```
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org