[GitHub] [accumulo] dlmarion commented on a change in pull request #2422: Initial implementation of my vision for implementation of Scan Servers.

[GitBox <gi...@apache.org>]

dlmarion commented on a change in pull request #2422:
URL: https://github.com/apache/accumulo/pull/2422#discussion_r791015464



##########
File path: core/src/main/java/org/apache/accumulo/core/clientImpl/TabletServerBatchReaderIterator.java
##########
@@ -497,26 +499,44 @@ private void doLookups(Map<String,Map<KeyExtent,List<Range>>> binnedRanges,
     for (final String tsLocation : locations) {
 
       final Map<KeyExtent,List<Range>> tabletsRanges = binnedRanges.get(tsLocation);
-      if (maxTabletsPerRequest == Integer.MAX_VALUE || tabletsRanges.size() == 1) {
-        QueryTask queryTask = new QueryTask(tsLocation, tabletsRanges, failures, receiver, columns);
-        queryTasks.add(queryTask);
+      if (options.isUseScanServer()) {
+        // Ignore the tablets location and find a scan server to use
+        ScanServerLocator ssl = context.getScanServerLocator();
+        tabletsRanges.forEach((k, v) -> {
+          try {
+            String location = ssl.reserveScanServer(new TabletIdImpl(k));

Review comment:
       As currently implemented a ScanServer performs one scan at a time, much like the Compactor performs one compaction at a time. I figured that if a ScanServer had many threads and performed more than one scan at a time, then we would potentially run into the same situation we have in in the TabletServer w/r/t memory usage.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org