[jira] Commented: (HBASE-2066) Perf: parallelize puts

["stack (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12793807#action_12793807 ] 

stack commented on HBASE-2066:
------------------------------

Patch looks great.  

We can't do a version bump in 0.20 branch.  Adding a new method to the interface w/o version bumping doesn't work I suppose.  How about a version in 0.20 that doesn't pass ExcecutorService and a timeout and whose method is named processBatchOfRows rather processBatchOfPuts?

Any chance of some tests?  

Will this fix help in 0.20 branch?

{code}
@@ -845,8 +855,9 @@ public class HConnectionManager implements HConstants {
 
           // by nature of the map, we know that the start key has to be < 
           // otherwise it wouldn't be in the headMap. 
-          if (KeyValue.getRowComparator(tableName).compareRows(endKey, 0, endKey.length,
-              row, 0, row.length) <= 0) {
+          if (Bytes.equals(endKey, HConstants.EMPTY_END_ROW) ||
+              KeyValue.getRowComparator(tableName).compareRows(endKey, 0, endKey.length,
+              row, 0, row.length) > 0) {
             // delete any matching entry
             HRegionLocation rl =
               tableLocations.remove(matchingRegions.lastKey());
{code}

Do you want to change these:

{code}
+            LOG.debug("Failed all from " + request.address + " due to ExecutionException");
{code}

.. so the are instead:

{code}
+            LOG.debug("Failed all from " + request.address, e);
{code}

Is this done once, getCurrentNrHRS, in the HTable constructor?

looks really good

> Perf: parallelize puts
> ----------------------
>
>                 Key: HBASE-2066
>                 URL: https://issues.apache.org/jira/browse/HBASE-2066
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.2
>            Reporter: ryan rawson
>             Fix For: 0.20.3, 0.21.0
>
>         Attachments: HBASE-2066-branch.patch
>
>
> Right now with large region count tables, the write buffer is not efficient.  This is because we issue potentially N RPCs, where N is the # of regions in the table.  When N gets large (lets say 1200+) things become sloowwwww.
> Instead if we batch things up using a different RPC and use thread pools, we could see higher performance!
> This requires a RPC change...

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.