[jira] [Comment Edited] (PHOENIX-1751) Perform aggregations, sorting, etc, in the preScannerOpen instead of postScannerOpen

["Samarth Jain (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15432189#comment-15432189 ] 

Samarth Jain edited comment on PHOENIX-1751 at 8/23/16 5:53 AM:
----------------------------------------------------------------

[~jamestaylor] - I am looking at code in HBase and it looks like preScannerOpen is still executed under lease. 

{code}
if (region.getCoprocessorHost() != null) {
          scanner = region.getCoprocessorHost().preScannerOpen(scan);
        }
        if (scanner == null) {
          scanner = region.getScanner(scan);
        }
        if (region.getCoprocessorHost() != null) {
          scanner = region.getCoprocessorHost().postScannerOpen(scan, scanner);
        }
{code}

but preScannerNext() isn't

{code}
          // Remove lease while its being processed in server; protects against case
          // where processing of request takes > lease expiration time.
          lease = regionServer.leases.removeLease(scannerName);
          List<Result> results = new ArrayList<Result>();

          boolean done = false;
          // Call coprocessor. Get region info from scanner.
          if (region != null && region.getCoprocessorHost() != null) {
            Boolean bypass = region.getCoprocessorHost().preScannerNext(
              scanner, results, rows);
{code}



was (Author: samarthjain):
[~jamestaylor] - I am looking at code in HBase and it looks like preScannerOpen is still executed under lease. 

{code}
if (region.getCoprocessorHost() != null) {
          scanner = region.getCoprocessorHost().preScannerOpen(scan);
        }
        if (scanner == null) {
          scanner = region.getScanner(scan);
        }
        if (region.getCoprocessorHost() != null) {
          scanner = region.getCoprocessorHost().postScannerOpen(scan, scanner);
        }
{code}

but preScannerNext() isn't

{code}
lease = regionServer.leases.removeLease(scannerName);
          List<Result> results = new ArrayList<Result>();

          boolean done = false;
          // Call coprocessor. Get region info from scanner.
          if (region != null && region.getCoprocessorHost() != null) {
            Boolean bypass = region.getCoprocessorHost().preScannerNext(
              scanner, results, rows);
{code}


> Perform aggregations, sorting, etc, in the preScannerOpen instead of postScannerOpen
> ------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-1751
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1751
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>            Assignee: James Taylor
>         Attachments: 1751-WIP-v2.txt, 1751-WIP-v2b.patch, 1751-WIP.txt, PHOENIX-1751-v2c.patch, PHOENIX-1751.patch, PHOENIX-1751_v3.patch
>
>
> HBase retains a lease for every scanner. Then lease expires the scan will no longer (be allowed to) work. The leases guard against the client going away, and allow cleaning up resources if that happens.
> At various points HBase "suspends" the lease while the region server are working on behalf of this scanner, so that the lease won't expire even though the server is working on it.
> HBase does that during the scanning process. Crucially it suspends the leaser after the scanner is opened, before next() is issued on it.
> The outcome of all this is that Phoenix executes aggregates, sorts, etc, with the lease in place, and hence if these take a bit the lease can expire even though the server was working on it.
> Phoenix should do this work in preScannerNext, being careful that the precalculation is only performed once.
> I'll attach a sample patch soon.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)