You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Frens Jan Rumph <fr...@web-iq.com.INVALID> on 2025/05/15 13:15:36 UTC

Cancelling scan with AdvancedScanResultConsumer

Hi all,

When scanning HBase using the AdvancedScanResultConsumer, what’s the most prompt way to cancel/close an active scan? I only see the possibility of calling ScanController#terminate in the onNext or onHeartbeat callbacks. However, if, because of an external signal like a timeout, this approach seems to unnecessarily delay the termination as it needs to await onNext or onHeartbeat.

The documentation is very clear on not storing references to ScanController. However, it would be nice if org.apache.hadoop.hbase.client.AsyncTable#scan(scan, consumer) would return some kind of handle that allows for immediate cancellation. Am I overlooking something? Is there a better approach to this?

I reckon that this would require some wiring in AsyncClientScanner#startScan to store the current scanner id and some coordination to handle various concurrent control (scan just being opened, etc.).

I would appreciate any feedback.

Best regards,
Frens Jan

Re: Cancelling scan with AdvancedScanResultConsumer

Posted by Frens Jan Rumph <fr...@web-iq.com.INVALID>.
Dear Duo,

Apologies for not understanding this. I’m not very familiar with the HBase server internals. I thought there was a link between AsyncScanSingleRegionRpcRetryingCaller#closeScanner(…) and an actual *active* close in the region server through RSRpcServices#scan(…) to RSRpcSevices#closeScanner(…) and then InternalScanner#close(). But I understand now that such a ‘kill the current scanner next RPC’ doesn’t exist.

Thanks for your guidance on this!

BR,
Frens Jan




> On 16 May 2025, at 06:02, 张铎(Duo Zhang) <pa...@gmail.com> wrote:
> 
> As I explained above, there is no way to 'cancel' a running scan at
> the region server side, we will remove the scanner from the lease map
> while we are scanning, so sending a close scanner request to the
> region server can only get an unknown scanner exception.
> 
> You can configure 'hbase.client.scanner.timeout.period' at region
> server side to control how long a scan could run, even if no entries
> can be returned, the scan will stop and return a heartbeat message
> back to client when reaching the time limit.
> 
> And you can also configure a smaller scan timeout at client side, the
> time limit of a scan is determine by both the client scan timeout and
> 'hbase.client.scanner.timeout.period' config at region server side.
> 
> Thanks.
> 
> Frens Jan Rumph <fr...@web-iq.com.invalid> 于2025年5月16日周五 03:49写道：
>> 
>> Hi Duo,
>> 
>> Thanks for your explanation. This actually what currently is implemented; as in the pseudo code below.
>> 
>> This would be beneficial of course. E.g., if a scan would be a full-table scan with e.g. a column name filter that doesn’t return any results, this would waste a lot of effort in the HBase region server. More actively sending a closeScanner request would prevent this. Right now, we need to wait for the heartbeat interval.
>> 
>> Would such behaviour be desirable from an HBase project/community point of view?
>> 
>> Best regards,
>> Frens Jan
>> 
>> 
>> 
>> 
>> 
>> 
>> class subscription:
>> 
>>    closed = false
>> 
>>    close():
>>        close = true;
>> 
>>    onNext(results, controller):
>>        if(closed)
>>            controller.terminate();
>>        else
>>            ...
>> 
>>    onHeartbeat(controller):
>>        if(closed)
>>            controller.terminate();
>> 
>> 
>> 
>> 
>> 
>> 
>> On 15 May 2025, at 17:24, 张铎(Duo Zhang) <pa...@gmail.com> wrote:
>> 
>> For asynchronous scan, we will send scan request to region server in
>> background, and once there is a scan result returned from the region
>> server, we can issue the next scan request.
>> And in fact, we still do not have the ability to actually 'cancel' a
>> scan request which is running at region server side, what we call a
>> 'terminate', is to just stop issuing new scan request from client
>> side.
>> 
>> The terminate call in ScanController just tells the background task to
>> stop sending new request to region server, and it is only valid when
>> you have finished a scan request and get the result from region
>> server.
>> 
>> For actual implementation, since the scan request is in background,
>> you can just use a flag to record that you want to terminate the scan,
>> and then you are free to do anything you want. And in the next onNext
>> call, you just call terminate to stop sending new scan requests.
>> 
>> Thanks.
>> 
>>

Re: Cancelling scan with AdvancedScanResultConsumer

Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
As I explained above, there is no way to 'cancel' a running scan at
the region server side, we will remove the scanner from the lease map
while we are scanning, so sending a close scanner request to the
region server can only get an unknown scanner exception.

You can configure 'hbase.client.scanner.timeout.period' at region
server side to control how long a scan could run, even if no entries
can be returned, the scan will stop and return a heartbeat message
back to client when reaching the time limit.

And you can also configure a smaller scan timeout at client side, the
time limit of a scan is determine by both the client scan timeout and
'hbase.client.scanner.timeout.period' config at region server side.

Thanks.

Frens Jan Rumph <fr...@web-iq.com.invalid> 于2025年5月16日周五 03:49写道：
>
> Hi Duo,
>
> Thanks for your explanation. This actually what currently is implemented; as in the pseudo code below.
>
> This would be beneficial of course. E.g., if a scan would be a full-table scan with e.g. a column name filter that doesn’t return any results, this would waste a lot of effort in the HBase region server. More actively sending a closeScanner request would prevent this. Right now, we need to wait for the heartbeat interval.
>
> Would such behaviour be desirable from an HBase project/community point of view?
>
> Best regards,
> Frens Jan
>
>
>
>
>
>
> class subscription:
>
>     closed = false
>
>     close():
>         close = true;
>
>     onNext(results, controller):
>         if(closed)
>             controller.terminate();
>         else
>             ...
>
>     onHeartbeat(controller):
>         if(closed)
>             controller.terminate();
>
>
>
>
>
>
> On 15 May 2025, at 17:24, 张铎(Duo Zhang) <pa...@gmail.com> wrote:
>
> For asynchronous scan, we will send scan request to region server in
> background, and once there is a scan result returned from the region
> server, we can issue the next scan request.
> And in fact, we still do not have the ability to actually 'cancel' a
> scan request which is running at region server side, what we call a
> 'terminate', is to just stop issuing new scan request from client
> side.
>
> The terminate call in ScanController just tells the background task to
> stop sending new request to region server, and it is only valid when
> you have finished a scan request and get the result from region
> server.
>
> For actual implementation, since the scan request is in background,
> you can just use a flag to record that you want to terminate the scan,
> and then you are free to do anything you want. And in the next onNext
> call, you just call terminate to stop sending new scan requests.
>
> Thanks.
>
>

Re: Cancelling scan with AdvancedScanResultConsumer

Posted by Frens Jan Rumph <fr...@web-iq.com.INVALID>.
Hi Duo,

Thanks for your explanation. This actually what currently is implemented; as in the pseudo code below.

This would be beneficial of course. E.g., if a scan would be a full-table scan with e.g. a column name filter that doesn’t return any results, this would waste a lot of effort in the HBase region server. More actively sending a closeScanner request would prevent this. Right now, we need to wait for the heartbeat interval.

Would such behaviour be desirable from an HBase project/community point of view?

Best regards,
Frens Jan






class subscription:
    
    closed = false
    
    close():
        close = true;
    
    onNext(results, controller):
        if(closed)
            controller.terminate();
        else
            ...
    
    onHeartbeat(controller):
        if(closed)
            controller.terminate();






> On 15 May 2025, at 17:24, 张铎(Duo Zhang) <pa...@gmail.com> wrote:
> 
> For asynchronous scan, we will send scan request to region server in
> background, and once there is a scan result returned from the region
> server, we can issue the next scan request.
> And in fact, we still do not have the ability to actually 'cancel' a
> scan request which is running at region server side, what we call a
> 'terminate', is to just stop issuing new scan request from client
> side.
> 
> The terminate call in ScanController just tells the background task to
> stop sending new request to region server, and it is only valid when
> you have finished a scan request and get the result from region
> server.
> 
> For actual implementation, since the scan request is in background,
> you can just use a flag to record that you want to terminate the scan,
> and then you are free to do anything you want. And in the next onNext
> call, you just call terminate to stop sending new scan requests.
> 
> Thanks.

Re: Cancelling scan with AdvancedScanResultConsumer

Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
For asynchronous scan, we will send scan request to region server in
background, and once there is a scan result returned from the region
server, we can issue the next scan request.
And in fact, we still do not have the ability to actually 'cancel' a
scan request which is running at region server side, what we call a
'terminate', is to just stop issuing new scan request from client
side.

The terminate call in ScanController just tells the background task to
stop sending new request to region server, and it is only valid when
you have finished a scan request and get the result from region
server.

For actual implementation, since the scan request is in background,
you can just use a flag to record that you want to terminate the scan,
and then you are free to do anything you want. And in the next onNext
call, you just call terminate to stop sending new scan requests.

Thanks.