Scanner returning subset of data

[Randy Fox <ra...@connexity.com>]

I have a needle-in-the-haystack type scan.  I have tried to read all the 
issues with ScannerTimeoutException and LeaseException, but do have not 
seen anyone report what I am seeing.

Running 0.92.1-cdh4.1.1.  All config wrt to timeouts and periods are 
default: 60s.

When I run a scanner that will return few results and my cache setting 
is a bit too high for results to return in 60 seconds, i sometimes get a 
subset of results (the last few returnable rows) and no exception.  it 
may take a while to get those results.  Other times I get the 
LeaseException, the ScannerTimeoutException, or the 
RetriesExhaustedException. I can see throwExceptionIfCallerDisconnected 
in RS logs.

The incorrect return set has me very concerned.  I can easily reproduce 
this with my own code or hbase shell.

Any help is greatly appreciated.

Cheers,

Randy Fox

RE: Scanner returning subset of data

[Anoop Sam John <an...@huawei.com>]

Randy 
As Ted suggested can you see the client logs closely (RS side also)?  Is there next() call retries happening from the client side because of RPC timeouts?
In such a case this kind of issue can happen.   I doubt he hit HBASE-5974

-Anoop-
________________________________________
From: Ted Yu [yuzhihong@gmail.com]
Sent: Tuesday, April 09, 2013 2:48 AM
To: user@hbase.apache.org
Subject: Re: Scanner returning subset of data

0.92.1 is pretty old. Are you able to deploy newer release, e.g. 0.94.6.1
and see if the problem can be reproduced ?

Otherwise we have two choices:
1. write a unit / integration test that shows this bug
2. see more of the region server / client logs so that further analysis can
be performed.

Thanks

On Mon, Apr 8, 2013 at 2:07 PM, Randy Fox <ra...@connexity.com> wrote:

> I have a needle-in-the-haystack type scan.  I have tried to read all the
> issues with ScannerTimeoutException and LeaseException, but do have not
> seen anyone report what I am seeing.
>
> Running 0.92.1-cdh4.1.1.  All config wrt to timeouts and periods are
> default: 60s.
>
> When I run a scanner that will return few results and my cache setting is
> a bit too high for results to return in 60 seconds, i sometimes get a
> subset of results (the last few returnable rows) and no exception.  it may
> take a while to get those results.  Other times I get the LeaseException,
> the ScannerTimeoutException, or the RetriesExhaustedException. I can see
> throwExceptionIfCallerDisconne**cted in RS logs.
>
> The incorrect return set has me very concerned.  I can easily reproduce
> this with my own code or hbase shell.
>
> Any help is greatly appreciated.
>
> Cheers,
>
> Randy Fox
>
>
>
>
>

Re: Scanner returning subset of data

[Ted Yu <yu...@gmail.com>]

0.92.1 is pretty old. Are you able to deploy newer release, e.g. 0.94.6.1
and see if the problem can be reproduced ?

Otherwise we have two choices:
1. write a unit / integration test that shows this bug
2. see more of the region server / client logs so that further analysis can
be performed.

Thanks

On Mon, Apr 8, 2013 at 2:07 PM, Randy Fox <ra...@connexity.com> wrote:

> I have a needle-in-the-haystack type scan.  I have tried to read all the
> issues with ScannerTimeoutException and LeaseException, but do have not
> seen anyone report what I am seeing.
>
> Running 0.92.1-cdh4.1.1.  All config wrt to timeouts and periods are
> default: 60s.
>
> When I run a scanner that will return few results and my cache setting is
> a bit too high for results to return in 60 seconds, i sometimes get a
> subset of results (the last few returnable rows) and no exception.  it may
> take a while to get those results.  Other times I get the LeaseException,
> the ScannerTimeoutException, or the RetriesExhaustedException. I can see
> throwExceptionIfCallerDisconne**cted in RS logs.
>
> The incorrect return set has me very concerned.  I can easily reproduce
> this with my own code or hbase shell.
>
> Any help is greatly appreciated.
>
> Cheers,
>
> Randy Fox
>
>
>
>
>