Scanners with little matching data

[Ishan Chhabra <ic...@rocketfuel.com>]

Hi,

i have a mapreduce job that reads data from hbase. To minimize data
transfer, i have implemented a filter that aggressively filters out data to
be sent back. Now, I am running into a situation where the scanner doesn't
send back anything for the rpc.timwout value, and the client times out,
retries, and repeats. My tasks fail in the initialize phase itself because
it gets stuck in this loop for 10 minutes and then gives up.

I am currently running with hbase.rpc.timeout and
hbase.regionserver.lease.period as 120s. I can increase this further, but
want to understand the cons of doing that first.

Also, is there any other way of getting around this?

-- 
*Ishan Chhabra *| Rocket Scientist | RocketFuel Inc.**

Re: Scanners with little matching data

[Ted Yu <yu...@gmail.com>]

One option is to make the filter less aggressive so that some data is returned within the current timeout. 

Cheers

On Aug 15, 2013, at 1:31 AM, Ishan Chhabra <ic...@rocketfuel.com> wrote:

> Hi,
> 
> i have a mapreduce job that reads data from hbase. To minimize data
> transfer, i have implemented a filter that aggressively filters out data to
> be sent back. Now, I am running into a situation where the scanner doesn't
> send back anything for the rpc.timwout value, and the client times out,
> retries, and repeats. My tasks fail in the initialize phase itself because
> it gets stuck in this loop for 10 minutes and then gives up.
> 
> I am currently running with hbase.rpc.timeout and
> hbase.regionserver.lease.period as 120s. I can increase this further, but
> want to understand the cons of doing that first.
> 
> Also, is there any other way of getting around this?
> 
> -- 
> *Ishan Chhabra *| Rocket Scientist | RocketFuel Inc.**

Re: Scanners with little matching data

[Nicolas Liochon <nk...@gmail.com>]

It's not uncommon to bump these values to something like 5 minutes, for the
exact reason you mention.
The obvious impact is that if the clients don't close the connections the
server will have to keep the resources. It's usually manageable.
Another one is that if the machine running the server dies silently (i.e.
the process is not killed but the machine stops responding) the client will
wait until the timeout is reached. On 0.96 this is covered by HBASE-7590:
the connection will be cut whatever the timeout. It's not available on the
0.94.

Cheers,

Nicolas


On Thu, Aug 15, 2013 at 10:31 AM, Ishan Chhabra <ic...@rocketfuel.com>wrote:

> Hi,
>
> i have a mapreduce job that reads data from hbase. To minimize data
> transfer, i have implemented a filter that aggressively filters out data to
> be sent back. Now, I am running into a situation where the scanner doesn't
> send back anything for the rpc.timwout value, and the client times out,
> retries, and repeats. My tasks fail in the initialize phase itself because
> it gets stuck in this loop for 10 minutes and then gives up.
>
> I am currently running with hbase.rpc.timeout and
> hbase.regionserver.lease.period as 120s. I can increase this further, but
> want to understand the cons of doing that first.
>
> Also, is there any other way of getting around this?
>
> --
> *Ishan Chhabra *| Rocket Scientist | RocketFuel Inc.**
>