You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Rajeshkumar J <ra...@gmail.com> on 2016/12/27 11:21:29 UTC
How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Hi,
I have following property value as below
<property>
<name>hbase.client.scanner.timeout.period</name>
<value>60000</value>
</property>
<property>
<name>hbase.rpc.timeout</name>
<value>60000</value>
</property>
When scanning 30 tables in hbase(each table holds 45 million records) I got
WARN client.ScannerCallable [main]: Ignore, probably already closed
org.apache.hadoop.hbase.regionserver.LeaseException:
org.apache.hadoop.hbase.regionserver.LeaseException: lease does not exist
But when I change the value to
<property>
<name>hbase.client.scanner.timeout.period</name>
<value>70000</value>
</property>
<property>
<name>hbase.rpc.timeout</name>
<value>70000</value>
</property>
It runs successfully.
Can any one tell me the reason for this failure and also is there any
factor to calculate these property values for any data.
Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Posted by Yu Li <ca...@gmail.com>.
bq. number_of_rows: 2147483647
From the above line in your posted log, somehow your
hbase.client.scanner.caching
setting didn't take effect. Please make sure you modified the correct
hbase-site.xml on client side.
You could also set the caching in the application code, simply by invoking
Scan#setCaching
Best Regards,
Yu
On 11 January 2017 at 20:45, Rajeshkumar J <ra...@gmail.com>
wrote:
> this is the log I am getting while running
>
> 2017-01-05 11:41:49,629 DEBUG
> [B.defaultRpcServer.handler=15,queue=0,port=16020] ipc.RpcServer:
> B.defaultRpcServer.handler=15,queue=0,port=16020: callId: 3 service:
> ClientService methodName: Scan size: 23 connection: hostname:port
> org.apache.hadoop.hbase.regionserver.LeaseException: lease '706' does not
> exist
> at org.apache.hadoop.hbase.regionserver.Leases.
> removeLease(Leases.java:221)
> at org.apache.hadoop.hbase.regionserver.Leases.
> cancelLease(Leases.java:206)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.
> scan(RSRpcServices.java:2491)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.
> callBlockingMethod(ClientProtos.java:32205)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:744)
> 2017-01-05 11:41:49,629 TRACE
> [B.defaultRpcServer.handler=18,queue=0,port=16020] ipc.RpcServer: callId:
> 2
> service: ClientService methodName: Scan size: 29 connection: hostname:port
> param: scanner_id: 706 number_of_rows: 2147483647 close_scanner: false
> next_call_seq: 0 client_handles_partials: true client_handles_heartbeats:
> true connection: hostname:port, response scanner_id: 706 more_results: true
> stale: false more_results_in_region: false queueTime: 1 processingTime:
> 60136 totalTime: 60137
>
> On Wed, Jan 11, 2017 at 4:45 PM, Rajeshkumar J <
> rajeshkumarit8292@gmail.com>
> wrote:
>
> > I have tried setting hbase.client.scanner.caching to 10, 100, 1000 and
> > 10000. But it still fails.
> >
> > On Wed, Jan 11, 2017 at 2:17 PM, Yu Li <ca...@gmail.com> wrote:
> >
> >> Hi Rajeshkumar,
> >>
> >> Simply speaking, in your case I'd suggest to set
> >> hbase.client.scanner.caching
> >> to a small value like 10 but not zero, and leave the
> >> hbase.client.scanner.caching or hbase.rpc.timeout setting as default.
> >> Below
> >> are some detailed explanation:
> >>
> >> Setting hbase.client.scanner.caching to zero will cause the scan returns
> >> no
> >> data (please check whether you are getting a wrong result with this
> >> setting), so please don't make such settings unless you intend to renew
> >> the
> >> scan lease. See RsRpcServices#scan source code for more details if
> >> interested.
> >>
> >> For version 1.1.1 (actually after HBASE-16973), the default value of
> >> hbase.client.scanner.caching
> >> is Integer.MAX_VALUE, and it won't return (for a single scanner.next
> call)
> >> until the result size reached hbase.client.scanner.max.result.size or
> >> times
> >> out. If you are using a sparse filter, then it's strongly suggested to
> >> set hbase.client.scanner.caching
> >> to some small value but not zero. See HBASE-16973 for more details if
> >> interested.
> >>
> >> Regarding hbase.rpc.timeout and hbase.client.scanner.timeout.period:
> >> * For scan it's hbase.client.scanner.timeout.period taking effect
> >> * For all other kinds of non-admin operations like
> >> get/put/append/increment/multi hbase.rpc.timeout takes effect, unless
> you
> >> specified hbase.client.operation.timeout.
> >> Check ClientScanner#scannerTimeout, ScannerCallable#call,
> >> AbstractRpcClient#channelOperationTimeout and
> >> AbstractRpcClient#callBlockingMethod for more details if interested.
> >>
> >> Best Regards,
> >> Yu
> >>
> >> On 11 January 2017 at 15:29, Rajeshkumar J <rajeshkumarit8292@gmail.com
> >
> >> wrote:
> >>
> >> > Hi, As i got lease expire exception i have increased the values of
> >> these
> >> > properties hbase.client.scanner.timeout.period and hbase.rpc.timeout.
> >> Is
> >> > there any disadvantage increasing these two values?
> >> >
> >> > On Mon, Jan 2, 2017 at 11:01 AM, Rajeshkumar J <
> >> > rajeshkumarit8292@gmail.com>
> >> > wrote:
> >> >
> >> > > I am using hbase 1.1.1.
> >> > >
> >> > > I have following property value in hbase-site.xml and I didn't set
> >> > value
> >> > > hbase.client.scanner.caching property(so it must pick default value
> >> for
> >> > > this)
> >> > >
> >> > > <property>
> >> > > <name>hbase.client.scanner.timeout.period</name>
> >> > > <value>60000</value>
> >> > > </property>
> >> > >
> >> > > <property>
> >> > > <name>hbase.rpc.timeout</name>
> >> > > <value>60000</value>
> >> > > </property>
> >> > > When scanning 30 tables in hbase(each table holds 45 million
> records)
> >> I
> >> > > got the following error
> >> > >
> >> > > WARN client.ScannerCallable [main]: Ignore, probably already closed
> >> > > org.apache.hadoop.hbase.regionserver.LeaseException:
> >> > > org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
> >> > exist
> >> > >
> >> > > But when I added hbase.client.scanner.caching property value as
> zero I
> >> > > didn't get any error. Why this happened?
> >> > > Even though I gave zero it supposed to pick the default value? How
> can
> >> > > this process succeeds?
> >> > >
> >> > > Thanks
> >> > >
> >> > > On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <el...@apache.org>
> >> wrote:
> >> > >
> >> > >> I am having an extremely difficult time parsing this one,
> >> Rajeshkumar.
> >> > >>
> >> > >> If you still have a question/something you don't understand, I'd
> >> > >> appreciate it if you could try to re-state it a little more
> clearly.
> >> > >>
> >> > >>
> >> > >> Rajeshkumar J wrote:
> >> > >>
> >> > >>> I am using hbase 1.1.1. Yes while setting zero it must pick the
> >> default
> >> > >>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have
> set
> >> the
> >> > >>> value as INTEGER.MAX_VALUE only but earlier it had failed but now
> >> > >>> succeeded. So only I am asking this. Also I didn't change any
> >> property
> >> > >>> whatever I used in process which failed in the process which
> >> succeeded
> >> > >>> except assigning Zero value for this property
> >> > >>>
> >> > >>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<el...@apache.org>
> >> wrote:
> >> > >>>
> >> > >>> Most likely, since you gave a nonsensical value, HBase used a
> >> default
> >> > >>>> value instead of the one you provided. Since you have not shared
> >> the
> >> > >>>> version of HBase which you are using, I would recommend that you
> >> look
> >> > at
> >> > >>>> the code. It should be very obvious what value is being used
> >> instead
> >> > of
> >> > >>>> the
> >> > >>>> bogus value you provided.
> >> > >>>>
> >> > >>>>
> >> > >>>> Rajeshkumar J wrote:
> >> > >>>>
> >> > >>>> I have tried setting hbase.client.scanner.caching property to 0
> >> with
> >> > >>>>> above
> >> > >>>>> two values as 60000. It runs fine. But
> >> hbase.client.scanner.caching
> >> > >>>>> property tells no of rows to be fetched for every scanner but
> >> here I
> >> > >>>>> have
> >> > >>>>> given zero. How this worked?
> >> > >>>>>
> >> > >>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<el...@apache.org>
> >> > >>>>> wrote:
> >> > >>>>>
> >> > >>>>> hbase.client.scanner.timeout.period is a timeout specifically
> for
> >> > RPCs
> >> > >>>>>
> >> > >>>>>> that come from the HBase Scanner classes (e.g. ClientScanner)
> >> while
> >> > >>>>>> hbase.rpc.timeout is the default timeout for any RPC. I believe
> >> that
> >> > >>>>>> the
> >> > >>>>>> hbase.client.scanner.timeout.period is also used by the
> >> > >>>>>> RegionServers to
> >> > >>>>>> define the lifetime of the Lease (the cause of the
> LeaseException
> >> > >>>>>> you're
> >> > >>>>>> seeing).
> >> > >>>>>>
> >> > >>>>>> Generally, when you see these kinds of exceptions while
> scanning
> >> > data
> >> > >>>>>> in
> >> > >>>>>> HBase, it is just a factor of your hardware and current
> >> performance
> >> > >>>>>> (in
> >> > >>>>>> other words, how long it takes to read your data). I can't
> really
> >> > >>>>>> give a
> >> > >>>>>> firm answer because it is dependent on your system's
> performance.
> >> > You
> >> > >>>>>> should be able to approximate the performance with some
> >> > >>>>>> back-of-the-envelope math.
> >> > >>>>>>
> >> > >>>>>> Some changes like https://issues.apache.org/jira
> >> /browse/HBASE-13090
> >> > >>>>>> and
> >> > >>>>>> https://issues.apache.org/jira/browse/HBASE-13333 should help
> >> > reduce
> >> > >>>>>> the
> >> > >>>>>> need for you to tweak configuration properties in the future.
> >> > >>>>>>
> >> > >>>>>>
> >> > >>>>>> Rajeshkumar J wrote:
> >> > >>>>>>
> >> > >>>>>> Hi,
> >> > >>>>>>
> >> > >>>>>>> I have following property value as below
> >> > >>>>>>>
> >> > >>>>>>> <property>
> >> > >>>>>>> <name>hbase.client.scanner.timeout.period</name>
> >> > >>>>>>> <value>60000</value>
> >> > >>>>>>> </property>
> >> > >>>>>>>
> >> > >>>>>>> <property>
> >> > >>>>>>> <name>hbase.rpc.timeout</name>
> >> > >>>>>>> <value>60000</value>
> >> > >>>>>>> </property>
> >> > >>>>>>> When scanning 30 tables in hbase(each table holds 45 million
> >> > >>>>>>> records) I
> >> > >>>>>>> got
> >> > >>>>>>>
> >> > >>>>>>> WARN client.ScannerCallable [main]: Ignore, probably already
> >> closed
> >> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
> >> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease
> does
> >> > not
> >> > >>>>>>> exist
> >> > >>>>>>>
> >> > >>>>>>> But when I change the value to
> >> > >>>>>>>
> >> > >>>>>>> <property>
> >> > >>>>>>> <name>hbase.client.scanner.timeout.period</name>
> >> > >>>>>>> <value>70000</value>
> >> > >>>>>>> </property>
> >> > >>>>>>>
> >> > >>>>>>> <property>
> >> > >>>>>>> <name>hbase.rpc.timeout</name>
> >> > >>>>>>> <value>70000</value>
> >> > >>>>>>> </property>
> >> > >>>>>>> It runs successfully.
> >> > >>>>>>> Can any one tell me the reason for this failure and also is
> >> there
> >> > any
> >> > >>>>>>> factor to calculate these property values for any data.
> >> > >>>>>>>
> >> > >>>>>>>
> >> > >>>>>>>
> >> > >>>>>>>
> >> > >>>
> >> > >
> >> >
> >>
> >
> >
>
Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Posted by Rajeshkumar J <ra...@gmail.com>.
this is the log I am getting while running
2017-01-05 11:41:49,629 DEBUG
[B.defaultRpcServer.handler=15,queue=0,port=16020] ipc.RpcServer:
B.defaultRpcServer.handler=15,queue=0,port=16020: callId: 3 service:
ClientService methodName: Scan size: 23 connection: hostname:port
org.apache.hadoop.hbase.regionserver.LeaseException: lease '706' does not
exist
at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:221)
at org.apache.hadoop.hbase.regionserver.Leases.cancelLease(Leases.java:206)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2491)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32205)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2114)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:744)
2017-01-05 11:41:49,629 TRACE
[B.defaultRpcServer.handler=18,queue=0,port=16020] ipc.RpcServer: callId: 2
service: ClientService methodName: Scan size: 29 connection: hostname:port
param: scanner_id: 706 number_of_rows: 2147483647 close_scanner: false
next_call_seq: 0 client_handles_partials: true client_handles_heartbeats:
true connection: hostname:port, response scanner_id: 706 more_results: true
stale: false more_results_in_region: false queueTime: 1 processingTime:
60136 totalTime: 60137
On Wed, Jan 11, 2017 at 4:45 PM, Rajeshkumar J <ra...@gmail.com>
wrote:
> I have tried setting hbase.client.scanner.caching to 10, 100, 1000 and
> 10000. But it still fails.
>
> On Wed, Jan 11, 2017 at 2:17 PM, Yu Li <ca...@gmail.com> wrote:
>
>> Hi Rajeshkumar,
>>
>> Simply speaking, in your case I'd suggest to set
>> hbase.client.scanner.caching
>> to a small value like 10 but not zero, and leave the
>> hbase.client.scanner.caching or hbase.rpc.timeout setting as default.
>> Below
>> are some detailed explanation:
>>
>> Setting hbase.client.scanner.caching to zero will cause the scan returns
>> no
>> data (please check whether you are getting a wrong result with this
>> setting), so please don't make such settings unless you intend to renew
>> the
>> scan lease. See RsRpcServices#scan source code for more details if
>> interested.
>>
>> For version 1.1.1 (actually after HBASE-16973), the default value of
>> hbase.client.scanner.caching
>> is Integer.MAX_VALUE, and it won't return (for a single scanner.next call)
>> until the result size reached hbase.client.scanner.max.result.size or
>> times
>> out. If you are using a sparse filter, then it's strongly suggested to
>> set hbase.client.scanner.caching
>> to some small value but not zero. See HBASE-16973 for more details if
>> interested.
>>
>> Regarding hbase.rpc.timeout and hbase.client.scanner.timeout.period:
>> * For scan it's hbase.client.scanner.timeout.period taking effect
>> * For all other kinds of non-admin operations like
>> get/put/append/increment/multi hbase.rpc.timeout takes effect, unless you
>> specified hbase.client.operation.timeout.
>> Check ClientScanner#scannerTimeout, ScannerCallable#call,
>> AbstractRpcClient#channelOperationTimeout and
>> AbstractRpcClient#callBlockingMethod for more details if interested.
>>
>> Best Regards,
>> Yu
>>
>> On 11 January 2017 at 15:29, Rajeshkumar J <ra...@gmail.com>
>> wrote:
>>
>> > Hi, As i got lease expire exception i have increased the values of
>> these
>> > properties hbase.client.scanner.timeout.period and hbase.rpc.timeout.
>> Is
>> > there any disadvantage increasing these two values?
>> >
>> > On Mon, Jan 2, 2017 at 11:01 AM, Rajeshkumar J <
>> > rajeshkumarit8292@gmail.com>
>> > wrote:
>> >
>> > > I am using hbase 1.1.1.
>> > >
>> > > I have following property value in hbase-site.xml and I didn't set
>> > value
>> > > hbase.client.scanner.caching property(so it must pick default value
>> for
>> > > this)
>> > >
>> > > <property>
>> > > <name>hbase.client.scanner.timeout.period</name>
>> > > <value>60000</value>
>> > > </property>
>> > >
>> > > <property>
>> > > <name>hbase.rpc.timeout</name>
>> > > <value>60000</value>
>> > > </property>
>> > > When scanning 30 tables in hbase(each table holds 45 million records)
>> I
>> > > got the following error
>> > >
>> > > WARN client.ScannerCallable [main]: Ignore, probably already closed
>> > > org.apache.hadoop.hbase.regionserver.LeaseException:
>> > > org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
>> > exist
>> > >
>> > > But when I added hbase.client.scanner.caching property value as zero I
>> > > didn't get any error. Why this happened?
>> > > Even though I gave zero it supposed to pick the default value? How can
>> > > this process succeeds?
>> > >
>> > > Thanks
>> > >
>> > > On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <el...@apache.org>
>> wrote:
>> > >
>> > >> I am having an extremely difficult time parsing this one,
>> Rajeshkumar.
>> > >>
>> > >> If you still have a question/something you don't understand, I'd
>> > >> appreciate it if you could try to re-state it a little more clearly.
>> > >>
>> > >>
>> > >> Rajeshkumar J wrote:
>> > >>
>> > >>> I am using hbase 1.1.1. Yes while setting zero it must pick the
>> default
>> > >>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have set
>> the
>> > >>> value as INTEGER.MAX_VALUE only but earlier it had failed but now
>> > >>> succeeded. So only I am asking this. Also I didn't change any
>> property
>> > >>> whatever I used in process which failed in the process which
>> succeeded
>> > >>> except assigning Zero value for this property
>> > >>>
>> > >>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<el...@apache.org>
>> wrote:
>> > >>>
>> > >>> Most likely, since you gave a nonsensical value, HBase used a
>> default
>> > >>>> value instead of the one you provided. Since you have not shared
>> the
>> > >>>> version of HBase which you are using, I would recommend that you
>> look
>> > at
>> > >>>> the code. It should be very obvious what value is being used
>> instead
>> > of
>> > >>>> the
>> > >>>> bogus value you provided.
>> > >>>>
>> > >>>>
>> > >>>> Rajeshkumar J wrote:
>> > >>>>
>> > >>>> I have tried setting hbase.client.scanner.caching property to 0
>> with
>> > >>>>> above
>> > >>>>> two values as 60000. It runs fine. But
>> hbase.client.scanner.caching
>> > >>>>> property tells no of rows to be fetched for every scanner but
>> here I
>> > >>>>> have
>> > >>>>> given zero. How this worked?
>> > >>>>>
>> > >>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<el...@apache.org>
>> > >>>>> wrote:
>> > >>>>>
>> > >>>>> hbase.client.scanner.timeout.period is a timeout specifically for
>> > RPCs
>> > >>>>>
>> > >>>>>> that come from the HBase Scanner classes (e.g. ClientScanner)
>> while
>> > >>>>>> hbase.rpc.timeout is the default timeout for any RPC. I believe
>> that
>> > >>>>>> the
>> > >>>>>> hbase.client.scanner.timeout.period is also used by the
>> > >>>>>> RegionServers to
>> > >>>>>> define the lifetime of the Lease (the cause of the LeaseException
>> > >>>>>> you're
>> > >>>>>> seeing).
>> > >>>>>>
>> > >>>>>> Generally, when you see these kinds of exceptions while scanning
>> > data
>> > >>>>>> in
>> > >>>>>> HBase, it is just a factor of your hardware and current
>> performance
>> > >>>>>> (in
>> > >>>>>> other words, how long it takes to read your data). I can't really
>> > >>>>>> give a
>> > >>>>>> firm answer because it is dependent on your system's performance.
>> > You
>> > >>>>>> should be able to approximate the performance with some
>> > >>>>>> back-of-the-envelope math.
>> > >>>>>>
>> > >>>>>> Some changes like https://issues.apache.org/jira
>> /browse/HBASE-13090
>> > >>>>>> and
>> > >>>>>> https://issues.apache.org/jira/browse/HBASE-13333 should help
>> > reduce
>> > >>>>>> the
>> > >>>>>> need for you to tweak configuration properties in the future.
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> Rajeshkumar J wrote:
>> > >>>>>>
>> > >>>>>> Hi,
>> > >>>>>>
>> > >>>>>>> I have following property value as below
>> > >>>>>>>
>> > >>>>>>> <property>
>> > >>>>>>> <name>hbase.client.scanner.timeout.period</name>
>> > >>>>>>> <value>60000</value>
>> > >>>>>>> </property>
>> > >>>>>>>
>> > >>>>>>> <property>
>> > >>>>>>> <name>hbase.rpc.timeout</name>
>> > >>>>>>> <value>60000</value>
>> > >>>>>>> </property>
>> > >>>>>>> When scanning 30 tables in hbase(each table holds 45 million
>> > >>>>>>> records) I
>> > >>>>>>> got
>> > >>>>>>>
>> > >>>>>>> WARN client.ScannerCallable [main]: Ignore, probably already
>> closed
>> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
>> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease does
>> > not
>> > >>>>>>> exist
>> > >>>>>>>
>> > >>>>>>> But when I change the value to
>> > >>>>>>>
>> > >>>>>>> <property>
>> > >>>>>>> <name>hbase.client.scanner.timeout.period</name>
>> > >>>>>>> <value>70000</value>
>> > >>>>>>> </property>
>> > >>>>>>>
>> > >>>>>>> <property>
>> > >>>>>>> <name>hbase.rpc.timeout</name>
>> > >>>>>>> <value>70000</value>
>> > >>>>>>> </property>
>> > >>>>>>> It runs successfully.
>> > >>>>>>> Can any one tell me the reason for this failure and also is
>> there
>> > any
>> > >>>>>>> factor to calculate these property values for any data.
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>
>> > >
>> >
>>
>
>
Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Posted by Rajeshkumar J <ra...@gmail.com>.
I have tried setting hbase.client.scanner.caching to 10, 100, 1000 and
10000. But it still fails.
On Wed, Jan 11, 2017 at 2:17 PM, Yu Li <ca...@gmail.com> wrote:
> Hi Rajeshkumar,
>
> Simply speaking, in your case I'd suggest to set
> hbase.client.scanner.caching
> to a small value like 10 but not zero, and leave the
> hbase.client.scanner.caching or hbase.rpc.timeout setting as default. Below
> are some detailed explanation:
>
> Setting hbase.client.scanner.caching to zero will cause the scan returns no
> data (please check whether you are getting a wrong result with this
> setting), so please don't make such settings unless you intend to renew the
> scan lease. See RsRpcServices#scan source code for more details if
> interested.
>
> For version 1.1.1 (actually after HBASE-16973), the default value of
> hbase.client.scanner.caching
> is Integer.MAX_VALUE, and it won't return (for a single scanner.next call)
> until the result size reached hbase.client.scanner.max.result.size or
> times
> out. If you are using a sparse filter, then it's strongly suggested to
> set hbase.client.scanner.caching
> to some small value but not zero. See HBASE-16973 for more details if
> interested.
>
> Regarding hbase.rpc.timeout and hbase.client.scanner.timeout.period:
> * For scan it's hbase.client.scanner.timeout.period taking effect
> * For all other kinds of non-admin operations like
> get/put/append/increment/multi hbase.rpc.timeout takes effect, unless you
> specified hbase.client.operation.timeout.
> Check ClientScanner#scannerTimeout, ScannerCallable#call,
> AbstractRpcClient#channelOperationTimeout and
> AbstractRpcClient#callBlockingMethod for more details if interested.
>
> Best Regards,
> Yu
>
> On 11 January 2017 at 15:29, Rajeshkumar J <ra...@gmail.com>
> wrote:
>
> > Hi, As i got lease expire exception i have increased the values of these
> > properties hbase.client.scanner.timeout.period and hbase.rpc.timeout. Is
> > there any disadvantage increasing these two values?
> >
> > On Mon, Jan 2, 2017 at 11:01 AM, Rajeshkumar J <
> > rajeshkumarit8292@gmail.com>
> > wrote:
> >
> > > I am using hbase 1.1.1.
> > >
> > > I have following property value in hbase-site.xml and I didn't set
> > value
> > > hbase.client.scanner.caching property(so it must pick default value for
> > > this)
> > >
> > > <property>
> > > <name>hbase.client.scanner.timeout.period</name>
> > > <value>60000</value>
> > > </property>
> > >
> > > <property>
> > > <name>hbase.rpc.timeout</name>
> > > <value>60000</value>
> > > </property>
> > > When scanning 30 tables in hbase(each table holds 45 million records) I
> > > got the following error
> > >
> > > WARN client.ScannerCallable [main]: Ignore, probably already closed
> > > org.apache.hadoop.hbase.regionserver.LeaseException:
> > > org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
> > exist
> > >
> > > But when I added hbase.client.scanner.caching property value as zero I
> > > didn't get any error. Why this happened?
> > > Even though I gave zero it supposed to pick the default value? How can
> > > this process succeeds?
> > >
> > > Thanks
> > >
> > > On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <el...@apache.org> wrote:
> > >
> > >> I am having an extremely difficult time parsing this one, Rajeshkumar.
> > >>
> > >> If you still have a question/something you don't understand, I'd
> > >> appreciate it if you could try to re-state it a little more clearly.
> > >>
> > >>
> > >> Rajeshkumar J wrote:
> > >>
> > >>> I am using hbase 1.1.1. Yes while setting zero it must pick the
> default
> > >>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have set
> the
> > >>> value as INTEGER.MAX_VALUE only but earlier it had failed but now
> > >>> succeeded. So only I am asking this. Also I didn't change any
> property
> > >>> whatever I used in process which failed in the process which
> succeeded
> > >>> except assigning Zero value for this property
> > >>>
> > >>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<el...@apache.org>
> wrote:
> > >>>
> > >>> Most likely, since you gave a nonsensical value, HBase used a default
> > >>>> value instead of the one you provided. Since you have not shared the
> > >>>> version of HBase which you are using, I would recommend that you
> look
> > at
> > >>>> the code. It should be very obvious what value is being used instead
> > of
> > >>>> the
> > >>>> bogus value you provided.
> > >>>>
> > >>>>
> > >>>> Rajeshkumar J wrote:
> > >>>>
> > >>>> I have tried setting hbase.client.scanner.caching property to 0 with
> > >>>>> above
> > >>>>> two values as 60000. It runs fine. But hbase.client.scanner.caching
> > >>>>> property tells no of rows to be fetched for every scanner but here
> I
> > >>>>> have
> > >>>>> given zero. How this worked?
> > >>>>>
> > >>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<el...@apache.org>
> > >>>>> wrote:
> > >>>>>
> > >>>>> hbase.client.scanner.timeout.period is a timeout specifically for
> > RPCs
> > >>>>>
> > >>>>>> that come from the HBase Scanner classes (e.g. ClientScanner)
> while
> > >>>>>> hbase.rpc.timeout is the default timeout for any RPC. I believe
> that
> > >>>>>> the
> > >>>>>> hbase.client.scanner.timeout.period is also used by the
> > >>>>>> RegionServers to
> > >>>>>> define the lifetime of the Lease (the cause of the LeaseException
> > >>>>>> you're
> > >>>>>> seeing).
> > >>>>>>
> > >>>>>> Generally, when you see these kinds of exceptions while scanning
> > data
> > >>>>>> in
> > >>>>>> HBase, it is just a factor of your hardware and current
> performance
> > >>>>>> (in
> > >>>>>> other words, how long it takes to read your data). I can't really
> > >>>>>> give a
> > >>>>>> firm answer because it is dependent on your system's performance.
> > You
> > >>>>>> should be able to approximate the performance with some
> > >>>>>> back-of-the-envelope math.
> > >>>>>>
> > >>>>>> Some changes like https://issues.apache.org/
> jira/browse/HBASE-13090
> > >>>>>> and
> > >>>>>> https://issues.apache.org/jira/browse/HBASE-13333 should help
> > reduce
> > >>>>>> the
> > >>>>>> need for you to tweak configuration properties in the future.
> > >>>>>>
> > >>>>>>
> > >>>>>> Rajeshkumar J wrote:
> > >>>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>>> I have following property value as below
> > >>>>>>>
> > >>>>>>> <property>
> > >>>>>>> <name>hbase.client.scanner.timeout.period</name>
> > >>>>>>> <value>60000</value>
> > >>>>>>> </property>
> > >>>>>>>
> > >>>>>>> <property>
> > >>>>>>> <name>hbase.rpc.timeout</name>
> > >>>>>>> <value>60000</value>
> > >>>>>>> </property>
> > >>>>>>> When scanning 30 tables in hbase(each table holds 45 million
> > >>>>>>> records) I
> > >>>>>>> got
> > >>>>>>>
> > >>>>>>> WARN client.ScannerCallable [main]: Ignore, probably already
> closed
> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
> > >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease does
> > not
> > >>>>>>> exist
> > >>>>>>>
> > >>>>>>> But when I change the value to
> > >>>>>>>
> > >>>>>>> <property>
> > >>>>>>> <name>hbase.client.scanner.timeout.period</name>
> > >>>>>>> <value>70000</value>
> > >>>>>>> </property>
> > >>>>>>>
> > >>>>>>> <property>
> > >>>>>>> <name>hbase.rpc.timeout</name>
> > >>>>>>> <value>70000</value>
> > >>>>>>> </property>
> > >>>>>>> It runs successfully.
> > >>>>>>> Can any one tell me the reason for this failure and also is there
> > any
> > >>>>>>> factor to calculate these property values for any data.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>
> > >
> >
>
Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Posted by Yu Li <ca...@gmail.com>.
Hi Rajeshkumar,
Simply speaking, in your case I'd suggest to set hbase.client.scanner.caching
to a small value like 10 but not zero, and leave the
hbase.client.scanner.caching or hbase.rpc.timeout setting as default. Below
are some detailed explanation:
Setting hbase.client.scanner.caching to zero will cause the scan returns no
data (please check whether you are getting a wrong result with this
setting), so please don't make such settings unless you intend to renew the
scan lease. See RsRpcServices#scan source code for more details if
interested.
For version 1.1.1 (actually after HBASE-16973), the default value of
hbase.client.scanner.caching
is Integer.MAX_VALUE, and it won't return (for a single scanner.next call)
until the result size reached hbase.client.scanner.max.result.size or times
out. If you are using a sparse filter, then it's strongly suggested to
set hbase.client.scanner.caching
to some small value but not zero. See HBASE-16973 for more details if
interested.
Regarding hbase.rpc.timeout and hbase.client.scanner.timeout.period:
* For scan it's hbase.client.scanner.timeout.period taking effect
* For all other kinds of non-admin operations like
get/put/append/increment/multi hbase.rpc.timeout takes effect, unless you
specified hbase.client.operation.timeout.
Check ClientScanner#scannerTimeout, ScannerCallable#call,
AbstractRpcClient#channelOperationTimeout and
AbstractRpcClient#callBlockingMethod for more details if interested.
Best Regards,
Yu
On 11 January 2017 at 15:29, Rajeshkumar J <ra...@gmail.com>
wrote:
> Hi, As i got lease expire exception i have increased the values of these
> properties hbase.client.scanner.timeout.period and hbase.rpc.timeout. Is
> there any disadvantage increasing these two values?
>
> On Mon, Jan 2, 2017 at 11:01 AM, Rajeshkumar J <
> rajeshkumarit8292@gmail.com>
> wrote:
>
> > I am using hbase 1.1.1.
> >
> > I have following property value in hbase-site.xml and I didn't set
> value
> > hbase.client.scanner.caching property(so it must pick default value for
> > this)
> >
> > <property>
> > <name>hbase.client.scanner.timeout.period</name>
> > <value>60000</value>
> > </property>
> >
> > <property>
> > <name>hbase.rpc.timeout</name>
> > <value>60000</value>
> > </property>
> > When scanning 30 tables in hbase(each table holds 45 million records) I
> > got the following error
> >
> > WARN client.ScannerCallable [main]: Ignore, probably already closed
> > org.apache.hadoop.hbase.regionserver.LeaseException:
> > org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
> exist
> >
> > But when I added hbase.client.scanner.caching property value as zero I
> > didn't get any error. Why this happened?
> > Even though I gave zero it supposed to pick the default value? How can
> > this process succeeds?
> >
> > Thanks
> >
> > On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <el...@apache.org> wrote:
> >
> >> I am having an extremely difficult time parsing this one, Rajeshkumar.
> >>
> >> If you still have a question/something you don't understand, I'd
> >> appreciate it if you could try to re-state it a little more clearly.
> >>
> >>
> >> Rajeshkumar J wrote:
> >>
> >>> I am using hbase 1.1.1. Yes while setting zero it must pick the default
> >>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have set the
> >>> value as INTEGER.MAX_VALUE only but earlier it had failed but now
> >>> succeeded. So only I am asking this. Also I didn't change any property
> >>> whatever I used in process which failed in the process which succeeded
> >>> except assigning Zero value for this property
> >>>
> >>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<el...@apache.org> wrote:
> >>>
> >>> Most likely, since you gave a nonsensical value, HBase used a default
> >>>> value instead of the one you provided. Since you have not shared the
> >>>> version of HBase which you are using, I would recommend that you look
> at
> >>>> the code. It should be very obvious what value is being used instead
> of
> >>>> the
> >>>> bogus value you provided.
> >>>>
> >>>>
> >>>> Rajeshkumar J wrote:
> >>>>
> >>>> I have tried setting hbase.client.scanner.caching property to 0 with
> >>>>> above
> >>>>> two values as 60000. It runs fine. But hbase.client.scanner.caching
> >>>>> property tells no of rows to be fetched for every scanner but here I
> >>>>> have
> >>>>> given zero. How this worked?
> >>>>>
> >>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<el...@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>> hbase.client.scanner.timeout.period is a timeout specifically for
> RPCs
> >>>>>
> >>>>>> that come from the HBase Scanner classes (e.g. ClientScanner) while
> >>>>>> hbase.rpc.timeout is the default timeout for any RPC. I believe that
> >>>>>> the
> >>>>>> hbase.client.scanner.timeout.period is also used by the
> >>>>>> RegionServers to
> >>>>>> define the lifetime of the Lease (the cause of the LeaseException
> >>>>>> you're
> >>>>>> seeing).
> >>>>>>
> >>>>>> Generally, when you see these kinds of exceptions while scanning
> data
> >>>>>> in
> >>>>>> HBase, it is just a factor of your hardware and current performance
> >>>>>> (in
> >>>>>> other words, how long it takes to read your data). I can't really
> >>>>>> give a
> >>>>>> firm answer because it is dependent on your system's performance.
> You
> >>>>>> should be able to approximate the performance with some
> >>>>>> back-of-the-envelope math.
> >>>>>>
> >>>>>> Some changes like https://issues.apache.org/jira/browse/HBASE-13090
> >>>>>> and
> >>>>>> https://issues.apache.org/jira/browse/HBASE-13333 should help
> reduce
> >>>>>> the
> >>>>>> need for you to tweak configuration properties in the future.
> >>>>>>
> >>>>>>
> >>>>>> Rajeshkumar J wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>>> I have following property value as below
> >>>>>>>
> >>>>>>> <property>
> >>>>>>> <name>hbase.client.scanner.timeout.period</name>
> >>>>>>> <value>60000</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>> <property>
> >>>>>>> <name>hbase.rpc.timeout</name>
> >>>>>>> <value>60000</value>
> >>>>>>> </property>
> >>>>>>> When scanning 30 tables in hbase(each table holds 45 million
> >>>>>>> records) I
> >>>>>>> got
> >>>>>>>
> >>>>>>> WARN client.ScannerCallable [main]: Ignore, probably already closed
> >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
> >>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease does
> not
> >>>>>>> exist
> >>>>>>>
> >>>>>>> But when I change the value to
> >>>>>>>
> >>>>>>> <property>
> >>>>>>> <name>hbase.client.scanner.timeout.period</name>
> >>>>>>> <value>70000</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>> <property>
> >>>>>>> <name>hbase.rpc.timeout</name>
> >>>>>>> <value>70000</value>
> >>>>>>> </property>
> >>>>>>> It runs successfully.
> >>>>>>> Can any one tell me the reason for this failure and also is there
> any
> >>>>>>> factor to calculate these property values for any data.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>
> >
>
Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Posted by Rajeshkumar J <ra...@gmail.com>.
Hi, As i got lease expire exception i have increased the values of these
properties hbase.client.scanner.timeout.period and hbase.rpc.timeout. Is
there any disadvantage increasing these two values?
On Mon, Jan 2, 2017 at 11:01 AM, Rajeshkumar J <ra...@gmail.com>
wrote:
> I am using hbase 1.1.1.
>
> I have following property value in hbase-site.xml and I didn't set value
> hbase.client.scanner.caching property(so it must pick default value for
> this)
>
> <property>
> <name>hbase.client.scanner.timeout.period</name>
> <value>60000</value>
> </property>
>
> <property>
> <name>hbase.rpc.timeout</name>
> <value>60000</value>
> </property>
> When scanning 30 tables in hbase(each table holds 45 million records) I
> got the following error
>
> WARN client.ScannerCallable [main]: Ignore, probably already closed
> org.apache.hadoop.hbase.regionserver.LeaseException:
> org.apache.hadoop.hbase.regionserver.LeaseException: lease does not exist
>
> But when I added hbase.client.scanner.caching property value as zero I
> didn't get any error. Why this happened?
> Even though I gave zero it supposed to pick the default value? How can
> this process succeeds?
>
> Thanks
>
> On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <el...@apache.org> wrote:
>
>> I am having an extremely difficult time parsing this one, Rajeshkumar.
>>
>> If you still have a question/something you don't understand, I'd
>> appreciate it if you could try to re-state it a little more clearly.
>>
>>
>> Rajeshkumar J wrote:
>>
>>> I am using hbase 1.1.1. Yes while setting zero it must pick the default
>>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have set the
>>> value as INTEGER.MAX_VALUE only but earlier it had failed but now
>>> succeeded. So only I am asking this. Also I didn't change any property
>>> whatever I used in process which failed in the process which succeeded
>>> except assigning Zero value for this property
>>>
>>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<el...@apache.org> wrote:
>>>
>>> Most likely, since you gave a nonsensical value, HBase used a default
>>>> value instead of the one you provided. Since you have not shared the
>>>> version of HBase which you are using, I would recommend that you look at
>>>> the code. It should be very obvious what value is being used instead of
>>>> the
>>>> bogus value you provided.
>>>>
>>>>
>>>> Rajeshkumar J wrote:
>>>>
>>>> I have tried setting hbase.client.scanner.caching property to 0 with
>>>>> above
>>>>> two values as 60000. It runs fine. But hbase.client.scanner.caching
>>>>> property tells no of rows to be fetched for every scanner but here I
>>>>> have
>>>>> given zero. How this worked?
>>>>>
>>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<el...@apache.org>
>>>>> wrote:
>>>>>
>>>>> hbase.client.scanner.timeout.period is a timeout specifically for RPCs
>>>>>
>>>>>> that come from the HBase Scanner classes (e.g. ClientScanner) while
>>>>>> hbase.rpc.timeout is the default timeout for any RPC. I believe that
>>>>>> the
>>>>>> hbase.client.scanner.timeout.period is also used by the
>>>>>> RegionServers to
>>>>>> define the lifetime of the Lease (the cause of the LeaseException
>>>>>> you're
>>>>>> seeing).
>>>>>>
>>>>>> Generally, when you see these kinds of exceptions while scanning data
>>>>>> in
>>>>>> HBase, it is just a factor of your hardware and current performance
>>>>>> (in
>>>>>> other words, how long it takes to read your data). I can't really
>>>>>> give a
>>>>>> firm answer because it is dependent on your system's performance. You
>>>>>> should be able to approximate the performance with some
>>>>>> back-of-the-envelope math.
>>>>>>
>>>>>> Some changes like https://issues.apache.org/jira/browse/HBASE-13090
>>>>>> and
>>>>>> https://issues.apache.org/jira/browse/HBASE-13333 should help reduce
>>>>>> the
>>>>>> need for you to tweak configuration properties in the future.
>>>>>>
>>>>>>
>>>>>> Rajeshkumar J wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>> I have following property value as below
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>hbase.client.scanner.timeout.period</name>
>>>>>>> <value>60000</value>
>>>>>>> </property>
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>hbase.rpc.timeout</name>
>>>>>>> <value>60000</value>
>>>>>>> </property>
>>>>>>> When scanning 30 tables in hbase(each table holds 45 million
>>>>>>> records) I
>>>>>>> got
>>>>>>>
>>>>>>> WARN client.ScannerCallable [main]: Ignore, probably already closed
>>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
>>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
>>>>>>> exist
>>>>>>>
>>>>>>> But when I change the value to
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>hbase.client.scanner.timeout.period</name>
>>>>>>> <value>70000</value>
>>>>>>> </property>
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>hbase.rpc.timeout</name>
>>>>>>> <value>70000</value>
>>>>>>> </property>
>>>>>>> It runs successfully.
>>>>>>> Can any one tell me the reason for this failure and also is there any
>>>>>>> factor to calculate these property values for any data.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>
>
Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Posted by Rajeshkumar J <ra...@gmail.com>.
I am using hbase 1.1.1.
I have following property value in hbase-site.xml and I didn't set
value hbase.client.scanner.caching
property(so it must pick default value for this)
<property>
<name>hbase.client.scanner.timeout.period</name>
<value>60000</value>
</property>
<property>
<name>hbase.rpc.timeout</name>
<value>60000</value>
</property>
When scanning 30 tables in hbase(each table holds 45 million records) I got
the following error
WARN client.ScannerCallable [main]: Ignore, probably already closed
org.apache.hadoop.hbase.regionserver.LeaseException:
org.apache.hadoop.hbase.regionserver.LeaseException: lease does not exist
But when I added hbase.client.scanner.caching property value as zero I
didn't get any error. Why this happened?
Even though I gave zero it supposed to pick the default value? How can this
process succeeds?
Thanks
On Mon, Jan 2, 2017 at 10:44 AM, Josh Elser <el...@apache.org> wrote:
> I am having an extremely difficult time parsing this one, Rajeshkumar.
>
> If you still have a question/something you don't understand, I'd
> appreciate it if you could try to re-state it a little more clearly.
>
>
> Rajeshkumar J wrote:
>
>> I am using hbase 1.1.1. Yes while setting zero it must pick the default
>> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have set the
>> value as INTEGER.MAX_VALUE only but earlier it had failed but now
>> succeeded. So only I am asking this. Also I didn't change any property
>> whatever I used in process which failed in the process which succeeded
>> except assigning Zero value for this property
>>
>> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<el...@apache.org> wrote:
>>
>> Most likely, since you gave a nonsensical value, HBase used a default
>>> value instead of the one you provided. Since you have not shared the
>>> version of HBase which you are using, I would recommend that you look at
>>> the code. It should be very obvious what value is being used instead of
>>> the
>>> bogus value you provided.
>>>
>>>
>>> Rajeshkumar J wrote:
>>>
>>> I have tried setting hbase.client.scanner.caching property to 0 with
>>>> above
>>>> two values as 60000. It runs fine. But hbase.client.scanner.caching
>>>> property tells no of rows to be fetched for every scanner but here I
>>>> have
>>>> given zero. How this worked?
>>>>
>>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<el...@apache.org> wrote:
>>>>
>>>> hbase.client.scanner.timeout.period is a timeout specifically for RPCs
>>>>
>>>>> that come from the HBase Scanner classes (e.g. ClientScanner) while
>>>>> hbase.rpc.timeout is the default timeout for any RPC. I believe that
>>>>> the
>>>>> hbase.client.scanner.timeout.period is also used by the RegionServers
>>>>> to
>>>>> define the lifetime of the Lease (the cause of the LeaseException
>>>>> you're
>>>>> seeing).
>>>>>
>>>>> Generally, when you see these kinds of exceptions while scanning data
>>>>> in
>>>>> HBase, it is just a factor of your hardware and current performance (in
>>>>> other words, how long it takes to read your data). I can't really give
>>>>> a
>>>>> firm answer because it is dependent on your system's performance. You
>>>>> should be able to approximate the performance with some
>>>>> back-of-the-envelope math.
>>>>>
>>>>> Some changes like https://issues.apache.org/jira/browse/HBASE-13090
>>>>> and
>>>>> https://issues.apache.org/jira/browse/HBASE-13333 should help reduce
>>>>> the
>>>>> need for you to tweak configuration properties in the future.
>>>>>
>>>>>
>>>>> Rajeshkumar J wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>>> I have following property value as below
>>>>>>
>>>>>> <property>
>>>>>> <name>hbase.client.scanner.timeout.period</name>
>>>>>> <value>60000</value>
>>>>>> </property>
>>>>>>
>>>>>> <property>
>>>>>> <name>hbase.rpc.timeout</name>
>>>>>> <value>60000</value>
>>>>>> </property>
>>>>>> When scanning 30 tables in hbase(each table holds 45 million records)
>>>>>> I
>>>>>> got
>>>>>>
>>>>>> WARN client.ScannerCallable [main]: Ignore, probably already closed
>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
>>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
>>>>>> exist
>>>>>>
>>>>>> But when I change the value to
>>>>>>
>>>>>> <property>
>>>>>> <name>hbase.client.scanner.timeout.period</name>
>>>>>> <value>70000</value>
>>>>>> </property>
>>>>>>
>>>>>> <property>
>>>>>> <name>hbase.rpc.timeout</name>
>>>>>> <value>70000</value>
>>>>>> </property>
>>>>>> It runs successfully.
>>>>>> Can any one tell me the reason for this failure and also is there any
>>>>>> factor to calculate these property values for any data.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>
Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Posted by Josh Elser <el...@apache.org>.
I am having an extremely difficult time parsing this one, Rajeshkumar.
If you still have a question/something you don't understand, I'd
appreciate it if you could try to re-state it a little more clearly.
Rajeshkumar J wrote:
> I am using hbase 1.1.1. Yes while setting zero it must pick the default
> value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have set the
> value as INTEGER.MAX_VALUE only but earlier it had failed but now
> succeeded. So only I am asking this. Also I didn't change any property
> whatever I used in process which failed in the process which succeeded
> except assigning Zero value for this property
>
> On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser<el...@apache.org> wrote:
>
>> Most likely, since you gave a nonsensical value, HBase used a default
>> value instead of the one you provided. Since you have not shared the
>> version of HBase which you are using, I would recommend that you look at
>> the code. It should be very obvious what value is being used instead of the
>> bogus value you provided.
>>
>>
>> Rajeshkumar J wrote:
>>
>>> I have tried setting hbase.client.scanner.caching property to 0 with above
>>> two values as 60000. It runs fine. But hbase.client.scanner.caching
>>> property tells no of rows to be fetched for every scanner but here I have
>>> given zero. How this worked?
>>>
>>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<el...@apache.org> wrote:
>>>
>>> hbase.client.scanner.timeout.period is a timeout specifically for RPCs
>>>> that come from the HBase Scanner classes (e.g. ClientScanner) while
>>>> hbase.rpc.timeout is the default timeout for any RPC. I believe that the
>>>> hbase.client.scanner.timeout.period is also used by the RegionServers to
>>>> define the lifetime of the Lease (the cause of the LeaseException you're
>>>> seeing).
>>>>
>>>> Generally, when you see these kinds of exceptions while scanning data in
>>>> HBase, it is just a factor of your hardware and current performance (in
>>>> other words, how long it takes to read your data). I can't really give a
>>>> firm answer because it is dependent on your system's performance. You
>>>> should be able to approximate the performance with some
>>>> back-of-the-envelope math.
>>>>
>>>> Some changes like https://issues.apache.org/jira/browse/HBASE-13090 and
>>>> https://issues.apache.org/jira/browse/HBASE-13333 should help reduce the
>>>> need for you to tweak configuration properties in the future.
>>>>
>>>>
>>>> Rajeshkumar J wrote:
>>>>
>>>> Hi,
>>>>> I have following property value as below
>>>>>
>>>>> <property>
>>>>> <name>hbase.client.scanner.timeout.period</name>
>>>>> <value>60000</value>
>>>>> </property>
>>>>>
>>>>> <property>
>>>>> <name>hbase.rpc.timeout</name>
>>>>> <value>60000</value>
>>>>> </property>
>>>>> When scanning 30 tables in hbase(each table holds 45 million records) I
>>>>> got
>>>>>
>>>>> WARN client.ScannerCallable [main]: Ignore, probably already closed
>>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
>>>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
>>>>> exist
>>>>>
>>>>> But when I change the value to
>>>>>
>>>>> <property>
>>>>> <name>hbase.client.scanner.timeout.period</name>
>>>>> <value>70000</value>
>>>>> </property>
>>>>>
>>>>> <property>
>>>>> <name>hbase.rpc.timeout</name>
>>>>> <value>70000</value>
>>>>> </property>
>>>>> It runs successfully.
>>>>> Can any one tell me the reason for this failure and also is there any
>>>>> factor to calculate these property values for any data.
>>>>>
>>>>>
>>>>>
>
Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Posted by Rajeshkumar J <ra...@gmail.com>.
I am using hbase 1.1.1. Yes while setting zero it must pick the default
value that is INTEGER.MAX_VALUE in hbase 1.1.1. Already we have set the
value as INTEGER.MAX_VALUE only but earlier it had failed but now
succeeded. So only I am asking this. Also I didn't change any property
whatever I used in process which failed in the process which succeeded
except assigning Zero value for this property
On Thu, Dec 29, 2016 at 1:33 AM, Josh Elser <el...@apache.org> wrote:
> Most likely, since you gave a nonsensical value, HBase used a default
> value instead of the one you provided. Since you have not shared the
> version of HBase which you are using, I would recommend that you look at
> the code. It should be very obvious what value is being used instead of the
> bogus value you provided.
>
>
> Rajeshkumar J wrote:
>
>> I have tried setting hbase.client.scanner.caching property to 0 with above
>> two values as 60000. It runs fine. But hbase.client.scanner.caching
>> property tells no of rows to be fetched for every scanner but here I have
>> given zero. How this worked?
>>
>> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<el...@apache.org> wrote:
>>
>> hbase.client.scanner.timeout.period is a timeout specifically for RPCs
>>> that come from the HBase Scanner classes (e.g. ClientScanner) while
>>> hbase.rpc.timeout is the default timeout for any RPC. I believe that the
>>> hbase.client.scanner.timeout.period is also used by the RegionServers to
>>> define the lifetime of the Lease (the cause of the LeaseException you're
>>> seeing).
>>>
>>> Generally, when you see these kinds of exceptions while scanning data in
>>> HBase, it is just a factor of your hardware and current performance (in
>>> other words, how long it takes to read your data). I can't really give a
>>> firm answer because it is dependent on your system's performance. You
>>> should be able to approximate the performance with some
>>> back-of-the-envelope math.
>>>
>>> Some changes like https://issues.apache.org/jira/browse/HBASE-13090 and
>>> https://issues.apache.org/jira/browse/HBASE-13333 should help reduce the
>>> need for you to tweak configuration properties in the future.
>>>
>>>
>>> Rajeshkumar J wrote:
>>>
>>> Hi,
>>>> I have following property value as below
>>>>
>>>> <property>
>>>> <name>hbase.client.scanner.timeout.period</name>
>>>> <value>60000</value>
>>>> </property>
>>>>
>>>> <property>
>>>> <name>hbase.rpc.timeout</name>
>>>> <value>60000</value>
>>>> </property>
>>>> When scanning 30 tables in hbase(each table holds 45 million records) I
>>>> got
>>>>
>>>> WARN client.ScannerCallable [main]: Ignore, probably already closed
>>>> org.apache.hadoop.hbase.regionserver.LeaseException:
>>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease does not
>>>> exist
>>>>
>>>> But when I change the value to
>>>>
>>>> <property>
>>>> <name>hbase.client.scanner.timeout.period</name>
>>>> <value>70000</value>
>>>> </property>
>>>>
>>>> <property>
>>>> <name>hbase.rpc.timeout</name>
>>>> <value>70000</value>
>>>> </property>
>>>> It runs successfully.
>>>> Can any one tell me the reason for this failure and also is there any
>>>> factor to calculate these property values for any data.
>>>>
>>>>
>>>>
>>
Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Posted by Josh Elser <el...@apache.org>.
Most likely, since you gave a nonsensical value, HBase used a default
value instead of the one you provided. Since you have not shared the
version of HBase which you are using, I would recommend that you look at
the code. It should be very obvious what value is being used instead of
the bogus value you provided.
Rajeshkumar J wrote:
> I have tried setting hbase.client.scanner.caching property to 0 with above
> two values as 60000. It runs fine. But hbase.client.scanner.caching
> property tells no of rows to be fetched for every scanner but here I have
> given zero. How this worked?
>
> On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser<el...@apache.org> wrote:
>
>> hbase.client.scanner.timeout.period is a timeout specifically for RPCs
>> that come from the HBase Scanner classes (e.g. ClientScanner) while
>> hbase.rpc.timeout is the default timeout for any RPC. I believe that the
>> hbase.client.scanner.timeout.period is also used by the RegionServers to
>> define the lifetime of the Lease (the cause of the LeaseException you're
>> seeing).
>>
>> Generally, when you see these kinds of exceptions while scanning data in
>> HBase, it is just a factor of your hardware and current performance (in
>> other words, how long it takes to read your data). I can't really give a
>> firm answer because it is dependent on your system's performance. You
>> should be able to approximate the performance with some
>> back-of-the-envelope math.
>>
>> Some changes like https://issues.apache.org/jira/browse/HBASE-13090 and
>> https://issues.apache.org/jira/browse/HBASE-13333 should help reduce the
>> need for you to tweak configuration properties in the future.
>>
>>
>> Rajeshkumar J wrote:
>>
>>> Hi,
>>> I have following property value as below
>>>
>>> <property>
>>> <name>hbase.client.scanner.timeout.period</name>
>>> <value>60000</value>
>>> </property>
>>>
>>> <property>
>>> <name>hbase.rpc.timeout</name>
>>> <value>60000</value>
>>> </property>
>>> When scanning 30 tables in hbase(each table holds 45 million records) I
>>> got
>>>
>>> WARN client.ScannerCallable [main]: Ignore, probably already closed
>>> org.apache.hadoop.hbase.regionserver.LeaseException:
>>> org.apache.hadoop.hbase.regionserver.LeaseException: lease does not exist
>>>
>>> But when I change the value to
>>>
>>> <property>
>>> <name>hbase.client.scanner.timeout.period</name>
>>> <value>70000</value>
>>> </property>
>>>
>>> <property>
>>> <name>hbase.rpc.timeout</name>
>>> <value>70000</value>
>>> </property>
>>> It runs successfully.
>>> Can any one tell me the reason for this failure and also is there any
>>> factor to calculate these property values for any data.
>>>
>>>
>
Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Posted by Rajeshkumar J <ra...@gmail.com>.
I have tried setting hbase.client.scanner.caching property to 0 with above
two values as 60000. It runs fine. But hbase.client.scanner.caching
property tells no of rows to be fetched for every scanner but here I have
given zero. How this worked?
On Tue, Dec 27, 2016 at 8:05 PM, Josh Elser <el...@apache.org> wrote:
> hbase.client.scanner.timeout.period is a timeout specifically for RPCs
> that come from the HBase Scanner classes (e.g. ClientScanner) while
> hbase.rpc.timeout is the default timeout for any RPC. I believe that the
> hbase.client.scanner.timeout.period is also used by the RegionServers to
> define the lifetime of the Lease (the cause of the LeaseException you're
> seeing).
>
> Generally, when you see these kinds of exceptions while scanning data in
> HBase, it is just a factor of your hardware and current performance (in
> other words, how long it takes to read your data). I can't really give a
> firm answer because it is dependent on your system's performance. You
> should be able to approximate the performance with some
> back-of-the-envelope math.
>
> Some changes like https://issues.apache.org/jira/browse/HBASE-13090 and
> https://issues.apache.org/jira/browse/HBASE-13333 should help reduce the
> need for you to tweak configuration properties in the future.
>
>
> Rajeshkumar J wrote:
>
>> Hi,
>> I have following property value as below
>>
>> <property>
>> <name>hbase.client.scanner.timeout.period</name>
>> <value>60000</value>
>> </property>
>>
>> <property>
>> <name>hbase.rpc.timeout</name>
>> <value>60000</value>
>> </property>
>> When scanning 30 tables in hbase(each table holds 45 million records) I
>> got
>>
>> WARN client.ScannerCallable [main]: Ignore, probably already closed
>> org.apache.hadoop.hbase.regionserver.LeaseException:
>> org.apache.hadoop.hbase.regionserver.LeaseException: lease does not exist
>>
>> But when I change the value to
>>
>> <property>
>> <name>hbase.client.scanner.timeout.period</name>
>> <value>70000</value>
>> </property>
>>
>> <property>
>> <name>hbase.rpc.timeout</name>
>> <value>70000</value>
>> </property>
>> It runs successfully.
>> Can any one tell me the reason for this failure and also is there any
>> factor to calculate these property values for any data.
>>
>>
Re: How to Configure hbase.client.scanner.timeout.period and hbase.rpc.timeout
Posted by Josh Elser <el...@apache.org>.
hbase.client.scanner.timeout.period is a timeout specifically for RPCs
that come from the HBase Scanner classes (e.g. ClientScanner) while
hbase.rpc.timeout is the default timeout for any RPC. I believe that the
hbase.client.scanner.timeout.period is also used by the RegionServers to
define the lifetime of the Lease (the cause of the LeaseException you're
seeing).
Generally, when you see these kinds of exceptions while scanning data in
HBase, it is just a factor of your hardware and current performance (in
other words, how long it takes to read your data). I can't really give a
firm answer because it is dependent on your system's performance. You
should be able to approximate the performance with some
back-of-the-envelope math.
Some changes like https://issues.apache.org/jira/browse/HBASE-13090 and
https://issues.apache.org/jira/browse/HBASE-13333 should help reduce the
need for you to tweak configuration properties in the future.
Rajeshkumar J wrote:
> Hi,
> I have following property value as below
>
> <property>
> <name>hbase.client.scanner.timeout.period</name>
> <value>60000</value>
> </property>
>
> <property>
> <name>hbase.rpc.timeout</name>
> <value>60000</value>
> </property>
> When scanning 30 tables in hbase(each table holds 45 million records) I got
>
> WARN client.ScannerCallable [main]: Ignore, probably already closed
> org.apache.hadoop.hbase.regionserver.LeaseException:
> org.apache.hadoop.hbase.regionserver.LeaseException: lease does not exist
>
> But when I change the value to
>
> <property>
> <name>hbase.client.scanner.timeout.period</name>
> <value>70000</value>
> </property>
>
> <property>
> <name>hbase.rpc.timeout</name>
> <value>70000</value>
> </property>
> It runs successfully.
> Can any one tell me the reason for this failure and also is there any
> factor to calculate these property values for any data.
>