You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by yeshwanth kumar <ye...@gmail.com> on 2014/08/12 13:16:25 UTC
Error: (state=08000,code=101) while indexing
hi i am running a single node hortonworks HDP2.1
i have 150Million rows in a table, while indexing the table on a column, it
is giving the error
0: jdbc:phoenix:localhost:/hbase-unsecure> CREATE INDEX INDEX_MONTHS ON
MONTHS_DATA (VCOUNT);
Error: (state=08000,code=101)
i increased the phoenix keepalive and querytimeout to 1hr
anypointers how to overcome this issue?
in regionserver logs i see this exception,
org.apache.hadoop.hbase.NotServingRegionException:
INDEX_MONTHS,\x80\x00\x80\x00\x00\x00\x00\x0F\xB1\x0D03/3,1407840676629.0158a3bd2e191361e2950ae3eb0
a9016. is closing
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5742)
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5706)
at
org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:3438)
at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2323)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2190)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2142)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2146)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4240)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3463)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3367)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28867)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
at java.lang.Thread.run(Thread.java:744)
thanks,
yeshwanth
Re: Error: (state=08000,code=101) while indexing
Posted by Jeffrey Zhong <jz...@hortonworks.com>.
Are you asking ³alter index with rebuild option²? it will drop existing
index data & rebuild index from beginning.
-Jeffrey
From: yeshwanth kumar <ye...@gmail.com>
Reply-To: <us...@phoenix.apache.org>
Date: Wednesday, August 13, 2014 at 11:07 PM
To: <us...@phoenix.apache.org>
Subject: Fwd: Error: (state=08000,code=101) while indexing
hi Jeffrey,
just want to know whether the indexing will start from initial state or will
it be resumed from the last checkpoint.
thanks,
yeshwanth
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Error: (state=08000,code=101) while indexing
Posted by yeshwanth kumar <ye...@gmail.com>.
the problem is with region splitting on the index table,
2014-08-14 10:07:50,654 WARN [RpcServer.handler=35,port=60020]
regionserver.HRegion: Failed getting lock in batch put,
row=\x80\x00\x80\x00\x00\x00\x
00\x0B\xAE201/23/2013 12:00:00 AM\x80\x00\x00\x03
is there any way i can presplit the index table, or any configuration
property in hbase can fix this issue.
thanks,
yeshwanth
On Thu, Aug 14, 2014 at 11:37 AM, yeshwanth kumar <ye...@gmail.com>
wrote:
>
> hi Jeffrey,
>
> just want to know whether the indexing will start from initial state or
> will it be resumed from the last checkpoint.
>
> thanks,
> yeshwanth
>
>
Fwd: Error: (state=08000,code=101) while indexing
Posted by yeshwanth kumar <ye...@gmail.com>.
hi Jeffrey,
just want to know whether the indexing will start from initial state or
will it be resumed from the last checkpoint.
thanks,
yeshwanth
Re: Error: (state=08000,code=101) while indexing
Posted by Jeffrey Zhong <jz...@hortonworks.com>.
Hey Yeshwanth,
There are two ways to get out of this state is 1) using alter index
statement with rebuild option to rebuild index2) (hack way) use hbase client
to update index_state to OEa¹ so that the index can be used for queries. In
the meantime, I will file a JIRA to log more errors in client side to have
more info in these situations.
Thanks,
-Jeffrey
From: yeshwanth kumar <ye...@gmail.com>
Reply-To: <us...@phoenix.apache.org>
Date: Tuesday, August 12, 2014 at 10:28 PM
To: <us...@phoenix.apache.org>
Subject: Re: Error: (state=08000,code=101) while indexing
hi Jeffrey,
\x00\x00INDEX_MONTHS column=0:INDEX_STATE, timestamp=1407840409969, value=b
i found this in phoenix system table.
how is it going to affect the index,
is there any way we can resume indexing after some failures,
-yeshwanth
On Wed, Aug 13, 2014 at 7:23 AM, Jeffrey Zhong <jz...@hortonworks.com>
wrote:
>
> Hey Yeshwanth,
>
> Could you check if the index¹s INDEX_STATE is set to OEa¹(active) or still be
> OEb¹(building)? Since there is not much error info, I guess the error is thrown
> from cleanup phase after index data had been built.
>
> Thanks,
> -Jeffrey
>
> From: yeshwanth kumar <ye...@gmail.com>
> Reply-To: <us...@phoenix.apache.org>
> Date: Tuesday, August 12, 2014 at 2:32 PM
> To: <us...@phoenix.apache.org>
> Subject: Re: Error: (state=08000,code=101) while indexing
>
> hi Jeffrey,
>
> that was the only error i saw in the regionserver logs,
>
> after the exception i made a count on index table and the main table ,
> both counts are equal and whenever i update some values in main table,
> index is getting updated
> describe of main table on hbase shell gives the phoenix index coprocessor
> details.
>
> evenafter getting the errorcode index is working,
> i don't understand this behavior.
>
> as the index table is getting larger and larger while indexing, it is
> compacting as well,
>
> any suggestions?
>
> thanks,
> yeshwanth
>
>
> On Wed, Aug 13, 2014 at 1:23 AM, Jeffrey Zhong <jz...@hortonworks.com> wrote:
>>
>>
>> In normal case underlying hbase client will retry(managed by configuration
>> setting hbase.client.retries.number and its default value=31) for errors you
>> pasted below. Therefore, the operation normally will succeed after retries
>> unless you see RetriesExhaustedException. Is the only error you saw around
>> that time or the index region couldn¹t be re-opened from some reason?
>>
>> -Jeffrey
>>
>> From: yeshwanth kumar <ye...@gmail.com>
>> Reply-To: <us...@phoenix.apache.org>
>> Date: Tuesday, August 12, 2014 at 4:16 AM
>> To: <us...@phoenix.apache.org>
>> Subject: Error: (state=08000,code=101) while indexing
>>
>> hi i am running a single node hortonworks HDP2.1
>>
>>
>> i have 150Million rows in a table, while indexing the table on a column, it
>> is giving the error
>>
>> 0: jdbc:phoenix:localhost:/hbase-unsecure> CREATE INDEX INDEX_MONTHS ON
>> MONTHS_DATA (VCOUNT);
>> Error: (state=08000,code=101)
>>
>> i increased the phoenix keepalive and querytimeout to 1hr
>>
>> anypointers how to overcome this issue?
>>
>> in regionserver logs i see this exception,
>>
>> org.apache.hadoop.hbase.NotServingRegionException:
>> INDEX_MONTHS,\x80\x00\x80\x00\x00\x00\x00\x0F\xB1\x0D03/3,1407840676629.0158a
>> 3bd2e191361e2950ae3eb0
>> a9016. is closing
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.jav
>> a:5742)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.jav
>> a:5706)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:3438)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java
>> :2323)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2190)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2142)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2146)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.ja
>> va:4240)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(
>> HRegionServer.java:3463)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3
>> 367)
>> at
>> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callB
>> lockingMethod(ClientProtos.java:28867)
>> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
>> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
>> at
>> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcSchedule
>> r.java:160)
>> at
>> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.
>> java:38)
>> at
>> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:
>> 110)
>> at java.lang.Thread.run(Thread.java:744)
>>
>> thanks,
>> yeshwanth
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader of
>> this message is not the intended recipient, you are hereby notified that any
>> printing, copying, dissemination, distribution, disclosure or forwarding of
>> this communication is strictly prohibited. If you have received this
>> communication in error, please contact the sender immediately and delete it
>> from your system. Thank You.
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader of
> this message is not the intended recipient, you are hereby notified that any
> printing, copying, dissemination, distribution, disclosure or forwarding of
> this communication is strictly prohibited. If you have received this
> communication in error, please contact the sender immediately and delete it
> from your system. Thank You.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Error: (state=08000,code=101) while indexing
Posted by yeshwanth kumar <ye...@gmail.com>.
hi Jeffrey,
\x00\x00INDEX_MONTHS column=0:INDEX_STATE, timestamp=1407840409969, value=b
i found this in phoenix system table.
how is it going to affect the index,
is there any way we can resume indexing after some failures,
-yeshwanth
On Wed, Aug 13, 2014 at 7:23 AM, Jeffrey Zhong <jz...@hortonworks.com>
wrote:
>
> Hey Yeshwanth,
>
> Could you check if the index’s INDEX_STATE is set to ‘a’(active) or still
> be ‘b’(building)? Since there is not much error info, I guess the error is
> thrown from cleanup phase after index data had been built.
>
> Thanks,
> -Jeffrey
>
> From: yeshwanth kumar <ye...@gmail.com>
> Reply-To: <us...@phoenix.apache.org>
> Date: Tuesday, August 12, 2014 at 2:32 PM
> To: <us...@phoenix.apache.org>
> Subject: Re: Error: (state=08000,code=101) while indexing
>
> hi Jeffrey,
>
> that was the only error i saw in the regionserver logs,
>
> after the exception i made a count on index table and the main table ,
> both counts are equal and whenever i update some values in main table,
> index is getting updated
> describe of main table on hbase shell gives the phoenix index coprocessor
> details.
>
> evenafter getting the errorcode index is working,
> i don't understand this behavior.
>
> as the index table is getting larger and larger while indexing, it is
> compacting as well,
>
> any suggestions?
>
> thanks,
> yeshwanth
>
>
> On Wed, Aug 13, 2014 at 1:23 AM, Jeffrey Zhong <jz...@hortonworks.com>
> wrote:
>
>>
>>
>> In normal case underlying hbase client will retry(managed by
>> configuration setting hbase.client.retries.number and its default value=31)
>> for errors you pasted below. Therefore, the operation normally will succeed
>> after retries unless you see RetriesExhaustedException. Is the only
>> error you saw around that time or the index region couldn’t be re-opened
>> from some reason?
>>
>> -Jeffrey
>>
>> From: yeshwanth kumar <ye...@gmail.com>
>> Reply-To: <us...@phoenix.apache.org>
>> Date: Tuesday, August 12, 2014 at 4:16 AM
>> To: <us...@phoenix.apache.org>
>> Subject: Error: (state=08000,code=101) while indexing
>>
>> hi i am running a single node hortonworks HDP2.1
>>
>>
>> i have 150Million rows in a table, while indexing the table on a column,
>> it is giving the error
>>
>> 0: jdbc:phoenix:localhost:/hbase-unsecure> CREATE INDEX INDEX_MONTHS ON
>> MONTHS_DATA (VCOUNT);
>> Error: (state=08000,code=101)
>>
>> i increased the phoenix keepalive and querytimeout to 1hr
>>
>> anypointers how to overcome this issue?
>>
>> in regionserver logs i see this exception,
>>
>> org.apache.hadoop.hbase.NotServingRegionException:
>> INDEX_MONTHS,\x80\x00\x80\x00\x00\x00\x00\x0F\xB1\x0D03/3,1407840676629.0158a3bd2e191361e2950ae3eb0
>> a9016. is closing
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5742)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5706)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:3438)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2323)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2190)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2142)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2146)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4240)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3463)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3367)
>> at
>> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28867)
>> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
>> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
>> at
>> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
>> at
>> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
>> at
>> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
>> at java.lang.Thread.run(Thread.java:744)
>>
>> thanks,
>> yeshwanth
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: Error: (state=08000,code=101) while indexing
Posted by Jeffrey Zhong <jz...@hortonworks.com>.
Hey Yeshwanth,
Could you check if the index¹s INDEX_STATE is set to OEa¹(active) or still be
OEb¹(building)? Since there is not much error info, I guess the error is
thrown from cleanup phase after index data had been built.
Thanks,
-Jeffrey
From: yeshwanth kumar <ye...@gmail.com>
Reply-To: <us...@phoenix.apache.org>
Date: Tuesday, August 12, 2014 at 2:32 PM
To: <us...@phoenix.apache.org>
Subject: Re: Error: (state=08000,code=101) while indexing
hi Jeffrey,
that was the only error i saw in the regionserver logs,
after the exception i made a count on index table and the main table ,
both counts are equal and whenever i update some values in main table,
index is getting updated
describe of main table on hbase shell gives the phoenix index coprocessor
details.
evenafter getting the errorcode index is working,
i don't understand this behavior.
as the index table is getting larger and larger while indexing, it is
compacting as well,
any suggestions?
thanks,
yeshwanth
On Wed, Aug 13, 2014 at 1:23 AM, Jeffrey Zhong <jz...@hortonworks.com>
wrote:
>
>
> In normal case underlying hbase client will retry(managed by configuration
> setting hbase.client.retries.number and its default value=31) for errors you
> pasted below. Therefore, the operation normally will succeed after retries
> unless you see RetriesExhaustedException. Is the only error you saw around
> that time or the index region couldn¹t be re-opened from some reason?
>
> -Jeffrey
>
> From: yeshwanth kumar <ye...@gmail.com>
> Reply-To: <us...@phoenix.apache.org>
> Date: Tuesday, August 12, 2014 at 4:16 AM
> To: <us...@phoenix.apache.org>
> Subject: Error: (state=08000,code=101) while indexing
>
> hi i am running a single node hortonworks HDP2.1
>
>
> i have 150Million rows in a table, while indexing the table on a column, it is
> giving the error
>
> 0: jdbc:phoenix:localhost:/hbase-unsecure> CREATE INDEX INDEX_MONTHS ON
> MONTHS_DATA (VCOUNT);
> Error: (state=08000,code=101)
>
> i increased the phoenix keepalive and querytimeout to 1hr
>
> anypointers how to overcome this issue?
>
> in regionserver logs i see this exception,
>
> org.apache.hadoop.hbase.NotServingRegionException:
> INDEX_MONTHS,\x80\x00\x80\x00\x00\x00\x00\x0F\xB1\x0D03/3,1407840676629.0158a3
> bd2e191361e2950ae3eb0
> a9016. is closing
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java
> :5742)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java
> :5706)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:3438)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:
> 2323)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2190)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2142)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2146)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.jav
> a:4240)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(H
> RegionServer.java:3463)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:33
> 67)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBl
> ockingMethod(ClientProtos.java:28867)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler
> .java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.j
> ava:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:1
> 10)
> at java.lang.Thread.run(Thread.java:744)
>
> thanks,
> yeshwanth
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader of
> this message is not the intended recipient, you are hereby notified that any
> printing, copying, dissemination, distribution, disclosure or forwarding of
> this communication is strictly prohibited. If you have received this
> communication in error, please contact the sender immediately and delete it
> from your system. Thank You.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Error: (state=08000,code=101) while indexing
Posted by yeshwanth kumar <ye...@gmail.com>.
hi Jeffrey,
that was the only error i saw in the regionserver logs,
after the exception i made a count on index table and the main table ,
both counts are equal and whenever i update some values in main table,
index is getting updated
describe of main table on hbase shell gives the phoenix index coprocessor
details.
evenafter getting the errorcode index is working,
i don't understand this behavior.
as the index table is getting larger and larger while indexing, it is
compacting as well,
any suggestions?
thanks,
yeshwanth
On Wed, Aug 13, 2014 at 1:23 AM, Jeffrey Zhong <jz...@hortonworks.com>
wrote:
>
>
> In normal case underlying hbase client will retry(managed by
> configuration setting hbase.client.retries.number and its default value=31)
> for errors you pasted below. Therefore, the operation normally will succeed
> after retries unless you see RetriesExhaustedException. Is the only error
> you saw around that time or the index region couldn’t be re-opened from
> some reason?
>
> -Jeffrey
>
> From: yeshwanth kumar <ye...@gmail.com>
> Reply-To: <us...@phoenix.apache.org>
> Date: Tuesday, August 12, 2014 at 4:16 AM
> To: <us...@phoenix.apache.org>
> Subject: Error: (state=08000,code=101) while indexing
>
> hi i am running a single node hortonworks HDP2.1
>
>
> i have 150Million rows in a table, while indexing the table on a column,
> it is giving the error
>
> 0: jdbc:phoenix:localhost:/hbase-unsecure> CREATE INDEX INDEX_MONTHS ON
> MONTHS_DATA (VCOUNT);
> Error: (state=08000,code=101)
>
> i increased the phoenix keepalive and querytimeout to 1hr
>
> anypointers how to overcome this issue?
>
> in regionserver logs i see this exception,
>
> org.apache.hadoop.hbase.NotServingRegionException:
> INDEX_MONTHS,\x80\x00\x80\x00\x00\x00\x00\x0F\xB1\x0D03/3,1407840676629.0158a3bd2e191361e2950ae3eb0
> a9016. is closing
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5742)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5706)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:3438)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2323)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2190)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2142)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2146)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.java:4240)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation(HRegionServer.java:3463)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3367)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28867)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
> at
> org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
> at java.lang.Thread.run(Thread.java:744)
>
> thanks,
> yeshwanth
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: Error: (state=08000,code=101) while indexing
Posted by Jeffrey Zhong <jz...@hortonworks.com>.
In normal case underlying hbase client will retry(managed by configuration
setting hbase.client.retries.number and its default value=31) for errors you
pasted below. Therefore, the operation normally will succeed after retries
unless you see RetriesExhaustedException. Is the only error you saw around
that time or the index region couldn¹t be re-opened from some reason?
-Jeffrey
From: yeshwanth kumar <ye...@gmail.com>
Reply-To: <us...@phoenix.apache.org>
Date: Tuesday, August 12, 2014 at 4:16 AM
To: <us...@phoenix.apache.org>
Subject: Error: (state=08000,code=101) while indexing
hi i am running a single node hortonworks HDP2.1
i have 150Million rows in a table, while indexing the table on a column, it
is giving the error
0: jdbc:phoenix:localhost:/hbase-unsecure> CREATE INDEX INDEX_MONTHS ON
MONTHS_DATA (VCOUNT);
Error: (state=08000,code=101)
i increased the phoenix keepalive and querytimeout to 1hr
anypointers how to overcome this issue?
in regionserver logs i see this exception,
org.apache.hadoop.hbase.NotServingRegionException:
INDEX_MONTHS,\x80\x00\x80\x00\x00\x00\x00\x0F\xB1\x0D03/3,1407840676629.0158
a3bd2e191361e2950ae3eb0
a9016. is closing
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.ja
va:5742)
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.ja
va:5706)
at
org.apache.hadoop.hbase.regionserver.HRegion.getRowLock(HRegion.java:3438)
at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.jav
a:2323)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2190)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2142)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2146)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doBatchOp(HRegionServer.j
ava:4240)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.doNonAtomicRegionMutation
(HRegionServer.java:3463)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:
3367)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.call
BlockingMethod(ClientProtos.java:28867)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcSchedul
er.java:160)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler
.java:38)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java
:110)
at java.lang.Thread.run(Thread.java:744)
thanks,
yeshwanth
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.