You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Boris Emelyanov <em...@post.km.ru> on 2013/10/17 11:25:04 UTC

CDH4.4 and HBASE-8912 issue

Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 
with hbase-0.94.6 and have faced 
https://issues.apache.org/jira/browse/HBASE-8912 issue.

Suggested solution was to update hbase to version 0.94.13, which is 
absent in cloudera distribution.

Is it possible to run pure hbase over cloudera hadoop?

Or how can i find if this bug is present in previous versions of cdh?

-- 
Best regards,

Boris Emelyanov.


Re: CDH4.4 and HBASE-8912 issue

Posted by Ted Yu <yu...@gmail.com>.
If I read Lars' comment on the JIRA correctly, HBASE-8912's target was moved to 0.94.13

It is still open. Meaning, if there is no patch, the target may move to next release. 

Cheers

On Oct 17, 2013, at 2:25 AM, Boris Emelyanov <em...@post.km.ru> wrote:

> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912 issue.
> 
> Suggested solution was to update hbase to version 0.94.13, which is absent in cloudera distribution.
> 
> Is it possible to run pure hbase over cloudera hadoop?
> 
> Or how can i find if this bug is present in previous versions of cdh?
> -- 
> Best regards,
> 
> Boris Emelyanov.

Re: CDH4.4 and HBASE-8912 issue

Posted by Ted Yu <yu...@gmail.com>.
If I read Lars' comment on the JIRA correctly, HBASE-8912's target was moved to 0.94.13

It is still open. Meaning, if there is no patch, the target may move to next release. 

Cheers

On Oct 17, 2013, at 2:25 AM, Boris Emelyanov <em...@post.km.ru> wrote:

> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912 issue.
> 
> Suggested solution was to update hbase to version 0.94.13, which is absent in cloudera distribution.
> 
> Is it possible to run pure hbase over cloudera hadoop?
> 
> Or how can i find if this bug is present in previous versions of cdh?
> -- 
> Best regards,
> 
> Boris Emelyanov.

Re: CDH4.4 and HBASE-8912 issue

Posted by Ted Yu <yu...@gmail.com>.
If I read Lars' comment on the JIRA correctly, HBASE-8912's target was moved to 0.94.13

It is still open. Meaning, if there is no patch, the target may move to next release. 

Cheers

On Oct 17, 2013, at 2:25 AM, Boris Emelyanov <em...@post.km.ru> wrote:

> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912 issue.
> 
> Suggested solution was to update hbase to version 0.94.13, which is absent in cloudera distribution.
> 
> Is it possible to run pure hbase over cloudera hadoop?
> 
> Or how can i find if this bug is present in previous versions of cdh?
> -- 
> Best regards,
> 
> Boris Emelyanov.

Re: CDH4.4 and HBASE-8912 issue

Posted by Boris Emelyanov <em...@post.km.ru>.

On 17.10.2013 13:43, Harsh J wrote:
> Moving touser@hbase.apache.org.
>
> HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
> version presently.
>
> Are you certain this is the exact issue you're hitting? I believe you
> can workaround this by removing the specific bad znode in ZK or so.
> When starting up after your major upgrade, did you ensure cleaning
> your ZK /hbase znode?
>
> On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov<em...@post.km.ru>  wrote:
>> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
>> hbase-0.94.6 and have facedhttps://issues.apache.org/jira/browse/HBASE-8912
>> issue.
>>
>> Suggested solution was to update hbase to version 0.94.13, which is absent
>> in cloudera distribution.
>>
>> Is it possible to run pure hbase over cloudera hadoop?
>>
>> Or how can i find if this bug is present in previous versions of cdh?
>>
>> --
>> Best regards,
>>
>> Boris Emelyanov.
>

> Are you certain this is the exact issue you're hitting?

I gues so. Description suits just fine!

> I believe you can workaround this by removing the specific bad znode
in ZK or so.

When I disable loadbalancer just after master startup (balance_switch
false), cluster works fine.
But when loadbalancer is enabled and it starts, it allways fails with
exception described in issue, allways with different regions.

>When starting up after your major upgrade, did you ensure cleaning
your ZK /hbase znode?

Yes, of course.

-- 
Best regards,

Boris Emelyanov.




Re: CDH4.4 and HBASE-8912 issue

Posted by Boris Emelyanov <em...@post.km.ru>.

On 17.10.2013 13:43, Harsh J wrote:
> Moving touser@hbase.apache.org.
>
> HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
> version presently.
>
> Are you certain this is the exact issue you're hitting? I believe you
> can workaround this by removing the specific bad znode in ZK or so.
> When starting up after your major upgrade, did you ensure cleaning
> your ZK /hbase znode?
>
> On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov<em...@post.km.ru>  wrote:
>> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
>> hbase-0.94.6 and have facedhttps://issues.apache.org/jira/browse/HBASE-8912
>> issue.
>>
>> Suggested solution was to update hbase to version 0.94.13, which is absent
>> in cloudera distribution.
>>
>> Is it possible to run pure hbase over cloudera hadoop?
>>
>> Or how can i find if this bug is present in previous versions of cdh?
>>
>> --
>> Best regards,
>>
>> Boris Emelyanov.
>

> Are you certain this is the exact issue you're hitting?

I gues so. Description suits just fine!

> I believe you can workaround this by removing the specific bad znode
in ZK or so.

When I disable loadbalancer just after master startup (balance_switch
false), cluster works fine.
But when loadbalancer is enabled and it starts, it allways fails with
exception described in issue, allways with different regions.

>When starting up after your major upgrade, did you ensure cleaning
your ZK /hbase znode?

Yes, of course.

-- 
Best regards,

Boris Emelyanov.




Re: CDH4.4 and HBASE-8912 issue

Posted by Boris Emelyanov <em...@post.km.ru>.

On 17.10.2013 13:43, Harsh J wrote:
> Moving touser@hbase.apache.org.
>
> HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
> version presently.
>
> Are you certain this is the exact issue you're hitting? I believe you
> can workaround this by removing the specific bad znode in ZK or so.
> When starting up after your major upgrade, did you ensure cleaning
> your ZK /hbase znode?
>
> On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov<em...@post.km.ru>  wrote:
>> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
>> hbase-0.94.6 and have facedhttps://issues.apache.org/jira/browse/HBASE-8912
>> issue.
>>
>> Suggested solution was to update hbase to version 0.94.13, which is absent
>> in cloudera distribution.
>>
>> Is it possible to run pure hbase over cloudera hadoop?
>>
>> Or how can i find if this bug is present in previous versions of cdh?
>>
>> --
>> Best regards,
>>
>> Boris Emelyanov.
>

> Are you certain this is the exact issue you're hitting?

I gues so. Description suits just fine!

> I believe you can workaround this by removing the specific bad znode
in ZK or so.

When I disable loadbalancer just after master startup (balance_switch
false), cluster works fine.
But when loadbalancer is enabled and it starts, it allways fails with
exception described in issue, allways with different regions.

>When starting up after your major upgrade, did you ensure cleaning
your ZK /hbase znode?

Yes, of course.

-- 
Best regards,

Boris Emelyanov.




Re: CDH4.4 and HBASE-8912 issue

Posted by Boris Emelyanov <em...@post.km.ru>.

On 17.10.2013 13:43, Harsh J wrote:
> Moving touser@hbase.apache.org.
>
> HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
> version presently.
>
> Are you certain this is the exact issue you're hitting? I believe you
> can workaround this by removing the specific bad znode in ZK or so.
> When starting up after your major upgrade, did you ensure cleaning
> your ZK /hbase znode?
>
> On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov<em...@post.km.ru>  wrote:
>> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
>> hbase-0.94.6 and have facedhttps://issues.apache.org/jira/browse/HBASE-8912
>> issue.
>>
>> Suggested solution was to update hbase to version 0.94.13, which is absent
>> in cloudera distribution.
>>
>> Is it possible to run pure hbase over cloudera hadoop?
>>
>> Or how can i find if this bug is present in previous versions of cdh?
>>
>> --
>> Best regards,
>>
>> Boris Emelyanov.
>

> Are you certain this is the exact issue you're hitting?

I gues so. Description suits just fine!

> I believe you can workaround this by removing the specific bad znode
in ZK or so.

When I disable loadbalancer just after master startup (balance_switch
false), cluster works fine.
But when loadbalancer is enabled and it starts, it allways fails with
exception described in issue, allways with different regions.

>When starting up after your major upgrade, did you ensure cleaning
your ZK /hbase znode?

Yes, of course.

-- 
Best regards,

Boris Emelyanov.




RE: CDH4.4 and HBASE-8912 issue

Posted by Vladimir Rodionov <vr...@carrieriq.com>.
Boris, what does hbck say?

We have had this issue a couple times before. To fix it I had to stop the cluster, run offline meta repair tool, delete zk-store on each zk quorum node
Offline Meta repair tool will not work if there are  inconsistencies  in HBase - you better try hbase hbck -fixAll first.


Best regards,
Vladimir Rodionov
Principal Platform Engineer
Carrier IQ, www.carrieriq.com
e-mail: vrodionov@carrieriq.com

________________________________________
From: lars hofhansl [larsh@apache.org]
Sent: Thursday, October 17, 2013 10:47 AM
To: user@hbase.apache.org; Harsh J
Subject: Re: CDH4.4 and HBASE-8912 issue

Hi Boris,

could you add the exact exceptions/logs you've seen to HBASE-8912?

Thanks.

-- Lars



________________________________
 From: Boris Emelyanov <em...@post.km.ru>
To: Harsh J <ha...@cloudera.com>; "user@hbase.apache.org" <us...@hbase.apache.org>
Sent: Thursday, October 17, 2013 3:54 AM
Subject: Re: CDH4.4 and HBASE-8912 issue


On 17.10.2013 13:43, Harsh J wrote:
> Moving touser@hbase.apache.org.
>
> HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
> version presently.
>
> Are you certain this is the exact issue you're hitting? I believe you
> can workaround this by removing the specific bad znode in ZK or so.
> When starting up after your major upgrade, did you ensure cleaning
> your ZK /hbase znode?
>
> On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov<em...@post.km.ru>  wrote:
>> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
>> hbase-0.94.6 and have facedhttps://issues.apache.org/jira/browse/HBASE-8912
>> issue.
>>
>> Suggested solution was to update hbase to version 0.94.13, which is absent
>> in cloudera distribution.
>>
>> Is it possible to run pure hbase over cloudera hadoop?
>>
>> Or how can i find if this bug is present in previous versions of cdh?
>>
>> --
>> Best regards,
>>
>> Boris Emelyanov.
>

> Are you certain this is the exact issue you're hitting?

I gues so. Description suits just fine!

> I believe you can workaround this by removing the specific bad znode
in ZK or so.

When I disable loadbalancer just after master startup (balance_switch
false), cluster works fine.
But when loadbalancer is enabled and it starts, it allways fails with
exception described in issue, allways with different regions.

>When starting up after your major upgrade, did you ensure cleaning
your ZK /hbase znode?

Yes, of course.


--
Best regards,

Boris Emelyanov.

Confidentiality Notice:  The information contained in this message, including any attachments hereto, may be confidential and is intended to be read only by the individual or entity to whom this message is addressed. If the reader of this message is not the intended recipient or an agent or designee of the intended recipient, please note that any review, use, disclosure or distribution of this message or its attachments, in any form, is strictly prohibited.  If you have received this message in error, please immediately notify the sender and/or Notifications@carrieriq.com and delete or destroy any copy of this message and its attachments.

Re: CDH4.4 and HBASE-8912 issue

Posted by lars hofhansl <la...@apache.org>.
Hi Boris,

could you add the exact exceptions/logs you've seen to HBASE-8912?

Thanks.

-- Lars



________________________________
 From: Boris Emelyanov <em...@post.km.ru>
To: Harsh J <ha...@cloudera.com>; "user@hbase.apache.org" <us...@hbase.apache.org> 
Sent: Thursday, October 17, 2013 3:54 AM
Subject: Re: CDH4.4 and HBASE-8912 issue
 

On 17.10.2013 13:43, Harsh J wrote:
> Moving touser@hbase.apache.org.
>
> HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
> version presently.
>
> Are you certain this is the exact issue you're hitting? I believe you
> can workaround this by removing the specific bad znode in ZK or so.
> When starting up after your major upgrade, did you ensure cleaning
> your ZK /hbase znode?
>
> On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov<em...@post.km.ru>  wrote:
>> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
>> hbase-0.94.6 and have facedhttps://issues.apache.org/jira/browse/HBASE-8912
>> issue.
>>
>> Suggested solution was to update hbase to version 0.94.13, which is absent
>> in cloudera distribution.
>>
>> Is it possible to run pure hbase over cloudera hadoop?
>>
>> Or how can i find if this bug is present in previous versions of cdh?
>>
>> --
>> Best regards,
>>
>> Boris Emelyanov.
>

> Are you certain this is the exact issue you're hitting?

I gues so. Description suits just fine!

> I believe you can workaround this by removing the specific bad znode 
in ZK or so.

When I disable loadbalancer just after master startup (balance_switch 
false), cluster works fine.
But when loadbalancer is enabled and it starts, it allways fails with 
exception described in issue, allways with different regions.

>When starting up after your major upgrade, did you ensure cleaning 
your ZK /hbase znode?

Yes, of course.


-- 
Best regards,

Boris Emelyanov.

Re: CDH4.4 and HBASE-8912 issue

Posted by Boris Emelyanov <em...@post.km.ru>.
On 17.10.2013 13:43, Harsh J wrote:
> Moving touser@hbase.apache.org.
>
> HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
> version presently.
>
> Are you certain this is the exact issue you're hitting? I believe you
> can workaround this by removing the specific bad znode in ZK or so.
> When starting up after your major upgrade, did you ensure cleaning
> your ZK /hbase znode?
>
> On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov<em...@post.km.ru>  wrote:
>> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
>> hbase-0.94.6 and have facedhttps://issues.apache.org/jira/browse/HBASE-8912
>> issue.
>>
>> Suggested solution was to update hbase to version 0.94.13, which is absent
>> in cloudera distribution.
>>
>> Is it possible to run pure hbase over cloudera hadoop?
>>
>> Or how can i find if this bug is present in previous versions of cdh?
>>
>> --
>> Best regards,
>>
>> Boris Emelyanov.
>

 > Are you certain this is the exact issue you're hitting?

I gues so. Description suits just fine!

 > I believe you can workaround this by removing the specific bad znode 
in ZK or so.

When I disable loadbalancer just after master startup (balance_switch 
false), cluster works fine.
But when loadbalancer is enabled and it starts, it allways fails with 
exception described in issue, allways with different regions.

 >When starting up after your major upgrade, did you ensure cleaning 
your ZK /hbase znode?

Yes, of course.

-- 
Best regards,

Boris Emelyanov.


Re: CDH4.4 and HBASE-8912 issue

Posted by Harsh J <ha...@cloudera.com>.
Moving to user@hbase.apache.org.

HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
version presently.

Are you certain this is the exact issue you're hitting? I believe you
can workaround this by removing the specific bad znode in ZK or so.
When starting up after your major upgrade, did you ensure cleaning
your ZK /hbase znode?

On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov <em...@post.km.ru> wrote:
> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
> hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912
> issue.
>
> Suggested solution was to update hbase to version 0.94.13, which is absent
> in cloudera distribution.
>
> Is it possible to run pure hbase over cloudera hadoop?
>
> Or how can i find if this bug is present in previous versions of cdh?
>
> --
> Best regards,
>
> Boris Emelyanov.



-- 
Harsh J

Re: CDH4.4 and HBASE-8912 issue

Posted by Ted Yu <yu...@gmail.com>.
If I read Lars' comment on the JIRA correctly, HBASE-8912's target was moved to 0.94.13

It is still open. Meaning, if there is no patch, the target may move to next release. 

Cheers

On Oct 17, 2013, at 2:25 AM, Boris Emelyanov <em...@post.km.ru> wrote:

> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912 issue.
> 
> Suggested solution was to update hbase to version 0.94.13, which is absent in cloudera distribution.
> 
> Is it possible to run pure hbase over cloudera hadoop?
> 
> Or how can i find if this bug is present in previous versions of cdh?
> -- 
> Best regards,
> 
> Boris Emelyanov.

Re: CDH4.4 and HBASE-8912 issue

Posted by Harsh J <ha...@cloudera.com>.
Moving to user@hbase.apache.org.

HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
version presently.

Are you certain this is the exact issue you're hitting? I believe you
can workaround this by removing the specific bad znode in ZK or so.
When starting up after your major upgrade, did you ensure cleaning
your ZK /hbase znode?

On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov <em...@post.km.ru> wrote:
> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
> hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912
> issue.
>
> Suggested solution was to update hbase to version 0.94.13, which is absent
> in cloudera distribution.
>
> Is it possible to run pure hbase over cloudera hadoop?
>
> Or how can i find if this bug is present in previous versions of cdh?
>
> --
> Best regards,
>
> Boris Emelyanov.



-- 
Harsh J

Re: CDH4.4 and HBASE-8912 issue

Posted by Harsh J <ha...@cloudera.com>.
Moving to user@hbase.apache.org.

HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
version presently.

Are you certain this is the exact issue you're hitting? I believe you
can workaround this by removing the specific bad znode in ZK or so.
When starting up after your major upgrade, did you ensure cleaning
your ZK /hbase znode?

On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov <em...@post.km.ru> wrote:
> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
> hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912
> issue.
>
> Suggested solution was to update hbase to version 0.94.13, which is absent
> in cloudera distribution.
>
> Is it possible to run pure hbase over cloudera hadoop?
>
> Or how can i find if this bug is present in previous versions of cdh?
>
> --
> Best regards,
>
> Boris Emelyanov.



-- 
Harsh J

Re: CDH4.4 and HBASE-8912 issue

Posted by Harsh J <ha...@cloudera.com>.
Moving to user@hbase.apache.org.

HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
version presently.

Are you certain this is the exact issue you're hitting? I believe you
can workaround this by removing the specific bad znode in ZK or so.
When starting up after your major upgrade, did you ensure cleaning
your ZK /hbase znode?

On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov <em...@post.km.ru> wrote:
> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
> hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912
> issue.
>
> Suggested solution was to update hbase to version 0.94.13, which is absent
> in cloudera distribution.
>
> Is it possible to run pure hbase over cloudera hadoop?
>
> Or how can i find if this bug is present in previous versions of cdh?
>
> --
> Best regards,
>
> Boris Emelyanov.



-- 
Harsh J

Re: CDH4.4 and HBASE-8912 issue

Posted by Harsh J <ha...@cloudera.com>.
Moving to user@hbase.apache.org.

HBASE-8912 is still unresolved, and 0.94.13 is just the targeted
version presently.

Are you certain this is the exact issue you're hitting? I believe you
can workaround this by removing the specific bad znode in ZK or so.
When starting up after your major upgrade, did you ensure cleaning
your ZK /hbase znode?

On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov <em...@post.km.ru> wrote:
> Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with
> hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912
> issue.
>
> Suggested solution was to update hbase to version 0.94.13, which is absent
> in cloudera distribution.
>
> Is it possible to run pure hbase over cloudera hadoop?
>
> Or how can i find if this bug is present in previous versions of cdh?
>
> --
> Best regards,
>
> Boris Emelyanov.



-- 
Harsh J