You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by kisalay <ki...@gmail.com> on 2011/11/23 13:39:34 UTC
Help for Online Merge
This query is regarding the online_merge script from
https://issues.apache.org/jira/browse/HBASE-1621
My setup is hbase - cdh3u1, hadoop-cdh3-u0.
When I am running this script, I encounter the problem at the following
line:
info =
Writables.getHRegionInfo(result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER))
The result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER)
returns a null and Writable.getHRegionInfo throws a null-pointer-exception.
I am wondering in what cases can I get null returned
from result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER)
?
Thanks
Kisalay
Re: Help for Online Merge
Posted by kisalay <ki...@gmail.com>.
Thanks Stack.. will be trying this out.
I had one more question: What are the scenarios under which I can get null
regioninfo. Can it be attributed to manual maintenance tasks like
major-compact or offline merge ? If I want to do a online periodic merge of
all my empty regions inoder to keep the number of regions under control,
shud I also, always look for these regions with null-region info and delete
them from META periodically ?
On Wed, Nov 23, 2011 at 10:01 PM, Stack <st...@duboce.net> wrote:
> On Wed, Nov 23, 2011 at 4:39 AM, kisalay <ki...@gmail.com> wrote:
> > This query is regarding the online_merge script from
> > https://issues.apache.org/jira/browse/HBASE-1621
> >
> > My setup is hbase - cdh3u1, hadoop-cdh3-u0.
> >
> > When I am running this script, I encounter the problem at the following
> > line:
> >
> > info =
> >
> Writables.getHRegionInfo(result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER))
> >
> > The
> result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER)
> > returns a null and Writable.getHRegionInfo throws a
> null-pointer-exception.
> >
> > I am wondering in what cases can I get null returned
> > from
> result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER)
> > ?
>
> Change the script to use
>
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/util/Writables.html#getHRegionInfoOrNull(byte[])
> instead. If it returns null, continue rather than proceed. Or clean
> out these rows with null regioninfo before running the script by
> deleting them from .META. You should see a list of what to delete if
> you look in your master log. It should have complaint about rows with
> empty info:regioninfos.
>
> St.Ack
>
Re: Help for Online Merge
Posted by kisalay <ki...@gmail.com>.
no Mark. You do not need to apply the patch of the jira for running the
script. The script runs on vanilla cdh3u1 hbase. Havent tried it with
hbase-u0.
On Wed, Nov 23, 2011 at 10:09 PM, Mark <st...@gmail.com> wrote:
> Do you need to patch Cloudera before running this script?
>
>
> On 11/23/11 8:31 AM, Stack wrote:
>
>> On Wed, Nov 23, 2011 at 4:39 AM, kisalay<ki...@gmail.com> wrote:
>>
>>> This query is regarding the online_merge script from
>>> https://issues.apache.org/**jira/browse/HBASE-1621<https://issues.apache.org/jira/browse/HBASE-1621>
>>>
>>> My setup is hbase - cdh3u1, hadoop-cdh3-u0.
>>>
>>> When I am running this script, I encounter the problem at the following
>>> line:
>>>
>>> info =
>>> Writables.getHRegionInfo(**result.getValue(HConstants::**
>>> CATALOG_FAMILY,HConstants::**REGIONINFO_QUALIFIER))
>>>
>>> The result.getValue(HConstants::**CATALOG_FAMILY,HConstants::**
>>> REGIONINFO_QUALIFIER)
>>> returns a null and Writable.getHRegionInfo throws a
>>> null-pointer-exception.
>>>
>>> I am wondering in what cases can I get null returned
>>> from result.getValue(HConstants::**CATALOG_FAMILY,HConstants::**
>>> REGIONINFO_QUALIFIER)
>>> ?
>>>
>> Change the script to use
>> http://hbase.apache.org/**apidocs/org/apache/hadoop/**
>> hbase/util/Writables.html#**getHRegionInfoOrNull(byte[])<http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/util/Writables.html#getHRegionInfoOrNull(byte[])>
>> instead. If it returns null, continue rather than proceed. Or clean
>> out these rows with null regioninfo before running the script by
>> deleting them from .META. You should see a list of what to delete if
>> you look in your master log. It should have complaint about rows with
>> empty info:regioninfos.
>>
>> St.Ack
>>
>
Re: Help for Online Merge
Posted by Stack <st...@duboce.net>.
On Wed, Nov 23, 2011 at 8:39 AM, Mark <st...@gmail.com> wrote:
> Do you need to patch Cloudera before running this script?
>
You need to change the script (Not sure what you mean by patch 'Cloudera').
St.Ack
> On 11/23/11 8:31 AM, Stack wrote:
>>
>> On Wed, Nov 23, 2011 at 4:39 AM, kisalay<ki...@gmail.com> wrote:
>>>
>>> This query is regarding the online_merge script from
>>> https://issues.apache.org/jira/browse/HBASE-1621
>>>
>>> My setup is hbase - cdh3u1, hadoop-cdh3-u0.
>>>
>>> When I am running this script, I encounter the problem at the following
>>> line:
>>>
>>> info =
>>>
>>> Writables.getHRegionInfo(result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER))
>>>
>>> The
>>> result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER)
>>> returns a null and Writable.getHRegionInfo throws a
>>> null-pointer-exception.
>>>
>>> I am wondering in what cases can I get null returned
>>> from
>>> result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER)
>>> ?
>>
>> Change the script to use
>>
>> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/util/Writables.html#getHRegionInfoOrNull(byte[])
>> instead. If it returns null, continue rather than proceed. Or clean
>> out these rows with null regioninfo before running the script by
>> deleting them from .META. You should see a list of what to delete if
>> you look in your master log. It should have complaint about rows with
>> empty info:regioninfos.
>>
>> St.Ack
>
Re: Help for Online Merge
Posted by Mark <st...@gmail.com>.
Do you need to patch Cloudera before running this script?
On 11/23/11 8:31 AM, Stack wrote:
> On Wed, Nov 23, 2011 at 4:39 AM, kisalay<ki...@gmail.com> wrote:
>> This query is regarding the online_merge script from
>> https://issues.apache.org/jira/browse/HBASE-1621
>>
>> My setup is hbase - cdh3u1, hadoop-cdh3-u0.
>>
>> When I am running this script, I encounter the problem at the following
>> line:
>>
>> info =
>> Writables.getHRegionInfo(result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER))
>>
>> The result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER)
>> returns a null and Writable.getHRegionInfo throws a null-pointer-exception.
>>
>> I am wondering in what cases can I get null returned
>> from result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER)
>> ?
> Change the script to use
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/util/Writables.html#getHRegionInfoOrNull(byte[])
> instead. If it returns null, continue rather than proceed. Or clean
> out these rows with null regioninfo before running the script by
> deleting them from .META. You should see a list of what to delete if
> you look in your master log. It should have complaint about rows with
> empty info:regioninfos.
>
> St.Ack
Re: Help for Online Merge
Posted by Stack <st...@duboce.net>.
On Wed, Nov 23, 2011 at 4:39 AM, kisalay <ki...@gmail.com> wrote:
> This query is regarding the online_merge script from
> https://issues.apache.org/jira/browse/HBASE-1621
>
> My setup is hbase - cdh3u1, hadoop-cdh3-u0.
>
> When I am running this script, I encounter the problem at the following
> line:
>
> info =
> Writables.getHRegionInfo(result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER))
>
> The result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER)
> returns a null and Writable.getHRegionInfo throws a null-pointer-exception.
>
> I am wondering in what cases can I get null returned
> from result.getValue(HConstants::CATALOG_FAMILY,HConstants::REGIONINFO_QUALIFIER)
> ?
Change the script to use
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/util/Writables.html#getHRegionInfoOrNull(byte[])
instead. If it returns null, continue rather than proceed. Or clean
out these rows with null regioninfo before running the script by
deleting them from .META. You should see a list of what to delete if
you look in your master log. It should have complaint about rows with
empty info:regioninfos.
St.Ack