You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Billy <sa...@pearsonwholesale.com> on 2007/12/03 06:17:07 UTC
hbase meta question
Is there a way to query the meta to tell what server has what region?
I tried select from .META. and -ROOT- but no luck from shell.
Thanks
Billy
Re: hbase meta question
Posted by stack <st...@duboce.net>.
Billy wrote:
> I will download the latest trunk and give it a try.
>
> I been working on getting a php class written around the REST interface so I
> can still use hadoop-streaming
> to work with my project.
>
Good stuff.
> Here what I am doing and needed to know the meta info for.
>
> I pull the regions from rest interface for the table I am working with
> start one rest scanner for each region
> load a map reduce job using hadoop-streaming to process the data from the
> scanners
>
> The problem is making sure each map task knows what scanner to use based on
> what its local regions are. So there is no data needed to be passed over the
> network. Got any suggestions on pulling what the local regions are for a
> region server using rest or non java method? If there was a way the rest
> could interact with the meta and root tables to get region info that would
> be nice.
>
No support for this yet Billy (HADOOP-2138). Once implemented, I
imagine that it would somehow all be just hidden from you and
hosting regionserver would be doing its best to scan data off adjacent
datanode -- if one were running on same machine.
St.Ack
> Billy
>
>
> "stack" <st...@duboce.net> wrote in message
> news:47544F65.5030509@duboce.net...
>
>> Billy:
>>
>> I just committed Edward Yoon's fix for the below. Also, regards your
>> comments in HADOOP-2068, try recent TRUNK. HADOOP-2315 should fix the
>> issue you reported with slashes in rows.
>>
>> (Stay tuned to this channel for more on answers/fixes to your queries
>> posted to HADOOP-2068...)
>>
>> Yours,
>> St.Ack
>>
>>
>> Billy wrote:
>>
>>> Is there a way to query the meta to tell what server has what region?
>>>
>>> I tried select from .META. and -ROOT- but no luck from shell.
>>>
>>> Thanks
>>> Billy
>>>
>>>
>>>
>>>
>>>
>>
>
>
>
>
Re: hbase meta question
Posted by Billy <sa...@pearsonwholesale.com>.
I will download the latest trunk and give it a try.
I been working on getting a php class written around the REST interface so I
can still use hadoop-streaming
to work with my project.
Here what I am doing and needed to know the meta info for.
I pull the regions from rest interface for the table I am working with
start one rest scanner for each region
load a map reduce job using hadoop-streaming to process the data from the
scanners
The problem is making sure each map task knows what scanner to use based on
what its local regions are. So there is no data needed to be passed over the
network. Got any suggestions on pulling what the local regions are for a
region server using rest or non java method? If there was a way the rest
could interact with the meta and root tables to get region info that would
be nice.
Billy
"stack" <st...@duboce.net> wrote in message
news:47544F65.5030509@duboce.net...
> Billy:
>
> I just committed Edward Yoon's fix for the below. Also, regards your
> comments in HADOOP-2068, try recent TRUNK. HADOOP-2315 should fix the
> issue you reported with slashes in rows.
>
> (Stay tuned to this channel for more on answers/fixes to your queries
> posted to HADOOP-2068...)
>
> Yours,
> St.Ack
>
>
> Billy wrote:
>> Is there a way to query the meta to tell what server has what region?
>>
>> I tried select from .META. and -ROOT- but no luck from shell.
>>
>> Thanks
>> Billy
>>
>>
>>
>>
>
>
Re: hbase meta question
Posted by Billy <sa...@pearsonwholesale.com>.
Thanks I forgot about the encode url part it works fine now the keys are
stored in url format there not converted back. thanks for the follow-up
Billy
"Bryan Duxbury" <br...@rapleaf.com> wrote in
message news:70C73E60-52C1-4733-AF80-BF6593E07836@rapleaf.com...
> Can you send me some of the keys you are using? I've been able to test
> putting slashes in the key with no problems. Are you certain that
> whatever you're using to URL encode the keys is correct?
>
> -Bryan
>
> On Dec 17, 2007, at 1:16 AM, Billy wrote:
>
>> Any chance someone can look over the rest Code to see if how its still
>> effected I tried to add rows via rest interface with /'s in them and
>> they
>> error out as they did before is there something hard coded in the rest
>> that
>> should be changed?
>>
>> Billy
>>
>> "stack" <st...@duboce.net> wrote in message
>> news:47544F65.5030509@duboce.net...
>>> Billy:
>>>
>>> I just committed Edward Yoon's fix for the below. Also, regards your
>>> comments in HADOOP-2068, try recent TRUNK. HADOOP-2315 should fix the
>>> issue you reported with slashes in rows.
>>>
>>> (Stay tuned to this channel for more on answers/fixes to your queries
>>> posted to HADOOP-2068...)
>>>
>>> Yours,
>>> St.Ack
>>>
>>>
>>> Billy wrote:
>>>> Is there a way to query the meta to tell what server has what region?
>>>>
>>>> I tried select from .META. and -ROOT- but no luck from shell.
>>>>
>>>> Thanks
>>>> Billy
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>
Re: hbase meta question
Posted by Bryan Duxbury <br...@rapleaf.com>.
Can you send me some of the keys you are using? I've been able to
test putting slashes in the key with no problems. Are you certain
that whatever you're using to URL encode the keys is correct?
-Bryan
On Dec 17, 2007, at 1:16 AM, Billy wrote:
> Any chance someone can look over the rest Code to see if how its still
> effected I tried to add rows via rest interface with /'s in them
> and they
> error out as they did before is there something hard coded in the
> rest that
> should be changed?
>
> Billy
>
> "stack" <st...@duboce.net> wrote in message
> news:47544F65.5030509@duboce.net...
>> Billy:
>>
>> I just committed Edward Yoon's fix for the below. Also, regards your
>> comments in HADOOP-2068, try recent TRUNK. HADOOP-2315 should fix
>> the
>> issue you reported with slashes in rows.
>>
>> (Stay tuned to this channel for more on answers/fixes to your queries
>> posted to HADOOP-2068...)
>>
>> Yours,
>> St.Ack
>>
>>
>> Billy wrote:
>>> Is there a way to query the meta to tell what server has what
>>> region?
>>>
>>> I tried select from .META. and -ROOT- but no luck from shell.
>>>
>>> Thanks
>>> Billy
>>>
>>>
>>>
>>>
>>
>>
>
>
>
Re: hbase meta question
Posted by Billy <sa...@pearsonwholesale.com>.
Any chance someone can look over the rest Code to see if how its still
effected I tried to add rows via rest interface with /'s in them and they
error out as they did before is there something hard coded in the rest that
should be changed?
Billy
"stack" <st...@duboce.net> wrote in message
news:47544F65.5030509@duboce.net...
> Billy:
>
> I just committed Edward Yoon's fix for the below. Also, regards your
> comments in HADOOP-2068, try recent TRUNK. HADOOP-2315 should fix the
> issue you reported with slashes in rows.
>
> (Stay tuned to this channel for more on answers/fixes to your queries
> posted to HADOOP-2068...)
>
> Yours,
> St.Ack
>
>
> Billy wrote:
>> Is there a way to query the meta to tell what server has what region?
>>
>> I tried select from .META. and -ROOT- but no luck from shell.
>>
>> Thanks
>> Billy
>>
>>
>>
>>
>
>
Re: hbase meta question
Posted by stack <st...@duboce.net>.
Billy:
I just committed Edward Yoon's fix for the below. Also, regards your
comments in HADOOP-2068, try recent TRUNK. HADOOP-2315 should fix the
issue you reported with slashes in rows.
(Stay tuned to this channel for more on answers/fixes to your queries
posted to HADOOP-2068...)
Yours,
St.Ack
Billy wrote:
> Is there a way to query the meta to tell what server has what region?
>
> I tried select from .META. and -ROOT- but no luck from shell.
>
> Thanks
> Billy
>
>
>
>
RE: hbase meta question
Posted by edward yoon <we...@udanax.org>.
Sorry, it seems to be a shell code's problem.
I'll fix it.
------------------------------
B. Regards,
Edward yoon @ NHN, corp.
Home : http://www.udanax.org
> To: hadoop-user@lucene.apache.org
> From: sales@pearsonwholesale.com
> Subject: hbase meta question
> Date: Sun, 2 Dec 2007 23:17:07 -0600
>
> Is there a way to query the meta to tell what server has what region?
>
> I tried select from .META. and -ROOT- but no luck from shell.
>
> Thanks
> Billy
>
>
>
_________________________________________________________________
Put your friends on the big screen with Windows Vista® + Windows Live™.
http://www.microsoft.com/windows/shop/specialoffers.mspx?ocid=TXT_TAGLM_CPC_MediaCtr_bigscreen_102007