You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by David Koch <og...@googlemail.com> on 2014/04/25 11:49:36 UTC
Getting all rows from .META. for a specific table.
Hi,
How do I get all rows from .META. pertaining to a specific table? I don't
want to scan all of .META. and then do a grep on the output since .META. is
quite large but I don't know how to use HBase filters from hbase shell.
Thanks,
/David
Re: Getting all rows from .META. for a specific table.
Posted by Ted Yu <yu...@gmail.com>.
Given the following tables in a deployment :
IntegrationTestBulkLoad
IntegrationTestIngest
IntegrationTestIngestStripeCompactions
IntegrationTestLoadAndVerify
IntegrationTestMTTR
If I pass IntegrationTestIngest as parameter, regions
of IntegrationTestIngestStripeCompactions would be picked up.
Here is modified version of Samir's script that I tested on 0.98 deployment
(note the comma following table name):
#!/bin/bash
table="$1"
echo "scan 'hbase:meta', {FILTER=>\"PrefixFilter('$table,')\"}" > cmd_file
hbase shell < cmd_file | grep $table |awk '{print $1}' | cut -d "," -f3 |
cut -d "." -f2 |tail -n+2 | uniq
On Fri, Apr 25, 2014 at 7:05 AM, Samir Ahmic <ah...@gmail.com> wrote:
> Hi David,
>
> Here is quick bash script that will extract encoded region name.
>
> #!/bin/bash
> table="$1"
> echo "scan '.META.', {FILTER=>\"PrefixFilter('$table')\"}" > cmd_file
> hbase shell < cmd_file | grep $table |awk '{print $1}' | cut -d "," -f3 |
> cut -d "." -f2 |tail -n+2 | uniq
>
> Regards
>
>
>
>
> On Fri, Apr 25, 2014 at 3:21 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Minor comment about usage of the PrefixFilter, if there is table 't11' in
> > your cluster, rows for 't11' would also be returned.
> >
> > You can specify PrefixFilter('t1,')
> >
> > Cheers
> >
> >
> > On Fri, Apr 25, 2014 at 3:19 AM, David Koch <og...@googlemail.com>
> wrote:
> >
> > > Hi, cool - thanks, it works nicely - taking this even further. Is
> there a
> > > way to get only the "plain text" region name - for instance, doing a
> > plain
> > > scan results in something like:
> > >
> > > my_table,\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\x
> > > column=info:regioninfo, timestamp=1398333985425, value={NAME =>
> > >
> > >
> >
> 'my_table,\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\xBC\x0E,1392713217395.09326af68a5973257f8d88155
> > > BC\x0E,1392713217395.09326af68a5973257f8d881552f809cd. 2f809cd.',
> > > STARTKEY => '\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\xBC\x0E',
> > ENDKEY
> > > => '\xBF\xB9\x92\xA4N\x11C\x8E\x9A$a\xEF\xB2\xB7R\xC6', ENCODED =>
> > > 09326af68a5973
> > >
> > > 257f8d881552f809cd,}
> > >
> > > and all I require is "09326af68a5973257f8d881552f809cd" - which I
> believe
> > > is the HFile name. I currently extract this using some ugly stuff on
> the
> > > command line on the hbase shell output.
> > >
> > > Thanks,
> > >
> > > /David
> > >
> > >
> > >
> > > On Fri, Apr 25, 2014 at 12:05 PM, Bharath Vissapragada <
> > > bharathv@cloudera.com> wrote:
> > >
> > > > You can use a prefix filter something like as follows
> > > >
> > > > hbase(main):013:0> scan '.META.', {FILTER => "PrefixFilter('t1')"}
> > > >
> > > > Gives all the rows which corresponds to table 't1' as all the row
> keys
> > > > start with "t1".
> > > >
> > > >
> > > > On Fri, Apr 25, 2014 at 3:19 PM, David Koch <og...@googlemail.com>
> > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > How do I get all rows from .META. pertaining to a specific table? I
> > > don't
> > > > > want to scan all of .META. and then do a grep on the output since
> > > .META.
> > > > is
> > > > > quite large but I don't know how to use HBase filters from hbase
> > shell.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > /David
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Bharath Vissapragada
> > > > <http://www.cloudera.com>
> > > >
> > >
> >
>
Re: Getting all rows from .META. for a specific table.
Posted by Samir Ahmic <ah...@gmail.com>.
Hi David,
Here is quick bash script that will extract encoded region name.
#!/bin/bash
table="$1"
echo "scan '.META.', {FILTER=>\"PrefixFilter('$table')\"}" > cmd_file
hbase shell < cmd_file | grep $table |awk '{print $1}' | cut -d "," -f3 |
cut -d "." -f2 |tail -n+2 | uniq
Regards
On Fri, Apr 25, 2014 at 3:21 PM, Ted Yu <yu...@gmail.com> wrote:
> Minor comment about usage of the PrefixFilter, if there is table 't11' in
> your cluster, rows for 't11' would also be returned.
>
> You can specify PrefixFilter('t1,')
>
> Cheers
>
>
> On Fri, Apr 25, 2014 at 3:19 AM, David Koch <og...@googlemail.com> wrote:
>
> > Hi, cool - thanks, it works nicely - taking this even further. Is there a
> > way to get only the "plain text" region name - for instance, doing a
> plain
> > scan results in something like:
> >
> > my_table,\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\x
> > column=info:regioninfo, timestamp=1398333985425, value={NAME =>
> >
> >
> 'my_table,\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\xBC\x0E,1392713217395.09326af68a5973257f8d88155
> > BC\x0E,1392713217395.09326af68a5973257f8d881552f809cd. 2f809cd.',
> > STARTKEY => '\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\xBC\x0E',
> ENDKEY
> > => '\xBF\xB9\x92\xA4N\x11C\x8E\x9A$a\xEF\xB2\xB7R\xC6', ENCODED =>
> > 09326af68a5973
> >
> > 257f8d881552f809cd,}
> >
> > and all I require is "09326af68a5973257f8d881552f809cd" - which I believe
> > is the HFile name. I currently extract this using some ugly stuff on the
> > command line on the hbase shell output.
> >
> > Thanks,
> >
> > /David
> >
> >
> >
> > On Fri, Apr 25, 2014 at 12:05 PM, Bharath Vissapragada <
> > bharathv@cloudera.com> wrote:
> >
> > > You can use a prefix filter something like as follows
> > >
> > > hbase(main):013:0> scan '.META.', {FILTER => "PrefixFilter('t1')"}
> > >
> > > Gives all the rows which corresponds to table 't1' as all the row keys
> > > start with "t1".
> > >
> > >
> > > On Fri, Apr 25, 2014 at 3:19 PM, David Koch <og...@googlemail.com>
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > How do I get all rows from .META. pertaining to a specific table? I
> > don't
> > > > want to scan all of .META. and then do a grep on the output since
> > .META.
> > > is
> > > > quite large but I don't know how to use HBase filters from hbase
> shell.
> > > >
> > > > Thanks,
> > > >
> > > > /David
> > > >
> > >
> > >
> > >
> > > --
> > > Bharath Vissapragada
> > > <http://www.cloudera.com>
> > >
> >
>
Re: Getting all rows from .META. for a specific table.
Posted by Ted Yu <yu...@gmail.com>.
Minor comment about usage of the PrefixFilter, if there is table 't11' in
your cluster, rows for 't11' would also be returned.
You can specify PrefixFilter('t1,')
Cheers
On Fri, Apr 25, 2014 at 3:19 AM, David Koch <og...@googlemail.com> wrote:
> Hi, cool - thanks, it works nicely - taking this even further. Is there a
> way to get only the "plain text" region name - for instance, doing a plain
> scan results in something like:
>
> my_table,\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\x
> column=info:regioninfo, timestamp=1398333985425, value={NAME =>
>
> 'my_table,\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\xBC\x0E,1392713217395.09326af68a5973257f8d88155
> BC\x0E,1392713217395.09326af68a5973257f8d881552f809cd. 2f809cd.',
> STARTKEY => '\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\xBC\x0E', ENDKEY
> => '\xBF\xB9\x92\xA4N\x11C\x8E\x9A$a\xEF\xB2\xB7R\xC6', ENCODED =>
> 09326af68a5973
>
> 257f8d881552f809cd,}
>
> and all I require is "09326af68a5973257f8d881552f809cd" - which I believe
> is the HFile name. I currently extract this using some ugly stuff on the
> command line on the hbase shell output.
>
> Thanks,
>
> /David
>
>
>
> On Fri, Apr 25, 2014 at 12:05 PM, Bharath Vissapragada <
> bharathv@cloudera.com> wrote:
>
> > You can use a prefix filter something like as follows
> >
> > hbase(main):013:0> scan '.META.', {FILTER => "PrefixFilter('t1')"}
> >
> > Gives all the rows which corresponds to table 't1' as all the row keys
> > start with "t1".
> >
> >
> > On Fri, Apr 25, 2014 at 3:19 PM, David Koch <og...@googlemail.com>
> wrote:
> >
> > > Hi,
> > >
> > > How do I get all rows from .META. pertaining to a specific table? I
> don't
> > > want to scan all of .META. and then do a grep on the output since
> .META.
> > is
> > > quite large but I don't know how to use HBase filters from hbase shell.
> > >
> > > Thanks,
> > >
> > > /David
> > >
> >
> >
> >
> > --
> > Bharath Vissapragada
> > <http://www.cloudera.com>
> >
>
Re: Getting all rows from .META. for a specific table.
Posted by Ted Yu <yu...@gmail.com>.
Some post processing is needed to get the encoded region name.
Cheers
On Apr 25, 2014, at 3:19 AM, David Koch <og...@googlemail.com> wrote:
> Hi, cool - thanks, it works nicely - taking this even further. Is there a
> way to get only the "plain text" region name - for instance, doing a plain
> scan results in something like:
>
> my_table,\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\x
> column=info:regioninfo, timestamp=1398333985425, value={NAME =>
> 'my_table,\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\xBC\x0E,1392713217395.09326af68a5973257f8d88155
> BC\x0E,1392713217395.09326af68a5973257f8d881552f809cd. 2f809cd.',
> STARTKEY => '\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\xBC\x0E', ENDKEY
> => '\xBF\xB9\x92\xA4N\x11C\x8E\x9A$a\xEF\xB2\xB7R\xC6', ENCODED =>
> 09326af68a5973
>
> 257f8d881552f809cd,}
>
> and all I require is "09326af68a5973257f8d881552f809cd" - which I believe
> is the HFile name. I currently extract this using some ugly stuff on the
> command line on the hbase shell output.
>
> Thanks,
>
> /David
>
>
>
> On Fri, Apr 25, 2014 at 12:05 PM, Bharath Vissapragada <
> bharathv@cloudera.com> wrote:
>
>> You can use a prefix filter something like as follows
>>
>> hbase(main):013:0> scan '.META.', {FILTER => "PrefixFilter('t1')"}
>>
>> Gives all the rows which corresponds to table 't1' as all the row keys
>> start with "t1".
>>
>>
>> On Fri, Apr 25, 2014 at 3:19 PM, David Koch <og...@googlemail.com> wrote:
>>
>>> Hi,
>>>
>>> How do I get all rows from .META. pertaining to a specific table? I don't
>>> want to scan all of .META. and then do a grep on the output since .META.
>> is
>>> quite large but I don't know how to use HBase filters from hbase shell.
>>>
>>> Thanks,
>>>
>>> /David
>>
>>
>>
>> --
>> Bharath Vissapragada
>> <http://www.cloudera.com>
>>
Re: Getting all rows from .META. for a specific table.
Posted by David Koch <og...@googlemail.com>.
Hi, cool - thanks, it works nicely - taking this even further. Is there a
way to get only the "plain text" region name - for instance, doing a plain
scan results in something like:
my_table,\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\x
column=info:regioninfo, timestamp=1398333985425, value={NAME =>
'my_table,\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\xBC\x0E,1392713217395.09326af68a5973257f8d88155
BC\x0E,1392713217395.09326af68a5973257f8d881552f809cd. 2f809cd.',
STARTKEY => '\xA0\x03\x93\xC7\x81aBb\x8Cr\x1D\xB0\x93\x9D\xBC\x0E', ENDKEY
=> '\xBF\xB9\x92\xA4N\x11C\x8E\x9A$a\xEF\xB2\xB7R\xC6', ENCODED =>
09326af68a5973
257f8d881552f809cd,}
and all I require is "09326af68a5973257f8d881552f809cd" - which I believe
is the HFile name. I currently extract this using some ugly stuff on the
command line on the hbase shell output.
Thanks,
/David
On Fri, Apr 25, 2014 at 12:05 PM, Bharath Vissapragada <
bharathv@cloudera.com> wrote:
> You can use a prefix filter something like as follows
>
> hbase(main):013:0> scan '.META.', {FILTER => "PrefixFilter('t1')"}
>
> Gives all the rows which corresponds to table 't1' as all the row keys
> start with "t1".
>
>
> On Fri, Apr 25, 2014 at 3:19 PM, David Koch <og...@googlemail.com> wrote:
>
> > Hi,
> >
> > How do I get all rows from .META. pertaining to a specific table? I don't
> > want to scan all of .META. and then do a grep on the output since .META.
> is
> > quite large but I don't know how to use HBase filters from hbase shell.
> >
> > Thanks,
> >
> > /David
> >
>
>
>
> --
> Bharath Vissapragada
> <http://www.cloudera.com>
>
Re: Getting all rows from .META. for a specific table.
Posted by Bharath Vissapragada <bh...@cloudera.com>.
You can use a prefix filter something like as follows
hbase(main):013:0> scan '.META.', {FILTER => "PrefixFilter('t1')"}
Gives all the rows which corresponds to table 't1' as all the row keys
start with "t1".
On Fri, Apr 25, 2014 at 3:19 PM, David Koch <og...@googlemail.com> wrote:
> Hi,
>
> How do I get all rows from .META. pertaining to a specific table? I don't
> want to scan all of .META. and then do a grep on the output since .META. is
> quite large but I don't know how to use HBase filters from hbase shell.
>
> Thanks,
>
> /David
>
--
Bharath Vissapragada
<http://www.cloudera.com>
Re: Getting all rows from .META. for a specific table.
Posted by Samir Ahmic <ah...@gmail.com>.
Hi David,
You may try this in hbase shell:
scan '.META.', {FILTER=>"PrefixFilter('table_name')"}
this will scan all regions in .META. table with table_name prefix.
Cheers:)
Samir
On Fri, Apr 25, 2014 at 11:49 AM, David Koch <og...@googlemail.com> wrote:
> Hi,
>
> How do I get all rows from .META. pertaining to a specific table? I don't
> want to scan all of .META. and then do a grep on the output since .META. is
> quite large but I don't know how to use HBase filters from hbase shell.
>
> Thanks,
>
> /David
>