You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Adam Phelps <am...@opendns.com> on 2012/12/19 20:05:45 UTC
Customizing hbase shell's table formatting
Is there a reasonably straight forward way to customize the shell's
output for a given table? While many of our tables use string based
data, we have some that use serialized data which is useless to view
directly in the shell on the occasions where it'd be useful for
debugging the output of a M/R job.
I'd think there'd be a way to write a object to serialize and
deserialize the keys and values that could be plugged into the shell,
but I'm not quickly finding how I'd do that.
(Note - I know almost nothing about ruby so I could easily be
overlooking something).
- Adam
Re: Customizing hbase shell's table formatting
Posted by Adam Phelps <am...@opendns.com>.
Thanks for the info. I see that info now that I look at the current
version's scan command, its just not in the 0.90.6 we're currently on
but should be updating before long.
This sounds like the formatting is for column only, our rowkeys
themselves are serialized, is there no way to specify a formatter for
the keys?
- Adam
On 12/19/12 1:17 PM, Stack wrote:
> In trunk, scan and get have formatting options. If you do help 'scan', in
> the middle you should see:
>
> "...Besides the default 'toStringBinary' format, 'scan' supports custom
> formatting
> by column. A user can define a FORMATTER by adding it to the column name in
> the scan specification. The FORMATTER can be stipulated:
>
> 1. either as a org.apache.hadoop.hbase.util.Bytes method name (e.g, toInt,
> toString)
> 2. or as a custom class followed by method name: e.g.
> 'c(MyFormatterClass).format'.
>
> Example formatting cf:qualifier1 and cf:qualifier2 both as Integers:
> hbase> scan 't1', {COLUMNS => ['cf:qualifier1:toInt',
> 'cf:qualifier2:c(org.apache.hadoop.hbase.util.Bytes).toInt'] }
>
> Note that you can specify a FORMATTER by column only (cf:qualifer). You
> cannot
> specify a FORMATTER for all columns of a column family...."
>
> Also, if you look in src, you fill
> find hbase-server/src/main/ruby/shell/formatter.rb.
>
> Check it out. See how there are a few formatters therein. Formatters are
> responsible for outputting results.
>
> There is a the base abstract class and then a Console and unimplemented
> xhtml and json formatters.
>
> You used to be able to specify an alternate 'formatter' to use on the
> commmand line.
>
> durruti:hbase stack$ ./bin/hbase shell --help
> Usage: shell [OPTIONS] [SCRIPTFILE [ARGUMENTS]]
>
> --format=OPTION Formatter for outputting results.
> Valid options are: console, html.
> (Default: console)
>
> -d | --debug Set DEBUG log levels.
> -h | --help This help.
>
> My guess is this facility has rotted from lack of use but if you wanted to
> hook it all up again, the ghosts of an alternate formatter might guide you
> some.
>
> Good luck,
> St.Ack
>
>
> On Wed, Dec 19, 2012 at 11:05 AM, Adam Phelps <am...@opendns.com> wrote:
>
>> Is there a reasonably straight forward way to customize the shell's
>> output for a given table? While many of our tables use string based
>> data, we have some that use serialized data which is useless to view
>> directly in the shell on the occasions where it'd be useful for
>> debugging the output of a M/R job.
>>
>> I'd think there'd be a way to write a object to serialize and
>> deserialize the keys and values that could be plugged into the shell,
>> but I'm not quickly finding how I'd do that.
>>
>> (Note - I know almost nothing about ruby so I could easily be
>> overlooking something).
>>
>> - Adam
>>
>
Re: Customizing hbase shell's table formatting
Posted by Stack <st...@duboce.net>.
In trunk, scan and get have formatting options. If you do help 'scan', in
the middle you should see:
"...Besides the default 'toStringBinary' format, 'scan' supports custom
formatting
by column. A user can define a FORMATTER by adding it to the column name in
the scan specification. The FORMATTER can be stipulated:
1. either as a org.apache.hadoop.hbase.util.Bytes method name (e.g, toInt,
toString)
2. or as a custom class followed by method name: e.g.
'c(MyFormatterClass).format'.
Example formatting cf:qualifier1 and cf:qualifier2 both as Integers:
hbase> scan 't1', {COLUMNS => ['cf:qualifier1:toInt',
'cf:qualifier2:c(org.apache.hadoop.hbase.util.Bytes).toInt'] }
Note that you can specify a FORMATTER by column only (cf:qualifer). You
cannot
specify a FORMATTER for all columns of a column family...."
Also, if you look in src, you fill
find hbase-server/src/main/ruby/shell/formatter.rb.
Check it out. See how there are a few formatters therein. Formatters are
responsible for outputting results.
There is a the base abstract class and then a Console and unimplemented
xhtml and json formatters.
You used to be able to specify an alternate 'formatter' to use on the
commmand line.
durruti:hbase stack$ ./bin/hbase shell --help
Usage: shell [OPTIONS] [SCRIPTFILE [ARGUMENTS]]
--format=OPTION Formatter for outputting results.
Valid options are: console, html.
(Default: console)
-d | --debug Set DEBUG log levels.
-h | --help This help.
My guess is this facility has rotted from lack of use but if you wanted to
hook it all up again, the ghosts of an alternate formatter might guide you
some.
Good luck,
St.Ack
On Wed, Dec 19, 2012 at 11:05 AM, Adam Phelps <am...@opendns.com> wrote:
> Is there a reasonably straight forward way to customize the shell's
> output for a given table? While many of our tables use string based
> data, we have some that use serialized data which is useless to view
> directly in the shell on the occasions where it'd be useful for
> debugging the output of a M/R job.
>
> I'd think there'd be a way to write a object to serialize and
> deserialize the keys and values that could be plugged into the shell,
> but I'm not quickly finding how I'd do that.
>
> (Note - I know almost nothing about ruby so I could easily be
> overlooking something).
>
> - Adam
>
Re: Customizing hbase shell's table formatting
Posted by Michael Segel <mi...@hotmail.com>.
Well the shell is written in JRuby which is pretty much Ruby.
That's the best way to enhance the shell.
On Dec 19, 2012, at 2:56 PM, Adam Phelps <am...@opendns.com> wrote:
> Right up there with my ruby skills (ie I've done a very little editing
> of ruby or jruby scripts). I'm fine with the J part of that, but thats
> about it.
>
> - Adam
>
> On 12/19/12 11:17 AM, Michael Segel wrote:
>> How is your JRuby skills?
>>
>>
>> On Dec 19, 2012, at 1:05 PM, Adam Phelps <am...@opendns.com> wrote:
>>
>>> Is there a reasonably straight forward way to customize the shell's
>>> output for a given table? While many of our tables use string based
>>> data, we have some that use serialized data which is useless to view
>>> directly in the shell on the occasions where it'd be useful for
>>> debugging the output of a M/R job.
>>>
>>> I'd think there'd be a way to write a object to serialize and
>>> deserialize the keys and values that could be plugged into the shell,
>>> but I'm not quickly finding how I'd do that.
>>>
>>> (Note - I know almost nothing about ruby so I could easily be
>>> overlooking something).
>>>
>>> - Adam
>>>
>>
>
>
Re: Customizing hbase shell's table formatting
Posted by Adam Phelps <am...@opendns.com>.
Right up there with my ruby skills (ie I've done a very little editing
of ruby or jruby scripts). I'm fine with the J part of that, but thats
about it.
- Adam
On 12/19/12 11:17 AM, Michael Segel wrote:
> How is your JRuby skills?
>
>
> On Dec 19, 2012, at 1:05 PM, Adam Phelps <am...@opendns.com> wrote:
>
>> Is there a reasonably straight forward way to customize the shell's
>> output for a given table? While many of our tables use string based
>> data, we have some that use serialized data which is useless to view
>> directly in the shell on the occasions where it'd be useful for
>> debugging the output of a M/R job.
>>
>> I'd think there'd be a way to write a object to serialize and
>> deserialize the keys and values that could be plugged into the shell,
>> but I'm not quickly finding how I'd do that.
>>
>> (Note - I know almost nothing about ruby so I could easily be
>> overlooking something).
>>
>> - Adam
>>
>
Re: Customizing hbase shell's table formatting
Posted by Michael Segel <mi...@hotmail.com>.
How is your JRuby skills?
On Dec 19, 2012, at 1:05 PM, Adam Phelps <am...@opendns.com> wrote:
> Is there a reasonably straight forward way to customize the shell's
> output for a given table? While many of our tables use string based
> data, we have some that use serialized data which is useless to view
> directly in the shell on the occasions where it'd be useful for
> debugging the output of a M/R job.
>
> I'd think there'd be a way to write a object to serialize and
> deserialize the keys and values that could be plugged into the shell,
> but I'm not quickly finding how I'd do that.
>
> (Note - I know almost nothing about ruby so I could easily be
> overlooking something).
>
> - Adam
>