You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by David Koch <og...@googlemail.com> on 2012/08/11 17:38:46 UTC
Printing integers in the Hbase shell
Hello,
I have a table whose qualifiers are always integer values. They are show in
the shell in \x notation. Is there a way to have the shell show the decimal
representation?
Thank you,
/David
Re: Printing integers in the Hbase shell
Posted by Stack <st...@duboce.net>.
On Mon, Aug 13, 2012 at 9:33 AM, Stack <st...@duboce.net> wrote:
> On Mon, Aug 13, 2012 at 5:08 PM, Jacques <wh...@gmail.com> wrote:
>> I was thinking that an easier way might even be to just add the conversion
>> capability at the ruby shell level. Something like the following where you
>> can give a third qualifier that describes how you want it interpreted.
>>
>> get|scan 'table1', {COLUMNS => ['fam:qual1:toInt', 'fam:qual2:toUTF8',
>> 'fam:qual3:c(com.example.MySpecialConverter)']}
>>
>> Pushing the schema into the server side seems like a much bigger task...
>>
>
> We should do that too....
>
I made a noob feature here: https://issues.apache.org/jira/browse/HBASE-6592
St.Ack
Re: Printing integers in the Hbase shell
Posted by Stack <st...@duboce.net>.
On Mon, Aug 13, 2012 at 5:08 PM, Jacques <wh...@gmail.com> wrote:
> I was thinking that an easier way might even be to just add the conversion
> capability at the ruby shell level. Something like the following where you
> can give a third qualifier that describes how you want it interpreted.
>
> get|scan 'table1', {COLUMNS => ['fam:qual1:toInt', 'fam:qual2:toUTF8',
> 'fam:qual3:c(com.example.MySpecialConverter)']}
>
> Pushing the schema into the server side seems like a much bigger task...
>
We should do that too....
St.Ack
Re: Printing integers in the Hbase shell
Posted by Jacques <wh...@gmail.com>.
I was thinking that an easier way might even be to just add the conversion
capability at the ruby shell level. Something like the following where you
can give a third qualifier that describes how you want it interpreted.
get|scan 'table1', {COLUMNS => ['fam:qual1:toInt', 'fam:qual2:toUTF8',
'fam:qual3:c(com.example.MySpecialConverter)']}
Pushing the schema into the server side seems like a much bigger task...
Jacques
On Mon, Aug 13, 2012 at 1:15 AM, Stack <st...@duboce.net> wrote:
> On Sun, Aug 12, 2012 at 12:52 PM, David Koch <og...@googlemail.com>
> wrote:
> > Hi Anil,
> >
> > Thank you for your advice.
> >
>
> We don't have a native column typing metadata facility in HBase
> currently and so there is nothing for the shell to leverage undoing
> the bytes returned in a scan. HBase in this case just does the
> lowest common denominator toStringsBinary which will escape the
> non-printables (An exception is what the shell does when you scan the
> .META. Here the column types are 'known' and so they are interpreted
> appropriately).
>
> We do have a metadata structure for a column family,
> HColumnDescriptor. It has a Map into which arbitrary keyvalues can be
> stuffed. It shouldn't be too hard coming up w/ a convention for
> typing. The shell could consult the HColumnDescriptor before
> outputting a cell value to see if the HCD had info on how to format
> it.
>
> St.Ack
>
Re: Printing integers in the Hbase shell
Posted by Stack <st...@duboce.net>.
On Sun, Aug 12, 2012 at 12:52 PM, David Koch <og...@googlemail.com> wrote:
> Hi Anil,
>
> Thank you for your advice.
>
We don't have a native column typing metadata facility in HBase
currently and so there is nothing for the shell to leverage undoing
the bytes returned in a scan. HBase in this case just does the
lowest common denominator toStringsBinary which will escape the
non-printables (An exception is what the shell does when you scan the
.META. Here the column types are 'known' and so they are interpreted
appropriately).
We do have a metadata structure for a column family,
HColumnDescriptor. It has a Map into which arbitrary keyvalues can be
stuffed. It shouldn't be too hard coming up w/ a convention for
typing. The shell could consult the HColumnDescriptor before
outputting a cell value to see if the HCD had info on how to format
it.
St.Ack
Re: Printing integers in the Hbase shell
Posted by David Koch <og...@googlemail.com>.
Hi Anil,
Thank you for your advice.
On Sat, Aug 11, 2012 at 10:12 PM, anil gupta <an...@gmail.com> wrote:
> Hi David,
>
> As i understand that you want to print the Integer values as Strings in
> HBase shell. There are two ways to do it:
> 1. You can write a ruby script to interpret the value as bytes. This might
> give you some pointers to do stuff in Hbase shell:
>
> http://stackoverflow.com/questions/7256100/how-to-scan-hbase-from-hbase-shell-using-filter
> I dont know anything about Ruby.
>
> 2. You can write a java program to interpret the row/columns as you want.
> If you write your java program then you will need to put the jar in
> classpath of HBase, invoke HBase shell and then use the class to scan the
> table. Here is a sample snippet from a class i use myself: Here i am
> trying to print as Double as a String in HBase shell :--
>
> package com.intuit.ihub.hbase.poc.filters;
> import java.io.IOException;
> import java.text.ParseException;
> import java.text.SimpleDateFormat;
> public class MyRowKeyRangeFilter1 {
>
> static final long TIME_MAX= 4102473600000L; // Epoch time at
> "01/01/2100 00:00:00"
> private static SimpleDateFormat sdf = new SimpleDateFormat ("yyyy-MM-dd
> HH:mm:ss");
>
> public MyRowKeyRangeFilter1(String tableName, String merchantId, String
> startDate, String endDate) throws IOException
> {
> Configuration conf = HBaseConfiguration.create();
> HTable table = new HTable(conf, tableName);
> scanTable(table, merchantId, startDate, endDate);
> }
>
> private void scanTable(HTable table, String merchantId, String endDate,
> String startDate) {
> long starttime = System.currentTimeMillis();
> Scan scan = new Scan();
> // endDate is the startRow because the data is stored in
> reverse chronological order.
> scan.setStartRow(getReverseTimestamp(merchantId,endDate));
> scan.setStopRow(getReverseTimestamp(merchantId,startDate));
> * ResultScanner scanner = null;
> try {
> scanner = table.getScanner(scan);
> } catch (IOException e) {
> System.out.println("Unable to get the scanner for
> filter.");
> }
> System.out.println("===============Results of
> scan===============");
>
> for (Result result : scanner) {
> for (KeyValue kv : result.raw()) {
> if(Bytes.toString(kv.getQualifier()).equals("amt"))
> {
> System.out.println("Printing the value for amt");
> System.out.println("KV: " + kv + ", Value: " +
> Bytes.toDouble(kv.getValue()));
> }
> else
> {
> System.out.println("KV: " + kv + ", Value: " +
> Bytes.toString(kv.getValue()));
> }
> }
> }*
> scanner.close();
> System.out.println("===============SCAN
> COMPLETED===============");
> System.out.println("TIme Taken:"+ (System.currentTimeMillis() -
> starttime));
> //
>
> System.out.println(Bytes.toString(getReverseTimestamp(merchantId,endDate)));
> //
>
> System.out.println(Bytes.toString(getReverseTimestamp(merchantId,startDate)));
> }
> }
> *
> If you are invoking a class in HBase shell then you need to specify:
> <fullclassname>.new(<Constructor args>)
> *
> HTH,
> Anil
>
> On Sat, Aug 11, 2012 at 9:30 AM, Ioakim Perros <im...@gmail.com> wrote:
>
> > I see your point - but I thought it was necessary only for debugging
> > purposes - I use this conversion for this reason. If anyone else is aware
> > of a more efficient way, please answer.
> >
> > Regards,
> > Ioakim
> >
> >
> > On 08/11/2012 07:26 PM, David Koch wrote:
> >
> >> Hello Ioakim,
> >>
> >> Yes, that would work but sacrificing performance by doing int/String/int
> >> conversions and also space just to be able to inspect the odd row from
> >> shell is not worth it :-/
> >>
> >> /David
> >>
> >> On Sat, Aug 11, 2012 at 5:41 PM, Ioakim Perros <im...@gmail.com>
> >> wrote:
> >>
> >> On 08/11/2012 06:38 PM, David Koch wrote:
> >>>
> >>> Hello,
> >>>>
> >>>> I have a table whose qualifiers are always integer values. They are
> show
> >>>> in
> >>>> the shell in \x notation. Is there a way to have the shell show the
> >>>> decimal
> >>>> representation?
> >>>>
> >>>> Thank you,
> >>>>
> >>>> /David
> >>>>
> >>>> Hello David,
> >>>>
> >>> You could take the String.valueOf(yourInt) and store the
> >>> Bytes.toBytes(yourString) representation of this amount as column
> >>> qualifier.
> >>>
> >>> Regards,
> >>> Ioakim
> >>>
> >>>
> >
>
>
> --
> Thanks & Regards,
> Anil Gupta
>
Re: Printing integers in the Hbase shell
Posted by anil gupta <an...@gmail.com>.
Hi David,
As i understand that you want to print the Integer values as Strings in
HBase shell. There are two ways to do it:
1. You can write a ruby script to interpret the value as bytes. This might
give you some pointers to do stuff in Hbase shell:
http://stackoverflow.com/questions/7256100/how-to-scan-hbase-from-hbase-shell-using-filter
I dont know anything about Ruby.
2. You can write a java program to interpret the row/columns as you want.
If you write your java program then you will need to put the jar in
classpath of HBase, invoke HBase shell and then use the class to scan the
table. Here is a sample snippet from a class i use myself: Here i am
trying to print as Double as a String in HBase shell :--
package com.intuit.ihub.hbase.poc.filters;
import java.io.IOException;
import java.text.ParseException;
import java.text.SimpleDateFormat;
public class MyRowKeyRangeFilter1 {
static final long TIME_MAX= 4102473600000L; // Epoch time at
"01/01/2100 00:00:00"
private static SimpleDateFormat sdf = new SimpleDateFormat ("yyyy-MM-dd
HH:mm:ss");
public MyRowKeyRangeFilter1(String tableName, String merchantId, String
startDate, String endDate) throws IOException
{
Configuration conf = HBaseConfiguration.create();
HTable table = new HTable(conf, tableName);
scanTable(table, merchantId, startDate, endDate);
}
private void scanTable(HTable table, String merchantId, String endDate,
String startDate) {
long starttime = System.currentTimeMillis();
Scan scan = new Scan();
// endDate is the startRow because the data is stored in
reverse chronological order.
scan.setStartRow(getReverseTimestamp(merchantId,endDate));
scan.setStopRow(getReverseTimestamp(merchantId,startDate));
* ResultScanner scanner = null;
try {
scanner = table.getScanner(scan);
} catch (IOException e) {
System.out.println("Unable to get the scanner for filter.");
}
System.out.println("===============Results of
scan===============");
for (Result result : scanner) {
for (KeyValue kv : result.raw()) {
if(Bytes.toString(kv.getQualifier()).equals("amt"))
{
System.out.println("Printing the value for amt");
System.out.println("KV: " + kv + ", Value: " +
Bytes.toDouble(kv.getValue()));
}
else
{
System.out.println("KV: " + kv + ", Value: " +
Bytes.toString(kv.getValue()));
}
}
}*
scanner.close();
System.out.println("===============SCAN
COMPLETED===============");
System.out.println("TIme Taken:"+ (System.currentTimeMillis() -
starttime));
//
System.out.println(Bytes.toString(getReverseTimestamp(merchantId,endDate)));
//
System.out.println(Bytes.toString(getReverseTimestamp(merchantId,startDate)));
}
}
*
If you are invoking a class in HBase shell then you need to specify:
<fullclassname>.new(<Constructor args>)
*
HTH,
Anil
On Sat, Aug 11, 2012 at 9:30 AM, Ioakim Perros <im...@gmail.com> wrote:
> I see your point - but I thought it was necessary only for debugging
> purposes - I use this conversion for this reason. If anyone else is aware
> of a more efficient way, please answer.
>
> Regards,
> Ioakim
>
>
> On 08/11/2012 07:26 PM, David Koch wrote:
>
>> Hello Ioakim,
>>
>> Yes, that would work but sacrificing performance by doing int/String/int
>> conversions and also space just to be able to inspect the odd row from
>> shell is not worth it :-/
>>
>> /David
>>
>> On Sat, Aug 11, 2012 at 5:41 PM, Ioakim Perros <im...@gmail.com>
>> wrote:
>>
>> On 08/11/2012 06:38 PM, David Koch wrote:
>>>
>>> Hello,
>>>>
>>>> I have a table whose qualifiers are always integer values. They are show
>>>> in
>>>> the shell in \x notation. Is there a way to have the shell show the
>>>> decimal
>>>> representation?
>>>>
>>>> Thank you,
>>>>
>>>> /David
>>>>
>>>> Hello David,
>>>>
>>> You could take the String.valueOf(yourInt) and store the
>>> Bytes.toBytes(yourString) representation of this amount as column
>>> qualifier.
>>>
>>> Regards,
>>> Ioakim
>>>
>>>
>
--
Thanks & Regards,
Anil Gupta
Re: Printing integers in the Hbase shell
Posted by Ioakim Perros <im...@gmail.com>.
I see your point - but I thought it was necessary only for debugging
purposes - I use this conversion for this reason. If anyone else is
aware of a more efficient way, please answer.
Regards,
Ioakim
On 08/11/2012 07:26 PM, David Koch wrote:
> Hello Ioakim,
>
> Yes, that would work but sacrificing performance by doing int/String/int
> conversions and also space just to be able to inspect the odd row from
> shell is not worth it :-/
>
> /David
>
> On Sat, Aug 11, 2012 at 5:41 PM, Ioakim Perros <im...@gmail.com> wrote:
>
>> On 08/11/2012 06:38 PM, David Koch wrote:
>>
>>> Hello,
>>>
>>> I have a table whose qualifiers are always integer values. They are show
>>> in
>>> the shell in \x notation. Is there a way to have the shell show the
>>> decimal
>>> representation?
>>>
>>> Thank you,
>>>
>>> /David
>>>
>>> Hello David,
>> You could take the String.valueOf(yourInt) and store the
>> Bytes.toBytes(yourString) representation of this amount as column qualifier.
>>
>> Regards,
>> Ioakim
>>
Re: Printing integers in the Hbase shell
Posted by David Koch <og...@googlemail.com>.
Hello Ioakim,
Yes, that would work but sacrificing performance by doing int/String/int
conversions and also space just to be able to inspect the odd row from
shell is not worth it :-/
/David
On Sat, Aug 11, 2012 at 5:41 PM, Ioakim Perros <im...@gmail.com> wrote:
> On 08/11/2012 06:38 PM, David Koch wrote:
>
>> Hello,
>>
>> I have a table whose qualifiers are always integer values. They are show
>> in
>> the shell in \x notation. Is there a way to have the shell show the
>> decimal
>> representation?
>>
>> Thank you,
>>
>> /David
>>
>> Hello David,
>
> You could take the String.valueOf(yourInt) and store the
> Bytes.toBytes(yourString) representation of this amount as column qualifier.
>
> Regards,
> Ioakim
>
Re: Printing integers in the Hbase shell
Posted by Ioakim Perros <im...@gmail.com>.
On 08/11/2012 06:38 PM, David Koch wrote:
> Hello,
>
> I have a table whose qualifiers are always integer values. They are show in
> the shell in \x notation. Is there a way to have the shell show the decimal
> representation?
>
> Thank you,
>
> /David
>
Hello David,
You could take the String.valueOf(yourInt) and store the
Bytes.toBytes(yourString) representation of this amount as column qualifier.
Regards,
Ioakim