You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Guillermo Ortiz <ko...@gmail.com> on 2014/06/09 11:02:25 UTC
Delete rowKey in hexadecimal array bytes.
Hi,
I'm generating key with SHA1, as it's a hex representation after generating
the keys, I use Hex.decode to save memory since I could store them in half
space.
I have a MapReduce process which deletes some of these keys, the problem
it's that it's that when I try to delete them, but I don't get it. If I
don't do the parse to Hex, it works.
So, For example, I put the keys in SHA like
b343664e210e7a7abff3625a005e65e2b0d4616 works, but if I parse this key with
Hex.decode to *\xB3CfN!\x0Ezz\xBF\xF3bZ\x00^e\xE2\xB0\ *column=l:dd,
timestamp=1384317115000 .... it doesn't.
I have been checked the code a lot but I think it's right, plus, if I
comments the decode to Hex it works.
Any clue about it? is there any problem with I am trying to??
Re: Delete rowKey in hexadecimal array bytes.
Posted by Ted Yu <yu...@gmail.com>.
Can you pastebin your junit test ?
Thanks
On Jun 10, 2014, at 3:07 AM, gortiz <go...@pragsis.com> wrote:
> I think we are in different points :).
>
> The problem is when the comparation of the keys when I use Hex.decode
>
> I emit keyValues on this way and works, but it spends double of memory to store keys.
>
> rowKey = Bytes.toBytes(DigestUtils.*shaHex*(pOutput.getRow()));
> KeyValue kv =
> new KeyValue(Bytes.toBytes(pOutput.getRow()), family, null, ts, type, null);
>
> If I use for rowKey:
> rowKey = DigestUtils.*sha*(pOutput.getRow());
> it doesn't work, I don't know why, since it's an byte array. But, I coded some Junit test and it never deletes the keys.
>
>
>
>
> On 09/06/14 13:43, Ted Yu wrote:
>> Decode rowkey has timestamp.
>> KeyValue has timestamp field.
>>
>> Do these two timestamps carry the same value ?
>>
>> Cheers
>>
>> On Jun 9, 2014, at 2:02 AM, Guillermo Ortiz <ko...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I'm generating key with SHA1, as it's a hex representation after generating
>>> the keys, I use Hex.decode to save memory since I could store them in half
>>> space.
>>>
>>> I have a MapReduce process which deletes some of these keys, the problem
>>> it's that it's that when I try to delete them, but I don't get it. If I
>>> don't do the parse to Hex, it works.
>>>
>>> So, For example, I put the keys in SHA like
>>> b343664e210e7a7abff3625a005e65e2b0d4616 works, but if I parse this key with
>>> Hex.decode to *\xB3CfN!\x0Ezz\xBF\xF3bZ\x00^e\xE2\xB0\ *column=l:dd,
>>> timestamp=1384317115000 .... it doesn't.
>>>
>>> I have been checked the code a lot but I think it's right, plus, if I
>>> comments the decode to Hex it works.
>>>
>>> Any clue about it? is there any problem with I am trying to??
>
>
> --
> *Guillermo Ortiz*
> /Big Data Developer/
>
> Telf.: +34 917 680 490
> Fax: +34 913 833 301
> C/ Manuel Tovar, 49-53 - 28034 Madrid - Spain
>
> _http://www.bidoop.es_
>
Re: Delete rowKey in hexadecimal array bytes.
Posted by gortiz <go...@pragsis.com>.
I found the error in my code, thanks.
On 10/06/14 12:07, gortiz wrote:
> I think we are in different points :).
>
> The problem is when the comparation of the keys when I use Hex.decode
>
> I emit keyValues on this way and works, but it spends double of memory
> to store keys.
>
> rowKey = Bytes.toBytes(DigestUtils.*shaHex*(pOutput.getRow()));
> KeyValue kv =
> new KeyValue(Bytes.toBytes(pOutput.getRow()),
> family, null, ts, type, null);
>
> If I use for rowKey:
> rowKey = DigestUtils.*sha*(pOutput.getRow());
> it doesn't work, I don't know why, since it's an byte array. But, I
> coded some Junit test and it never deletes the keys.
>
>
>
>
> On 09/06/14 13:43, Ted Yu wrote:
>> Decode rowkey has timestamp.
>> KeyValue has timestamp field.
>>
>> Do these two timestamps carry the same value ?
>>
>> Cheers
>>
>> On Jun 9, 2014, at 2:02 AM, Guillermo Ortiz <ko...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I'm generating key with SHA1, as it's a hex representation after
>>> generating
>>> the keys, I use Hex.decode to save memory since I could store them
>>> in half
>>> space.
>>>
>>> I have a MapReduce process which deletes some of these keys, the
>>> problem
>>> it's that it's that when I try to delete them, but I don't get it. If I
>>> don't do the parse to Hex, it works.
>>>
>>> So, For example, I put the keys in SHA like
>>> b343664e210e7a7abff3625a005e65e2b0d4616 works, but if I parse this
>>> key with
>>> Hex.decode to *\xB3CfN!\x0Ezz\xBF\xF3bZ\x00^e\xE2\xB0\ *column=l:dd,
>>> timestamp=1384317115000 .... it doesn't.
>>>
>>> I have been checked the code a lot but I think it's right, plus, if I
>>> comments the decode to Hex it works.
>>>
>>> Any clue about it? is there any problem with I am trying to??
>
>
--
*Guillermo Ortiz*
/Big Data Developer/
Telf.: +34 917 680 490
Fax: +34 913 833 301
C/ Manuel Tovar, 49-53 - 28034 Madrid - Spain
_http://www.bidoop.es_
Re: Delete rowKey in hexadecimal array bytes.
Posted by gortiz <go...@pragsis.com>.
I think we are in different points :).
The problem is when the comparation of the keys when I use Hex.decode
I emit keyValues on this way and works, but it spends double of memory
to store keys.
rowKey = Bytes.toBytes(DigestUtils.*shaHex*(pOutput.getRow()));
KeyValue kv =
new KeyValue(Bytes.toBytes(pOutput.getRow()),
family, null, ts, type, null);
If I use for rowKey:
rowKey = DigestUtils.*sha*(pOutput.getRow());
it doesn't work, I don't know why, since it's an byte array. But, I
coded some Junit test and it never deletes the keys.
On 09/06/14 13:43, Ted Yu wrote:
> Decode rowkey has timestamp.
> KeyValue has timestamp field.
>
> Do these two timestamps carry the same value ?
>
> Cheers
>
> On Jun 9, 2014, at 2:02 AM, Guillermo Ortiz <ko...@gmail.com> wrote:
>
>> Hi,
>>
>> I'm generating key with SHA1, as it's a hex representation after generating
>> the keys, I use Hex.decode to save memory since I could store them in half
>> space.
>>
>> I have a MapReduce process which deletes some of these keys, the problem
>> it's that it's that when I try to delete them, but I don't get it. If I
>> don't do the parse to Hex, it works.
>>
>> So, For example, I put the keys in SHA like
>> b343664e210e7a7abff3625a005e65e2b0d4616 works, but if I parse this key with
>> Hex.decode to *\xB3CfN!\x0Ezz\xBF\xF3bZ\x00^e\xE2\xB0\ *column=l:dd,
>> timestamp=1384317115000 .... it doesn't.
>>
>> I have been checked the code a lot but I think it's right, plus, if I
>> comments the decode to Hex it works.
>>
>> Any clue about it? is there any problem with I am trying to??
--
*Guillermo Ortiz*
/Big Data Developer/
Telf.: +34 917 680 490
Fax: +34 913 833 301
C/ Manuel Tovar, 49-53 - 28034 Madrid - Spain
_http://www.bidoop.es_
Re: Delete rowKey in hexadecimal array bytes.
Posted by Ted Yu <yu...@gmail.com>.
Decode rowkey has timestamp.
KeyValue has timestamp field.
Do these two timestamps carry the same value ?
Cheers
On Jun 9, 2014, at 2:02 AM, Guillermo Ortiz <ko...@gmail.com> wrote:
> Hi,
>
> I'm generating key with SHA1, as it's a hex representation after generating
> the keys, I use Hex.decode to save memory since I could store them in half
> space.
>
> I have a MapReduce process which deletes some of these keys, the problem
> it's that it's that when I try to delete them, but I don't get it. If I
> don't do the parse to Hex, it works.
>
> So, For example, I put the keys in SHA like
> b343664e210e7a7abff3625a005e65e2b0d4616 works, but if I parse this key with
> Hex.decode to *\xB3CfN!\x0Ezz\xBF\xF3bZ\x00^e\xE2\xB0\ *column=l:dd,
> timestamp=1384317115000 .... it doesn't.
>
> I have been checked the code a lot but I think it's right, plus, if I
> comments the decode to Hex it works.
>
> Any clue about it? is there any problem with I am trying to??
Re: Delete rowKey in hexadecimal array bytes.
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Hi Guillermo,
Any chance to share your code to see where this can come from?
JM
2014-06-09 5:02 GMT-04:00 Guillermo Ortiz <ko...@gmail.com>:
> Hi,
>
> I'm generating key with SHA1, as it's a hex representation after generating
> the keys, I use Hex.decode to save memory since I could store them in half
> space.
>
> I have a MapReduce process which deletes some of these keys, the problem
> it's that it's that when I try to delete them, but I don't get it. If I
> don't do the parse to Hex, it works.
>
> So, For example, I put the keys in SHA like
> b343664e210e7a7abff3625a005e65e2b0d4616 works, but if I parse this key with
> Hex.decode to *\xB3CfN!\x0Ezz\xBF\xF3bZ\x00^e\xE2\xB0\ *column=l:dd,
> timestamp=1384317115000 .... it doesn't.
>
> I have been checked the code a lot but I think it's right, plus, if I
> comments the decode to Hex it works.
>
> Any clue about it? is there any problem with I am trying to??
>