You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by Alfonso Nishikawa <al...@gmail.com> on 2014/10/27 15:24:09 UTC
Working with arrays in HBase. Do they only grow?
Hi!
I am working with arrays serialized in HBase and I suspect that when
persisting arrays to HBase, you can't delete elements (
https://github.com/apache/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L312).
The array can only grow.
Am I missing something or am I right?
Thanks!
Regards,
Alfonso
Re: Working with arrays in HBase. Do they only grow?
Posted by Alfonso Nishikawa <al...@gmail.com>.
Hi, Renato,
I will open a jira to comment there :)
Thanks!
Regards,
Alfonso
2014-10-27 20:38 GMT+01:00 Renato Marroquín Mogrovejo <
renatoj.marroquin@gmail.com>:
> Hi Alfonso,
>
> You are right, the array can not be shrinked at the moment and yes, it is
> wrong having to write the whole array back if you just want to change a
> single element.
> The column qualifier used for each item is the original index that means if
> your original array had 10 elements, then you'd have 10 column qualfiers to
> store those 10 items. But if then you delete the third element, Gora will
> end up with 9 actual elements (without the third), but there will be a 10th
> element inside HBase :( and when modifying a specific element, we will end
> up rewriting all of the elements :(
> Maybe we should do the same thing, we do with the maps and rewrite them all
> into HBase. At least it will work correctly. Could you open a JIRA for
> that?
> Thanks for pointing this out!
>
>
> Renato M.
>
> 2014-10-27 16:47 GMT+01:00 Alfonso Nishikawa <alfonso.nishikawa@gmail.com
> >:
>
> > Hi, Talat.
> >
> > Thank you for your response.
> >
> > I lost that detail. Never knew that in order to shrink an array we had to
> > nullify first.
> >
> > I don't know how to do in the Pig connector when updating an array... :S
> >
> > So, let me see if I understood. In order to delete an element (shrink) an
> > array you have to:
> >
> > 1 - Define the array as nullable ['null', ...]
> > 2 - Set the array as null in the persistent's field and persist.
> > 3 - Set the new array in the persistent's field and persist.
> >
> > Although it works, I don't know what/why but I find something inherent
> > wrong.
> >
> > Well, at least it is a solution and I can't think about a better way at
> > this moment. Thank you very much :)
> >
> > Regards,
> >
> > Alfonso
> >
> >
> > 2014-10-27 16:28 GMT+01:00 Talat Uyarer <ta...@uyarer.com>:
> >
> > > Hi Alfonso,
> > >
> > > As i know when you want to change array's item you should replace with
> > > null. An then you should set array again. We write over it.
> > >
> > > Talat
> > > On Oct 27, 2014 4:25 PM, "Alfonso Nishikawa" <
> > alfonso.nishikawa@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hi!
> > > >
> > > > I am working with arrays serialized in HBase and I suspect that when
> > > > persisting arrays to HBase, you can't delete elements (
> > > >
> > > >
> > >
> >
> https://github.com/apache/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L312
> > > > ).
> > > > The array can only grow.
> > > > Am I missing something or am I right?
> > > >
> > > > Thanks!
> > > >
> > > > Regards,
> > > >
> > > > Alfonso
> > > >
> > >
> >
>
Re: Working with arrays in HBase. Do they only grow?
Posted by Renato Marroquín Mogrovejo <re...@gmail.com>.
Hi Alfonso,
You are right, the array can not be shrinked at the moment and yes, it is
wrong having to write the whole array back if you just want to change a
single element.
The column qualifier used for each item is the original index that means if
your original array had 10 elements, then you'd have 10 column qualfiers to
store those 10 items. But if then you delete the third element, Gora will
end up with 9 actual elements (without the third), but there will be a 10th
element inside HBase :( and when modifying a specific element, we will end
up rewriting all of the elements :(
Maybe we should do the same thing, we do with the maps and rewrite them all
into HBase. At least it will work correctly. Could you open a JIRA for that?
Thanks for pointing this out!
Renato M.
2014-10-27 16:47 GMT+01:00 Alfonso Nishikawa <al...@gmail.com>:
> Hi, Talat.
>
> Thank you for your response.
>
> I lost that detail. Never knew that in order to shrink an array we had to
> nullify first.
>
> I don't know how to do in the Pig connector when updating an array... :S
>
> So, let me see if I understood. In order to delete an element (shrink) an
> array you have to:
>
> 1 - Define the array as nullable ['null', ...]
> 2 - Set the array as null in the persistent's field and persist.
> 3 - Set the new array in the persistent's field and persist.
>
> Although it works, I don't know what/why but I find something inherent
> wrong.
>
> Well, at least it is a solution and I can't think about a better way at
> this moment. Thank you very much :)
>
> Regards,
>
> Alfonso
>
>
> 2014-10-27 16:28 GMT+01:00 Talat Uyarer <ta...@uyarer.com>:
>
> > Hi Alfonso,
> >
> > As i know when you want to change array's item you should replace with
> > null. An then you should set array again. We write over it.
> >
> > Talat
> > On Oct 27, 2014 4:25 PM, "Alfonso Nishikawa" <
> alfonso.nishikawa@gmail.com
> > >
> > wrote:
> >
> > > Hi!
> > >
> > > I am working with arrays serialized in HBase and I suspect that when
> > > persisting arrays to HBase, you can't delete elements (
> > >
> > >
> >
> https://github.com/apache/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L312
> > > ).
> > > The array can only grow.
> > > Am I missing something or am I right?
> > >
> > > Thanks!
> > >
> > > Regards,
> > >
> > > Alfonso
> > >
> >
>
Re: Working with arrays in HBase. Do they only grow?
Posted by Alfonso Nishikawa <al...@gmail.com>.
Hi, Talat.
Thank you for your response.
I lost that detail. Never knew that in order to shrink an array we had to
nullify first.
I don't know how to do in the Pig connector when updating an array... :S
So, let me see if I understood. In order to delete an element (shrink) an
array you have to:
1 - Define the array as nullable ['null', ...]
2 - Set the array as null in the persistent's field and persist.
3 - Set the new array in the persistent's field and persist.
Although it works, I don't know what/why but I find something inherent
wrong.
Well, at least it is a solution and I can't think about a better way at
this moment. Thank you very much :)
Regards,
Alfonso
2014-10-27 16:28 GMT+01:00 Talat Uyarer <ta...@uyarer.com>:
> Hi Alfonso,
>
> As i know when you want to change array's item you should replace with
> null. An then you should set array again. We write over it.
>
> Talat
> On Oct 27, 2014 4:25 PM, "Alfonso Nishikawa" <alfonso.nishikawa@gmail.com
> >
> wrote:
>
> > Hi!
> >
> > I am working with arrays serialized in HBase and I suspect that when
> > persisting arrays to HBase, you can't delete elements (
> >
> >
> https://github.com/apache/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L312
> > ).
> > The array can only grow.
> > Am I missing something or am I right?
> >
> > Thanks!
> >
> > Regards,
> >
> > Alfonso
> >
>
Re: Working with arrays in HBase. Do they only grow?
Posted by Talat Uyarer <ta...@uyarer.com>.
Hi Alfonso,
As i know when you want to change array's item you should replace with
null. An then you should set array again. We write over it.
Talat
On Oct 27, 2014 4:25 PM, "Alfonso Nishikawa" <al...@gmail.com>
wrote:
> Hi!
>
> I am working with arrays serialized in HBase and I suspect that when
> persisting arrays to HBase, you can't delete elements (
>
> https://github.com/apache/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L312
> ).
> The array can only grow.
> Am I missing something or am I right?
>
> Thanks!
>
> Regards,
>
> Alfonso
>