Anyone working on Hbase + twitter ??

[praveenesh kumar <pr...@gmail.com>]

hey guys..!!

I am new to hbase and basically wants hbase to store twitter data ??
Anyone working similar to this kind of problem..
If yes,. I am looking forward to understand how to design Hbase database
schema to store twitter data..!!

Any suggestions ..??

Thanks.
Praveenesh

Re: Anyone working on Hbase + twitter ??

[Jean-Daniel Cryans <jd...@apache.org>]

There are multiple ways of doing, depends on what you are trying to
achieve. You could have separate tables for keeping the followers and
following such as userid_followerid and user_id_followingid so that
scans could quickly give you the data you need. One thing to keep in
mind is that HBase doesn't support multi-row transactions.

Another way is to keep those lists directly in a cell with the user's
row (like in a json), but then you need to retrieve full lists to
update them and put them back in place.

A third way is to have a different family for followers and following,
and then store the id in the qualifier.

Regarding people with a lot of followers, the first solution would
probably scale better.

You might be able to think of more sophisticated designs, but that
really depends on what you need to do (if you do know that).

J-D

On Thu, May 26, 2011 at 9:29 PM, praveenesh kumar <pr...@gmail.com> wrote:
> Actually I was more interested to know how to design schema to store
> followers and following data..
>
> what I read from the tutorials.. I came to this conclusion..
>
> that if I make userid+timestamp as my rowkey..then I can make followers and
> following as column families with followers_id and following_id as index..
>
>  if one person is having 1000 followers and 100 followings only.. then how I
> will be storing it ?
> Can anyone help in designing schema of this type..??
>
> Thanks.
>
>
>
> On Thu, May 26, 2011 at 11:05 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> 1 row per tweet? The row key might be something like user+timestamp.
>>
>> J-D
>>
>> On Thu, May 26, 2011 at 12:02 AM, praveenesh kumar <pr...@gmail.com>
>> wrote:
>> > hey guys..!!
>> >
>> > I am new to hbase and basically wants hbase to store twitter data ??
>> > Anyone working similar to this kind of problem..
>> > If yes,. I am looking forward to understand how to design Hbase database
>> > schema to store twitter data..!!
>> >
>> > Any suggestions ..??
>> >
>> > Thanks.
>> > Praveenesh
>> >
>>
>

Re: Anyone working on Hbase + twitter ??

[praveenesh kumar <pr...@gmail.com>]

Actually I was more interested to know how to design schema to store
followers and following data..

what I read from the tutorials.. I came to this conclusion..

that if I make userid+timestamp as my rowkey..then I can make followers and
following as column families with followers_id and following_id as index..

 if one person is having 1000 followers and 100 followings only.. then how I
will be storing it ?
Can anyone help in designing schema of this type..??

Thanks.



On Thu, May 26, 2011 at 11:05 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:

> 1 row per tweet? The row key might be something like user+timestamp.
>
> J-D
>
> On Thu, May 26, 2011 at 12:02 AM, praveenesh kumar <pr...@gmail.com>
> wrote:
> > hey guys..!!
> >
> > I am new to hbase and basically wants hbase to store twitter data ??
> > Anyone working similar to this kind of problem..
> > If yes,. I am looking forward to understand how to design Hbase database
> > schema to store twitter data..!!
> >
> > Any suggestions ..??
> >
> > Thanks.
> > Praveenesh
> >
>

Re: Anyone working on Hbase + twitter ??

[Jean-Daniel Cryans <jd...@apache.org>]

1 row per tweet? The row key might be something like user+timestamp.

J-D

On Thu, May 26, 2011 at 12:02 AM, praveenesh kumar <pr...@gmail.com> wrote:
> hey guys..!!
>
> I am new to hbase and basically wants hbase to store twitter data ??
> Anyone working similar to this kind of problem..
> If yes,. I am looking forward to understand how to design Hbase database
> schema to store twitter data..!!
>
> Any suggestions ..??
>
> Thanks.
> Praveenesh
>