You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Serega Sheypak <se...@gmail.com> on 2015/01/14 16:51:55 UTC
Show last 10 (100/1000) events
Hi, I have event-processing system which uses hbase + a pack of tomcat
web-apps as front-end.
tomcat web-apps are similar and used for front-end load-balancing.
tomcat apps write events to hbase.
What is good pattern to show last 10/100/1000 events?
events table schema is:
row_key=user_id
each user_id has 128 versions. So I keep history for the last 128 user
events.
There is no way to get last events, I can get last event only for concrete
user.
I had an idea to create separate table named 'last_events'
and force all tomcats write there copy of event with the same key and set
versions count to 1000.
HBase would automatically delete old events.
Drawbacks are:
1. x2 traffic
2. x2 write ops on hbase to single region
Solution is bad.
Are they any good patterns to resolve such problem. The other option is to
use some kind of memcache or stuff like that.
Re: Show last 10 (100/1000) events
Posted by Serega Sheypak <se...@gmail.com>.
no it would be 10/100/10000
10000 is absolute limit. I understand that simple threadsafe java
collection can handle this.
2015-01-14 22:17 GMT+03:00 Serega Sheypak <se...@gmail.com>:
> Ok, I got it, thanks.
>
> 2015-01-14 19:22 GMT+03:00 Wilm Schumacher <wi...@gmail.com>:
>
>> Will the number of "last" will be much larger than 10 (100/1000)?
>>
>> If not, then I wouldn't bother with a real database after all and would
>> hold the data in RAM.
>>
>> Either:
>> * in an object in your "gateway" to hbase. E.g. simple java Array list
>> in your java server which serves the api to the web servers. This would
>> be super easy and super fast
>>
>> Or:
>> * in-memory-db like redis if you haven't something like above
>>
>> Or you redisgn your datamodel to something timestamp based. Then it's a
>> scan.
>>
>> Best wishes,
>>
>> Wilm
>>
>> Am 14.01.2015 um 16:51 schrieb Serega Sheypak:
>> > Hi, I have event-processing system which uses hbase + a pack of tomcat
>> > web-apps as front-end.
>> > tomcat web-apps are similar and used for front-end load-balancing.
>> > tomcat apps write events to hbase.
>> >
>> > What is good pattern to show last 10/100/1000 events?
>> > events table schema is:
>> > row_key=user_id
>> > each user_id has 128 versions. So I keep history for the last 128 user
>> > events.
>> >
>> > There is no way to get last events, I can get last event only for
>> concrete
>> > user.
>> >
>> > I had an idea to create separate table named 'last_events'
>> > and force all tomcats write there copy of event with the same key and
>> set
>> > versions count to 1000.
>> > HBase would automatically delete old events.
>> > Drawbacks are:
>> > 1. x2 traffic
>> > 2. x2 write ops on hbase to single region
>> >
>> > Solution is bad.
>> > Are they any good patterns to resolve such problem. The other option is
>> to
>> > use some kind of memcache or stuff like that.
>> >
>>
>>
>
Re: Show last 10 (100/1000) events
Posted by Serega Sheypak <se...@gmail.com>.
Ok, I got it, thanks.
2015-01-14 19:22 GMT+03:00 Wilm Schumacher <wi...@gmail.com>:
> Will the number of "last" will be much larger than 10 (100/1000)?
>
> If not, then I wouldn't bother with a real database after all and would
> hold the data in RAM.
>
> Either:
> * in an object in your "gateway" to hbase. E.g. simple java Array list
> in your java server which serves the api to the web servers. This would
> be super easy and super fast
>
> Or:
> * in-memory-db like redis if you haven't something like above
>
> Or you redisgn your datamodel to something timestamp based. Then it's a
> scan.
>
> Best wishes,
>
> Wilm
>
> Am 14.01.2015 um 16:51 schrieb Serega Sheypak:
> > Hi, I have event-processing system which uses hbase + a pack of tomcat
> > web-apps as front-end.
> > tomcat web-apps are similar and used for front-end load-balancing.
> > tomcat apps write events to hbase.
> >
> > What is good pattern to show last 10/100/1000 events?
> > events table schema is:
> > row_key=user_id
> > each user_id has 128 versions. So I keep history for the last 128 user
> > events.
> >
> > There is no way to get last events, I can get last event only for
> concrete
> > user.
> >
> > I had an idea to create separate table named 'last_events'
> > and force all tomcats write there copy of event with the same key and set
> > versions count to 1000.
> > HBase would automatically delete old events.
> > Drawbacks are:
> > 1. x2 traffic
> > 2. x2 write ops on hbase to single region
> >
> > Solution is bad.
> > Are they any good patterns to resolve such problem. The other option is
> to
> > use some kind of memcache or stuff like that.
> >
>
>
Re: Show last 10 (100/1000) events
Posted by Wilm Schumacher <wi...@gmail.com>.
Will the number of "last" will be much larger than 10 (100/1000)?
If not, then I wouldn't bother with a real database after all and would
hold the data in RAM.
Either:
* in an object in your "gateway" to hbase. E.g. simple java Array list
in your java server which serves the api to the web servers. This would
be super easy and super fast
Or:
* in-memory-db like redis if you haven't something like above
Or you redisgn your datamodel to something timestamp based. Then it's a
scan.
Best wishes,
Wilm
Am 14.01.2015 um 16:51 schrieb Serega Sheypak:
> Hi, I have event-processing system which uses hbase + a pack of tomcat
> web-apps as front-end.
> tomcat web-apps are similar and used for front-end load-balancing.
> tomcat apps write events to hbase.
>
> What is good pattern to show last 10/100/1000 events?
> events table schema is:
> row_key=user_id
> each user_id has 128 versions. So I keep history for the last 128 user
> events.
>
> There is no way to get last events, I can get last event only for concrete
> user.
>
> I had an idea to create separate table named 'last_events'
> and force all tomcats write there copy of event with the same key and set
> versions count to 1000.
> HBase would automatically delete old events.
> Drawbacks are:
> 1. x2 traffic
> 2. x2 write ops on hbase to single region
>
> Solution is bad.
> Are they any good patterns to resolve such problem. The other option is to
> use some kind of memcache or stuff like that.
>