HBase for matching engine

[barrymac <ba...@gmail.com>]

Hi folks,

I'm currently investigating ways to scale a matching engine which is
currently based on stored procedures. The kind of matching it does would be
several columns from multiple sources but usually only two. The data
similarity is quite high in most cases so the match numbers are high. The
current approach would need many hours in some cases to run the stored
procedure that does the row matching run. The system has been built so that
the sources are already combined into one database table and the procedure
iterates over this table.

Is a the type of workload that Hbase would be suited to well? 

Thanks in advance

-- 
View this message in context: http://old.nabble.com/HBase-for-matching-engine-tp30250015p30250015.html
Sent from the HBase User mailing list archive at Nabble.com.

Re: HBase for matching engine

[Ted Yu <yu...@gmail.com>]

Can the data in this table be partitioned ?
If so, you can use hbase TableInputFormat to consume it.

On Thu, Nov 18, 2010 at 9:15 AM, barrymac <ba...@gmail.com> wrote:

>
> Hi folks,
>
> I'm currently investigating ways to scale a matching engine which is
> currently based on stored procedures. The kind of matching it does would be
> several columns from multiple sources but usually only two. The data
> similarity is quite high in most cases so the match numbers are high. The
> current approach would need many hours in some cases to run the stored
> procedure that does the row matching run. The system has been built so that
> the sources are already combined into one database table and the procedure
> iterates over this table.
>
> Is a the type of workload that Hbase would be suited to well?
>
> Thanks in advance
>
> --
> View this message in context:
> http://old.nabble.com/HBase-for-matching-engine-tp30250015p30250015.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>