You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Oliver Plohmann <ol...@objectscape.org> on 2012/08/12 21:24:43 UTC
Loading data on-demand in Cassandra
Hello,
I'm looking a bit into Cassandra to see whether it would be something to
go with for my company. I searched through the Internet, looked through
the FAQs, etc. but there are still some few open questions. Hope I don't
bother anybody with the usual beginner questions ...
Is there a way to do load-on-demand of data in Cassandra? For the time
being, we cannot afford to built up a cluster that holds our 700 GB
SQL-Database in RAM. So we need to be able to load data on-demand from
our relational database. Can this be done in Cassandra? Then there also
needs to be a way to unload data in order to reclaim RAM space. Would be
nice if it were possible to register for an asynchronous notification in
case some value was changed. Can this be done?
Thanks for any answers.
Regards, Oliver
Re: Loading data on-demand in Cassandra
Posted by sal <sa...@yahoo.com>.
Pierre Chalamet <pierre <at> chalamet.net> writes:
>
> Hi,You do not need to have 700 Go of data in RAM. Cassandra is able to store
on disks and query from there if data is not cached in memory. Caches are
maintained by C* by itself but you still have to some configuration.Supposing
you want to store around 800 Go and with a RF=3, you will need at least 6
servers if you want to store all data of your db (keeping max 400 Go per server)
: 800x3/400=6.There is no native implementation of trigger in C*. Anyway, there
is an extension bringing this feature:
https://github.com/hmsonline/cassandra-triggers. This should allow you to be
notified of mutations (ie: not query). Some peoples on this ML are involved in
this, maybe they could help on this.Cheers,- Pierre
> From: Oliver Plohmann <oliver <at> objectscape.org>
>
> Date: Sun, 12 Aug 2012 21:24:43 +0200
> To: <user <at> cassandra.apache.org>
> ReplyTo: user <at> cassandra.apache.org
>
> Subject: Loading data on-demand in Cassandra
>
> Hello,
> I'm looking a bit into Cassandra to
> see whether it would be something to go with for my company. I
> searched through the Internet, looked through the FAQs, etc. but
> there are still some few open questions. Hope I don't bother
> anybody with the usual beginner questions ...
> Is there a way to do load-on-demand
> of data in Cassandra? For the time being, we cannot afford to
> built up a cluster that holds our 700 GB SQL-Database in RAM. So
> we need to be able to load data on-demand from our relational
> database. Can this be done in Cassandra? Then there also needs to
> be a way to unload data in order to reclaim RAM space. Would be
> nice if it were possible to register for an asynchronous
> notification in case some value was changed. Can this be done?
> Thanks for any answers.
> Regards, Oliver
>
I would consider looking into distributed caching technology (ehcache, gemfire)
Re: Loading data on-demand in Cassandra
Posted by Pierre Chalamet <pi...@chalamet.net>.
Hi,
You do not need to have 700 Go of data in RAM. Cassandra is able to store on disks and query from there if data is not cached in memory. Caches are maintained by C* by itself but you still have to some configuration.
Supposing you want to store around 800 Go and with a RF=3, you will need at least 6 servers if you want to store all data of your db (keeping max 400 Go per server) : 800x3/400=6.
There is no native implementation of trigger in C*. Anyway, there is an extension bringing this feature: https://github.com/hmsonline/cassandra-triggers. This should allow you to be notified of mutations (ie: not query). Some peoples on this ML are involved in this, maybe they could help on this.
Cheers,
- Pierre
-----Original Message-----
From: Oliver Plohmann <ol...@objectscape.org>
Date: Sun, 12 Aug 2012 21:24:43
To: <us...@cassandra.apache.org>
Reply-To: user@cassandra.apache.org
Subject: Loading data on-demand in Cassandra
Hello,
I'm looking a bit into Cassandra to see whether it would be something to
go with for my company. I searched through the Internet, looked through
the FAQs, etc. but there are still some few open questions. Hope I don't
bother anybody with the usual beginner questions ...
Is there a way to do load-on-demand of data in Cassandra? For the time
being, we cannot afford to built up a cluster that holds our 700 GB
SQL-Database in RAM. So we need to be able to load data on-demand from
our relational database. Can this be done in Cassandra? Then there also
needs to be a way to unload data in order to reclaim RAM space. Would be
nice if it were possible to register for an asynchronous notification in
case some value was changed. Can this be done?
Thanks for any answers.
Regards, Oliver
Re: Loading data on-demand in Cassandra
Posted by Oliver Plohmann <ol...@objectscape.org>.
Thanks Dave. Does anybody know of a distributed in-memory system that can do this and that supports structured data (e.g. tables)?
/Oliver
Am 12.08.2012 um 21:39 schrieb Dave Brosius <db...@mebigfatguy.com>:
> When data is first written it does remain in memory until that memory is flushed. After the data is only on disk, it remains there until a read for that row-key/column is requested so in essense it's always load on demand.
>
> Currently there is no support for async notifications of changes.
>
>
>
> On 08/12/2012 03:24 PM, Oliver Plohmann wrote:
>>
>> Hello,
>>
>> I'm looking a bit into Cassandra to see whether it would be something to go with for my company. I searched through the Internet, looked through the FAQs, etc. but there are still some few open questions. Hope I don't bother anybody with the usual beginner questions ...
>>
>> Is there a way to do load-on-demand of data in Cassandra? For the time being, we cannot afford to built up a cluster that holds our 700 GB SQL-Database in RAM. So we need to be able to load data on-demand from our relational database. Can this be done in Cassandra? Then there also needs to be a way to unload data in order to reclaim RAM space. Would be nice if it were possible to register for an asynchronous notification in case some value was changed. Can this be done?
>>
>> Thanks for any answers.
>> Regards, Oliver
>>
>
Re: Loading data on-demand in Cassandra
Posted by Dave Brosius <db...@mebigfatguy.com>.
When data is first written it does remain in memory until that memory is
flushed. After the data is only on disk, it remains there until a read
for that row-key/column is requested so in essense it's always load on
demand.
Currently there is no support for async notifications of changes.
On 08/12/2012 03:24 PM, Oliver Plohmann wrote:
>
> Hello,
>
> I'm looking a bit into Cassandra to see whether it would be something
> to go with for my company. I searched through the Internet, looked
> through the FAQs, etc. but there are still some few open questions.
> Hope I don't bother anybody with the usual beginner questions ...
>
> Is there a way to do load-on-demand of data in Cassandra? For the time
> being, we cannot afford to built up a cluster that holds our 700 GB
> SQL-Database in RAM. So we need to be able to load data on-demand from
> our relational database. Can this be done in Cassandra? Then there
> also needs to be a way to unload data in order to reclaim RAM space.
> Would be nice if it were possible to register for an asynchronous
> notification in case some value was changed. Can this be done?
>
> Thanks for any answers.
> Regards, Oliver
>