You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Marcin <ma...@simpletags.org> on 2010/03/19 15:40:17 UTC

Startup issue when big data in.

Hi guys,

is there a way to avoid compacting, flushing and all of this thing on 
startup and perform it while node is running ?

It takes a lot of on startup.


cheers,
/Marcin

Re: Startup issue when big data in.

Posted by Marcin <ma...@simpletags.org>.
Probably that was the reason of course ;-)


thanks for pointing it out.


cheers,
/Marcin


> You have to wait for the flush to finish, of course.
>
> On Fri, Mar 19, 2010 at 11:44 AM, Marcin<ma...@simpletags.org>  wrote:
>    
>> I have done that before plus compact and it didn't help? (I have been using
>> nodeprobe)
>>
>>
>> cheers,
>> /Marcin
>>
>>      
>>> Flush before you kill the process and restart will be much faster.
>>>
>>> On Fri, Mar 19, 2010 at 9:40 AM, Marcin<ma...@simpletags.org>    wrote:
>>>
>>>        
>>>> Hi guys,
>>>>
>>>> is there a way to avoid compacting, flushing and all of this thing on
>>>> startup and perform it while node is running ?
>>>>
>>>> It takes a lot of on startup.
>>>>
>>>>
>>>> cheers,
>>>> /Marcin
>>>>
>>>>
>>>>          
>>
>>      


Re: Startup issue when big data in.

Posted by Jonathan Ellis <jb...@gmail.com>.
You have to wait for the flush to finish, of course.

On Fri, Mar 19, 2010 at 11:44 AM, Marcin <ma...@simpletags.org> wrote:
> I have done that before plus compact and it didn't help? (I have been using
> nodeprobe)
>
>
> cheers,
> /Marcin
>
>> Flush before you kill the process and restart will be much faster.
>>
>> On Fri, Mar 19, 2010 at 9:40 AM, Marcin<ma...@simpletags.org>  wrote:
>>
>>>
>>> Hi guys,
>>>
>>> is there a way to avoid compacting, flushing and all of this thing on
>>> startup and perform it while node is running ?
>>>
>>> It takes a lot of on startup.
>>>
>>>
>>> cheers,
>>> /Marcin
>>>
>>>
>
>

Re: Startup issue when big data in.

Posted by Marcin <ma...@simpletags.org>.
I have done that before plus compact and it didn't help? (I have been 
using nodeprobe)


cheers,
/Marcin

> Flush before you kill the process and restart will be much faster.
>
> On Fri, Mar 19, 2010 at 9:40 AM, Marcin<ma...@simpletags.org>  wrote:
>    
>> Hi guys,
>>
>> is there a way to avoid compacting, flushing and all of this thing on
>> startup and perform it while node is running ?
>>
>> It takes a lot of on startup.
>>
>>
>> cheers,
>> /Marcin
>>
>>      


Re: Startup issue when big data in.

Posted by Jonathan Ellis <jb...@gmail.com>.
Flush before you kill the process and restart will be much faster.

On Fri, Mar 19, 2010 at 9:40 AM, Marcin <ma...@simpletags.org> wrote:
> Hi guys,
>
> is there a way to avoid compacting, flushing and all of this thing on
> startup and perform it while node is running ?
>
> It takes a lot of on startup.
>
>
> cheers,
> /Marcin
>

Re: Startup issue when big data in.

Posted by Tatu Saloranta <ts...@gmail.com>.
On Fri, Mar 19, 2010 at 11:25 AM, Stu Hood <st...@rackspace.com> wrote:
> All write patterns should provide the same performance with Cassandra, since all writes to disk occur sequentially.

Ok that makes sense.

> The only variance might be in the data structure used for the Memtable (a concurrent skip list), but I expect that it is quite stable.
>
> See http://www.mikeperham.com/2010/03/13/cassandra-internals-writing/

Thanks!

-+ Tatu +-

Re: Startup issue when big data in.

Posted by Lenin Gali <ga...@gmail.com>.
Thanx Joe. Great stuff and very encouraging.

Lenin
Sent from my BlackBerry® wireless handheld

-----Original Message-----
From: Joe Stump <jo...@joestump.net>
Date: Sat, 20 Mar 2010 09:27:21 
To: <us...@cassandra.apache.org>
Subject: Re: Startup issue when big data in.


On Mar 20, 2010, at 3:33 AM, Lenin Gali wrote:

> 1.what kind of performance are you getting, how many writes vs reads do you do per min?

Our performance is quite good. Here are some HDD benchmarks I've ran:
http://stu.mp/2009/12/disk-io-and-throughput-benchmarks-on-amazons-ec2.html

> 2. have you used EBS or local disks and 

We use RAID0 ephemeral drives on large instances. XFS for a filesystem. 

> 3. what would you recommend from your experience to those who would like to try out cassandra on EC2?

We have a rack aware strategy specifically for EC2 that we're working with jbellis to get committed. It basically ensures your keys are equally spread across each AZ. So my recommendation is to use ephemeral drives, spread your machines across AZ's, and keep your reads in a given AZ.

> 4. Some one suggested to use Zookeeper for adding locks on top Cassandra? what are pros and cons of doing that?

Locks suck. Avoid them.

--Joe

Re: Startup issue when big data in.

Posted by Joe Stump <jo...@joestump.net>.
On Mar 20, 2010, at 3:33 AM, Lenin Gali wrote:

> 1.what kind of performance are you getting, how many writes vs reads do you do per min?

Our performance is quite good. Here are some HDD benchmarks I've ran:
http://stu.mp/2009/12/disk-io-and-throughput-benchmarks-on-amazons-ec2.html

> 2. have you used EBS or local disks and 

We use RAID0 ephemeral drives on large instances. XFS for a filesystem. 

> 3. what would you recommend from your experience to those who would like to try out cassandra on EC2?

We have a rack aware strategy specifically for EC2 that we're working with jbellis to get committed. It basically ensures your keys are equally spread across each AZ. So my recommendation is to use ephemeral drives, spread your machines across AZ's, and keep your reads in a given AZ.

> 4. Some one suggested to use Zookeeper for adding locks on top Cassandra? what are pros and cons of doing that?

Locks suck. Avoid them.

--Joe

Re: Startup issue when big data in.

Posted by Lenin Gali <ga...@gmail.com>.
Thanks Stu for point the article. The really helped me and also ask the
following questions to user group.

 I am curious about your users EC2 experience, if you can share some details
on

1.what kind of performance are you getting, how many writes vs reads do you
do per min?
2. have you used EBS or local disks and
3. what would you recommend from your experience to those who would like to
try out cassandra on EC2?
4. Some one suggested to use Zookeeper for adding locks on top Cassandra?
what are pros and cons of doing that?

Thanks
Lenin

On Fri, Mar 19, 2010 at 11:25 AM, Stu Hood <st...@rackspace.com> wrote:

> All write patterns should provide the same performance with Cassandra,
> since all writes to disk occur sequentially. The only variance might be in
> the data structure used for the Memtable (a concurrent skip list), but I
> expect that it is quite stable.
>
> See http://www.mikeperham.com/2010/03/13/cassandra-internals-writing/
>
> Thanks,
> Stu
>
> -----Original Message-----
> From: "Tatu Saloranta" <ts...@gmail.com>
> Sent: Friday, March 19, 2010 1:17pm
> To: user@cassandra.apache.org
> Subject: Re: Startup issue when big data in.
>
> On Fri, Mar 19, 2010 at 10:56 AM, Jonathan Ellis <jb...@gmail.com>
> wrote:
> > On Fri, Mar 19, 2010 at 12:52 PM, Tatu Saloranta <ts...@gmail.com>
> wrote:
> >> One sort of related question: given that order of insertions has huge
> >> effects on some stores, like BDB (where inserting in key order is 10x
> >> faster than arbitrary order), would insertion order possibly have
> >> significant effect on Cassandra as well if using such stores?
> >
> > I'm not really interested in pluggable storage systems, so I don't know.
> :)
>
> ... ?
>
> Maybe I should rephrase my question: is it known that certain pattern
> of puts can be significantly faster than others, for same data set? I
> did not mean to imply anything regarding work-arounds or such, just
> that there may be features underlying storage has that would affect
> higher-level performance.
>
> -+ Tatu +-
>
>
>


-- 
twitter: leningali
skype: galilenin
Cell:513.382.3371

Re: Startup issue when big data in.

Posted by Stu Hood <st...@rackspace.com>.
All write patterns should provide the same performance with Cassandra, since all writes to disk occur sequentially. The only variance might be in the data structure used for the Memtable (a concurrent skip list), but I expect that it is quite stable.

See http://www.mikeperham.com/2010/03/13/cassandra-internals-writing/

Thanks,
Stu 

-----Original Message-----
From: "Tatu Saloranta" <ts...@gmail.com>
Sent: Friday, March 19, 2010 1:17pm
To: user@cassandra.apache.org
Subject: Re: Startup issue when big data in.

On Fri, Mar 19, 2010 at 10:56 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> On Fri, Mar 19, 2010 at 12:52 PM, Tatu Saloranta <ts...@gmail.com> wrote:
>> One sort of related question: given that order of insertions has huge
>> effects on some stores, like BDB (where inserting in key order is 10x
>> faster than arbitrary order), would insertion order possibly have
>> significant effect on Cassandra as well if using such stores?
>
> I'm not really interested in pluggable storage systems, so I don't know. :)

... ?

Maybe I should rephrase my question: is it known that certain pattern
of puts can be significantly faster than others, for same data set? I
did not mean to imply anything regarding work-arounds or such, just
that there may be features underlying storage has that would affect
higher-level performance.

-+ Tatu +-



Re: Startup issue when big data in.

Posted by Tatu Saloranta <ts...@gmail.com>.
On Fri, Mar 19, 2010 at 10:56 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> On Fri, Mar 19, 2010 at 12:52 PM, Tatu Saloranta <ts...@gmail.com> wrote:
>> One sort of related question: given that order of insertions has huge
>> effects on some stores, like BDB (where inserting in key order is 10x
>> faster than arbitrary order), would insertion order possibly have
>> significant effect on Cassandra as well if using such stores?
>
> I'm not really interested in pluggable storage systems, so I don't know. :)

... ?

Maybe I should rephrase my question: is it known that certain pattern
of puts can be significantly faster than others, for same data set? I
did not mean to imply anything regarding work-arounds or such, just
that there may be features underlying storage has that would affect
higher-level performance.

-+ Tatu +-

Re: Startup issue when big data in.

Posted by Jonathan Ellis <jb...@gmail.com>.
On Fri, Mar 19, 2010 at 12:52 PM, Tatu Saloranta <ts...@gmail.com> wrote:
> One sort of related question: given that order of insertions has huge
> effects on some stores, like BDB (where inserting in key order is 10x
> faster than arbitrary order), would insertion order possibly have
> significant effect on Cassandra as well if using such stores?

I'm not really interested in pluggable storage systems, so I don't know. :)

-Jonathan

Re: Startup issue when big data in.

Posted by Tatu Saloranta <ts...@gmail.com>.
On Fri, Mar 19, 2010 at 7:40 AM, Marcin <ma...@simpletags.org> wrote:
> Hi guys,
>
> is there a way to avoid compacting, flushing and all of this thing on
> startup and perform it while node is running ?
>
> It takes a lot of on startup.

One sort of related question: given that order of insertions has huge
effects on some stores, like BDB (where inserting in key order is 10x
faster than arbitrary order), would insertion order possibly have
significant effect on Cassandra as well if using such stores? I bring
this up because what we found out when creating local bdbs was that
doing disk-sort before inserting big data sets allowed radically
faster creation of these (read-only) databases.

-+ Tatu +-