You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by 郭鹏 <gp...@gmail.com> on 2010/03/20 14:44:17 UTC
why Cassandra 0.5.1 write speed is very slow
Hi:
I'm doing a research on move our data from MySQL to Cassandra 0.5.1
At first, I am doing it in the Windows XP, and read the record from MySQL
then insert it into Cassandra. The write speed is ok, about 18,000 records
per second.
But when I changed it into the 5 Linux Red Hat 5 Server, doing the same
opertation, the write speed in downto 1000 records per second.
Why would this happend?
I am using the default config.
--
Regards
Peng Guo
Re: why Cassandra 0.5.1 write speed is very slow
Posted by 郭鹏 <gp...@gmail.com>.
The replication factor is One,
The consistency level also is One.
在 2010年3月23日 上午10:21，Benjamin Black <b...@b3k.us>写道：
> Maybe I missed this: what replication factor and consistency level are
> you using?
>
> 2010/3/21 郭鹏 <gp...@gmail.com>:
> > Thx, I will try it in the multi-thread mode.
> >
> > What's the best practice in the production env?
> >
> >
> >
> > 在 2010年3月21日 下午12:04，Jonathan Ellis <jb...@gmail.com>写道：
> >>
> >> If you're benchmarking throughput, then you absolutely have to be
> >> multithreaded.
> >>
> >> 2010/3/20 郭鹏 <gp...@gmail.com>:
> >> > You mean I waste lots of time waiting for the write reply?
> >> >
> >> > How could I avoid this problem? Using multi-thread for very write
> >> > operation?
> >> >
> >> > I am totally confused.
> >> >
> >> > Thx.
> >> >
> >> > 在 2010年3月21日 上午11:16，Jonathan Ellis <jb...@gmail.com>写道：
> >> >>
> >> >> time a single-threaded client is spending waiting for a reply, is
> time
> >> >> it's not spending sending another request
> >> >>
> >> >> 2010/3/20 郭鹏 <gp...@gmail.com>:
> >> >> > Good ideal, I will try it with client multithread write.
> >> >> >
> >> >> > But I don't understand the meaning of "d the latency in cluster
> mode
> >> >> > hurts
> >> >> > that much more."
> >> >> >
> >> >> > Thx.
> >> >> >
> >> >> > 2010/3/21 Jonathan Ellis <jb...@gmail.com>
> >> >> >>
> >> >> >> 2010/3/20 郭鹏 <gp...@gmail.com>:
> >> >> >> > :)
> >> >> >> >
> >> >> >> > I just want to know why the write speed is ok in the standalone
> >> >> >> > mode
> >> >> >> > but
> >> >> >> > too
> >> >> >> > slow in the cluster mode.
> >> >> >> >
> >> >> >> > Am I doing something wrong?
> >> >> >>
> >> >> >> Probably, but it's hard to say what.
> >> >> >>
> >> >> >> Maybe you're just not using enough threads in your benchmark and
> the
> >> >> >> latency in cluster mode hurts that much more.
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Regards
> >> >> > Peng Guo
> >> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Regards
> >> > Peng Guo
> >> >
> >
> >
> >
> > --
> > Regards
> > Peng Guo
> >
>
--
Regards
Peng Guo
Re: why Cassandra 0.5.1 write speed is very slow
Posted by Benjamin Black <b...@b3k.us>.
Maybe I missed this: what replication factor and consistency level are
you using?
2010/3/21 郭鹏 <gp...@gmail.com>:
> Thx, I will try it in the multi-thread mode.
>
> What's the best practice in the production env?
>
>
>
> 在 2010年3月21日 下午12:04，Jonathan Ellis <jb...@gmail.com>写道：
>>
>> If you're benchmarking throughput, then you absolutely have to be
>> multithreaded.
>>
>> 2010/3/20 郭鹏 <gp...@gmail.com>:
>> > You mean I waste lots of time waiting for the write reply?
>> >
>> > How could I avoid this problem? Using multi-thread for very write
>> > operation?
>> >
>> > I am totally confused.
>> >
>> > Thx.
>> >
>> > 在 2010年3月21日 上午11:16，Jonathan Ellis <jb...@gmail.com>写道：
>> >>
>> >> time a single-threaded client is spending waiting for a reply, is time
>> >> it's not spending sending another request
>> >>
>> >> 2010/3/20 郭鹏 <gp...@gmail.com>:
>> >> > Good ideal, I will try it with client multithread write.
>> >> >
>> >> > But I don't understand the meaning of "d the latency in cluster mode
>> >> > hurts
>> >> > that much more."
>> >> >
>> >> > Thx.
>> >> >
>> >> > 2010/3/21 Jonathan Ellis <jb...@gmail.com>
>> >> >>
>> >> >> 2010/3/20 郭鹏 <gp...@gmail.com>:
>> >> >> > :)
>> >> >> >
>> >> >> > I just want to know why the write speed is ok in the standalone
>> >> >> > mode
>> >> >> > but
>> >> >> > too
>> >> >> > slow in the cluster mode.
>> >> >> >
>> >> >> > Am I doing something wrong?
>> >> >>
>> >> >> Probably, but it's hard to say what.
>> >> >>
>> >> >> Maybe you're just not using enough threads in your benchmark and the
>> >> >> latency in cluster mode hurts that much more.
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Regards
>> >> > Peng Guo
>> >> >
>> >
>> >
>> >
>> > --
>> > Regards
>> > Peng Guo
>> >
>
>
>
> --
> Regards
> Peng Guo
>
Re: why Cassandra 0.5.1 write speed is very slow
Posted by 郭鹏 <gp...@gmail.com>.
Thx, I will try it in the multi-thread mode.
What's the best practice in the production env?
在 2010年3月21日 下午12:04，Jonathan Ellis <jb...@gmail.com>写道：
> If you're benchmarking throughput, then you absolutely have to be
> multithreaded.
>
> 2010/3/20 郭鹏 <gp...@gmail.com>:
> > You mean I waste lots of time waiting for the write reply?
> >
> > How could I avoid this problem? Using multi-thread for very write
> operation?
> >
> > I am totally confused.
> >
> > Thx.
> >
> > 在 2010年3月21日 上午11:16，Jonathan Ellis <jb...@gmail.com>写道：
> >>
> >> time a single-threaded client is spending waiting for a reply, is time
> >> it's not spending sending another request
> >>
> >> 2010/3/20 郭鹏 <gp...@gmail.com>:
> >> > Good ideal, I will try it with client multithread write.
> >> >
> >> > But I don't understand the meaning of "d the latency in cluster mode
> >> > hurts
> >> > that much more."
> >> >
> >> > Thx.
> >> >
> >> > 2010/3/21 Jonathan Ellis <jb...@gmail.com>
> >> >>
> >> >> 2010/3/20 郭鹏 <gp...@gmail.com>:
> >> >> > :)
> >> >> >
> >> >> > I just want to know why the write speed is ok in the standalone
> mode
> >> >> > but
> >> >> > too
> >> >> > slow in the cluster mode.
> >> >> >
> >> >> > Am I doing something wrong?
> >> >>
> >> >> Probably, but it's hard to say what.
> >> >>
> >> >> Maybe you're just not using enough threads in your benchmark and the
> >> >> latency in cluster mode hurts that much more.
> >> >
> >> >
> >> >
> >> > --
> >> > Regards
> >> > Peng Guo
> >> >
> >
> >
> >
> > --
> > Regards
> > Peng Guo
> >
>
--
Regards
Peng Guo
Re: why Cassandra 0.5.1 write speed is very slow
Posted by Jonathan Ellis <jb...@gmail.com>.
If you're benchmarking throughput, then you absolutely have to be multithreaded.
2010/3/20 郭鹏 <gp...@gmail.com>:
> You mean I waste lots of time waiting for the write reply?
>
> How could I avoid this problem? Using multi-thread for very write operation?
>
> I am totally confused.
>
> Thx.
>
> 在 2010年3月21日 上午11:16，Jonathan Ellis <jb...@gmail.com>写道：
>>
>> time a single-threaded client is spending waiting for a reply, is time
>> it's not spending sending another request
>>
>> 2010/3/20 郭鹏 <gp...@gmail.com>:
>> > Good ideal, I will try it with client multithread write.
>> >
>> > But I don't understand the meaning of "d the latency in cluster mode
>> > hurts
>> > that much more."
>> >
>> > Thx.
>> >
>> > 2010/3/21 Jonathan Ellis <jb...@gmail.com>
>> >>
>> >> 2010/3/20 郭鹏 <gp...@gmail.com>:
>> >> > :)
>> >> >
>> >> > I just want to know why the write speed is ok in the standalone mode
>> >> > but
>> >> > too
>> >> > slow in the cluster mode.
>> >> >
>> >> > Am I doing something wrong?
>> >>
>> >> Probably, but it's hard to say what.
>> >>
>> >> Maybe you're just not using enough threads in your benchmark and the
>> >> latency in cluster mode hurts that much more.
>> >
>> >
>> >
>> > --
>> > Regards
>> > Peng Guo
>> >
>
>
>
> --
> Regards
> Peng Guo
>
Re: why Cassandra 0.5.1 write speed is very slow
Posted by 郭鹏 <gp...@gmail.com>.
You mean I waste lots of time waiting for the write reply?
How could I avoid this problem? Using multi-thread for very write operation?
I am totally confused.
Thx.
在 2010年3月21日 上午11:16，Jonathan Ellis <jb...@gmail.com>写道：
> time a single-threaded client is spending waiting for a reply, is time
> it's not spending sending another request
>
> 2010/3/20 郭鹏 <gp...@gmail.com>:
> > Good ideal, I will try it with client multithread write.
> >
> > But I don't understand the meaning of "d the latency in cluster mode
> hurts
> > that much more."
> >
> > Thx.
> >
> > 2010/3/21 Jonathan Ellis <jb...@gmail.com>
> >>
> >> 2010/3/20 郭鹏 <gp...@gmail.com>:
> >> > :)
> >> >
> >> > I just want to know why the write speed is ok in the standalone mode
> but
> >> > too
> >> > slow in the cluster mode.
> >> >
> >> > Am I doing something wrong?
> >>
> >> Probably, but it's hard to say what.
> >>
> >> Maybe you're just not using enough threads in your benchmark and the
> >> latency in cluster mode hurts that much more.
> >
> >
> >
> > --
> > Regards
> > Peng Guo
> >
>
--
Regards
Peng Guo
Re: why Cassandra 0.5.1 write speed is very slow
Posted by Jonathan Ellis <jb...@gmail.com>.
time a single-threaded client is spending waiting for a reply, is time
it's not spending sending another request
2010/3/20 郭鹏 <gp...@gmail.com>:
> Good ideal, I will try it with client multithread write.
>
> But I don't understand the meaning of "d the latency in cluster mode hurts
> that much more."
>
> Thx.
>
> 2010/3/21 Jonathan Ellis <jb...@gmail.com>
>>
>> 2010/3/20 郭鹏 <gp...@gmail.com>:
>> > :)
>> >
>> > I just want to know why the write speed is ok in the standalone mode but
>> > too
>> > slow in the cluster mode.
>> >
>> > Am I doing something wrong?
>>
>> Probably, but it's hard to say what.
>>
>> Maybe you're just not using enough threads in your benchmark and the
>> latency in cluster mode hurts that much more.
>
>
>
> --
> Regards
> Peng Guo
>
Re: why Cassandra 0.5.1 write speed is very slow
Posted by 郭鹏 <gp...@gmail.com>.
Good ideal, I will try it with client multithread write.
But I don't understand the meaning of "d the latency in cluster mode hurts
that much more."
Thx.
2010/3/21 Jonathan Ellis <jb...@gmail.com>
> 2010/3/20 郭鹏 <gp...@gmail.com>:
> > :)
> >
> > I just want to know why the write speed is ok in the standalone mode but
> too
> > slow in the cluster mode.
> >
> > Am I doing something wrong?
>
> Probably, but it's hard to say what.
>
> Maybe you're just not using enough threads in your benchmark and the
> latency in cluster mode hurts that much more.
>
--
Regards
Peng Guo
Re: why Cassandra 0.5.1 write speed is very slow
Posted by Jonathan Ellis <jb...@gmail.com>.
2010/3/20 郭鹏 <gp...@gmail.com>:
> :)
>
> I just want to know why the write speed is ok in the standalone mode but too
> slow in the cluster mode.
>
> Am I doing something wrong?
Probably, but it's hard to say what.
Maybe you're just not using enough threads in your benchmark and the
latency in cluster mode hurts that much more.
Re: why Cassandra 0.5.1 write speed is very slow
Posted by 郭鹏 <gp...@gmail.com>.
:)
I just want to know why the write speed is ok in the standalone mode but too
slow in the cluster mode.
Am I doing something wrong?
Thx.
Peng Guo
2010/3/21 Jonathan Ellis <jb...@gmail.com>
> Looks like the right question to ask is "what _else_ changed during the
> move?"
>
> On Sat, Mar 20, 2010 at 8:44 AM, 郭鹏 <gp...@gmail.com> wrote:
> > Hi:
> >
> > I'm doing a research on move our data from MySQL to Cassandra 0.5.1
> >
> > At first, I am doing it in the Windows XP, and read the record from MySQL
> > then insert it into Cassandra. The write speed is ok, about 18,000
> records
> > per second.
> >
> > But when I changed it into the 5 Linux Red Hat 5 Server, doing the same
> > opertation, the write speed in downto 1000 records per second.
> >
> > Why would this happend?
> >
> > I am using the default config.
> >
> > --
> > Regards
> > Peng Guo
> >
>
--
Regards
Peng Guo
Re: why Cassandra 0.5.1 write speed is very slow
Posted by Jonathan Ellis <jb...@gmail.com>.
Looks like the right question to ask is "what _else_ changed during the move?"
On Sat, Mar 20, 2010 at 8:44 AM, 郭鹏 <gp...@gmail.com> wrote:
> Hi:
>
> I'm doing a research on move our data from MySQL to Cassandra 0.5.1
>
> At first, I am doing it in the Windows XP, and read the record from MySQL
> then insert it into Cassandra. The write speed is ok, about 18,000 records
> per second.
>
> But when I changed it into the 5 Linux Red Hat 5 Server, doing the same
> opertation, the write speed in downto 1000 records per second.
>
> Why would this happend?
>
> I am using the default config.
>
> --
> Regards
> Peng Guo
>