You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Evgeny <er...@cmcrc.com> on 2011/10/31 02:33:24 UTC
Very slow writes in Cassandra
Hello Cassandra users,
I'm newbie in NoSQL and Cassandara in particular. At the moment doing some
benchmarking with Cassandra and experiencing very slow write throughput.
As it is said, Cassandra can perform hundreds of thousands of inserts per
second, however I'm not observing this: 1) when I send 100 thousand inserts
simultaneously via 8 CQL clients, then throughput is ~14470 inserts per
second.
2) when I do the same via 8 Thrift clients, then throughput is ~16300 inserts
per seconds.
I think Cassandra performance can be improved, but I don't know what to tune.
Please take a look at the test conditions below and advise something.
Thank you.
Tests conditions:
1. Cassandra cluster is deployed on three machines, each machine has 8
cores Intel(R) Xeon(R) CPU E5420 @ 2.50GHz, RAM is 16GB,
network speed is 1000Mb/s.
2. The data sample is
set MM[utf8('1:exc_source_algo:20100105000000.000000:ENTER:0')]['order_id'] =
'1.0';
set MM[utf8('1:exc_source_algo:20100105000000.000000:ENTER:0')]['security'] =
'AA1';
set MM[utf8('1:exc_source_algo:20100105000000.000000:ENTER:0')]['price'] =
'47.1';
set MM[utf8('1:exc_source_algo:20100105000000.000000:ENTER:0')]['volume'] =
'300.0';
set MM[utf8('1:exc_source_algo:20100105000000.000000:ENTER:0')]['se'] = '1';
set MM[utf8('2:exc_source_algo:20100105000000.000000:ENTER:0')]['order_id'] =
'2.0';
set MM[utf8('2:exc_source_algo:20100105000000.000000:ENTER:0')]['security'] =
'AA1';
set MM[utf8('2:exc_source_algo:20100105000000.000000:ENTER:0')]['price'] =
'44.89';
set MM[utf8('2:exc_source_algo:20100105000000.000000:ENTER:0')]['volume'] =
'310.0';
set MM[utf8('2:exc_source_algo:20100105000000.000000:ENTER:0')]['se'] = '1';
set MM[utf8('3:exc_source_algo:20100105000000.000000:ENTER:0')]['order_id'] =
'3.0';
set MM[utf8('3:exc_source_algo:20100105000000.000000:ENTER:0')]['security'] =
'AA2';
set MM[utf8('3:exc_source_algo:20100105000000.000000:ENTER:0')]['price'] =
'0.35';
3. Commit log is written on the local hard drive, the data is written on
Lustre.
4. Keyspace description Keyspace: MD: Replication Strategy:
org.apache.cassandra.locator.NetworkTopologyStrategy Durable Writes: true
Options: [datacenter1:1]
Column Families: ColumnFamily: MM Key Validation Class:
org.apache.cassandra.db.marshal.BytesType Default column value validator:
org.apache.cassandra.db.marshal.BytesType Columns sorted by:
org.apache.cassandra.db.marshal.BytesType Row cache size / save period in
seconds: 0.0/0 Key cache size / save period in seconds:200000.0/14400 Memtable
thresholds: 2.3249999999999997/1440/496 (millions of ops/minutes/MB) GC grace
seconds: 864000 Compaction min/max thresholds: 4/32 Read repair chance: 1.0
Replicate on write: true Built indexes: []
Thanks in advance.
Evgeny.
Re: Very slow writes in Cassandra
Posted by Adrian Cockcroft <ad...@gmail.com>.
You are using replication factor of one and the Lustre clustered
filesystem over the network. Not good practice.
Try RF=3 and local disks. Lustre duplicates much of the functionality
of Cassandra, there is no point using both. Make your Lustre server
nodes into Cassandra nodes instead.
Adrian
On Sun, Oct 30, 2011 at 6:33 PM, Evgeny <er...@cmcrc.com> wrote:
> Hello Cassandra users,
>
> I'm newbie in NoSQL and Cassandara in particular. At the moment doing some
> benchmarking with Cassandra and experiencing very slow write throughput.
>
> As it is said, Cassandra can perform hundreds of thousands of inserts per
> second, however I'm not observing this: 1) when I send 100 thousand inserts
> simultaneously via 8 CQL clients, then throughput is ~14470 inserts per
> second.
> 2) when I do the same via 8 Thrift clients, then throughput is ~16300 inserts
> per seconds.
>
> I think Cassandra performance can be improved, but I don't know what to tune.
> Please take a look at the test conditions below and advise something.
> Thank you.
>
> Tests conditions:
>
> 1. Cassandra cluster is deployed on three machines, each machine has 8
> cores Intel(R) Xeon(R) CPU E5420 @ 2.50GHz, RAM is 16GB,
> network speed is 1000Mb/s.
>
> 2. The data sample is
>
> set MM[utf8('1:exc_source_algo:20100105000000.000000:ENTER:0')]['order_id'] =
> '1.0';
> set MM[utf8('1:exc_source_algo:20100105000000.000000:ENTER:0')]['security'] =
> 'AA1';
> set MM[utf8('1:exc_source_algo:20100105000000.000000:ENTER:0')]['price'] =
> '47.1';
> set MM[utf8('1:exc_source_algo:20100105000000.000000:ENTER:0')]['volume'] =
> '300.0';
> set MM[utf8('1:exc_source_algo:20100105000000.000000:ENTER:0')]['se'] = '1';
> set MM[utf8('2:exc_source_algo:20100105000000.000000:ENTER:0')]['order_id'] =
> '2.0';
> set MM[utf8('2:exc_source_algo:20100105000000.000000:ENTER:0')]['security'] =
> 'AA1';
> set MM[utf8('2:exc_source_algo:20100105000000.000000:ENTER:0')]['price'] =
> '44.89';
> set MM[utf8('2:exc_source_algo:20100105000000.000000:ENTER:0')]['volume'] =
> '310.0';
> set MM[utf8('2:exc_source_algo:20100105000000.000000:ENTER:0')]['se'] = '1';
> set MM[utf8('3:exc_source_algo:20100105000000.000000:ENTER:0')]['order_id'] =
> '3.0';
> set MM[utf8('3:exc_source_algo:20100105000000.000000:ENTER:0')]['security'] =
> 'AA2';
> set MM[utf8('3:exc_source_algo:20100105000000.000000:ENTER:0')]['price'] =
> '0.35';
>
> 3. Commit log is written on the local hard drive, the data is written on
> Lustre.
>
> 4. Keyspace description Keyspace: MD: Replication Strategy:
> org.apache.cassandra.locator.NetworkTopologyStrategy Durable Writes: true
> Options: [datacenter1:1]
> Column Families: ColumnFamily: MM Key Validation Class:
> org.apache.cassandra.db.marshal.BytesType Default column value validator:
> org.apache.cassandra.db.marshal.BytesType Columns sorted by:
> org.apache.cassandra.db.marshal.BytesType Row cache size / save period in
> seconds: 0.0/0 Key cache size / save period in seconds:200000.0/14400 Memtable
> thresholds: 2.3249999999999997/1440/496 (millions of ops/minutes/MB) GC grace
> seconds: 864000 Compaction min/max thresholds: 4/32 Read repair chance: 1.0
> Replicate on write: true Built indexes: []
>
> Thanks in advance.
>
> Evgeny.
>
>