You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Jonathan Ellis <jb...@gmail.com> on 2012/07/01 09:25:24 UTC
Re: Memtable tuning in 1.0 and higher
On Thu, Jun 28, 2012 at 1:39 PM, Joost van de Wijgerd
<jw...@gmail.com> wrote:
> the currentThoughput is increased even before the data is merged into the
> memtable so it is actually measuring the throughput afaik.
You're right. I've attached a patch to
https://issues.apache.org/jira/browse/CASSANDRA-4399 to fix this.
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: Memtable tuning in 1.0 and higher
Posted by aaron morton <aa...@thelastpickle.com>.
Joost,
Part of Jonathans explanation for flushing every approx 5 minutes was to reduce the size of the commit log, and reduce the replay time.
Even with the patch flushing memtables is necessary at some point to truncate the log. If this is an issue consider disabling durable_writes on the KS to bypass the commit log. Obviously there are some dangers involved, but it sounds like it may be acceptable to you.
Hope that helps.
-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com
On 2/07/2012, at 7:43 PM, Jonathan Ellis wrote:
> I'm afraid not. It's too much change for an oldstable release series,
> and the bulk of the change is to AtomicSortedColumns which doesn't
> exist in 1.0, so even if we wanted to take a "maybe it's okay if we
> release it first in 1.1.3 and then backport" approach it wouldn't
> improve our safety margin since you'd basically need to rewrite the
> patch.
>
> On Sun, Jul 1, 2012 at 6:40 AM, Joost Van De Wijgerd <jw...@gmail.com> wrote:
>> Hi Jonathan,
>>
>> Looks good, any chance of porting this fix to the 1.0 branch?
>>
>> Kind regards
>>
>> Joost
>>
>> Sent from my iPhone
>>
>>
>> On 1 jul. 2012, at 09:25, Jonathan Ellis <jb...@gmail.com> wrote:
>>
>>> On Thu, Jun 28, 2012 at 1:39 PM, Joost van de Wijgerd
>>> <jw...@gmail.com> wrote:
>>>> the currentThoughput is increased even before the data is merged into the
>>>> memtable so it is actually measuring the throughput afaik.
>>>
>>> You're right. I've attached a patch to
>>> https://issues.apache.org/jira/browse/CASSANDRA-4399 to fix this.
>>>
>>> --
>>> Jonathan Ellis
>>> Project Chair, Apache Cassandra
>>> co-founder of DataStax, the source for professional Cassandra support
>>> http://www.datastax.com
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
Re: Memtable tuning in 1.0 and higher
Posted by Jonathan Ellis <jb...@gmail.com>.
I'm afraid not. It's too much change for an oldstable release series,
and the bulk of the change is to AtomicSortedColumns which doesn't
exist in 1.0, so even if we wanted to take a "maybe it's okay if we
release it first in 1.1.3 and then backport" approach it wouldn't
improve our safety margin since you'd basically need to rewrite the
patch.
On Sun, Jul 1, 2012 at 6:40 AM, Joost Van De Wijgerd <jw...@gmail.com> wrote:
> Hi Jonathan,
>
> Looks good, any chance of porting this fix to the 1.0 branch?
>
> Kind regards
>
> Joost
>
> Sent from my iPhone
>
>
> On 1 jul. 2012, at 09:25, Jonathan Ellis <jb...@gmail.com> wrote:
>
>> On Thu, Jun 28, 2012 at 1:39 PM, Joost van de Wijgerd
>> <jw...@gmail.com> wrote:
>>> the currentThoughput is increased even before the data is merged into the
>>> memtable so it is actually measuring the throughput afaik.
>>
>> You're right. I've attached a patch to
>> https://issues.apache.org/jira/browse/CASSANDRA-4399 to fix this.
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: Memtable tuning in 1.0 and higher
Posted by Jonathan Ellis <jb...@gmail.com>.
I'm afraid not. It's too much change for an oldstable release series,
and the bulk of the change is to AtomicSortedColumns which doesn't
exist in 1.0, so even if we wanted to take a "maybe it's okay if we
release it first in 1.1.3 and then backport" approach it wouldn't
improve our safety margin since you'd basically need to rewrite the
patch.
On Sun, Jul 1, 2012 at 6:40 AM, Joost Van De Wijgerd <jw...@gmail.com> wrote:
> Hi Jonathan,
>
> Looks good, any chance of porting this fix to the 1.0 branch?
>
> Kind regards
>
> Joost
>
> Sent from my iPhone
>
>
> On 1 jul. 2012, at 09:25, Jonathan Ellis <jb...@gmail.com> wrote:
>
>> On Thu, Jun 28, 2012 at 1:39 PM, Joost van de Wijgerd
>> <jw...@gmail.com> wrote:
>>> the currentThoughput is increased even before the data is merged into the
>>> memtable so it is actually measuring the throughput afaik.
>>
>> You're right. I've attached a patch to
>> https://issues.apache.org/jira/browse/CASSANDRA-4399 to fix this.
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: Memtable tuning in 1.0 and higher
Posted by Joost Van De Wijgerd <jw...@gmail.com>.
Hi Jonathan,
Looks good, any chance of porting this fix to the 1.0 branch?
Kind regards
Joost
Sent from my iPhone
On 1 jul. 2012, at 09:25, Jonathan Ellis <jb...@gmail.com> wrote:
> On Thu, Jun 28, 2012 at 1:39 PM, Joost van de Wijgerd
> <jw...@gmail.com> wrote:
>> the currentThoughput is increased even before the data is merged into the
>> memtable so it is actually measuring the throughput afaik.
>
> You're right. I've attached a patch to
> https://issues.apache.org/jira/browse/CASSANDRA-4399 to fix this.
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
Re: Memtable tuning in 1.0 and higher
Posted by Joost Van De Wijgerd <jw...@gmail.com>.
Hi Jonathan,
Looks good, any chance of porting this fix to the 1.0 branch?
Kind regards
Joost
Sent from my iPhone
On 1 jul. 2012, at 09:25, Jonathan Ellis <jb...@gmail.com> wrote:
> On Thu, Jun 28, 2012 at 1:39 PM, Joost van de Wijgerd
> <jw...@gmail.com> wrote:
>> the currentThoughput is increased even before the data is merged into the
>> memtable so it is actually measuring the throughput afaik.
>
> You're right. I've attached a patch to
> https://issues.apache.org/jira/browse/CASSANDRA-4399 to fix this.
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com