You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@cassandra.apache.org by Yang <te...@gmail.com> on 2011/08/19 02:41:14 UTC

help with CommitLog code?

I'm reading the commitLog code since I have some similar logic in my
application code,
so that I could benefit from the same techniques that CommitLog code uses.

I see that
CommitLog.add(RowMutation rowMutation) {
        executor.add(new LogRecordAdder(rowMutation));
}

while executor could be the parallel  BatchCommitLogExecutorService(),
which by default
has 32 writer threads.

I wonder how the CommitLog code is able to avoid writing garbled
result on the file,
since if multiple LogRecordAdder() are all executed at the same time,  they run
currentSegment().write() ====> SequentialWriter.write() , is it
possible that their output will overlap??


Thanks  l lot
Yang

Re: help with CommitLog code?

Posted by Yang <te...@gmail.com>.

thanks Jonathan, found it


    public BatchCommitLogExecutorService(int queueSize)
    {
        queue = new LinkedBlockingQueue<CheaterFutureTask>(queueSize);
        ...........
        appendingThread = new Thread(runnable, "COMMIT-LOG-WRITER");
        appendingThread.start();

    }

On Thu, Aug 18, 2011 at 5:58 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> There is always a single CLES thread whether for Batch or Periodic.
> (For Batch, look for the appendingThread field.)
>
> On Thu, Aug 18, 2011 at 7:41 PM, Yang <te...@gmail.com> wrote:
>> I'm reading the commitLog code since I have some similar logic in my
>> application code,
>> so that I could benefit from the same techniques that CommitLog code uses.
>>
>> I see that
>> CommitLog.add(RowMutation rowMutation) {
>>        executor.add(new LogRecordAdder(rowMutation));
>> }
>>
>> while executor could be the parallel  BatchCommitLogExecutorService(),
>> which by default
>> has 32 writer threads.
>>
>> I wonder how the CommitLog code is able to avoid writing garbled
>> result on the file,
>> since if multiple LogRecordAdder() are all executed at the same time,  they run
>> currentSegment().write() ====> SequentialWriter.write() , is it
>> possible that their output will overlap??
>>
>>
>> Thanks  l lot
>> Yang
>>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>

Re: help with CommitLog code?

Posted by Jonathan Ellis <jb...@gmail.com>.

There is always a single CLES thread whether for Batch or Periodic.
(For Batch, look for the appendingThread field.)

On Thu, Aug 18, 2011 at 7:41 PM, Yang <te...@gmail.com> wrote:
> I'm reading the commitLog code since I have some similar logic in my
> application code,
> so that I could benefit from the same techniques that CommitLog code uses.
>
> I see that
> CommitLog.add(RowMutation rowMutation) {
>        executor.add(new LogRecordAdder(rowMutation));
> }
>
> while executor could be the parallel  BatchCommitLogExecutorService(),
> which by default
> has 32 writer threads.
>
> I wonder how the CommitLog code is able to avoid writing garbled
> result on the file,
> since if multiple LogRecordAdder() are all executed at the same time,  they run
> currentSegment().write() ====> SequentialWriter.write() , is it
> possible that their output will overlap??
>
>
> Thanks  l lot
> Yang
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com