You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Zhu Han <sc...@gmail.com> on 2011/07/15 04:04:00 UTC
Commit log is not emptied after "nodetool drain"
The deployed version is based on 0.6.13.
After "nodetool drain" is invoked on one of the nodes, the commit log is not
emptied. Is this the expected behavior? If so, how can I rename a column
family on 0.6.x branch?
Here is the log output:
"
INFO [COMMIT-LOG-WRITER] 2011-07-15 00:39:49,541 CommitLogSegment.java (line
50) Creating new commitlog segment
/var/lib/cassandra/commitlog/CommitLog-1310661589541.log
INFO [RMI TCP Connection(8)-202.120.2.16] 2011-07-15 00:39:49,544
StorageService.java (line 391) Node is drained
"
I saw an issue here, but it was reported against 0.8.x branch.
https://issues.apache.org/jira/browse/CASSANDRA-2874
best regards,
韩竹(Zhu Han)
坚果铺子 <https://jianguopuzi.com>, 最简单易用的云存储
同步文件, 分享照片, 文档备份!
Re: Commit log is not emptied after "nodetool drain"
Posted by Jonathan Ellis <jb...@gmail.com>.
You can create a ticket, but (1) the community has pretty much moved
on from maintaining 0.6 and (2) this looks like the kind of
non-critical problem that isn't worth jeopardizing stability to
attempt a fix for.
2011/7/15 Zhu Han <sc...@gmail.com>:
> 2011/7/15 Zhu Han <sc...@gmail.com>
>>
>> 2011/7/15 Jonathan Ellis <jb...@gmail.com>
>>>
>>> If you have non-empty segments post-drain that is a bug. Is it
>>> reproducible?
>>
>> I think it is always reproducible on 0.6.x branch. Here is a simple
>> experiment:
>
> Should I raise an issue ticket on it?
>
>>
>> 1) "bin/nodetool -h localhost"
>>
>> 2) During flush the memtables, we can observe the name of the old commit
>> log from log:
>>>
>>> " INFO [main] 2011-07-15 11:57:46,742 ColumnFamilyStore.java (line 478)
>>> Data has reached its threshold; switching in a fresh Memtable at
>>> CommitLogContext(file='/var/lib/cassandra/commitlog/CommitLog-1310702265959.log',
>>> position=125) "
>>
>> 3) Before the node is drained, new commitlog is created:
>>>
>>> INFO [COMMIT-LOG-WRITER] 2011-07-15 11:58:11,383 CommitLogSegment.java
>>> (line 50) Creating new commitlog segment
>>> /var/lib/cassandra/commitlog/CommitLog-1310702291383.log
>>> INFO [RMI TCP Connection(2)-192.168.1.101] 2011-07-15 11:58:11,413
>>> StorageService.java (line 391) Node is drained
>>
>> 4) After the node is drained and killed, there are still two commit log
>> under the directory
>>>
>>> $ ls -lh /var/lib/cassandra/commitlog/
>>> total 128K
>>> -rw-r--r-- 1 root root 439 2011-07-15 11:57 CommitLog-1310702265959.log
>>> -rw-r--r-- 1 root root 125 2011-07-15 11:58 CommitLog-1310702291383.log
>>
>>
>>
>>>
>>> 2011/7/14 Zhu Han <sc...@gmail.com>:
>>> > Jonathan,
>>> >
>>> > But all the old non-empty log segments are kept on the disk. And
>>> > cassandra
>>> > takes some time to apply the operations from these closed log segments
>>> > after
>>> > restart of the process.
>>> >
>>> > Is it expected?
>>> >
>>> > best regards,
>>> > 韩竹(Zhu Han)
>>> >
>>> > 坚果铺子, 最简单易用的云存储
>>> > 同步文件, 分享照片, 文档备份!
>>> >
>>> >
>>> >
>>> > 2011/7/15 Jonathan Ellis <jb...@gmail.com>
>>> >>
>>> >> It's expected to have a new, empty segment after drain completes.
>>> >>
>>> >> 2011/7/14 Zhu Han <sc...@gmail.com>:
>>> >> > The deployed version is based on 0.6.13.
>>> >> >
>>> >> > After "nodetool drain" is invoked on one of the nodes, the commit
>>> >> > log is
>>> >> > not
>>> >> > emptied. Is this the expected behavior? If so, how can I rename a
>>> >> > column
>>> >> > family on 0.6.x branch?
>>> >> >
>>> >> > Here is the log output:
>>> >> > "
>>> >> > INFO [COMMIT-LOG-WRITER] 2011-07-15 00:39:49,541
>>> >> > CommitLogSegment.java
>>> >> > (line
>>> >> > 50) Creating new commitlog segment
>>> >> > /var/lib/cassandra/commitlog/CommitLog-1310661589541.log
>>> >> > INFO [RMI TCP Connection(8)-202.120.2.16] 2011-07-15 00:39:49,544
>>> >> > StorageService.java (line 391) Node is drained
>>> >> > "
>>> >> >
>>> >> > I saw an issue here, but it was reported against 0.8.x branch.
>>> >> > https://issues.apache.org/jira/browse/CASSANDRA-2874
>>> >> >
>>> >> > best regards,
>>> >> > 韩竹(Zhu Han)
>>> >> >
>>> >> > 坚果铺子, 最简单易用的云存储
>>> >> > 同步文件, 分享照片, 文档备份!
>>> >> >
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Jonathan Ellis
>>> >> Project Chair, Apache Cassandra
>>> >> co-founder of DataStax, the source for professional Cassandra support
>>> >> http://www.datastax.com
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Jonathan Ellis
>>> Project Chair, Apache Cassandra
>>> co-founder of DataStax, the source for professional Cassandra support
>>> http://www.datastax.com
>>
>
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: Commit log is not emptied after "nodetool drain"
Posted by Zhu Han <sc...@gmail.com>.
2011/7/15 Zhu Han <sc...@gmail.com>
>
> 2011/7/15 Jonathan Ellis <jb...@gmail.com>
>
>> If you have non-empty segments post-drain that is a bug. Is it
>> reproducible?
>>
>
> I think it is always reproducible on 0.6.x branch. Here is a simple
> experiment:
>
Should I raise an issue ticket on it?
>
> 1) "bin/nodetool -h localhost"
>
> 2) During flush the memtables, we can observe the name of the old commit
> log from log:
>
>> " INFO [main] 2011-07-15 11:57:46,742 ColumnFamilyStore.java (line 478)
>> Data has reached its threshold; switching in a fresh Memtable at
>> CommitLogContext(file='/var/lib/cassandra/commitlog/CommitLog-1310702265959.log',
>> position=125) "
>>
>
> 3) Before the node is drained, new commitlog is created:
>
>> INFO [COMMIT-LOG-WRITER] 2011-07-15 11:58:11,383 CommitLogSegment.java
>> (line 50) Creating new commitlog segment
>> /var/lib/cassandra/commitlog/CommitLog-1310702291383.log
>> INFO [RMI TCP Connection(2)-192.168.1.101] 2011-07-15 11:58:11,413
>> StorageService.java (line 391) Node is drained
>>
>
> 4) After the node is drained and killed, there are still two commit log
> under the directory
>
>> $ ls -lh /var/lib/cassandra/commitlog/
>> total 128K
>> -rw-r--r-- 1 root root 439 2011-07-15 11:57 CommitLog-1310702265959.log
>> -rw-r--r-- 1 root root 125 2011-07-15 11:58 CommitLog-1310702291383.log
>>
>
>
>
>
>> 2011/7/14 Zhu Han <sc...@gmail.com>:
>> > Jonathan,
>> >
>> > But all the old non-empty log segments are kept on the disk. And
>> cassandra
>> > takes some time to apply the operations from these closed log segments
>> after
>> > restart of the process.
>> >
>> > Is it expected?
>> >
>> > best regards,
>> > 韩竹(Zhu Han)
>> >
>> > 坚果铺子, 最简单易用的云存储
>> > 同步文件, 分享照片, 文档备份!
>> >
>> >
>> >
>> > 2011/7/15 Jonathan Ellis <jb...@gmail.com>
>> >>
>> >> It's expected to have a new, empty segment after drain completes.
>> >>
>> >> 2011/7/14 Zhu Han <sc...@gmail.com>:
>> >> > The deployed version is based on 0.6.13.
>> >> >
>> >> > After "nodetool drain" is invoked on one of the nodes, the commit log
>> is
>> >> > not
>> >> > emptied. Is this the expected behavior? If so, how can I rename a
>> column
>> >> > family on 0.6.x branch?
>> >> >
>> >> > Here is the log output:
>> >> > "
>> >> > INFO [COMMIT-LOG-WRITER] 2011-07-15 00:39:49,541
>> CommitLogSegment.java
>> >> > (line
>> >> > 50) Creating new commitlog segment
>> >> > /var/lib/cassandra/commitlog/CommitLog-1310661589541.log
>> >> > INFO [RMI TCP Connection(8)-202.120.2.16] 2011-07-15 00:39:49,544
>> >> > StorageService.java (line 391) Node is drained
>> >> > "
>> >> >
>> >> > I saw an issue here, but it was reported against 0.8.x branch.
>> >> > https://issues.apache.org/jira/browse/CASSANDRA-2874
>> >> >
>> >> > best regards,
>> >> > 韩竹(Zhu Han)
>> >> >
>> >> > 坚果铺子, 最简单易用的云存储
>> >> > 同步文件, 分享照片, 文档备份!
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Jonathan Ellis
>> >> Project Chair, Apache Cassandra
>> >> co-founder of DataStax, the source for professional Cassandra support
>> >> http://www.datastax.com
>> >
>> >
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>>
>
>
Re: Commit log is not emptied after "nodetool drain"
Posted by Zhu Han <sc...@gmail.com>.
2011/7/15 Jonathan Ellis <jb...@gmail.com>
> If you have non-empty segments post-drain that is a bug. Is it
> reproducible?
>
I think it is always reproducible on 0.6.x branch. Here is a simple
experiment:
1) "bin/nodetool -h localhost"
2) During flush the memtables, we can observe the name of the old commit
log from log:
> " INFO [main] 2011-07-15 11:57:46,742 ColumnFamilyStore.java (line 478)
> Data has reached its threshold; switching in a fresh Memtable at
> CommitLogContext(file='/var/lib/cassandra/commitlog/CommitLog-1310702265959.log',
> position=125) "
>
3) Before the node is drained, new commitlog is created:
> INFO [COMMIT-LOG-WRITER] 2011-07-15 11:58:11,383 CommitLogSegment.java
> (line 50) Creating new commitlog segment
> /var/lib/cassandra/commitlog/CommitLog-1310702291383.log
> INFO [RMI TCP Connection(2)-192.168.1.101] 2011-07-15 11:58:11,413
> StorageService.java (line 391) Node is drained
>
4) After the node is drained and killed, there are still two commit log
under the directory
> $ ls -lh /var/lib/cassandra/commitlog/
> total 128K
> -rw-r--r-- 1 root root 439 2011-07-15 11:57 CommitLog-1310702265959.log
> -rw-r--r-- 1 root root 125 2011-07-15 11:58 CommitLog-1310702291383.log
>
> 2011/7/14 Zhu Han <sc...@gmail.com>:
> > Jonathan,
> >
> > But all the old non-empty log segments are kept on the disk. And
> cassandra
> > takes some time to apply the operations from these closed log segments
> after
> > restart of the process.
> >
> > Is it expected?
> >
> > best regards,
> > 韩竹(Zhu Han)
> >
> > 坚果铺子, 最简单易用的云存储
> > 同步文件, 分享照片, 文档备份!
> >
> >
> >
> > 2011/7/15 Jonathan Ellis <jb...@gmail.com>
> >>
> >> It's expected to have a new, empty segment after drain completes.
> >>
> >> 2011/7/14 Zhu Han <sc...@gmail.com>:
> >> > The deployed version is based on 0.6.13.
> >> >
> >> > After "nodetool drain" is invoked on one of the nodes, the commit log
> is
> >> > not
> >> > emptied. Is this the expected behavior? If so, how can I rename a
> column
> >> > family on 0.6.x branch?
> >> >
> >> > Here is the log output:
> >> > "
> >> > INFO [COMMIT-LOG-WRITER] 2011-07-15 00:39:49,541 CommitLogSegment.java
> >> > (line
> >> > 50) Creating new commitlog segment
> >> > /var/lib/cassandra/commitlog/CommitLog-1310661589541.log
> >> > INFO [RMI TCP Connection(8)-202.120.2.16] 2011-07-15 00:39:49,544
> >> > StorageService.java (line 391) Node is drained
> >> > "
> >> >
> >> > I saw an issue here, but it was reported against 0.8.x branch.
> >> > https://issues.apache.org/jira/browse/CASSANDRA-2874
> >> >
> >> > best regards,
> >> > 韩竹(Zhu Han)
> >> >
> >> > 坚果铺子, 最简单易用的云存储
> >> > 同步文件, 分享照片, 文档备份!
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> Jonathan Ellis
> >> Project Chair, Apache Cassandra
> >> co-founder of DataStax, the source for professional Cassandra support
> >> http://www.datastax.com
> >
> >
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>
Re: Commit log is not emptied after "nodetool drain"
Posted by Jonathan Ellis <jb...@gmail.com>.
If you have non-empty segments post-drain that is a bug. Is it reproducible?
2011/7/14 Zhu Han <sc...@gmail.com>:
> Jonathan,
>
> But all the old non-empty log segments are kept on the disk. And cassandra
> takes some time to apply the operations from these closed log segments after
> restart of the process.
>
> Is it expected?
>
> best regards,
> 韩竹(Zhu Han)
>
> 坚果铺子, 最简单易用的云存储
> 同步文件, 分享照片, 文档备份!
>
>
>
> 2011/7/15 Jonathan Ellis <jb...@gmail.com>
>>
>> It's expected to have a new, empty segment after drain completes.
>>
>> 2011/7/14 Zhu Han <sc...@gmail.com>:
>> > The deployed version is based on 0.6.13.
>> >
>> > After "nodetool drain" is invoked on one of the nodes, the commit log is
>> > not
>> > emptied. Is this the expected behavior? If so, how can I rename a column
>> > family on 0.6.x branch?
>> >
>> > Here is the log output:
>> > "
>> > INFO [COMMIT-LOG-WRITER] 2011-07-15 00:39:49,541 CommitLogSegment.java
>> > (line
>> > 50) Creating new commitlog segment
>> > /var/lib/cassandra/commitlog/CommitLog-1310661589541.log
>> > INFO [RMI TCP Connection(8)-202.120.2.16] 2011-07-15 00:39:49,544
>> > StorageService.java (line 391) Node is drained
>> > "
>> >
>> > I saw an issue here, but it was reported against 0.8.x branch.
>> > https://issues.apache.org/jira/browse/CASSANDRA-2874
>> >
>> > best regards,
>> > 韩竹(Zhu Han)
>> >
>> > 坚果铺子, 最简单易用的云存储
>> > 同步文件, 分享照片, 文档备份!
>> >
>> >
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder of DataStax, the source for professional Cassandra support
>> http://www.datastax.com
>
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: Commit log is not emptied after "nodetool drain"
Posted by Zhu Han <sc...@gmail.com>.
Jonathan,
But all the old non-empty log segments are kept on the disk. And cassandra
takes some time to apply the operations from these closed log segments after
restart of the process.
Is it expected?
best regards,
韩竹(Zhu Han)
坚果铺子 <https://jianguopuzi.com>, 最简单易用的云存储
同步文件, 分享照片, 文档备份!
2011/7/15 Jonathan Ellis <jb...@gmail.com>
> It's expected to have a new, empty segment after drain completes.
>
> 2011/7/14 Zhu Han <sc...@gmail.com>:
> > The deployed version is based on 0.6.13.
> >
> > After "nodetool drain" is invoked on one of the nodes, the commit log is
> not
> > emptied. Is this the expected behavior? If so, how can I rename a column
> > family on 0.6.x branch?
> >
> > Here is the log output:
> > "
> > INFO [COMMIT-LOG-WRITER] 2011-07-15 00:39:49,541 CommitLogSegment.java
> (line
> > 50) Creating new commitlog segment
> > /var/lib/cassandra/commitlog/CommitLog-1310661589541.log
> > INFO [RMI TCP Connection(8)-202.120.2.16] 2011-07-15 00:39:49,544
> > StorageService.java (line 391) Node is drained
> > "
> >
> > I saw an issue here, but it was reported against 0.8.x branch.
> > https://issues.apache.org/jira/browse/CASSANDRA-2874
> >
> > best regards,
> > 韩竹(Zhu Han)
> >
> > 坚果铺子, 最简单易用的云存储
> > 同步文件, 分享照片, 文档备份!
> >
> >
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>
Re: Commit log is not emptied after "nodetool drain"
Posted by Jonathan Ellis <jb...@gmail.com>.
It's expected to have a new, empty segment after drain completes.
2011/7/14 Zhu Han <sc...@gmail.com>:
> The deployed version is based on 0.6.13.
>
> After "nodetool drain" is invoked on one of the nodes, the commit log is not
> emptied. Is this the expected behavior? If so, how can I rename a column
> family on 0.6.x branch?
>
> Here is the log output:
> "
> INFO [COMMIT-LOG-WRITER] 2011-07-15 00:39:49,541 CommitLogSegment.java (line
> 50) Creating new commitlog segment
> /var/lib/cassandra/commitlog/CommitLog-1310661589541.log
> INFO [RMI TCP Connection(8)-202.120.2.16] 2011-07-15 00:39:49,544
> StorageService.java (line 391) Node is drained
> "
>
> I saw an issue here, but it was reported against 0.8.x branch.
> https://issues.apache.org/jira/browse/CASSANDRA-2874
>
> best regards,
> 韩竹(Zhu Han)
>
> 坚果铺子, 最简单易用的云存储
> 同步文件, 分享照片, 文档备份!
>
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com