You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Radim Kolar <hs...@filez.com> on 2012/04/06 02:51:17 UTC

leveled compaction - improve log message

it would be really helpfull if leveled compaction prints level into syslog.

Demo:

INFO [CompactionExecutor:891] 2012-04-05 22:39:27,043 
CompactionTask.java (line 113) Compacting ***LEVEL 1*** 
[SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19690-Data.db'), 
SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19688-Data.db'), 
SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19691-Data.db'), 
SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19700-Data.db'), 
SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19686-Data.db'), 
SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19696-Data.db'), 
SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19687-Data.db'), 
SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19695-Data.db'), 
SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19689-Data.db'), 
SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19694-Data.db'), 
SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19693-Data.db')]

  INFO [CompactionExecutor:891] 2012-04-05 22:39:57,299 
CompactionTask.java (line 221) *** LEVEL 1 *** Compacted to 
[/var/lib/cassandra/data/rapidshare/querycache-hc-19701-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19702-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19703-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19704-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19705-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19706-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19707-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19708-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19709-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19710-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19711-Data.db,].  
59,643,011 to 57,564,216 (~96% of original) bytes for 590,909 keys at 
1.814434MB/s.  Time: 30,256ms.



Re: leveled compaction - improve log message

Posted by aaron morton <aa...@thelastpickle.com>.
If you would like to see a change create a request for an improvement here https://issues.apache.org/jira/browse/CASSANDRA

Cheers


-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 6/04/2012, at 12:51 PM, Radim Kolar wrote:

> it would be really helpfull if leveled compaction prints level into syslog.
> 
> Demo:
> 
> INFO [CompactionExecutor:891] 2012-04-05 22:39:27,043 CompactionTask.java (line 113) Compacting ***LEVEL 1*** [SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19690-Data.db'), SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19688-Data.db'), SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19691-Data.db'), SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19700-Data.db'), SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19686-Data.db'), SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19696-Data.db'), SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19687-Data.db'), SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19695-Data.db'), SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19689-Data.db'), SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19694-Data.db'), SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19693-Data.db')]
> 
> INFO [CompactionExecutor:891] 2012-04-05 22:39:57,299 CompactionTask.java (line 221) *** LEVEL 1 *** Compacted to [/var/lib/cassandra/data/rapidshare/querycache-hc-19701-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19702-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19703-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19704-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19705-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19706-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19707-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19708-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19709-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19710-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19711-Data.db,].  59,643,011 to 57,564,216 (~96% of original) bytes for 590,909 keys at 1.814434MB/s.  Time: 30,256ms.
> 
> 


Re: leveled compaction - improve log message

Posted by Maki Watanabe <wa...@gmail.com>.
for details, open conf/log4j-server.properties and add following configuration:

log4j.logger.org.apache.cassandra.db.compaction.LeveledManifest=DEBUG

fyi.

maki


2012/4/10 Jonathan Ellis <jb...@gmail.com>:
> CompactionExecutor doesn't have level information available to it; it
> just compacts the sstables it's told to.  But if you enable debug
> logging on LeveledManifest you'd see what you want.  ("Compaction
> candidates for L{} are {}")
>
> 2012/4/5 Radim Kolar <hs...@filez.com>:
>> it would be really helpfull if leveled compaction prints level into syslog.
>>
>> Demo:
>>
>> INFO [CompactionExecutor:891] 2012-04-05 22:39:27,043 CompactionTask.java
>> (line 113) Compacting ***LEVEL 1***
>> [SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19690-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19688-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19691-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19700-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19686-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19696-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19687-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19695-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19689-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19694-Data.db'),
>> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19693-Data.db')]
>>
>>  INFO [CompactionExecutor:891] 2012-04-05 22:39:57,299 CompactionTask.java
>> (line 221) *** LEVEL 1 *** Compacted to
>> [/var/lib/cassandra/data/rapidshare/querycache-hc-19701-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19702-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19703-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19704-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19705-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19706-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19707-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19708-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19709-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19710-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19711-Data.db,].
>>  59,643,011 to 57,564,216 (~96% of original) bytes for 590,909 keys at
>> 1.814434MB/s.  Time: 30,256ms.
>>
>>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com

Re: leveled compaction - improve log message

Posted by Jonathan Ellis <jb...@gmail.com>.
CompactionExecutor doesn't have level information available to it; it
just compacts the sstables it's told to.  But if you enable debug
logging on LeveledManifest you'd see what you want.  ("Compaction
candidates for L{} are {}")

2012/4/5 Radim Kolar <hs...@filez.com>:
> it would be really helpfull if leveled compaction prints level into syslog.
>
> Demo:
>
> INFO [CompactionExecutor:891] 2012-04-05 22:39:27,043 CompactionTask.java
> (line 113) Compacting ***LEVEL 1***
> [SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19690-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19688-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19691-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19700-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19686-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19696-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19687-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19695-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19689-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19694-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/rapidshare/querycache-hc-19693-Data.db')]
>
>  INFO [CompactionExecutor:891] 2012-04-05 22:39:57,299 CompactionTask.java
> (line 221) *** LEVEL 1 *** Compacted to
> [/var/lib/cassandra/data/rapidshare/querycache-hc-19701-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19702-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19703-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19704-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19705-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19706-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19707-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19708-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19709-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19710-Data.db,/var/lib/cassandra/data/rapidshare/querycache-hc-19711-Data.db,].
>  59,643,011 to 57,564,216 (~96% of original) bytes for 590,909 keys at
> 1.814434MB/s.  Time: 30,256ms.
>
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com