You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by "Han,Meng" <me...@ufl.edu> on 2014/03/18 19:25:11 UTC
How to extract information from commit log?
Hi Cassandra hackers!
I have a question regarding extracting useful information from commit
log.
Since its a binary log, how should I extract information such as
timestamp, values from it? Does anyone know any binary log reader that I
can use directly to read commit log?
If there is no such reader, could someone give me some advice hwo I can
wrote such a reader?
Particularly, I want to know the order that write operations happens at
each replica(cassandra server node) along with their timestamps, Does
anyone know other methods how I can get this information without
instrumenting Cassandra code?
Any help is appreciated!
Cheers,
Meng
Re: How to extract information from commit log?
Posted by Artur Kronenberg <ar...@openmarket.com>.
Hi,
we did something similar. We did utilize some cassandra code though and
wrote a custom commitlog reader that outputs our data into a readable form.
You can look here:
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.cassandra/cassandra-all/1.1.9/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
This code is used to replay commitlogs when starting up cassandra. It
has the ability to deserialize and transform the data into what you'll need.
-- artur
On 18/03/14 19:32, Han,Meng wrote:
>
> Hi Jonathan,
>
> Thank you for the timely reply. I am doing this experiment on a
> continuous basis. To be more specific, I will issue a large amount of
> read and write operations to a particular key in a short time
> interval. I'd like to know the order that write operations happens at
> each replica. TImestamps definitely help to determine order, but the
> WRITETIME and SStable2Json both looks me only return the timestamps
> when that key was updated the moment the WRITETIME/SStable2Json is
> issued. It looks like a one time thing to me. Or put in another way,
> if I want to get the write time for all write operations in that short
> invertal to determine a total order for write on that replia I have to
> constantly issue WRITETIME to this replica? Correct me if I am wrong
> here.
>
> Light me up pleeeeeeeeeease!
>
> On Tue, 18 Mar 2014 15:05:07 -0400, Jonathan Lacefield wrote:
>
>> Hello,
>> Is this a one time investigative item or are you looking to set
>> something up to do this continuously? Don't recommend trying to read
>> the commit log.
>> You can always use the WRITETIME function in CQL or look within
>> SSTables via the SStable2Json utility to see write times for
>> particular versions of partitions.
>> Jonathan
>>
>> Jonathan Lacefield
>> Solutions Architect, DataStax
>> (404) 822 3487
>> <http://www.linkedin.com/in/jlacefield>
>> <http://www.datastax.com/what-we-offer/products-services/training/virtual-training>
>>
>>
>> On Tue, Mar 18, 2014 at 2:25 PM, Han,Meng <menghan@ufl.edu
>> <ma...@ufl.edu>> wrote:
>>
>> Hi Cassandra hackers!
>>
>> I have a question regarding extracting useful information from
>> commit log.
>>
>> Since its a binary log, how should I extract information such as
>> timestamp, values from it? Does anyone know any binary log reader
>> that I can use directly to read commit log?
>> If there is no such reader, could someone give me some advice hwo
>> I can wrote such a reader?
>>
>> Particularly, I want to know the order that write operations
>> happens at each replica(cassandra server node) along with their
>> timestamps, Does anyone know other methods how I can get this
>> information without instrumenting Cassandra code?
>>
>> Any help is appreciated!
>>
>> Cheers,
>> Meng
>>
Re: How to extract information from commit log?
Posted by "Han,Meng" <me...@ufl.edu>.
Hi Jonathan,
Thank you for the timely reply. I am doing this
experiment on a continuous basis. To be more specific, I will issue a
large amount of read and write operations to a particular key in a short
time interval. I'd like to know the order that write operations happens
at each replica. TImestamps definitely help to determine order, but the
WRITETIME and SStable2Json both looks me only return the timestamps when
that key was updated the moment the WRITETIME/SStable2Json is issued. It
looks like a one time thing to me. Or put in another way, if I want to
get the write time for all write operations in that short invertal to
determine a total order for write on that replia I have to constantly
issue WRITETIME to this replica? Correct me if I am wrong here.
Light
me up pleeeeeeeeeease!
On Tue, 18 Mar 2014 15:05:07 -0400, Jonathan
Lacefield wrote:
> Hello,
> Is this a one time investigative item or
are you looking to set something up to do this continuously? Don't
recommend trying to read the commit log.
> You can always use the
WRITETIME function in CQL or look within SSTables via the SStable2Json
utility to see write times for particular versions of partitions.
>
Jonathan
>
> Jonathan Lacefield
> Solutions Architect, DataStax
>
(404) 822 3487
> [1]
> [2]
>
> On Tue, Mar 18, 2014 at 2:25 PM,
Han,Meng wrote:
>
>> Hi Cassandra hackers!
>>
>> I have a question
regarding extracting useful information from commit log.
>>
>> Since
its a binary log, how should I extract information such as timestamp,
values from it? Does anyone know any binary log reader that I can use
directly to read commit log?
>> If there is no such reader, could
someone give me some advice hwo I can wrote such a reader?
>>
>>
Particularly, I want to know the order that write operations happens at
each replica(cassandra server node) along with their timestamps, Does
anyone know other methods how I can get this information without
instrumenting Cassandra code?
>>
>> Any help is appreciated!
>>
>>
Cheers,
>> Meng
Links:
------
[1]
http://www.linkedin.com/in/jlacefield
[2]
http://www.datastax.com/what-we-offer/products-services/training/virtual-training
[3]
mailto:menghan@ufl.edu
Re: How to extract information from commit log?
Posted by Jonathan Lacefield <jl...@datastax.com>.
Hello,
Is this a one time investigative item or are you looking to set something
up to do this continuously? Don't recommend trying to read the commit log.
You can always use the WRITETIME function in CQL or look within SSTables
via the SStable2Json utility to see write times for particular versions of
partitions.
Jonathan
Jonathan Lacefield
Solutions Architect, DataStax
(404) 822 3487
<http://www.linkedin.com/in/jlacefield>
<http://www.datastax.com/what-we-offer/products-services/training/virtual-training>
On Tue, Mar 18, 2014 at 2:25 PM, Han,Meng <me...@ufl.edu> wrote:
> Hi Cassandra hackers!
>
> I have a question regarding extracting useful information from commit log.
>
> Since its a binary log, how should I extract information such as
> timestamp, values from it? Does anyone know any binary log reader that I
> can use directly to read commit log?
> If there is no such reader, could someone give me some advice hwo I can
> wrote such a reader?
>
> Particularly, I want to know the order that write operations happens at
> each replica(cassandra server node) along with their timestamps, Does
> anyone know other methods how I can get this information without
> instrumenting Cassandra code?
>
> Any help is appreciated!
>
> Cheers,
> Meng
>