You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "kishore gopalakrishna (JIRA)" <ji...@apache.org> on 2011/09/14 20:58:08 UTC
[jira] [Created] (ZOOKEEPER-1183) Enhance LogFormatter to output
additional detail from transaction log
Enhance LogFormatter to output additional detail from transaction log
---------------------------------------------------------------------
Key: ZOOKEEPER-1183
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1183
Project: ZooKeeper
Issue Type: Improvement
Affects Versions: 3.4.0
Reporter: kishore gopalakrishna
Priority: Minor
Current LogFormatter prints the following information
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
8/15/11 1:55:36 PM PDT session 0x131cf1a236f0014 cxid 0x0 zxid 0xf01 createSession
8/15/11 1:55:57 PM PDT session 0x131cf1a236f0000 cxid 0x55f zxid 0xf02 setData
8/15/11 1:56:00 PM PDT session 0x131cf1a236f0015 cxid 0x0 zxid 0xf03 createSession
...
..
8/15/11 2:00:33 PM PDT session 0x131cf1a236f001c cxid 0x36 zxid 0xf6b setData
8/15/11 2:00:33 PM PDT session 0x131cf1a236f0021 cxid 0xa1 zxid 0xf6c create
8/15/11 2:00:33 PM PDT session 0x131cf1a236f001b cxid 0x3e zxid 0xf6d setData
8/15/11 2:00:33 PM PDT session 0x131cf1a236f001e cxid 0x3e zxid 0xf6e setData
8/15/11 2:00:33 PM PDT session 0x131cf1a236f001d cxid 0x41 zxid 0xf6f setData
Though this is good information, it does not provide additional information like
createSession: which ip created the session and its time out
set|get|delete: the path and data
create: path created and createmode along with data
We can add additional parameter -detail and provide detailed output of the transaction.
Outputting data is slightly tricky since we cant print data without understanding the format. We need not print this for now.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1183) Enhance LogFormatter to output
additional detail from transaction log
Posted by "kishore gopalakrishna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13107589#comment-13107589 ]
kishore gopalakrishna commented on ZOOKEEPER-1183:
--------------------------------------------------
New output format
time:8/15/11 2:52:10 PM PDT session:0x131cf1a236f001e cxid:0x0 zxid:0x1189 type:closeSession path:/test version:10 data:48656C6C6F20576F726C6421
Used HexBinaryAdapter to marshal data byte[]
To get the byte[] from data one can do new HexBinaryAdapter().unmarshal("48656C6C6F20576F726C6421").
Can some one tell me when will a MultiTxn happen. Currently that is not formatted using the above logic.
> Enhance LogFormatter to output additional detail from transaction log
> ---------------------------------------------------------------------
>
> Key: ZOOKEEPER-1183
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1183
> Project: ZooKeeper
> Issue Type: Improvement
> Affects Versions: 3.4.0
> Reporter: kishore gopalakrishna
> Assignee: kishore gopalakrishna
> Priority: Minor
> Attachments: ZOOKEEPER-1183.patch
>
>
> Current LogFormatter prints the following information
> ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
> 8/15/11 1:55:36 PM PDT session 0x131cf1a236f0014 cxid 0x0 zxid 0xf01 createSession
> 8/15/11 1:55:57 PM PDT session 0x131cf1a236f0000 cxid 0x55f zxid 0xf02 setData
> 8/15/11 1:56:00 PM PDT session 0x131cf1a236f0015 cxid 0x0 zxid 0xf03 createSession
> ...
> ..
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001c cxid 0x36 zxid 0xf6b setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f0021 cxid 0xa1 zxid 0xf6c create
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001b cxid 0x3e zxid 0xf6d setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001e cxid 0x3e zxid 0xf6e setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001d cxid 0x41 zxid 0xf6f setData
> Though this is good information, it does not provide additional information like
> createSession: which ip created the session and its time out
> set|get|delete: the path and data
> create: path created and createmode along with data
> We can add additional parameter -detail and provide detailed output of the transaction.
> Outputting data is slightly tricky since we cant print data without understanding the format. We need not print this for now.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (ZOOKEEPER-1183) Enhance LogFormatter to output
additional detail from transaction log
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt reassigned ZOOKEEPER-1183:
---------------------------------------
Assignee: kishore gopalakrishna
> Enhance LogFormatter to output additional detail from transaction log
> ---------------------------------------------------------------------
>
> Key: ZOOKEEPER-1183
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1183
> Project: ZooKeeper
> Issue Type: Improvement
> Affects Versions: 3.4.0
> Reporter: kishore gopalakrishna
> Assignee: kishore gopalakrishna
> Priority: Minor
>
> Current LogFormatter prints the following information
> ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
> 8/15/11 1:55:36 PM PDT session 0x131cf1a236f0014 cxid 0x0 zxid 0xf01 createSession
> 8/15/11 1:55:57 PM PDT session 0x131cf1a236f0000 cxid 0x55f zxid 0xf02 setData
> 8/15/11 1:56:00 PM PDT session 0x131cf1a236f0015 cxid 0x0 zxid 0xf03 createSession
> ...
> ..
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001c cxid 0x36 zxid 0xf6b setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f0021 cxid 0xa1 zxid 0xf6c create
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001b cxid 0x3e zxid 0xf6d setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001e cxid 0x3e zxid 0xf6e setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001d cxid 0x41 zxid 0xf6f setData
> Though this is good information, it does not provide additional information like
> createSession: which ip created the session and its time out
> set|get|delete: the path and data
> create: path created and createmode along with data
> We can add additional parameter -detail and provide detailed output of the transaction.
> Outputting data is slightly tricky since we cant print data without understanding the format. We need not print this for now.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1183) Enhance LogFormatter to output
additional detail from transaction log
Posted by "kishore gopalakrishna (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182183#comment-13182183 ]
kishore gopalakrishna commented on ZOOKEEPER-1183:
--------------------------------------------------
Hi Camille,
Yes, I wanted to enhance it as well. I was actually planning enhance it to a audit log and have a tool called zk data tree timeline.
Requirements were
* For a given path show the create, modify, delete sequence for a given range of time.
* Given a session tell when was it created, who created it (unfortunately i cant get ip from transaction log), when was it closed.
There were few more things I wanted but dont remember now.
The Zk time line tool i had in mind was a applet with a time slider. It show the zk data tree. One should be able to use the slider to move back and forth in time and see the tree. This is very useful in debugging issues.
I have written a tool(not ui) for my own project which parses the ZK transaction log and actually gives us the ability to know sequence of events that happened in the application. Unfortunately this is very specific to my application and others may not be able to use this.
Let me know what you about a tool like this. If there is enough interest, I can spend time making it generic enough for others to use.
> Enhance LogFormatter to output additional detail from transaction log
> ---------------------------------------------------------------------
>
> Key: ZOOKEEPER-1183
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1183
> Project: ZooKeeper
> Issue Type: Improvement
> Affects Versions: 3.4.0
> Reporter: kishore gopalakrishna
> Assignee: kishore gopalakrishna
> Priority: Minor
> Attachments: ZOOKEEPER-1183.patch
>
>
> Current LogFormatter prints the following information
> ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
> 8/15/11 1:55:36 PM PDT session 0x131cf1a236f0014 cxid 0x0 zxid 0xf01 createSession
> 8/15/11 1:55:57 PM PDT session 0x131cf1a236f0000 cxid 0x55f zxid 0xf02 setData
> 8/15/11 1:56:00 PM PDT session 0x131cf1a236f0015 cxid 0x0 zxid 0xf03 createSession
> ...
> ..
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001c cxid 0x36 zxid 0xf6b setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f0021 cxid 0xa1 zxid 0xf6c create
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001b cxid 0x3e zxid 0xf6d setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001e cxid 0x3e zxid 0xf6e setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001d cxid 0x41 zxid 0xf6f setData
> Though this is good information, it does not provide additional information like
> createSession: which ip created the session and its time out
> set|get|delete: the path and data
> create: path created and createmode along with data
> We can add additional parameter -detail and provide detailed output of the transaction.
> Outputting data is slightly tricky since we cant print data without understanding the format. We need not print this for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1183) Enhance LogFormatter to output
additional detail from transaction log
Posted by "kishore gopalakrishna (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182358#comment-13182358 ]
kishore gopalakrishna commented on ZOOKEEPER-1183:
--------------------------------------------------
The patch I have provided adds the ability to print the data if the transaction modifies the data along with the path that change. You can check out the patch and free to either use it, modify it or start from scratch.( It does not handle multi transaction)
The requirements I added were from the experience I had when we used this patched LogFormatter to provide additional data needed to debug https://issues.apache.org/jira/browse/ZOOKEEPER-1208.
I dont think its ambitious, its based on whether you want to say its not a bug(I guess thats what you said in that 1208) because the user cant provide additional/relevant data or have the ability to ask user to run a command(which would have been a better thing) and give its output and you can reason about it.
We had to spend quite some time to actually provide the relevant data needed for the ticket to actually find the issue and be solved. Don't know if its the right thing to expect every one to understand the transaction log to provide the relevant data.
> Enhance LogFormatter to output additional detail from transaction log
> ---------------------------------------------------------------------
>
> Key: ZOOKEEPER-1183
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1183
> Project: ZooKeeper
> Issue Type: Improvement
> Affects Versions: 3.4.0
> Reporter: kishore gopalakrishna
> Assignee: kishore gopalakrishna
> Priority: Minor
> Attachments: ZOOKEEPER-1183.patch
>
>
> Current LogFormatter prints the following information
> ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
> 8/15/11 1:55:36 PM PDT session 0x131cf1a236f0014 cxid 0x0 zxid 0xf01 createSession
> 8/15/11 1:55:57 PM PDT session 0x131cf1a236f0000 cxid 0x55f zxid 0xf02 setData
> 8/15/11 1:56:00 PM PDT session 0x131cf1a236f0015 cxid 0x0 zxid 0xf03 createSession
> ...
> ..
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001c cxid 0x36 zxid 0xf6b setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f0021 cxid 0xa1 zxid 0xf6c create
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001b cxid 0x3e zxid 0xf6d setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001e cxid 0x3e zxid 0xf6e setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001d cxid 0x41 zxid 0xf6f setData
> Though this is good information, it does not provide additional information like
> createSession: which ip created the session and its time out
> set|get|delete: the path and data
> create: path created and createmode along with data
> We can add additional parameter -detail and provide detailed output of the transaction.
> Outputting data is slightly tricky since we cant print data without understanding the format. We need not print this for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1183) Enhance LogFormatter to output
additional detail from transaction log
Posted by "kishore gopalakrishna (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
kishore gopalakrishna updated ZOOKEEPER-1183:
---------------------------------------------
Attachment: ZOOKEEPER-1183.patch
Patch to provide additional information about the transaction
> Enhance LogFormatter to output additional detail from transaction log
> ---------------------------------------------------------------------
>
> Key: ZOOKEEPER-1183
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1183
> Project: ZooKeeper
> Issue Type: Improvement
> Affects Versions: 3.4.0
> Reporter: kishore gopalakrishna
> Assignee: kishore gopalakrishna
> Priority: Minor
> Attachments: ZOOKEEPER-1183.patch
>
>
> Current LogFormatter prints the following information
> ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
> 8/15/11 1:55:36 PM PDT session 0x131cf1a236f0014 cxid 0x0 zxid 0xf01 createSession
> 8/15/11 1:55:57 PM PDT session 0x131cf1a236f0000 cxid 0x55f zxid 0xf02 setData
> 8/15/11 1:56:00 PM PDT session 0x131cf1a236f0015 cxid 0x0 zxid 0xf03 createSession
> ...
> ..
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001c cxid 0x36 zxid 0xf6b setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f0021 cxid 0xa1 zxid 0xf6c create
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001b cxid 0x3e zxid 0xf6d setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001e cxid 0x3e zxid 0xf6e setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001d cxid 0x41 zxid 0xf6f setData
> Though this is good information, it does not provide additional information like
> createSession: which ip created the session and its time out
> set|get|delete: the path and data
> create: path created and createmode along with data
> We can add additional parameter -detail and provide detailed output of the transaction.
> Outputting data is slightly tricky since we cant print data without understanding the format. We need not print this for now.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1183) Enhance LogFormatter to output
additional detail from transaction log
Posted by "Camille Fournier (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182344#comment-13182344 ]
Camille Fournier commented on ZOOKEEPER-1183:
---------------------------------------------
Honestly, I think you're getting a bit ambitious for this ticket. I think you should simply enhance the logformatter to a degree that makes sense, and any additional tooling either make a new ticket or perhaps a github project for the work.
> Enhance LogFormatter to output additional detail from transaction log
> ---------------------------------------------------------------------
>
> Key: ZOOKEEPER-1183
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1183
> Project: ZooKeeper
> Issue Type: Improvement
> Affects Versions: 3.4.0
> Reporter: kishore gopalakrishna
> Assignee: kishore gopalakrishna
> Priority: Minor
> Attachments: ZOOKEEPER-1183.patch
>
>
> Current LogFormatter prints the following information
> ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
> 8/15/11 1:55:36 PM PDT session 0x131cf1a236f0014 cxid 0x0 zxid 0xf01 createSession
> 8/15/11 1:55:57 PM PDT session 0x131cf1a236f0000 cxid 0x55f zxid 0xf02 setData
> 8/15/11 1:56:00 PM PDT session 0x131cf1a236f0015 cxid 0x0 zxid 0xf03 createSession
> ...
> ..
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001c cxid 0x36 zxid 0xf6b setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f0021 cxid 0xa1 zxid 0xf6c create
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001b cxid 0x3e zxid 0xf6d setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001e cxid 0x3e zxid 0xf6e setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001d cxid 0x41 zxid 0xf6f setData
> Though this is good information, it does not provide additional information like
> createSession: which ip created the session and its time out
> set|get|delete: the path and data
> create: path created and createmode along with data
> We can add additional parameter -detail and provide detailed output of the transaction.
> Outputting data is slightly tricky since we cant print data without understanding the format. We need not print this for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1183) Enhance LogFormatter to output
additional detail from transaction log
Posted by "Camille Fournier (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13182133#comment-13182133 ]
Camille Fournier commented on ZOOKEEPER-1183:
---------------------------------------------
Kishore, are you still interested in working on this? I'm thinking of enhancing the LogFormatter a bit more cleanly, debating whether to work on your patch or start from scratch.
> Enhance LogFormatter to output additional detail from transaction log
> ---------------------------------------------------------------------
>
> Key: ZOOKEEPER-1183
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1183
> Project: ZooKeeper
> Issue Type: Improvement
> Affects Versions: 3.4.0
> Reporter: kishore gopalakrishna
> Assignee: kishore gopalakrishna
> Priority: Minor
> Attachments: ZOOKEEPER-1183.patch
>
>
> Current LogFormatter prints the following information
> ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
> 8/15/11 1:55:36 PM PDT session 0x131cf1a236f0014 cxid 0x0 zxid 0xf01 createSession
> 8/15/11 1:55:57 PM PDT session 0x131cf1a236f0000 cxid 0x55f zxid 0xf02 setData
> 8/15/11 1:56:00 PM PDT session 0x131cf1a236f0015 cxid 0x0 zxid 0xf03 createSession
> ...
> ..
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001c cxid 0x36 zxid 0xf6b setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f0021 cxid 0xa1 zxid 0xf6c create
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001b cxid 0x3e zxid 0xf6d setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001e cxid 0x3e zxid 0xf6e setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001d cxid 0x41 zxid 0xf6f setData
> Though this is good information, it does not provide additional information like
> createSession: which ip created the session and its time out
> set|get|delete: the path and data
> create: path created and createmode along with data
> We can add additional parameter -detail and provide detailed output of the transaction.
> Outputting data is slightly tricky since we cant print data without understanding the format. We need not print this for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira