You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by "Keith Turner (JIRA)" <ji...@apache.org> on 2012/04/20 23:13:33 UTC
[jira] [Created] (ACCUMULO-549) Create infrastructure that supports
debugging.
Keith Turner created ACCUMULO-549:
-------------------------------------
Summary: Create infrastructure that supports debugging.
Key: ACCUMULO-549
URL: https://issues.apache.org/jira/browse/ACCUMULO-549
Project: Accumulo
Issue Type: New Feature
Reporter: Keith Turner
Fix For: 1.5.0
Before a release an extensive amount of testing is done that usually involves running test like continuous ingest and random walk on a cluster for extended periods of time. When bugs are found by these test it can take a lot of time to track the issue down sometimes. Inorder to make tracking these issues down easier the write ahead logs are archived. These walogs archives make it possible to answer questions about a tablets history because everything ever written to the metadata table is there. It would be nice to always have this capability on an accumulo system, and have it be easy to use. Spelunking around in the write ahead logs is not an easy task.
It would be nice if accumulo could answer questions like the following.
* Where has a tablet been assigned
* What compactions has a tablet done
* What split or merge created a tablet
These questions can currently be answered with walogs and log4j logs, but its painful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-549) Create infrastructure that
supports debugging.
Posted by "Keith Turner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276774#comment-13276774 ]
Keith Turner commented on ACCUMULO-549:
---------------------------------------
Eric, I think offering a macroscopic view of the trace data would be very useful in addition to the current microscopic view. However I think this warrants a separate ticket?
> Create infrastructure that supports debugging.
> ----------------------------------------------
>
> Key: ACCUMULO-549
> URL: https://issues.apache.org/jira/browse/ACCUMULO-549
> Project: Accumulo
> Issue Type: New Feature
> Reporter: Keith Turner
> Fix For: 1.5.0
>
>
> Before a release an extensive amount of testing is done that usually involves running test like continuous ingest and random walk on a cluster for extended periods of time. When bugs are found by these test it can take a lot of time to track the issue down sometimes. Inorder to make tracking these issues down easier the write ahead logs are archived. These walogs archives make it possible to answer questions about a tablets history because everything ever written to the metadata table is there. It would be nice to always have this capability on an accumulo system, and have it be easy to use. Spelunking around in the write ahead logs is not an easy task.
> It would be nice if accumulo could answer questions like the following.
>
> * Where has a tablet been assigned
> * What compactions has a tablet done
> * What split or merge created a tablet
> These questions can currently be answered with walogs and log4j logs, but its painful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-549) Create infrastructure that
supports debugging.
Posted by "Eric Newton (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276391#comment-13276391 ]
Eric Newton commented on ACCUMULO-549:
--------------------------------------
I would add:
# ability to consume the logs into an accumulo instance so we can query and search 10G of logs with greater parallelism and speed
# statistical analysis of traces, which will probably require the accumulo instance to be be up and running
> Create infrastructure that supports debugging.
> ----------------------------------------------
>
> Key: ACCUMULO-549
> URL: https://issues.apache.org/jira/browse/ACCUMULO-549
> Project: Accumulo
> Issue Type: New Feature
> Reporter: Keith Turner
> Fix For: 1.5.0
>
>
> Before a release an extensive amount of testing is done that usually involves running test like continuous ingest and random walk on a cluster for extended periods of time. When bugs are found by these test it can take a lot of time to track the issue down sometimes. Inorder to make tracking these issues down easier the write ahead logs are archived. These walogs archives make it possible to answer questions about a tablets history because everything ever written to the metadata table is there. It would be nice to always have this capability on an accumulo system, and have it be easy to use. Spelunking around in the write ahead logs is not an easy task.
> It would be nice if accumulo could answer questions like the following.
>
> * Where has a tablet been assigned
> * What compactions has a tablet done
> * What split or merge created a tablet
> These questions can currently be answered with walogs and log4j logs, but its painful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-549) Create infrastructure that
supports debugging.
Posted by "Keith Turner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276794#comment-13276794 ]
Keith Turner commented on ACCUMULO-549:
---------------------------------------
For comment 1 Eric made, I was thinking of walogs and was slightly confused. I asked Eric about this, he was thinking about indexing the log4j debug logs in parallel from each machine in the cluster.
> Create infrastructure that supports debugging.
> ----------------------------------------------
>
> Key: ACCUMULO-549
> URL: https://issues.apache.org/jira/browse/ACCUMULO-549
> Project: Accumulo
> Issue Type: New Feature
> Reporter: Keith Turner
> Fix For: 1.5.0
>
>
> Before a release an extensive amount of testing is done that usually involves running test like continuous ingest and random walk on a cluster for extended periods of time. When bugs are found by these test it can take a lot of time to track the issue down sometimes. Inorder to make tracking these issues down easier the write ahead logs are archived. These walogs archives make it possible to answer questions about a tablets history because everything ever written to the metadata table is there. It would be nice to always have this capability on an accumulo system, and have it be easy to use. Spelunking around in the write ahead logs is not an easy task.
> It would be nice if accumulo could answer questions like the following.
>
> * Where has a tablet been assigned
> * What compactions has a tablet done
> * What split or merge created a tablet
> These questions can currently be answered with walogs and log4j logs, but its painful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-549) Create infrastructure that
supports debugging.
Posted by "Keith Turner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276261#comment-13276261 ]
Keith Turner commented on ACCUMULO-549:
---------------------------------------
I am thinking one possible way to accomplish this is with the following components.
# Mechanism that continuously collect and stores !METADATA mutations
#* ACCUMULO-212 or ACCUMULO-378 are possible ways to accomplish this
#* Need to ensure that latest mutations are available when needed.
#* Need to provide an easy way of copying mutations from a production system to a developers machine for analysis.
#* Mutations must be accessible even when accumulo is not functioning properly.
# Utility to index !METADATA table mutations into Accumulo tables
# Utility to query indexes
> Create infrastructure that supports debugging.
> ----------------------------------------------
>
> Key: ACCUMULO-549
> URL: https://issues.apache.org/jira/browse/ACCUMULO-549
> Project: Accumulo
> Issue Type: New Feature
> Reporter: Keith Turner
> Fix For: 1.5.0
>
>
> Before a release an extensive amount of testing is done that usually involves running test like continuous ingest and random walk on a cluster for extended periods of time. When bugs are found by these test it can take a lot of time to track the issue down sometimes. Inorder to make tracking these issues down easier the write ahead logs are archived. These walogs archives make it possible to answer questions about a tablets history because everything ever written to the metadata table is there. It would be nice to always have this capability on an accumulo system, and have it be easy to use. Spelunking around in the write ahead logs is not an easy task.
> It would be nice if accumulo could answer questions like the following.
>
> * Where has a tablet been assigned
> * What compactions has a tablet done
> * What split or merge created a tablet
> These questions can currently be answered with walogs and log4j logs, but its painful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira