You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Ari Rabkin (JIRA)" <ji...@apache.org> on 2010/01/03 20:44:54 UTC
[jira] Created: (CHUKWA-445) real-time buffer and display at
Collector
real-time buffer and display at Collector
-----------------------------------------
Key: CHUKWA-445
URL: https://issues.apache.org/jira/browse/CHUKWA-445
Project: Hadoop Chukwa
Issue Type: New Feature
Reporter: Ari Rabkin
It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-445:
------------------------------
Status: Patch Available (was: Open)
I'd like to commit this. Objections? if it turns out not to be very useful, we can deprecate it and nobody will the wiser. :)
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Attachments: CHUKWA-445.patch
>
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin reassigned CHUKWA-445:
---------------------------------
Assignee: Ari Rabkin
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Attachments: CHUKWA-445.patch
>
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-445:
------------------------------
Status: Open (was: Patch Available)
Want to test this at Berkeley before proposing to commit.
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Attachments: CHUKWA-445.patch
>
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-445:
------------------------------
Attachment: CHUKWA-445.patch
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
> Attachments: CHUKWA-445.patch
>
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-445:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed this to both branch and trunk.
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Attachments: CHUKWA-445.patch
>
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848351#action_12848351 ]
Eric Yang commented on CHUKWA-445:
----------------------------------
+1 Looks good.
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Attachments: CHUKWA-445.patch
>
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848344#action_12848344 ]
Ari Rabkin commented on CHUKWA-445:
-----------------------------------
Last call for objections here; I intend to commit this to both trunk and the 0.4 branch, barring comment.
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Attachments: CHUKWA-445.patch
>
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796022#action_12796022 ]
Ari Rabkin commented on CHUKWA-445:
-----------------------------------
In small to midsize deployments, it would be nice for collectors to cache, say, the last megabyte of received log data, and make it visible via the web.
Intended implementation: Have a web servlet coupled to a collector pipeline stage.
Keep a total of 1 MB of chunks. Keep an index by stream. Let users browse, via the web, to an arbitrary stream and show all the data in that stream.
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-445:
------------------------------
Status: Patch Available (was: Open)
A pipeline writer stage and a servlet to display the collected data.
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
> Attachments: CHUKWA-445.patch
>
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12850969#action_12850969 ]
Hudson commented on CHUKWA-445:
-------------------------------
Integrated in Chukwa-trunk #330 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/330/])
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Attachments: CHUKWA-445.patch
>
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796038#action_12796038 ]
Eric Yang commented on CHUKWA-445:
----------------------------------
I also have an implementation of this, but I made a queue buffering on the HICC side to keep the collector simple. This could be accomplished by using:
new SocketDataLoader("all");
Either implementation works. Let's think this over. What are the pros/cons of having the buffer in the collector?
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Attachments: CHUKWA-445.patch
>
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-445) real-time buffer and display at
Collector
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796041#action_12796041 ]
Ari Rabkin commented on CHUKWA-445:
-----------------------------------
I think the tradeoff is whether you think collectors are heavily loaded or mostly idle. At Berkeley, they're mostly idle. So it seemed extravagant to add a whole 'nother process to display logs. If your collectors are heavily loaded, it makes sense to pull this functionality onto the HICC server.
> real-time buffer and display at Collector
> -----------------------------------------
>
> Key: CHUKWA-445
> URL: https://issues.apache.org/jira/browse/CHUKWA-445
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Attachments: CHUKWA-445.patch
>
>
> It would be really cool to have the collector buffer a certain amount of received data, and make it available via the web interface.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.