You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2009/04/16 22:26:14 UTC
[jira] Commented: (CHUKWA-136) Duplicated adaptors
[ https://issues.apache.org/jira/browse/CHUKWA-136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699843#action_12699843 ]
Eric Yang commented on CHUKWA-136:
----------------------------------
Let's assume the checkpoint file writes:
add org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaAgent 21281 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
And next request adds:
add org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaAgent 0 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 0
This would not be considered as duplicated streams from the patch above. We may have to do duplicate detection at adaptor level because the parameters passed to the adaptor are non-uniform. However, be careful of the implementation. We don't want too much cost in doing the adaptor duplication detection. Add Ari, Jerome to charm in.
> Duplicated adaptors
> -------------------
>
> Key: CHUKWA-136
> URL: https://issues.apache.org/jira/browse/CHUKWA-136
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection
> Reporter: Cheng
> Assignee: Cheng
> Priority: Blocker
> Attachments: chukwa-136.patch
>
>
> I did a telnet list command, it shows
> telnet localhost 9093
> Trying 127.0.0.1...
> Connected to localhost (127.0.0.1).
> Escape character is '^]'.
> list
> 1) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped Iostat 0 /grid/0/chukwa/var/log/Iostat.log 145600
> 2) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped Top 0 /grid/0/chukwa/var/log/Top.log 1742264
> 3) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chunkQueue 9339 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 4) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaHttpSender 14777 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> 5) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaAgent 12449 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 6) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chunkQueue 12061 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 7) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped Sar 0 /grid/0/chukwa/var/log/Sar.log 271400
> 8) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaAgent 12817 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 9) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaHttpSender 19221 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> 10) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped Ps 0 /grid/0/chukwa/var/log/Ps.log 3332933
> 11) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaAgent 13185 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 12) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped Netstat 0 /grid/0/chukwa/var/log/Netstat.log 15537
> 13) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chunkQueue 11788 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 14) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chunkQueue 4767 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 15) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped Df 0 /grid/0/chukwa/var/log/Df.log 54540
> 17) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaAgent 9873 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 16) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaHttpSender 18887 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> 19) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaAgent 0 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 18) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chunkQueue 12334 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 21) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaHttpSender 6077 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> 20) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaHttpSender 18553 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> 23) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chunkQueue 20207 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chunkQueue.log 36825
> 22) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaAgent 21281 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaAgent.log 38209
> 25) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped CC 0 /grid/0/chukwa/var/log/collector.log 1040900
> 24) org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.CharFileTailingAdaptorUTF8NewLineEscaped chukwaHttpSender 31669 /grid/0/chukwa/var/log/metrics/chukwa-gmon-chukwaHttpSender.log 58005
> chukwAgent is listed 6 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.