You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Edward Capriolo (JIRA)" <ji...@apache.org> on 2008/11/05 16:31:44 UTC
[jira] Created: (HADOOP-4594) Monitoring Scripts for Nagios
Monitoring Scripts for Nagios
-----------------------------
Key: HADOOP-4594
URL: https://issues.apache.org/jira/browse/HADOOP-4594
Project: Hadoop Core
Issue Type: Wish
Reporter: Edward Capriolo
Priority: Minor
I would like to create a set of local via NRPE and remote check scripts that can be shipped with the hadoop distribution and used to monitor Hadoop. I already have completed the NRPE scripts. The second set of scripts would use wget to read the output of the hadoop web interfaces. Do these already exist?
I guess these would fall under a new contrib project.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4594) Monitoring Scripts for Nagios
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645555#action_12645555 ]
Hadoop QA commented on HADOOP-4594:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12393401/HADOOP-4594.patch
against trunk revision 711734.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3542/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3542/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3542/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3542/console
This message is automatically generated.
> Monitoring Scripts for Nagios
> -----------------------------
>
> Key: HADOOP-4594
> URL: https://issues.apache.org/jira/browse/HADOOP-4594
> Project: Hadoop Core
> Issue Type: Wish
> Reporter: Edward Capriolo
> Priority: Minor
> Attachments: HADOOP-4594.patch
>
>
> I would like to create a set of local via NRPE and remote check scripts that can be shipped with the hadoop distribution and used to monitor Hadoop. I already have completed the NRPE scripts. The second set of scripts would use wget to read the output of the hadoop web interfaces. Do these already exist?
> I guess these would fall under a new contrib project.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4594) Monitoring Scripts for Nagios
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Capriolo updated HADOOP-4594:
------------------------------------
Attachment: (was: HADOOP-4594.patch)
> Monitoring Scripts for Nagios
> -----------------------------
>
> Key: HADOOP-4594
> URL: https://issues.apache.org/jira/browse/HADOOP-4594
> Project: Hadoop Core
> Issue Type: Wish
> Reporter: Edward Capriolo
> Priority: Minor
> Attachments: HADOOP-4594.patch
>
>
> I would like to create a set of local via NRPE and remote check scripts that can be shipped with the hadoop distribution and used to monitor Hadoop. I already have completed the NRPE scripts. The second set of scripts would use wget to read the output of the hadoop web interfaces. Do these already exist?
> I guess these would fall under a new contrib project.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4594) Monitoring Scripts for Nagios
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645971#action_12645971 ]
Chris Douglas commented on HADOOP-4594:
---------------------------------------
Sorry, my previous comments were cursory and this deserves more attention. I ran this past someone who had worked with nagios and Hadoop and it's my understanding that- while a good base- the patch as-is is not generally applicable. Specifically, the feedback was that the configuration contained several hard-coded values (hostnames, ports, environment variables) and that a combination of jps and grep was not a sufficient check of node health.
What I can speak to directly: configuration files- even templates for common tools- are not usually checked into contrib. On the other hand: you're right, there isn't a good place for this right now. As far as monitoring goes, there are several efforts currently in progress (Chukwa, metrics, HADOOP-3628 (the {{ping()}} functionality specifically)) that would be compelling points of integration.
> Monitoring Scripts for Nagios
> -----------------------------
>
> Key: HADOOP-4594
> URL: https://issues.apache.org/jira/browse/HADOOP-4594
> Project: Hadoop Core
> Issue Type: Wish
> Reporter: Edward Capriolo
> Priority: Minor
> Attachments: HADOOP-4594.patch
>
>
> I would like to create a set of local via NRPE and remote check scripts that can be shipped with the hadoop distribution and used to monitor Hadoop. I already have completed the NRPE scripts. The second set of scripts would use wget to read the output of the hadoop web interfaces. Do these already exist?
> I guess these would fall under a new contrib project.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4594) Monitoring Scripts for Nagios
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-4594:
----------------------------------
Status: Open (was: Patch Available)
* The copyright is unacceptable:
{noformat}
+#!/bin/sh
+#Copyright 2008 Edward Capriolo
+#www.jointhegrid.com
{noformat}
* A number of parameters are hard-coded
* While potentially useful, new contrib modules need to be more complete and general than what the current patch provides.
> Monitoring Scripts for Nagios
> -----------------------------
>
> Key: HADOOP-4594
> URL: https://issues.apache.org/jira/browse/HADOOP-4594
> Project: Hadoop Core
> Issue Type: Wish
> Reporter: Edward Capriolo
> Priority: Minor
> Attachments: HADOOP-4594.patch
>
>
> I would like to create a set of local via NRPE and remote check scripts that can be shipped with the hadoop distribution and used to monitor Hadoop. I already have completed the NRPE scripts. The second set of scripts would use wget to read the output of the hadoop web interfaces. Do these already exist?
> I guess these would fall under a new contrib project.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4594) Monitoring Scripts for Nagios
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12646010#action_12646010 ]
Edward Capriolo commented on HADOOP-4594:
-----------------------------------------
I have read up on Chukwa and HADOOP-3628, I think ping() would work with nagios well. My goal was to provided something that works today. I agree that jps and grep is not a great way to monitor anything, but I also believe in the 80 / 20 rule. Checking the reply of each Component web interface is a step better.
I was thinking checks like this might be meaningful useful without being complicated.
NumberOfDeadNodes > X -- This alarm would go off if the number of dead nodes in the cluster goes higher then X
PercentageOfDeadNodes > X -- This would alarm if the % of dead nodes goes higher then X
WriteFileReadFile (String hdfspath ) -- This would attempt to read and write a file.
ReadFile (String hdfspath) -- would attempt to read a file
TotalFreeDFSPrecent < X -- Would alarm when the DFS spaces falls below a certain value.
These are some things that someone in an administrative role would want.
> Monitoring Scripts for Nagios
> -----------------------------
>
> Key: HADOOP-4594
> URL: https://issues.apache.org/jira/browse/HADOOP-4594
> Project: Hadoop Core
> Issue Type: Wish
> Reporter: Edward Capriolo
> Priority: Minor
> Attachments: HADOOP-4594.patch
>
>
> I would like to create a set of local via NRPE and remote check scripts that can be shipped with the hadoop distribution and used to monitor Hadoop. I already have completed the NRPE scripts. The second set of scripts would use wget to read the output of the hadoop web interfaces. Do these already exist?
> I guess these would fall under a new contrib project.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4594) Monitoring Scripts for Nagios
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Capriolo updated HADOOP-4594:
------------------------------------
Release Note: Used to monitor hadoop with nagios
Status: Patch Available (was: Open)
> Monitoring Scripts for Nagios
> -----------------------------
>
> Key: HADOOP-4594
> URL: https://issues.apache.org/jira/browse/HADOOP-4594
> Project: Hadoop Core
> Issue Type: Wish
> Reporter: Edward Capriolo
> Priority: Minor
>
> I would like to create a set of local via NRPE and remote check scripts that can be shipped with the hadoop distribution and used to monitor Hadoop. I already have completed the NRPE scripts. The second set of scripts would use wget to read the output of the hadoop web interfaces. Do these already exist?
> I guess these would fall under a new contrib project.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4594) Monitoring Scripts for Nagios
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645954#action_12645954 ]
Edward Capriolo commented on HADOOP-4594:
-----------------------------------------
I removed the extra text, sorry about that.
As far as I can tell, nothing in the scripts/plugins are hard coded. the SAMPLE file is just example usage. Not part that would be installed on the nagios server.
For example:
command_line $USER1$/check_http -H $HOSTADDRESS$ -u http://$HOSTADDRESS$:$ARG1$/jobtracker.jsp -p $ARG1$ -r RUNNING
This check follows the standard nagios conventions in that it allows the user to specify host and port. -r RUNNING is a specific regular expression being searched for on the page. Since the status pages are in more of a human then machine readable format extracting information from them is difficult, Complex parsing of dfshealth.jsp is also subject to change.
Trying to make nagios check scripts more complete is difficult as nagios deals with only simple states.
Critical, Warning, OK, UNKNOWN, and a plugin only can output one status line.
Let me know what you think would make the scripts better.
> Monitoring Scripts for Nagios
> -----------------------------
>
> Key: HADOOP-4594
> URL: https://issues.apache.org/jira/browse/HADOOP-4594
> Project: Hadoop Core
> Issue Type: Wish
> Reporter: Edward Capriolo
> Priority: Minor
> Attachments: HADOOP-4594.patch
>
>
> I would like to create a set of local via NRPE and remote check scripts that can be shipped with the hadoop distribution and used to monitor Hadoop. I already have completed the NRPE scripts. The second set of scripts would use wget to read the output of the hadoop web interfaces. Do these already exist?
> I guess these would fall under a new contrib project.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4594) Monitoring Scripts for Nagios
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Capriolo updated HADOOP-4594:
------------------------------------
Attachment: HADOOP-4594.patch
> Monitoring Scripts for Nagios
> -----------------------------
>
> Key: HADOOP-4594
> URL: https://issues.apache.org/jira/browse/HADOOP-4594
> Project: Hadoop Core
> Issue Type: Wish
> Reporter: Edward Capriolo
> Priority: Minor
> Attachments: HADOOP-4594.patch
>
>
> I would like to create a set of local via NRPE and remote check scripts that can be shipped with the hadoop distribution and used to monitor Hadoop. I already have completed the NRPE scripts. The second set of scripts would use wget to read the output of the hadoop web interfaces. Do these already exist?
> I guess these would fall under a new contrib project.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4594) Monitoring Scripts for Nagios
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Capriolo updated HADOOP-4594:
------------------------------------
Attachment: HADOOP-4594.patch
Second Version
> Monitoring Scripts for Nagios
> -----------------------------
>
> Key: HADOOP-4594
> URL: https://issues.apache.org/jira/browse/HADOOP-4594
> Project: Hadoop Core
> Issue Type: Wish
> Reporter: Edward Capriolo
> Priority: Minor
> Attachments: HADOOP-4594.patch
>
>
> I would like to create a set of local via NRPE and remote check scripts that can be shipped with the hadoop distribution and used to monitor Hadoop. I already have completed the NRPE scripts. The second set of scripts would use wget to read the output of the hadoop web interfaces. Do these already exist?
> I guess these would fall under a new contrib project.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.