You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by 麦树荣 <sh...@qunar.com> on 2013/09/09 04:39:45 UTC
FATAL org.apache.hadoop.mapred.JettyBugMonitor question
hi，
Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor ，tasktracker shut down”problem in some hadoop computers. The log information is as follows:
2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor: ************************************************************
Jetty CPU usage: 120.6%. This is greater than the fatal threshold mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
After google, I find the following relative information:
The TaskTracker now has a thread which monitors for a known Jetty bug in which the selector thread starts spinning and map output can no longer be served. If the bug is detected, the TaskTracker will shut itself down. This feature can be disabled by setting mapred.tasktracker.jetty.cpu.check.enabled to false.
How do you solve the problem usually ? Is there a simple method to deal with the problem ?
Thanks.
回复: Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by 麦树荣 <sh...@qunar.com>.
hi, thank you for your reply.
Hadoop version is hadoop-0.20.2-cdh3u4 ,
I guess the jetty version is jetty-6.1.26 ( because I see the files "jetty-6.1.26.cloudera.1.jar", "jetty-servlet-tester-6.1.26.cloudera.1.jar","jetty-util-6.1.26.cloudera.1.jar " in $HADOOP_HOME/lib/ )
how to ship a patched Jetty ? can you give me website ?
________________________________
麦树荣
发件人： Harsh J<ma...@cloudera.com>
发送时间： 2013-09-14 18:44
收件人： <ma...@hadoop.apache.org> ; user6d6b4dda<ma...@qunar.com>
主题： Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
What version of jetty are you using? We've not seen this lately (after
we began shipping a patched Jetty), but the check is valid and
protects your MR jobs from getting into a hung or slow state.
On Fri, Sep 13, 2013 at 1:26 PM, 麦树荣 <sh...@qunar.com> wrote:
> No one gives me help ?
>
> ________________________________
> 麦树荣
>
> From: 麦树荣
> Date: 2013-09-09 10:39
> To: user@hadoop.apache.org
> Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
> hi，
>
> Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor
> ，tasktracker shut down”problem in some hadoop computers. The log information
> is as follows:
> 2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor:
> ************************************************************
> Jetty CPU usage: 120.6%. This is greater than the fatal threshold
> mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
>
> After google, I find the following relative information:
> The TaskTracker now has a thread which monitors for a known Jetty bug in
> which the selector thread starts spinning and map output can no longer be
> served. If the bug is detected, the TaskTracker will shut itself down. This
> feature can be disabled by setting
> mapred.tasktracker.jetty.cpu.check.enabled to false.
>
> How do you solve the problem usually ? Is there a simple method to deal with
> the problem ?
> Thanks.
--
Harsh J
回复: Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by 麦树荣 <sh...@qunar.com>.
hi, thank you for your reply.
Hadoop version is hadoop-0.20.2-cdh3u4 ,
I guess the jetty version is jetty-6.1.26 ( because I see the files "jetty-6.1.26.cloudera.1.jar", "jetty-servlet-tester-6.1.26.cloudera.1.jar","jetty-util-6.1.26.cloudera.1.jar " in $HADOOP_HOME/lib/ )
how to ship a patched Jetty ? can you give me website ?
________________________________
麦树荣
发件人： Harsh J<ma...@cloudera.com>
发送时间： 2013-09-14 18:44
收件人： <ma...@hadoop.apache.org> ; user6d6b4dda<ma...@qunar.com>
主题： Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
What version of jetty are you using? We've not seen this lately (after
we began shipping a patched Jetty), but the check is valid and
protects your MR jobs from getting into a hung or slow state.
On Fri, Sep 13, 2013 at 1:26 PM, 麦树荣 <sh...@qunar.com> wrote:
> No one gives me help ?
>
> ________________________________
> 麦树荣
>
> From: 麦树荣
> Date: 2013-09-09 10:39
> To: user@hadoop.apache.org
> Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
> hi，
>
> Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor
> ，tasktracker shut down”problem in some hadoop computers. The log information
> is as follows:
> 2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor:
> ************************************************************
> Jetty CPU usage: 120.6%. This is greater than the fatal threshold
> mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
>
> After google, I find the following relative information:
> The TaskTracker now has a thread which monitors for a known Jetty bug in
> which the selector thread starts spinning and map output can no longer be
> served. If the bug is detected, the TaskTracker will shut itself down. This
> feature can be disabled by setting
> mapred.tasktracker.jetty.cpu.check.enabled to false.
>
> How do you solve the problem usually ? Is there a simple method to deal with
> the problem ?
> Thanks.
--
Harsh J
回复: Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by 麦树荣 <sh...@qunar.com>.
hi, thank you for your reply.
Hadoop version is hadoop-0.20.2-cdh3u4 ,
I guess the jetty version is jetty-6.1.26 ( because I see the files "jetty-6.1.26.cloudera.1.jar", "jetty-servlet-tester-6.1.26.cloudera.1.jar","jetty-util-6.1.26.cloudera.1.jar " in $HADOOP_HOME/lib/ )
how to ship a patched Jetty ? can you give me website ?
________________________________
麦树荣
发件人： Harsh J<ma...@cloudera.com>
发送时间： 2013-09-14 18:44
收件人： <ma...@hadoop.apache.org> ; user6d6b4dda<ma...@qunar.com>
主题： Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
What version of jetty are you using? We've not seen this lately (after
we began shipping a patched Jetty), but the check is valid and
protects your MR jobs from getting into a hung or slow state.
On Fri, Sep 13, 2013 at 1:26 PM, 麦树荣 <sh...@qunar.com> wrote:
> No one gives me help ?
>
> ________________________________
> 麦树荣
>
> From: 麦树荣
> Date: 2013-09-09 10:39
> To: user@hadoop.apache.org
> Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
> hi，
>
> Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor
> ，tasktracker shut down”problem in some hadoop computers. The log information
> is as follows:
> 2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor:
> ************************************************************
> Jetty CPU usage: 120.6%. This is greater than the fatal threshold
> mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
>
> After google, I find the following relative information:
> The TaskTracker now has a thread which monitors for a known Jetty bug in
> which the selector thread starts spinning and map output can no longer be
> served. If the bug is detected, the TaskTracker will shut itself down. This
> feature can be disabled by setting
> mapred.tasktracker.jetty.cpu.check.enabled to false.
>
> How do you solve the problem usually ? Is there a simple method to deal with
> the problem ?
> Thanks.
--
Harsh J
回复: Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by 麦树荣 <sh...@qunar.com>.
hi, thank you for your reply.
Hadoop version is hadoop-0.20.2-cdh3u4 ,
I guess the jetty version is jetty-6.1.26 ( because I see the files "jetty-6.1.26.cloudera.1.jar", "jetty-servlet-tester-6.1.26.cloudera.1.jar","jetty-util-6.1.26.cloudera.1.jar " in $HADOOP_HOME/lib/ )
how to ship a patched Jetty ? can you give me website ?
________________________________
麦树荣
发件人： Harsh J<ma...@cloudera.com>
发送时间： 2013-09-14 18:44
收件人： <ma...@hadoop.apache.org> ; user6d6b4dda<ma...@qunar.com>
主题： Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
What version of jetty are you using? We've not seen this lately (after
we began shipping a patched Jetty), but the check is valid and
protects your MR jobs from getting into a hung or slow state.
On Fri, Sep 13, 2013 at 1:26 PM, 麦树荣 <sh...@qunar.com> wrote:
> No one gives me help ?
>
> ________________________________
> 麦树荣
>
> From: 麦树荣
> Date: 2013-09-09 10:39
> To: user@hadoop.apache.org
> Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
> hi，
>
> Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor
> ，tasktracker shut down”problem in some hadoop computers. The log information
> is as follows:
> 2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor:
> ************************************************************
> Jetty CPU usage: 120.6%. This is greater than the fatal threshold
> mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
>
> After google, I find the following relative information:
> The TaskTracker now has a thread which monitors for a known Jetty bug in
> which the selector thread starts spinning and map output can no longer be
> served. If the bug is detected, the TaskTracker will shut itself down. This
> feature can be disabled by setting
> mapred.tasktracker.jetty.cpu.check.enabled to false.
>
> How do you solve the problem usually ? Is there a simple method to deal with
> the problem ?
> Thanks.
--
Harsh J
Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by Harsh J <ha...@cloudera.com>.
What version of jetty are you using? We've not seen this lately (after
we began shipping a patched Jetty), but the check is valid and
protects your MR jobs from getting into a hung or slow state.
On Fri, Sep 13, 2013 at 1:26 PM, 麦树荣 <sh...@qunar.com> wrote:
> No one gives me help ?
>
> ________________________________
> 麦树荣
>
> From: 麦树荣
> Date: 2013-09-09 10:39
> To: user@hadoop.apache.org
> Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
> hi，
>
> Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor
> ，tasktracker shut down”problem in some hadoop computers. The log information
> is as follows:
> 2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor:
> ************************************************************
> Jetty CPU usage: 120.6%. This is greater than the fatal threshold
> mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
>
> After google, I find the following relative information:
> The TaskTracker now has a thread which monitors for a known Jetty bug in
> which the selector thread starts spinning and map output can no longer be
> served. If the bug is detected, the TaskTracker will shut itself down. This
> feature can be disabled by setting
> mapred.tasktracker.jetty.cpu.check.enabled to false.
>
> How do you solve the problem usually ? Is there a simple method to deal with
> the problem ?
> Thanks.
--
Harsh J
Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by Harsh J <ha...@cloudera.com>.
What version of jetty are you using? We've not seen this lately (after
we began shipping a patched Jetty), but the check is valid and
protects your MR jobs from getting into a hung or slow state.
On Fri, Sep 13, 2013 at 1:26 PM, 麦树荣 <sh...@qunar.com> wrote:
> No one gives me help ?
>
> ________________________________
> 麦树荣
>
> From: 麦树荣
> Date: 2013-09-09 10:39
> To: user@hadoop.apache.org
> Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
> hi，
>
> Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor
> ，tasktracker shut down”problem in some hadoop computers. The log information
> is as follows:
> 2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor:
> ************************************************************
> Jetty CPU usage: 120.6%. This is greater than the fatal threshold
> mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
>
> After google, I find the following relative information:
> The TaskTracker now has a thread which monitors for a known Jetty bug in
> which the selector thread starts spinning and map output can no longer be
> served. If the bug is detected, the TaskTracker will shut itself down. This
> feature can be disabled by setting
> mapred.tasktracker.jetty.cpu.check.enabled to false.
>
> How do you solve the problem usually ? Is there a simple method to deal with
> the problem ?
> Thanks.
--
Harsh J
Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by Harsh J <ha...@cloudera.com>.
What version of jetty are you using? We've not seen this lately (after
we began shipping a patched Jetty), but the check is valid and
protects your MR jobs from getting into a hung or slow state.
On Fri, Sep 13, 2013 at 1:26 PM, 麦树荣 <sh...@qunar.com> wrote:
> No one gives me help ?
>
> ________________________________
> 麦树荣
>
> From: 麦树荣
> Date: 2013-09-09 10:39
> To: user@hadoop.apache.org
> Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
> hi，
>
> Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor
> ，tasktracker shut down”problem in some hadoop computers. The log information
> is as follows:
> 2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor:
> ************************************************************
> Jetty CPU usage: 120.6%. This is greater than the fatal threshold
> mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
>
> After google, I find the following relative information:
> The TaskTracker now has a thread which monitors for a known Jetty bug in
> which the selector thread starts spinning and map output can no longer be
> served. If the bug is detected, the TaskTracker will shut itself down. This
> feature can be disabled by setting
> mapred.tasktracker.jetty.cpu.check.enabled to false.
>
> How do you solve the problem usually ? Is there a simple method to deal with
> the problem ?
> Thanks.
--
Harsh J
Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by Harsh J <ha...@cloudera.com>.
What version of jetty are you using? We've not seen this lately (after
we began shipping a patched Jetty), but the check is valid and
protects your MR jobs from getting into a hung or slow state.
On Fri, Sep 13, 2013 at 1:26 PM, 麦树荣 <sh...@qunar.com> wrote:
> No one gives me help ?
>
> ________________________________
> 麦树荣
>
> From: 麦树荣
> Date: 2013-09-09 10:39
> To: user@hadoop.apache.org
> Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
> hi，
>
> Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor
> ，tasktracker shut down”problem in some hadoop computers. The log information
> is as follows:
> 2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor:
> ************************************************************
> Jetty CPU usage: 120.6%. This is greater than the fatal threshold
> mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
>
> After google, I find the following relative information:
> The TaskTracker now has a thread which monitors for a known Jetty bug in
> which the selector thread starts spinning and map output can no longer be
> served. If the bug is detected, the TaskTracker will shut itself down. This
> feature can be disabled by setting
> mapred.tasktracker.jetty.cpu.check.enabled to false.
>
> How do you solve the problem usually ? Is there a simple method to deal with
> the problem ?
> Thanks.
--
Harsh J
Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by 麦树荣 <sh...@qunar.com>.
No one gives me help ?
________________________________
麦树荣
From: 麦树荣<ma...@qunar.com>
Date: 2013-09-09 10:39
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
hi，
Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor ，tasktracker shut down”problem in some hadoop computers. The log information is as follows:
2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor: ************************************************************
Jetty CPU usage: 120.6%. This is greater than the fatal threshold mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
After google, I find the following relative information:
The TaskTracker now has a thread which monitors for a known Jetty bug in which the selector thread starts spinning and map output can no longer be served. If the bug is detected, the TaskTracker will shut itself down. This feature can be disabled by setting mapred.tasktracker.jetty.cpu.check.enabled to false.
How do you solve the problem usually ? Is there a simple method to deal with the problem ?
Thanks.
Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by 麦树荣 <sh...@qunar.com>.
No one gives me help ?
________________________________
麦树荣
From: 麦树荣<ma...@qunar.com>
Date: 2013-09-09 10:39
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
hi，
Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor ，tasktracker shut down”problem in some hadoop computers. The log information is as follows:
2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor: ************************************************************
Jetty CPU usage: 120.6%. This is greater than the fatal threshold mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
After google, I find the following relative information:
The TaskTracker now has a thread which monitors for a known Jetty bug in which the selector thread starts spinning and map output can no longer be served. If the bug is detected, the TaskTracker will shut itself down. This feature can be disabled by setting mapred.tasktracker.jetty.cpu.check.enabled to false.
How do you solve the problem usually ? Is there a simple method to deal with the problem ?
Thanks.
Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by 麦树荣 <sh...@qunar.com>.
No one gives me help ?
________________________________
麦树荣
From: 麦树荣<ma...@qunar.com>
Date: 2013-09-09 10:39
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
hi，
Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor ，tasktracker shut down”problem in some hadoop computers. The log information is as follows:
2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor: ************************************************************
Jetty CPU usage: 120.6%. This is greater than the fatal threshold mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
After google, I find the following relative information:
The TaskTracker now has a thread which monitors for a known Jetty bug in which the selector thread starts spinning and map output can no longer be served. If the bug is detected, the TaskTracker will shut itself down. This feature can be disabled by setting mapred.tasktracker.jetty.cpu.check.enabled to false.
How do you solve the problem usually ? Is there a simple method to deal with the problem ?
Thanks.
Re: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
Posted by 麦树荣 <sh...@qunar.com>.
No one gives me help ?
________________________________
麦树荣
From: 麦树荣<ma...@qunar.com>
Date: 2013-09-09 10:39
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: FATAL org.apache.hadoop.mapred.JettyBugMonitor question
hi，
Recently，I encounter “FATAL org.apache.hadoop.mapred.JettyBugMonitor ，tasktracker shut down”problem in some hadoop computers. The log information is as follows:
2013-09-02 19:33:53,015 FATAL org.apache.hadoop.mapred.JettyBugMonitor: ************************************************************
Jetty CPU usage: 120.6%. This is greater than the fatal threshold mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
After google, I find the following relative information:
The TaskTracker now has a thread which monitors for a known Jetty bug in which the selector thread starts spinning and map output can no longer be served. If the bug is detected, the TaskTracker will shut itself down. This feature can be disabled by setting mapred.tasktracker.jetty.cpu.check.enabled to false.
How do you solve the problem usually ? Is there a simple method to deal with the problem ?
Thanks.