You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Thomas Graves (JIRA)" <ji...@apache.org> on 2012/09/19 17:31:07 UTC
[jira] [Created] (MAPREDUCE-4668) JettyBugMonitor thread should
kill TT if can't find jetty selector thread
Thomas Graves created MAPREDUCE-4668:
----------------------------------------
Summary: JettyBugMonitor thread should kill TT if can't find jetty selector thread
Key: MAPREDUCE-4668
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4668
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 1.0.2
Reporter: Thomas Graves
We ran into a case where a tasktracker started but for some reason the jetty thread didn't start or died shortly after start. I couldn't find anything useful as to why.
Since Jetty wasn't running the TT couldn't serve up any map output. The JettyBugMonitor thread saw this but just spit out the warning. We should have it just kill the TT since its basically unusable without it.
> 2012-07-23 05:42:04,072 WARN org.apache.hadoop.mapred.JettyBugMonitor: Could
not locate Jetty selector threads
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4668) JettyBugMonitor thread should
kill TT if can't find jetty selector thread
Posted by "Thomas Graves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458903#comment-13458903 ]
Thomas Graves commented on MAPREDUCE-4668:
------------------------------------------
thanks Todd, yeah we still haven't had a chance to try your patched jetty version.
I agree with you about being careful for the race at startup. I haven't checked our clusters to see if we see the warning message but the jetty thread is actually there.
> JettyBugMonitor thread should kill TT if can't find jetty selector thread
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-4668
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4668
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.2
> Reporter: Thomas Graves
>
> We ran into a case where a tasktracker started but for some reason the jetty thread didn't start or died shortly after start. I couldn't find anything useful as to why.
> Since Jetty wasn't running the TT couldn't serve up any map output. The JettyBugMonitor thread saw this but just spit out the warning. We should have it just kill the TT since its basically unusable without it.
> > 2012-07-23 05:42:04,072 WARN org.apache.hadoop.mapred.JettyBugMonitor: Could
> not locate Jetty selector threads
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4668) JettyBugMonitor thread should
kill TT if can't find jetty selector thread
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458822#comment-13458822 ]
Todd Lipcon commented on MAPREDUCE-4668:
----------------------------------------
Hey Thomas. This bug is due to JETTY-1316. We have a patched build of Jetty on our github: https://github.com/cloudera/jetty-hadoop-fix/commits/6.1.26.cloudera and also in our maven repository (version 6.1.26.cloudera.2), if you want to try it out.
Fixing the bug monitor to abort would also work, but we should be careful that there isn't a race at startup time, where the selector is in the process of starting but hasn't yet done so.
> JettyBugMonitor thread should kill TT if can't find jetty selector thread
> -------------------------------------------------------------------------
>
> Key: MAPREDUCE-4668
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4668
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.2
> Reporter: Thomas Graves
>
> We ran into a case where a tasktracker started but for some reason the jetty thread didn't start or died shortly after start. I couldn't find anything useful as to why.
> Since Jetty wasn't running the TT couldn't serve up any map output. The JettyBugMonitor thread saw this but just spit out the warning. We should have it just kill the TT since its basically unusable without it.
> > 2012-07-23 05:42:04,072 WARN org.apache.hadoop.mapred.JettyBugMonitor: Could
> not locate Jetty selector threads
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira