You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Nigel Daley (JIRA)" <ji...@apache.org> on 2006/12/07 19:30:21 UTC
[jira] Created: (HADOOP-794) JobTracker crashes with
ArithmeticException
JobTracker crashes with ArithmeticException
-------------------------------------------
Key: HADOOP-794
URL: http://issues.apache.org/jira/browse/HADOOP-794
Project: Hadoop
Issue Type: Bug
Components: mapred
Affects Versions: 0.9.1
Reporter: Nigel Daley
The sort benchmark on 20 nodes failed for me with an ArithmeticException. Sort.java calculates the number of maps and reduces dynamically based on the cluster size. I'm guessing at the time it did the calculation, the jobtracker had heard from 0 task trackers (which seems odd since I wait 1 minute after starting the jobtracker before starting Sort).
>From the job output:
Running on 0 nodes to sort from /sortBenchmark20/input into /sortBenchmark20/output with 0 reduces.
>From the JobTracker log:
2006-12-07 07:10:06,709 ERROR org.apache.hadoop.mapred.JobTracker: Job initialization failed:
java.lang.ArithmeticException: / by zero
at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:130)
at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:125)
at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:326)
at java.lang.Thread.run(Thread.java:595)
The jobtacker should check for illegal inputs like this and not fall over.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-794) JobTracker crashes with
ArithmeticException
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-794?page=comments#action_12457992 ]
Hadoop QA commented on HADOOP-794:
----------------------------------
+1, http://issues.apache.org/jira/secure/attachment/12347050/div-zero.patch applied and successfully tested against trunk revision r486399
> JobTracker crashes with ArithmeticException
> -------------------------------------------
>
> Key: HADOOP-794
> URL: http://issues.apache.org/jira/browse/HADOOP-794
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.1
> Reporter: Nigel Daley
> Assigned To: Owen O'Malley
> Fix For: 0.10.0
>
> Attachments: div-zero.patch
>
>
> The sort benchmark on 20 nodes failed for me with an ArithmeticException. Sort.java calculates the number of maps and reduces dynamically based on the cluster size. I'm guessing at the time it did the calculation, the jobtracker had heard from 0 task trackers (which seems odd since I wait 1 minute after starting the jobtracker before starting Sort).
> From the job output:
> Running on 0 nodes to sort from /sortBenchmark20/input into /sortBenchmark20/output with 0 reduces.
> From the JobTracker log:
> 2006-12-07 07:10:06,709 ERROR org.apache.hadoop.mapred.JobTracker: Job initialization failed:
> java.lang.ArithmeticException: / by zero
> at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:130)
> at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:125)
> at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:326)
> at java.lang.Thread.run(Thread.java:595)
> The jobtacker should check for illegal inputs like this and not fall over.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-794) JobTracker crashes with
ArithmeticException
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-794?page=all ]
Owen O'Malley updated HADOOP-794:
---------------------------------
Attachment: div-zero.patch
This patch protects against requesting 0 maps. The sort benchmark still will have problems if it is run with 0 task trackers because it will request 0 reduces. The proper fix for tat is to wait until there are a reasonable number of task trackers before submitting the job. This patch also adds a test case that requests 0 maps.
> JobTracker crashes with ArithmeticException
> -------------------------------------------
>
> Key: HADOOP-794
> URL: http://issues.apache.org/jira/browse/HADOOP-794
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.1
> Reporter: Nigel Daley
> Assigned To: Owen O'Malley
> Fix For: 0.10.0
>
> Attachments: div-zero.patch
>
>
> The sort benchmark on 20 nodes failed for me with an ArithmeticException. Sort.java calculates the number of maps and reduces dynamically based on the cluster size. I'm guessing at the time it did the calculation, the jobtracker had heard from 0 task trackers (which seems odd since I wait 1 minute after starting the jobtracker before starting Sort).
> From the job output:
> Running on 0 nodes to sort from /sortBenchmark20/input into /sortBenchmark20/output with 0 reduces.
> From the JobTracker log:
> 2006-12-07 07:10:06,709 ERROR org.apache.hadoop.mapred.JobTracker: Job initialization failed:
> java.lang.ArithmeticException: / by zero
> at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:130)
> at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:125)
> at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:326)
> at java.lang.Thread.run(Thread.java:595)
> The jobtacker should check for illegal inputs like this and not fall over.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (HADOOP-794) JobTracker crashes with
ArithmeticException
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-794?page=all ]
Owen O'Malley reassigned HADOOP-794:
------------------------------------
Assignee: Owen O'Malley
> JobTracker crashes with ArithmeticException
> -------------------------------------------
>
> Key: HADOOP-794
> URL: http://issues.apache.org/jira/browse/HADOOP-794
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.1
> Reporter: Nigel Daley
> Assigned To: Owen O'Malley
>
> The sort benchmark on 20 nodes failed for me with an ArithmeticException. Sort.java calculates the number of maps and reduces dynamically based on the cluster size. I'm guessing at the time it did the calculation, the jobtracker had heard from 0 task trackers (which seems odd since I wait 1 minute after starting the jobtracker before starting Sort).
> From the job output:
> Running on 0 nodes to sort from /sortBenchmark20/input into /sortBenchmark20/output with 0 reduces.
> From the JobTracker log:
> 2006-12-07 07:10:06,709 ERROR org.apache.hadoop.mapred.JobTracker: Job initialization failed:
> java.lang.ArithmeticException: / by zero
> at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:130)
> at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:125)
> at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:326)
> at java.lang.Thread.run(Thread.java:595)
> The jobtacker should check for illegal inputs like this and not fall over.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-794) JobTracker crashes with
ArithmeticException
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-794?page=all ]
Owen O'Malley updated HADOOP-794:
---------------------------------
Status: Patch Available (was: Open)
Fix Version/s: 0.10.0
> JobTracker crashes with ArithmeticException
> -------------------------------------------
>
> Key: HADOOP-794
> URL: http://issues.apache.org/jira/browse/HADOOP-794
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.1
> Reporter: Nigel Daley
> Assigned To: Owen O'Malley
> Fix For: 0.10.0
>
> Attachments: div-zero.patch
>
>
> The sort benchmark on 20 nodes failed for me with an ArithmeticException. Sort.java calculates the number of maps and reduces dynamically based on the cluster size. I'm guessing at the time it did the calculation, the jobtracker had heard from 0 task trackers (which seems odd since I wait 1 minute after starting the jobtracker before starting Sort).
> From the job output:
> Running on 0 nodes to sort from /sortBenchmark20/input into /sortBenchmark20/output with 0 reduces.
> From the JobTracker log:
> 2006-12-07 07:10:06,709 ERROR org.apache.hadoop.mapred.JobTracker: Job initialization failed:
> java.lang.ArithmeticException: / by zero
> at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:130)
> at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:125)
> at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:326)
> at java.lang.Thread.run(Thread.java:595)
> The jobtacker should check for illegal inputs like this and not fall over.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-794) JobTracker crashes with
ArithmeticException
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-794?page=all ]
Doug Cutting updated HADOOP-794:
--------------------------------
Status: Resolved (was: Patch Available)
Resolution: Fixed
I just committed this. Thanks, Owen!
> JobTracker crashes with ArithmeticException
> -------------------------------------------
>
> Key: HADOOP-794
> URL: http://issues.apache.org/jira/browse/HADOOP-794
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.9.1
> Reporter: Nigel Daley
> Assigned To: Owen O'Malley
> Fix For: 0.10.0
>
> Attachments: div-zero.patch
>
>
> The sort benchmark on 20 nodes failed for me with an ArithmeticException. Sort.java calculates the number of maps and reduces dynamically based on the cluster size. I'm guessing at the time it did the calculation, the jobtracker had heard from 0 task trackers (which seems odd since I wait 1 minute after starting the jobtracker before starting Sort).
> From the job output:
> Running on 0 nodes to sort from /sortBenchmark20/input into /sortBenchmark20/output with 0 reduces.
> From the JobTracker log:
> 2006-12-07 07:10:06,709 ERROR org.apache.hadoop.mapred.JobTracker: Job initialization failed:
> java.lang.ArithmeticException: / by zero
> at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:130)
> at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:125)
> at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:326)
> at java.lang.Thread.run(Thread.java:595)
> The jobtacker should check for illegal inputs like this and not fall over.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira