You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Sanjay Dahiya (JIRA)" <ji...@apache.org> on 2006/09/12 18:39:22 UTC
[jira] Created: (HADOOP-523) TextInputformat .isSplittable() fails
with NullPointerException with hadoop 0.6.1
TextInputformat .isSplittable() fails with NullPointerException with hadoop 0.6.1
----------------------------------------------------------------------------------
Key: HADOOP-523
URL: http://issues.apache.org/jira/browse/HADOOP-523
Project: Hadoop
Issue Type: Bug
Components: mapred
Affects Versions: 0.6.0
Reporter: Sanjay Dahiya
Assigned To: Owen O'Malley
Fix For: 0.7.0
With hadoop 0.6.1, TextInputFormat fails with NullPointerException, causing Job with fail.
TextInputFormat.configure() is not called to initialize compressionCodecs and following method throws NullPointerException.
protected boolean isSplitable(FileSystem fs, Path file) {
return compressionCodecs.getCodec(file) == null;
}
java.lang.NullPointerException
at org.apache.hadoop.mapred.TextInputFormat.isSplitable(TextInputFormat.java:37)
at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:149)
at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:129)
at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:314)
at java.lang.Thread.run(Thread.java:613)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-523) TextInputformat .isSplittable() fails
with NullPointerException with hadoop 0.6.1
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-523?page=all ]
Owen O'Malley updated HADOOP-523:
---------------------------------
Attachment: configure-input-format.patch
This patch makes the InputFormat creation go through the standard utils to make sure it gets configured. It also replaces the redundant "jd" JobConf with "conf".
> TextInputformat .isSplittable() fails with NullPointerException with hadoop 0.6.1
> ---------------------------------------------------------------------------------
>
> Key: HADOOP-523
> URL: http://issues.apache.org/jira/browse/HADOOP-523
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.6.0
> Reporter: Sanjay Dahiya
> Assigned To: Owen O'Malley
> Fix For: 0.7.0
>
> Attachments: configure-input-format.patch
>
>
> With hadoop 0.6.1, TextInputFormat fails with NullPointerException, causing Job with fail.
> TextInputFormat.configure() is not called to initialize compressionCodecs and following method throws NullPointerException.
> protected boolean isSplitable(FileSystem fs, Path file) {
> return compressionCodecs.getCodec(file) == null;
> }
> java.lang.NullPointerException
> at org.apache.hadoop.mapred.TextInputFormat.isSplitable(TextInputFormat.java:37)
> at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:149)
> at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:129)
> at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:314)
> at java.lang.Thread.run(Thread.java:613)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (HADOOP-523) TextInputformat .isSplittable() fails
with NullPointerException with hadoop 0.6.1
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-523?page=all ]
Doug Cutting resolved HADOOP-523.
---------------------------------
Resolution: Fixed
I just committed this. Thanks, Owen!
> TextInputformat .isSplittable() fails with NullPointerException with hadoop 0.6.1
> ---------------------------------------------------------------------------------
>
> Key: HADOOP-523
> URL: http://issues.apache.org/jira/browse/HADOOP-523
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.6.0
> Reporter: Sanjay Dahiya
> Assigned To: Owen O'Malley
> Fix For: 0.6.1
>
> Attachments: configure-input-format.patch, test-configure-input-format.patch
>
>
> With hadoop 0.6.1, TextInputFormat fails with NullPointerException, causing Job with fail.
> TextInputFormat.configure() is not called to initialize compressionCodecs and following method throws NullPointerException.
> protected boolean isSplitable(FileSystem fs, Path file) {
> return compressionCodecs.getCodec(file) == null;
> }
> java.lang.NullPointerException
> at org.apache.hadoop.mapred.TextInputFormat.isSplitable(TextInputFormat.java:37)
> at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:149)
> at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:129)
> at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:314)
> at java.lang.Thread.run(Thread.java:613)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-523) TextInputformat .isSplittable() fails
with NullPointerException with hadoop 0.6.1
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-523?page=all ]
Owen O'Malley updated HADOOP-523:
---------------------------------
Fix Version/s: 0.6.1
(was: 0.7.0)
> TextInputformat .isSplittable() fails with NullPointerException with hadoop 0.6.1
> ---------------------------------------------------------------------------------
>
> Key: HADOOP-523
> URL: http://issues.apache.org/jira/browse/HADOOP-523
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.6.0
> Reporter: Sanjay Dahiya
> Assigned To: Owen O'Malley
> Fix For: 0.6.1
>
> Attachments: configure-input-format.patch
>
>
> With hadoop 0.6.1, TextInputFormat fails with NullPointerException, causing Job with fail.
> TextInputFormat.configure() is not called to initialize compressionCodecs and following method throws NullPointerException.
> protected boolean isSplitable(FileSystem fs, Path file) {
> return compressionCodecs.getCodec(file) == null;
> }
> java.lang.NullPointerException
> at org.apache.hadoop.mapred.TextInputFormat.isSplitable(TextInputFormat.java:37)
> at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:149)
> at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:129)
> at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:314)
> at java.lang.Thread.run(Thread.java:613)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-523) TextInputformat .isSplittable() fails
with NullPointerException with hadoop 0.6.1
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-523?page=all ]
Owen O'Malley updated HADOOP-523:
---------------------------------
Attachment: test-configure-input-format.patch
Here is a one line change to the regression tests that would have detected the problem.
> TextInputformat .isSplittable() fails with NullPointerException with hadoop 0.6.1
> ---------------------------------------------------------------------------------
>
> Key: HADOOP-523
> URL: http://issues.apache.org/jira/browse/HADOOP-523
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.6.0
> Reporter: Sanjay Dahiya
> Assigned To: Owen O'Malley
> Fix For: 0.6.1
>
> Attachments: configure-input-format.patch, test-configure-input-format.patch
>
>
> With hadoop 0.6.1, TextInputFormat fails with NullPointerException, causing Job with fail.
> TextInputFormat.configure() is not called to initialize compressionCodecs and following method throws NullPointerException.
> protected boolean isSplitable(FileSystem fs, Path file) {
> return compressionCodecs.getCodec(file) == null;
> }
> java.lang.NullPointerException
> at org.apache.hadoop.mapred.TextInputFormat.isSplitable(TextInputFormat.java:37)
> at org.apache.hadoop.mapred.InputFormatBase.getSplits(InputFormatBase.java:149)
> at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:129)
> at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:314)
> at java.lang.Thread.run(Thread.java:613)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira