You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by "Johannes.Lichtenberger" <Jo...@uni-konstanz.de> on 2010/10/08 14:01:36 UTC
NullPointerException
Hello,
I'm currently getting a NullPointerException:
java.lang.NullPointerException
at
org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:797)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:524)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
I've read it has something to do with the InputSplit (that doesn't
implement Writable), but I want to use FileSplit, which implements Writable.
public final class XMLInputFormat extends FileInputFormat<DateWritable,
Text> {
static {
System.setProperty("javax.xml.parsers.DocumentBuilderFactory",
"com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl");
}
/**
* Empty constructor.
*/
public XMLInputFormat() {
// Default constructor.
}
@Override
public RecordReader<DateWritable, Text> createRecordReader(final
InputSplit paramSplit,
final TaskAttemptContext paramContext) throws IOException,
InterruptedException {
return new XMLRecordReader();
// reader.initialize(paramSplit, paramContext);
// return reader;
}
}
In XMLRecordReader:
@Override
public void initialize(final InputSplit paramGenericSplit, final
TaskAttemptContext paramContext)
throws IOException {
final FileSplit split = (FileSplit)paramGenericSplit;
final Configuration conf = paramContext.getConfiguration();
...
Do you have any advice?
My DateWritable which is the key implements WritableComparable and my
value is Text, which also implements Writable.
regards,
Johannes
Re: NullPointerException
Posted by "Johannes.Lichtenberger" <Jo...@uni-konstanz.de>.
Just in case it matters, my DateWritable key class:
package com.treetank.wikipedia.hadoop;
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import java.text.DateFormat;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Locale;
import org.apache.hadoop.io.WritableComparable;
import org.slf4j.LoggerFactory;
import com.treetank.service.xml.shredder.XMLShredder;
import com.treetank.utils.LogWrapper;
/**
* <h1>DateWritable</h1>
*
* <p>
* Simple date wrapper which implements {@link WritableComparable}, as
such it can be used as a key.
* </p>
*
* @author Johannes Lichtenberger, University of Konstanz
*
*/
public final class DateWritable implements
WritableComparable<DateWritable> {
/**
* {@link LogWrapper} used for logging.
*/
private static final LogWrapper LOGWRAPPER = new
LogWrapper(LoggerFactory.getLogger(XMLShredder.class));
/** {@link DateFormat}. */
private final DateFormat mFormatter = new
SimpleDateFormat("yyyy.MM.dd HH.mm.ss", Locale.ENGLISH);
/** {@link Date} used as the key. */
private Date mTimestamp;
/**
* Constructor.
*/
public DateWritable() {
}
/**
* Constructor.
*
* @param paramTimestamp
* The {@link Date} to use as the key.
*/
public DateWritable(final Date paramTimestamp) {
mTimestamp = paramTimestamp;
}
/**
* Set timestamp.
*
* @param paramTimestamp
* The Timestamp to set.
*/
public void setTimestamp(final Date paramTimestamp) {
mTimestamp = paramTimestamp;
}
@Override
public void readFields(final DataInput paramIn) throws IOException {
try {
mTimestamp = mFormatter.parse(paramIn.readUTF());
} catch (final ParseException e) {
LOGWRAPPER.error(e.getMessage(), e);
}
}
@Override
public void write(final DataOutput paramOut) throws IOException {
paramOut.writeUTF(mFormatter.format(mTimestamp));
}
@Override
public int compareTo(final DateWritable paramDate) {
int retVal = 0;
if (paramDate.mTimestamp.before(this.mTimestamp)) {
retVal = -1;
} else if (paramDate.mTimestamp.after(this.mTimestamp)) {
retVal = 1;
}
return retVal;
}
}
Re: NullPointerException
Posted by "Johannes.Lichtenberger" <Jo...@uni-konstanz.de>.
And attached my XMLRecordReader.
regards,
Johannes
Re: NullPointerException
Posted by "Johannes.Lichtenberger" <Jo...@uni-konstanz.de>.
Hm, I've explicitly castet to FileSplit but still the same.
final RecordReader<DateWritable, Text> reader = new XMLRecordReader();
reader.initialize((FileSplit)paramSplit, paramContext);
return reader;
Anyone an idea? Would be really great :-/
regards,
Johannes