You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by "Johannes.Lichtenberger" <Jo...@uni-konstanz.de> on 2010/10/08 14:01:36 UTC

NullPointerException

Hello,

I'm currently getting a NullPointerException:

java.lang.NullPointerException
	at
org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73)
	at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:797)
	at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:524)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

I've read it has something to do with the InputSplit (that doesn't
implement Writable), but I want to use FileSplit, which implements Writable.

public final class XMLInputFormat extends FileInputFormat<DateWritable,
Text> {

    static {
        System.setProperty("javax.xml.parsers.DocumentBuilderFactory",

"com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl");
    }

    /**
     * Empty constructor.
     */
    public XMLInputFormat() {
        // Default constructor.
    }

    @Override
    public RecordReader<DateWritable, Text> createRecordReader(final
InputSplit paramSplit,
        final TaskAttemptContext paramContext) throws IOException,
InterruptedException {
       return new XMLRecordReader();
//        reader.initialize(paramSplit, paramContext);
//        return reader;
    }
}

In XMLRecordReader:

    @Override
    public void initialize(final InputSplit paramGenericSplit, final
TaskAttemptContext paramContext)
        throws IOException {
        final FileSplit split = (FileSplit)paramGenericSplit;
        final Configuration conf = paramContext.getConfiguration();
        ...

Do you have any advice?

My DateWritable which is the key implements WritableComparable and my
value is Text, which also implements Writable.

regards,
Johannes


Re: NullPointerException

Posted by "Johannes.Lichtenberger" <Jo...@uni-konstanz.de>.
Just in case it matters, my DateWritable key class:

package com.treetank.wikipedia.hadoop;

import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import java.text.DateFormat;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Locale;

import org.apache.hadoop.io.WritableComparable;
import org.slf4j.LoggerFactory;

import com.treetank.service.xml.shredder.XMLShredder;
import com.treetank.utils.LogWrapper;

/**
 * <h1>DateWritable</h1>
 *
 * <p>
 * Simple date wrapper which implements {@link WritableComparable}, as
such it can be used as a key.
 * </p>
 *
 * @author Johannes Lichtenberger, University of Konstanz
 *
 */
public final class DateWritable implements
WritableComparable<DateWritable> {

    /**
     * {@link LogWrapper} used for logging.
     */
    private static final LogWrapper LOGWRAPPER = new
LogWrapper(LoggerFactory.getLogger(XMLShredder.class));

    /** {@link DateFormat}. */
    private final DateFormat mFormatter = new
SimpleDateFormat("yyyy.MM.dd HH.mm.ss", Locale.ENGLISH);

    /** {@link Date} used as the key. */
    private Date mTimestamp;

    /**
     * Constructor.
     */
    public DateWritable() {

    }

    /**
     * Constructor.
     *
     * @param paramTimestamp
     *                  The {@link Date} to use as the key.
     */
    public DateWritable(final Date paramTimestamp) {
        mTimestamp = paramTimestamp;
    }

    /**
     * Set timestamp.
     *
     * @param paramTimestamp
     *            The Timestamp to set.
     */
    public void setTimestamp(final Date paramTimestamp) {
        mTimestamp = paramTimestamp;
    }

    @Override
    public void readFields(final DataInput paramIn) throws IOException {
        try {
            mTimestamp = mFormatter.parse(paramIn.readUTF());
        } catch (final ParseException e) {
            LOGWRAPPER.error(e.getMessage(), e);
        }
    }

    @Override
    public void write(final DataOutput paramOut) throws IOException {
        paramOut.writeUTF(mFormatter.format(mTimestamp));
    }

    @Override
    public int compareTo(final DateWritable paramDate) {
        int retVal = 0;

        if (paramDate.mTimestamp.before(this.mTimestamp)) {
            retVal = -1;
        } else if (paramDate.mTimestamp.after(this.mTimestamp)) {
            retVal = 1;
        }

        return retVal;
    }
}


Re: NullPointerException

Posted by "Johannes.Lichtenberger" <Jo...@uni-konstanz.de>.
And attached my XMLRecordReader.

regards,
Johannes


Re: NullPointerException

Posted by "Johannes.Lichtenberger" <Jo...@uni-konstanz.de>.
Hm, I've explicitly castet to FileSplit but still the same.

final RecordReader<DateWritable, Text> reader =  new XMLRecordReader();
reader.initialize((FileSplit)paramSplit, paramContext);
return reader;

Anyone an idea? Would be really great :-/

regards,
Johannes