How to use MapWritable?!

[Luiz Antonio Falaguasta Barbosa <la...@gmail.com>]

Hi people,

Please, I don't know what is happening with values I'm putting into a
MapWritable.

I think I have to call write method (or readFields?) but I don't know how.
Is it that I need to do to get values through 'LOG.info("freq: " +
docIDfreq.get(docIDfreqItr.next()));', for example?

Why does it work with key? Look: 'LOG.info("key: " +
(docIDfreqItr.next()));'

Look the code:

 private static final Log LOG = LogFactory.getLog(WordCount.class
.getName());


 public static class MapClass extends MapReduceBase

implements Mapper<LongWritable, Text, Text, MapWritable> {


 public void map(LongWritable key, Text value,

  OutputCollector<Text, MapWritable> output,

  Reporter reporter) throws IOException {


  String line = value.toString();

 StringTokenizer itr = new StringTokenizer(line);

 HashMap<String,Integer> map = new HashMap<String,Integer>();

 Integer valor = 0;


  while (itr.hasMoreTokens()) {

  try {

  valor =  map.get(itr.nextToken());


   if (valor == null) {

   map.put(itr.nextToken(), 1);

  }

  else {

   map.put(itr.nextToken(), valor + 1);

  }

  } catch (NoSuchElementException e) {

  e.printStackTrace();

  }

 }


  Iterator<String> mapItr = map.keySet().iterator();

 MapWritable docIDfreq = new MapWritable();


  while (mapItr.hasNext()){


  String term = mapItr.next();

  Integer freq = map.get(term);


  LOG.info("freq = " + freq);


  docIDfreq.put(key, new LongWritable(freq.longValue()));

    LOG.info("emitindo... " + "<" + term + "(" + key + "," + freq + ")>");

  output.collect(new Text(term), docIDfreq);

 }


  Iterator<Writable> docIDfreqItr = docIDfreq.keySet().iterator();


  while (docIDfreqItr.hasNext()){

  LOG.info("key: " + (docIDfreqItr.next()));

  LOG.info("freq: " + docIDfreq.get(docIDfreqItr.next()));

 }

 }

}

Thanks in advance!

Regards,

Luiz

Re: How to use MapWritable?!

[Luiz Antonio Falaguasta Barbosa <la...@gmail.com>]

That is what I get in Eclipse's console:

12/02/06 00:50:08 INFO WordCount: emitindo... <perif?rica(1,1)>

12/02/06 00:50:08 INFO WordCount: freq = 3

12/02/06 00:50:08 INFO WordCount: emitindo... <esp?cie(1,3)>

12/02/06 00:50:08 INFO WordCount: freq = 2

12/02/06 00:50:08 INFO WordCount: emitindo... <very(1,2)>

12/02/06 00:50:08 INFO WordCount: key: 1

12/02/06 00:50:08 WARN mapred.LocalJobRunner: job_local_0001

java.util.NoSuchElementException

at java.util.HashMap$HashIterator.nextEntry(HashMap.java:796)

at java.util.HashMap$KeyIterator.next(HashMap.java:828)

at WordCount$MapClass.map(WordCount.java:114)

at WordCount$MapClass.map(WordCount.java:1)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:435)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)

at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)

12/02/06 00:50:09 INFO mapred.JobClient: Job complete: job_local_0001


Where WordCount.java:114 is:

LOG.info("freq: " + docIDfreq.get(docIDfreqItr.next()));

If I put values there, why can't I recover them?


Regards,


Luiz

2012/2/5 Luiz Antonio Falaguasta Barbosa <la...@gmail.com>

> Hi people,
>
> Please, I don't know what is happening with values I'm putting into a
> MapWritable.
>
> I think I have to call write method (or readFields?) but I don't know how.
> Is it that I need to do to get values through 'LOG.info("freq: " +
> docIDfreq.get(docIDfreqItr.next()));', for example?
>
> Why does it work with key? Look: 'LOG.info("key: " +
> (docIDfreqItr.next()));'
>
> Look the code:
>
>  private static final Log LOG = LogFactory.getLog(WordCount.class
> .getName());
>
>
>  public static class MapClass extends MapReduceBase
>
> implements Mapper<LongWritable, Text, Text, MapWritable> {
>
>
>  public void map(LongWritable key, Text value,
>
>   OutputCollector<Text, MapWritable> output,
>
>   Reporter reporter) throws IOException {
>
>
>   String line = value.toString();
>
>  StringTokenizer itr = new StringTokenizer(line);
>
>  HashMap<String,Integer> map = new HashMap<String,Integer>();
>
>  Integer valor = 0;
>
>
>   while (itr.hasMoreTokens()) {
>
>   try {
>
>   valor =  map.get(itr.nextToken());
>
>
>    if (valor == null) {
>
>    map.put(itr.nextToken(), 1);
>
>   }
>
>   else {
>
>    map.put(itr.nextToken(), valor + 1);
>
>   }
>
>   } catch (NoSuchElementException e) {
>
>   e.printStackTrace();
>
>   }
>
>  }
>
>
>   Iterator<String> mapItr = map.keySet().iterator();
>
>  MapWritable docIDfreq = new MapWritable();
>
>
>   while (mapItr.hasNext()){
>
>
>   String term = mapItr.next();
>
>   Integer freq = map.get(term);
>
>
>   LOG.info("freq = " + freq);
>
>
>   docIDfreq.put(key, new LongWritable(freq.longValue()));
>
>     LOG.info("emitindo... " + "<" + term + "(" + key + "," + freq + ")>");
>
>   output.collect(new Text(term), docIDfreq);
>
>  }
>
>
>   Iterator<Writable> docIDfreqItr = docIDfreq.keySet().iterator();
>
>
>   while (docIDfreqItr.hasNext()){
>
>   LOG.info("key: " + (docIDfreqItr.next()));
>
>   LOG.info("freq: " + docIDfreq.get(docIDfreqItr.next()));
>
>  }
>
>  }
>
> }
>
> Thanks in advance!
>
> Regards,
>
> Luiz
>



-- 
[]s,

Luiz