You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by maha <ma...@umail.ucsb.edu> on 2011/02/14 19:57:03 UTC

Check/Compare mappers output

Hi all,

  I want to know how can I check/compare mappers key and values.

Example:

  My Mappers have the following to filter documents before being output:

  String doc1;

 if(!doc1.equals("d1"))
    output.collect(new Text('#'+doc1+'#'), new Text('#'+word1.substring(word1.indexOf(',')+1, word1.indexOf('>'))+'#')); 
		 

 Yet the intermediate output still includes "d1":
#d1#	#1#
#d1#	#2#
#d1#	#1#
#d1#	#5#
#d1#	#3#
......

  I put '#' to see if there was a space or newline included. Any ideas?

Thank you,

Maha