You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@mrunit.apache.org by bigdata <bi...@outlook.com> on 2013/02/27 04:26:02 UTC

HBase Result object problem when testing MapReduce

I have some mapreduce to calculate data in hbase. I try to use MRUnit to test it.









public void map(ImmutableBytesWritable row, Result value,
				Context context) throws IOException, InterruptedException {


			
			byte[] a = value.getValue(Bytes.toBytes("f"),
					Bytes.toBytes("a"));
			byte[] b = value.getValue(Bytes.toBytes("f"),
					Bytes.toBytes("b"));
			byte[] c = value.getValue(Bytes.toBytes("f"),
					Bytes.toBytes("c"));
			byte[] d = value.getValue(Bytes.toBytes("f"),
					Bytes.toBytes("e"));
			byte[] e = value.getValue(Bytes.toBytes("f"),
					Bytes.toBytes("e"));
connect(a,b,c,d,e);...}
in Test case code, I manually create a Result object to init some keyvalue data.







@Test
	public void testEventMap() throws IOException 
	{
		ArrayList<KeyValue> list = new ArrayList<KeyValue>();
		KeyValue k1 = KeyValueTestUtil.create("1", "f", "a", 1, "2013-02-01 12:23:23");
		KeyValue k2 = KeyValueTestUtil.create("1", "f", "b", 2, "1");
		KeyValue k3 = KeyValueTestUtil.create("1", "f", "c", 3, "1");
		KeyValue k4 = KeyValueTestUtil.create("1", "f", "d", 4,"1");
		KeyValue k5 = KeyValueTestUtil.create("1", "f", "e", 5, "1");









list.add(k1);
		list.add(k2);
		list.add(k3);
		
		list.add(k4);
		list.add(k5);

		
		
		ImmutableBytesWritable b = new ImmutableBytesWritable(Bytes.toBytes("1"));
		Result r = new Result();
		
		mapdriver.withInput(b, new Result(list));
		mapdriver.withOutput(new Text("1-20130201-1-1-1"), new IntWritable(1));}
The problem is Output is " 1-20130201-null-1-1" incorrect!!
I've adjust the list.add orders, to list.add(k2)list.add(k1)list.add(k3)list.add(k4)list.add(k5)
the result is changed to "null-null-null-null-null"
It seems KeyValue orders in Result object impacts the output. Why?
Anybody has suggestions about it?
Thanks.

HBase Result object problem when testing MapReduce

Posted by bigdata <bi...@outlook.com>.

I'm not sure this belongs to HBase or MRUnit.
It's about HBase Result object for MapReduce.





I have some mapreduce to calculate data in hbase. I try to use MRUnit to test it.









public void map(ImmutableBytesWritable row, Result value,
				Context context) throws IOException, InterruptedException {


			
			byte[] a = value.getValue(Bytes.toBytes("f"),
					Bytes.toBytes("a"));
			byte[] b = value.getValue(Bytes.toBytes("f"),
					Bytes.toBytes("b"));
			byte[] c = value.getValue(Bytes.toBytes("f"),
					Bytes.toBytes("c"));
			byte[] d = value.getValue(Bytes.toBytes("f"),
					Bytes.toBytes("e"));
			byte[] e = value.getValue(Bytes.toBytes("f"),
					Bytes.toBytes("e"));
connect(a,b,c,d,e);...}
in Test case code, I manually create a Result object to init some keyvalue data.







@Test
	public void testEventMap() throws IOException 
	{
		ArrayList<KeyValue> list = new ArrayList<KeyValue>();
		KeyValue k1 = KeyValueTestUtil.create("1", "f", "a", 1, "2013-02-01 12:23:23");
		KeyValue k2 = KeyValueTestUtil.create("1", "f", "b", 2, "1");
		KeyValue k3 = KeyValueTestUtil.create("1", "f", "c", 3, "1");
		KeyValue k4 = KeyValueTestUtil.create("1", "f", "d", 4,"1");
		KeyValue k5 = KeyValueTestUtil.create("1", "f", "e", 5, "1");









list.add(k1);
		list.add(k2);
		list.add(k3);
		
		list.add(k4);
		list.add(k5);

		
		
		ImmutableBytesWritable b = new ImmutableBytesWritable(Bytes.toBytes("1"));
		Result r = new Result();
		
		mapdriver.withInput(b, new Result(list));
		mapdriver.withOutput(new Text("1-20130201-1-1-1"), new IntWritable(1));}
The problem is Output is " 1-20130201-null-1-1" incorrect!!
I've adjust the list.add orders, to list.add(k2)list.add(k1)list.add(k3)list.add(k4)list.add(k5)
the result is changed to "null-null-null-null-null"
It seems KeyValue orders in Result object impacts the output. Why?
Anybody has suggestions about it?
Thanks.