You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Tom Melendez <to...@supertom.com> on 2012/04/01 00:50:05 UTC

Re: how to unit test my RawComparator

Hi Chris and all, hope you don't mind if I inject a question in here.
It's highly related IMO (famous last words).

On Sat, Mar 31, 2012 at 2:18 PM, Chris White <ch...@gmail.com> wrote:
> You can serialize your Writables to a ByteArrayOutputStream and then
> get it's underlying byte array:
>
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> DataOutputStream dos = new DataOutputStream(baos);
> Writable myWritable = new Text("text");
> myWritable.write(dos);
> byte[] bytes = baos.toByteArray();
>

I popped in this into a quick test and it failed.  What I want are the
exact bytes back from the Writable (in my case, BytesWritable).  So,
this fails for me:

	@Test
	public void byteswritabletest() {
		ByteArrayOutputStream baos = new ByteArrayOutputStream();
		DataOutputStream dos = new DataOutputStream(baos);
		BytesWritable myBW = new BytesWritable("test".getBytes());
		try {
			myBW.write(dos);
		} catch (IOException e) {
			e.printStackTrace();
		}
		byte[] bytes = baos.toByteArray();
		assertEquals("test".getBytes().length, bytes.length);  //I get
expected: 4, actual 8 with this assertion
	}


I see that in new versions of Text and BytesWritable, there is a
.copyBytes() method that is available that gives us that.
https://reviews.apache.org/r/182/diff/

Is there another way (without the upgrade) to achieve that?

Thanks,

Tom

Re: how to unit test my RawComparator

Posted by Chris White <ch...@gmail.com>.
BytesWritable writes the size of the byte array as an int (4 bytes)
then the contents of the byte array ("text".getBytes() == 4), so a
total of 8 bytes in total

On Sat, Mar 31, 2012 at 6:50 PM, Tom Melendez <to...@supertom.com> wrote:
> Hi Chris and all, hope you don't mind if I inject a question in here.
> It's highly related IMO (famous last words).
>
> On Sat, Mar 31, 2012 at 2:18 PM, Chris White <ch...@gmail.com> wrote:
>> You can serialize your Writables to a ByteArrayOutputStream and then
>> get it's underlying byte array:
>>
>> ByteArrayOutputStream baos = new ByteArrayOutputStream();
>> DataOutputStream dos = new DataOutputStream(baos);
>> Writable myWritable = new Text("text");
>> myWritable.write(dos);
>> byte[] bytes = baos.toByteArray();
>>
>
> I popped in this into a quick test and it failed.  What I want are the
> exact bytes back from the Writable (in my case, BytesWritable).  So,
> this fails for me:
>
>        @Test
>        public void byteswritabletest() {
>                ByteArrayOutputStream baos = new ByteArrayOutputStream();
>                DataOutputStream dos = new DataOutputStream(baos);
>                BytesWritable myBW = new BytesWritable("test".getBytes());
>                try {
>                        myBW.write(dos);
>                } catch (IOException e) {
>                        e.printStackTrace();
>                }
>                byte[] bytes = baos.toByteArray();
>                assertEquals("test".getBytes().length, bytes.length);  //I get
> expected: 4, actual 8 with this assertion
>        }
>
>
> I see that in new versions of Text and BytesWritable, there is a
> .copyBytes() method that is available that gives us that.
> https://reviews.apache.org/r/182/diff/
>
> Is there another way (without the upgrade) to achieve that?
>
> Thanks,
>
> Tom