You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Tom Melendez <to...@supertom.com> on 2012/04/01 00:50:05 UTC
Re: how to unit test my RawComparator
Hi Chris and all, hope you don't mind if I inject a question in here.
It's highly related IMO (famous last words).
On Sat, Mar 31, 2012 at 2:18 PM, Chris White <ch...@gmail.com> wrote:
> You can serialize your Writables to a ByteArrayOutputStream and then
> get it's underlying byte array:
>
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> DataOutputStream dos = new DataOutputStream(baos);
> Writable myWritable = new Text("text");
> myWritable.write(dos);
> byte[] bytes = baos.toByteArray();
>
I popped in this into a quick test and it failed. What I want are the
exact bytes back from the Writable (in my case, BytesWritable). So,
this fails for me:
@Test
public void byteswritabletest() {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(baos);
BytesWritable myBW = new BytesWritable("test".getBytes());
try {
myBW.write(dos);
} catch (IOException e) {
e.printStackTrace();
}
byte[] bytes = baos.toByteArray();
assertEquals("test".getBytes().length, bytes.length); //I get
expected: 4, actual 8 with this assertion
}
I see that in new versions of Text and BytesWritable, there is a
.copyBytes() method that is available that gives us that.
https://reviews.apache.org/r/182/diff/
Is there another way (without the upgrade) to achieve that?
Thanks,
Tom
Re: how to unit test my RawComparator
Posted by Chris White <ch...@gmail.com>.
BytesWritable writes the size of the byte array as an int (4 bytes)
then the contents of the byte array ("text".getBytes() == 4), so a
total of 8 bytes in total
On Sat, Mar 31, 2012 at 6:50 PM, Tom Melendez <to...@supertom.com> wrote:
> Hi Chris and all, hope you don't mind if I inject a question in here.
> It's highly related IMO (famous last words).
>
> On Sat, Mar 31, 2012 at 2:18 PM, Chris White <ch...@gmail.com> wrote:
>> You can serialize your Writables to a ByteArrayOutputStream and then
>> get it's underlying byte array:
>>
>> ByteArrayOutputStream baos = new ByteArrayOutputStream();
>> DataOutputStream dos = new DataOutputStream(baos);
>> Writable myWritable = new Text("text");
>> myWritable.write(dos);
>> byte[] bytes = baos.toByteArray();
>>
>
> I popped in this into a quick test and it failed. What I want are the
> exact bytes back from the Writable (in my case, BytesWritable). So,
> this fails for me:
>
> @Test
> public void byteswritabletest() {
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> DataOutputStream dos = new DataOutputStream(baos);
> BytesWritable myBW = new BytesWritable("test".getBytes());
> try {
> myBW.write(dos);
> } catch (IOException e) {
> e.printStackTrace();
> }
> byte[] bytes = baos.toByteArray();
> assertEquals("test".getBytes().length, bytes.length); //I get
> expected: 4, actual 8 with this assertion
> }
>
>
> I see that in new versions of Text and BytesWritable, there is a
> .copyBytes() method that is available that gives us that.
> https://reviews.apache.org/r/182/diff/
>
> Is there another way (without the upgrade) to achieve that?
>
> Thanks,
>
> Tom