retrieve ObjectWritable

["Peter W." <pe...@marketingbrokers.com>]

Hi,

After trying to pass objects to reduce using
ObjectWritable without success I learned the
class instead sends primitives such as float.

However, you can make it go as object by
passing it as byte[] with:

new ObjectWritable(serialize_method(obj))

but it's not easy to retrieve once inside reduce
because ((ObjectWritable)values.next()).get()
returns an object not an array, no deserialize.

Trying to cast this object to it's original form
delivers a 'ClassCastException b[' error
(no ending square bracket) and empty part file.

How do you retrieve ObjectWritable?

Regards,

Peter W.

Re: retrieve ObjectWritable

["Peter W." <pe...@marketingbrokers.com>]

Hi,

If you have a requirement to pass an object or collection
you can replace ObjectWritable with BytesWritable:

test code...

import java.io.*;
import java.util.*;

import org.apache.hadoop.io.BytesWritable;

public class bao
    {
    public static void main(String args[])
       {
       HashMap hm=new HashMap();
       Map bm=Collections.synchronizedMap(hm);
       bm.put("123",new Integer(123));
       bm.put("456",new Integer(456));

       try
          {
          BytesWritable bw=new BytesWritable(serialize_method(bm));	
          Map m=(Map)deserialize_method(bw.get());
	 System.out.println("MAP_SIZE: "+ m.size()); // works
          }

       catch(Exception e)
          {
          System.out.println(e);
          }
       }

// static serialize,deserialize...

/***

stuff like this can be sent to reduce,
but can't recast from ObjectWritable:

    private static MapWritable mw(Map m)
       {
       MapWritable rmw=new MapWritable();
       List l=new ArrayList(m.keySet());Iterator i=l.iterator();
       while(i.hasNext()){Object k=i.next();rmw.put(new Text 
(k.toString()),
       new ObjectWritable(serialize_method((Object)m.get(k))));}
       return rmw;
       }

***/

    }

Good Luck,

Peter W.




Peter W. wrote:

> Hi,
>
> After trying to pass objects to reduce using
> ObjectWritable without success I learned the
> class instead sends primitives such as float.
>
> However, you can make it go as object by
> passing it as byte[] with:
>
> new ObjectWritable(serialize_method(obj))
>
> but it's not easy to retrieve once inside reduce
> because ((ObjectWritable)values.next()).get()
> returns an object not an array, no deserialize.
>
> Trying to cast this object to it's original form
> delivers a 'ClassCastException b[' error
> (no ending square bracket) and empty part file.
>
> How do you retrieve ObjectWritable?
>
> Regards,
>
> Peter W.