Re: coprocessorExec got stucked with generic type

[Pavel Hančar <pa...@gmail.com>]

oops - I forgot to answer. The NameAndDistance doesn't implement
org.apache.hadoop.io.Writable, so thanks Gary, it looks like a good
alternative. But I stayed with SerializationUtils, because it looks simpler
than to implement the read and write methods.
 Pavel



2013/6/25 Kim Chew <kc...@gmail.com>

> I think I have run into a similar situation like Pavel.
>
> My method returns Map<Long, ArrayList<Foo>>, where Foo is
>
> public class Foo implements Writable {
>      String something;
>      long    counter1;
>      long    counter2;
>
>      blah........
> };
>
> And I got the following exception when I called my coprocessor method from
> a client,
>
> java.io.NotSerializableException: Foo
>     at
> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1164)
>     at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
>     at java.util.ArrayList.writeObject(ArrayList.java:570)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     at java.lang.reflect.Method.invoke(Method.java:597)
>     at
> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945)
>     at
> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1469)
>     at
>
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
>     at
> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
>     at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
>     at java.util.HashMap.writeObject(HashMap.java:1001)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>     at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>     at java.lang.reflect.Method.invoke(Method.java:597)
>     at
> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945)
>     at
> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1469)
>     at
>
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1400)
>     at
> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1158)
>     at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:330)
>     at
>
> org.apache.hadoop.hbase.io.HbaseObjectWritable.writeObject(HbaseObjectWritable.java:540)
>     at
>
> org.apache.hadoop.hbase.client.coprocessor.ExecResult.write(ExecResult.java:76)
>     at
>
> org.apache.hadoop.hbase.io.HbaseObjectWritable.writeObject(HbaseObjectWritable.java:525)
>     at
>
> org.apache.hadoop.hbase.io.HbaseObjectWritable.write(HbaseObjectWritable.java:335)
>     at
>
> org.apache.hadoop.hbase.ipc.HBaseServer$Call.setResponse(HBaseServer.java:365)
>     at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1442)
>
> What I do not understand is, how does the predicate at line 526 in
> HbaseObjectWritable is true if Foo does not implement Serializable?
>
>  } else if (Serializable.class.isAssignableFrom(declClass))
>
> BTW, making Foo serializable worked.
>
> Regards,
> Kim
>
>
> On Tue, Jun 11, 2013 at 9:48 AM, Gary Helmling <gh...@gmail.com>
> wrote:
>
> > Does your NameAndDistance class implement org.apache.hadoop.io.Writable?
> >  If so, it _should_ be serialized correctly.  There was a past issue
> > handling generic types in coprocessor endpoints, but that was fixed way
> > back (long before 0.94.2).  So, as far as I know, this should all be
> > working, assuming that NameAndDistance can be serialized.
> >
> >
> > On Mon, Jun 10, 2013 at 9:36 AM, Pavel Hančar <pavel.hancar@gmail.com
> > >wrote:
> >
> > >  It's org.apache.commons.lang.SerializationUtils
> > > I have it in hbase-0.94.2-cdh4.2.1/lib/commons-lang-2.5.jar
> > >  Pavel
> > >
> > >
> > > 2013/6/10 Ted Yu <yu...@gmail.com>
> > >
> > > > I searched for SerializationUtils class in hadoop (both branch-1 and
> > > > branch-2)
> > > > I also searched for SerializationUtils in hbase codebase.
> > > >
> > > > I didn't seem to find it.
> > > >
> > > > Is it an internal class of your project ?
> > > >
> > > > Cheers
> > > >
> > > > On Mon, Jun 10, 2013 at 6:11 AM, Pavel Hančar <
> pavel.hancar@gmail.com
> > > > >wrote:
> > > >
> > > > >  I see, that's probably big nonsense to return ArrayList (or array)
> > of
> > > > > another classes from coprocessor, because it's a list of pointers.
> > The
> > > > > solution is to serialize it to byte[] by
> > SerializationUtils.serialize(
> > > > > Serializable<
> > > > >
> > > >
> > >
> >
> http://java.sun.com/javase/6/docs/api/java/io/Serializable.html?is-external=true
> > > > > >
> > > > >  obj).
> > > > >   Pavel
> > > > >
> > > > >
> > > > > 2013/6/10 Pavel Hančar <pa...@gmail.com>
> > > > >
> > > > > >   Hello,
> > > > > > can I return from an EndPoint a generic type? I try to return
> > > > > > ArrayList<NameAndDistance> from an EndPoint method (where
> > > > NameAndDistance
> > > > > > is a simpe class with two public variables name and distance).
> But
> > > > when I
> > > > > > return  unempty ArrayList, the coprocessorExec get stucked.
> > > > > >   Thanks,
> > > > > >   Pavel Hančar
> > > > > >
> > > > >
> > > >
> > >
> >
>