Vertica & Storm Error

[satyajit sa <sa...@gmail.com>]

Hi ,

Have tried a simple program with storm, for inserting data into vertica
tables.

but facing the below error,

ERROR org.apache.zookeeper.server.NIOServerCnxnFactory - Thread
Thread[main,5,main] died
java.lang.RuntimeException: java.io.NotSerializableException:
com.vertica.jdbc.VerticaConnectionImpl
at backtype.storm.utils.Utils.serialize(Utils.java:81)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at
backtype.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:106)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at com.wordcount.example.HelloStorm.main(HelloStorm.java:21) ~[bin/:na]
Caused by: java.io.NotSerializableException:
com.vertica.jdbc.VerticaConnectionImpl
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
~[na:1.7.0_51]
at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
~[na:1.7.0_51]
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
~[na:1.7.0_51]
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
~[na:1.7.0_51]
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
~[na:1.7.0_51]
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
~[na:1.7.0_51]
at backtype.storm.utils.Utils.serialize(Utils.java:77)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
... 2 common frames omitted

com.vertica.jdbc.VerticaConnectionImpl is from VerticaJDBC driver.

is there any other way to avoid this problem and proceed with out erros.

Re: Vertica & Storm Error

[Nathan Leung <nc...@gmail.com>]

As Vladi says, you should initialize the connection in prepare(), not in
the constructor.  Any objects that are created in the constructor are
serialized and sent to the nimbus, so you will get this exception if you
create anything that is not serializable in your spout/bolt constructors.


On Fri, Jul 11, 2014 at 11:06 AM, Vladi Feigin <vl...@gmail.com> wrote:

> Probably you're trying to pass the VerticaConnectionImpl object as part of
> storm tuple but it's not serializable. This is why you fail
> Don't pass it - just initialize it in the persistence bolt (the bolt
> writing into Vertica) in the prepare method
> Vladi
>
>
> On Fri, Jul 11, 2014 at 2:46 PM, satyajit sa <sa...@gmail.com>
> wrote:
>
>> Hi ,
>>
>> Have tried a simple program with storm, for inserting data into vertica
>> tables.
>>
>> but facing the below error,
>>
>> ERROR org.apache.zookeeper.server.NIOServerCnxnFactory - Thread
>> Thread[main,5,main] died
>> java.lang.RuntimeException: java.io.NotSerializableException:
>> com.vertica.jdbc.VerticaConnectionImpl
>> at backtype.storm.utils.Utils.serialize(Utils.java:81)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> at
>> backtype.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:106)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> at com.wordcount.example.HelloStorm.main(HelloStorm.java:21) ~[bin/:na]
>> Caused by: java.io.NotSerializableException:
>> com.vertica.jdbc.VerticaConnectionImpl
>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
>> ~[na:1.7.0_51]
>> at
>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
>> ~[na:1.7.0_51]
>> at
>> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
>> ~[na:1.7.0_51]
>> at
>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
>> ~[na:1.7.0_51]
>> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
>> ~[na:1.7.0_51]
>> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
>> ~[na:1.7.0_51]
>> at backtype.storm.utils.Utils.serialize(Utils.java:77)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> ... 2 common frames omitted
>>
>> com.vertica.jdbc.VerticaConnectionImpl is from VerticaJDBC driver.
>>
>> is there any other way to avoid this problem and proceed with out erros.
>>
>>
>

Re: Vertica & Storm Error

[Vladi Feigin <vl...@gmail.com>]

Probably you're trying to pass the VerticaConnectionImpl object as part of
storm tuple but it's not serializable. This is why you fail
Don't pass it - just initialize it in the persistence bolt (the bolt
writing into Vertica) in the prepare method
Vladi


On Fri, Jul 11, 2014 at 2:46 PM, satyajit sa <sa...@gmail.com> wrote:

> Hi ,
>
> Have tried a simple program with storm, for inserting data into vertica
> tables.
>
> but facing the below error,
>
> ERROR org.apache.zookeeper.server.NIOServerCnxnFactory - Thread
> Thread[main,5,main] died
> java.lang.RuntimeException: java.io.NotSerializableException:
> com.vertica.jdbc.VerticaConnectionImpl
> at backtype.storm.utils.Utils.serialize(Utils.java:81)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:106)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at com.wordcount.example.HelloStorm.main(HelloStorm.java:21) ~[bin/:na]
> Caused by: java.io.NotSerializableException:
> com.vertica.jdbc.VerticaConnectionImpl
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
> ~[na:1.7.0_51]
> at
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> ~[na:1.7.0_51]
> at
> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> ~[na:1.7.0_51]
> at
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> ~[na:1.7.0_51]
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> ~[na:1.7.0_51]
> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
> ~[na:1.7.0_51]
> at backtype.storm.utils.Utils.serialize(Utils.java:77)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> ... 2 common frames omitted
>
> com.vertica.jdbc.VerticaConnectionImpl is from VerticaJDBC driver.
>
> is there any other way to avoid this problem and proceed with out erros.
>
>