java.io.InvalidClassException: backtype.storm.daemon.common.WorkerHeartbeat

[Shun KAWAHARA <sh...@gmail.com>]

Hello.

I started Storm by the following constitution.

server1: nimbus, supervisor
server2: supervisor
server3: supervisor
server4: supervisor

However, An error has occurred in only server4.
Supervisor's log of server4 is following.
Please tell me the solution.

Shun


-----

2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
GMT
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client environment:host.name=server4
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
environment:java.version=1.6.0_24
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
environment:java.vendor=Sun Microsystems Inc.
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
environment:java.class.path=/opt/storm/storm-core-*.jar:/opt/storm/storm-netty-*.jar:/opt/storm/storm-console-logging-*.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.2-incubating.jar:/opt/storm/lib/netty-3.6.3.Final.jar:/opt/storm/lib/meat-locker-0.3.1.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/lib/javax.mail.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/httpclient-4.3.3.jar:/opt/storm/lib/curator-client-2.4.0.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/junit-3.8.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/disruptor-2.10.1.jar:/opt/storm/lib/zookeeper-3.4.5.jar:/opt/storm/lib/clj-stacktrace-0.2.4.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/commons-beanutils-1.8.3.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/logback-core-1.0.6.jar:/opt/storm/lib/httpcore-4.3.2.jar:/opt/storm/lib/curator-framework-2.4.0.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/servlet-api-2.5-20081211.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/javamail.jar:/opt/storm/lib/netty-3.2.2.Final.jar:/opt/storm/lib/json-lib-2.4-jdk15.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/slf4j-api-1.6.5.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/logback-classic-1.0.6.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/guava-13.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/activation.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/ezmorph-1.0.6.jar:/opt/storm/lib/commons-collections-3.2.1.jar:/opt/storm/lib/stanford-ner.jar:/opt/storm/lib/ring-devel-0.3.11.jar::/opt/storm/conf
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
environment:java.io.tmpdir=/tmp
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client environment:java.compiler=<NA>
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client environment:os.name=Linux
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd64
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
environment:os.version=2.6.32-358.6.2.el6.x86_64
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client environment:user.name=storm
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
environment:user.home=/opt/storm
2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
environment:user.dir=/opt/storm
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
GMT
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:host.name=server4
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.version=1.6.0_24
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.vendor=Sun Microsystems Inc.
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.class.path=/opt/storm/storm-core-*.jar:/opt/storm/storm-netty-*.jar:/opt/storm/storm-console-logging-*.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.2-incubating.jar:/opt/storm/lib/netty-3.6.3.Final.jar:/opt/storm/lib/meat-locker-0.3.1.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/lib/javax.mail.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/httpclient-4.3.3.jar:/opt/storm/lib/curator-client-2.4.0.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/junit-3.8.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/disruptor-2.10.1.jar:/opt/storm/lib/zookeeper-3.4.5.jar:/opt/storm/lib/clj-stacktrace-0.2.4.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/commons-beanutils-1.8.3.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/logback-core-1.0.6.jar:/opt/storm/lib/httpcore-4.3.2.jar:/opt/storm/lib/curator-framework-2.4.0.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/servlet-api-2.5-20081211.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/javamail.jar:/opt/storm/lib/netty-3.2.2.Final.jar:/opt/storm/lib/json-lib-2.4-jdk15.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/slf4j-api-1.6.5.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/logback-classic-1.0.6.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/guava-13.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/activation.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/ezmorph-1.0.6.jar:/opt/storm/lib/commons-collections-3.2.1.jar:/opt/storm/lib/stanford-ner.jar:/opt/storm/lib/ring-devel-0.3.11.jar::/opt/storm/conf
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.io.tmpdir=/tmp
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.compiler=<NA>
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:os.name=Linux
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:os.arch=amd64
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:os.version=2.6.32-358.6.2.el6.x86_64
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:user.name=storm
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:user.home=/opt/storm
2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
environment:user.dir=/opt/storm
2014-08-07 22:47:25 b.s.d.supervisor [INFO] Starting Supervisor with
conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
"topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000,
"topology.skip.missing.kryo.registrations" false,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts"
"-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign"
true, "topology.trident.batch.emit.interval.millis" 500,
"drpc.servers" ["xxx.xxx.xxx.xxx"],
"storm.messaging.netty.flush.check.interval.ms" 10,
"nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
"java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib",
"topology.executor.send.buffer.size" 1024, "storm.local.dir"
"/opt/storm", "storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120,
"topology.enable.message.timeouts" true,
"nimbus.cleanup.inbox.freq.secs" 600,
"nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
"topology.worker.shared.thread.pool.size" 4, "nimbus.host"
"xxx.xxx.xxx.xxx", "storm.messaging.netty.min_wait_ms" 100,
"storm.zookeeper.port" 2181, "transactional.zookeeper.port" nil,
"topology.executor.receive.buffer.size" 1024,
"transactional.zookeeper.servers" nil, "storm.zookeeper.root"
"/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
"supervisor.enable" true,
"storm.messaging.netty.server_worker_threads" 1,
"storm.zookeeper.servers" ["xxx.xxx.xxx.xxx"],
"transactional.zookeeper.root" "/transactional",
"topology.acker.executors" nil, "topology.transfer.buffer.size" 1024,
"topology.worker.childopts" nil, "drpc.queue.size" 128,
"worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs"
5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0,
"drpc.port" 3772, "supervisor.monitor.frequency.secs" 3,
"drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8,
"task.heartbeat.frequency.secs" 3, "topology.tasks" nil,
"storm.messaging.netty.max_retries" 30, "topology.spout.wait.strategy"
"backtype.storm.spout.SleepSpoutWaitStrategy",
"nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending"
nil, "storm.zookeeper.retry.interval" 1000,
"topology.sleep.spout.wait.strategy.time.ms" 1,
"nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator",
"supervisor.slots.ports" [6700 6701 6702 6703 6704 6705 6706 6707 6708
6709 6710 6711 6712 6713 6714], "topology.debug" false,
"nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60,
"topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10,
"topology.workers" 1, "supervisor.childopts" "-Xmx256m",
"nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05,
"worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy"
"com.lmax.disruptor.BlockingWaitStrategy",
"topology.multilang.serializer"
"backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs"
30, "storm.zookeeper.connection.timeout" 15000,
"topology.kryo.factory"
"backtype.storm.serialization.DefaultKryoFactory",
"drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1,
"storm.zookeeper.retry.times" 5,
"topology.worker.receiver.thread.count" 1, "storm.thrift.transport"
"backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30,
"nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport"
"backtype.storm.messaging.netty.Context", "logviewer.appender.name"
"A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false,
"ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode"
"distributed", "topology.max.task.parallelism" nil,
"storm.messaging.netty.transfer.batch.size" 262144,
"supervisor.scheduler.meta" {"name" "sarajevo-supervisor"}}
2014-08-07 22:47:29 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2014-08-07 22:47:29 o.a.z.ZooKeeper [INFO] Initiating client
connection, connectString=xxx.xxx.xxx.xxx:2181 sessionTimeout=20000
watcher=org.apache.curator.ConnectionState@b4dadde
2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Opening socket connection
to server server1/xxx.xxx.xxx.xxx:2181. Will not attempt to
authenticate using SASL (unknown error)
2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Socket connection
established to server1/xxx.xxx.xxx.xxx:2181, initiating session
2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Session establishment
complete on server server1/xxx.xxx.xxx.xxx:2181, sessionid =
0x147391967370924, negotiated timeout = 20000
2014-08-07 22:47:29 o.a.c.f.s.ConnectionStateManager [INFO] State
change: CONNECTED
2014-08-07 22:47:29 o.a.c.f.s.ConnectionStateManager [WARN] There are
no ConnectionStateListeners registered.
2014-08-07 22:47:29 b.s.zookeeper [INFO] Zookeeper state update: :connected:none
2014-08-07 22:47:30 o.a.z.ZooKeeper [INFO] Session: 0x147391967370924 closed
2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] EventThread shut down
2014-08-07 22:47:30 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2014-08-07 22:47:30 o.a.z.ZooKeeper [INFO] Initiating client
connection, connectString=xxx.xxx.xxx.xxx:2181/storm
sessionTimeout=20000
watcher=org.apache.curator.ConnectionState@4900269d
2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Opening socket connection
to server server1/xxx.xxx.xxx.xxx:2181. Will not attempt to
authenticate using SASL (unknown error)
2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Socket connection
established to server1/xxx.xxx.xxx.xxx:2181, initiating session
2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Session establishment
complete on server server1/xxx.xxx.xxx.xxx:2181, sessionid =
0x147391967370925, negotiated timeout = 20000
2014-08-07 22:47:30 o.a.c.f.s.ConnectionStateManager [INFO] State
change: CONNECTED
2014-08-07 22:47:30 o.a.c.f.s.ConnectionStateManager [WARN] There are
no ConnectionStateListeners registered.
2014-08-07 22:47:30 b.s.d.supervisor [INFO] Starting supervisor with
id e7126247-5801-4337-9beb-c4e62ca3142b at host server4
2014-08-07 22:47:32 b.s.event [ERROR] Error when processing event
java.lang.RuntimeException: java.io.InvalidClassException:
backtype.storm.daemon.common.WorkerHeartbeat; local class
incompatible: stream classdesc serialVersionUID =
-6996865048894131652, local class serialVersionUID =
2074174925015471843
at backtype.storm.utils.Utils.deserialize(Utils.java:93)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.utils.LocalState.snapshot(LocalState.java:45)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.utils.LocalState.get(LocalState.java:56)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.daemon.supervisor$read_worker_heartbeat.invoke(supervisor.clj:77)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.daemon.supervisor$read_worker_heartbeats$iter__6167__6171$fn__6172.invoke(supervisor.clj:90)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na]
at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na]
at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na]
at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na]
at clojure.core$dorun.invoke(core.clj:2780) ~[clojure-1.5.1.jar:na]
at clojure.core$doall.invoke(core.clj:2796) ~[clojure-1.5.1.jar:na]
at backtype.storm.daemon.supervisor$read_worker_heartbeats.invoke(supervisor.clj:89)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.daemon.supervisor$read_allocated_workers.invoke(supervisor.clj:106)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:209)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.applyToHelper(AFn.java:161) [clojure-1.5.1.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
at clojure.core$apply.invoke(core.clj:619) ~[clojure-1.5.1.jar:na]
at clojure.core$partial$fn__445.doInvoke(core.clj:2396) ~[na:na]
at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.5.1.jar:na]
at backtype.storm.event$event_manager$fn__2378.invoke(event.clj:39)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:679) [na:1.6.0_24]
Caused by: java.io.InvalidClassException:
backtype.storm.daemon.common.WorkerHeartbeat; local class
incompatible: stream classdesc serialVersionUID =
-6996865048894131652, local class serialVersionUID =
2074174925015471843
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:579)
~[na:1.6.0_24]
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1619)
~[na:1.6.0_24]
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)
~[na:1.6.0_24]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1768)
~[na:1.6.0_24]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
~[na:1.6.0_24]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
~[na:1.6.0_24]
at java.util.HashMap.readObject(HashMap.java:1047) ~[na:1.6.0_24]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_24]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[na:1.6.0_24]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.6.0_24]
at java.lang.reflect.Method.invoke(Method.java:616) ~[na:1.6.0_24]
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:988)
~[na:1.6.0_24]
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1890)
~[na:1.6.0_24]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1795)
~[na:1.6.0_24]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
~[na:1.6.0_24]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
~[na:1.6.0_24]
at backtype.storm.utils.Utils.deserialize(Utils.java:89)
~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
... 21 common frames omitted
2014-08-07 22:47:32 b.s.util [INFO] Halting process: ("Error when
processing an event")

Re: java.io.InvalidClassException: backtype.storm.daemon.common.WorkerHeartbeat

[Shun KAWAHARA <sh...@gmail.com>]

Thank you for answering my questions.
I was able to solve a problem!

2014-08-08 14:10 GMT+09:00 Harsha <st...@harsha.io>:
> Make sure you bring down your topologies and stop all the storm daemons
> and zookeeper.
> From your config it looks like /opt/storm-local is your dir .Delete the
> this dir contents and check your zookeeper config zoo.cfg look for
> dataDir location and delete the content.  Restart your zookeeper and
> storm
> -Harsha
>
> On Thu, Aug 7, 2014, at 09:42 PM, Shun KAWAHARA wrote:
>> Thank you for answering my questions.
>>
>> However, I don't know how to clear storm-local and zookeeper data.
>> Sorry, please tell me how to clear theirs.
>>
>> Shun.
>>
>> 2014-08-08 13:17 GMT+09:00 Harsha <st...@harsha.io>:
>> > "local class
>> > incompatible: stream classdesc serialVersionUID =
>> > -6996865048894131652, local class serialVersionUID =
>> > 2074174925015471843"
>> >
>> > The above error usually happens when the storm versions (usually
>> > dependent jars ) differ.
>> > If its new installation make sure you've same version of storm on every
>> > node.
>> > If you are upgrading clear storm-local and zookeeper data and restart
>> > the daemons.
>> > If it persists I'll try clearing storm-local and zookeeper data.
>> > -Harsha
>> >
>> > On Thu, Aug 7, 2014, at 08:58 PM, Shun KAWAHARA wrote:
>> >> Hello.
>> >>
>> >> I started Storm by the following constitution.
>> >>
>> >> server1: nimbus, supervisor
>> >> server2: supervisor
>> >> server3: supervisor
>> >> server4: supervisor
>> >>
>> >> However, An error has occurred in only server4.
>> >> Supervisor's log of server4 is following.
>> >> Please tell me the solution.
>> >>
>> >> Shun
>> >>
>> >>
>> >> -----
>> >>
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
>> >> GMT
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:host.name=server4
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:java.version=1.6.0_24
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:java.vendor=Sun Microsystems Inc.
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:java.class.path=/opt/storm/storm-core-*.jar:/opt/storm/storm-netty-*.jar:/opt/storm/storm-console-logging-*.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.2-incubating.jar:/opt/storm/lib/netty-3.6.3.Final.jar:/opt/storm/lib/meat-locker-0.3.1.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/lib/javax.mail.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/httpclient-4.3.3.jar:/opt/storm/lib/curator-client-2.4.0.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/junit-3.8.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/disruptor-2.10.1.jar:/opt/storm/lib/zookeeper-3.4.5.jar:/opt/storm/lib/clj-stacktrace-0.2.4.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/commons-beanutils-1.8.3.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/logback-core-1.0.6.jar:/opt/storm/lib/httpcore-4.3.2.jar:/opt/storm/lib/curator-framework-2.4.0.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/servlet-api-2.5-20081211.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/javamail.jar:/opt/storm/lib/netty-3.2.2.Final.jar:/opt/storm/lib/json-lib-2.4-jdk15.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/slf4j-api-1.6.5.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/logback-classic-1.0.6.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/guava-13.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/activation.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/ezmorph-1.0.6.jar:/opt/storm/lib/commons-collections-3.2.1.jar:/opt/storm/lib/stanford-ner.jar:/opt/storm/lib/ring-devel-0.3.11.jar::/opt/storm/conf
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:java.io.tmpdir=/tmp
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:java.compiler=<NA>
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:os.name=Linux
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:os.arch=amd64
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:os.version=2.6.32-358.6.2.el6.x86_64
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:user.name=storm
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:user.home=/opt/storm
>> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> >> environment:user.dir=/opt/storm
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
>> >> GMT
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:host.name=server4
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:java.version=1.6.0_24
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:java.vendor=Sun Microsystems Inc.
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:java.class.path=/opt/storm/storm-core-*.jar:/opt/storm/storm-netty-*.jar:/opt/storm/storm-console-logging-*.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.2-incubating.jar:/opt/storm/lib/netty-3.6.3.Final.jar:/opt/storm/lib/meat-locker-0.3.1.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/lib/javax.mail.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/httpclient-4.3.3.jar:/opt/storm/lib/curator-client-2.4.0.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/junit-3.8.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/disruptor-2.10.1.jar:/opt/storm/lib/zookeeper-3.4.5.jar:/opt/storm/lib/clj-stacktrace-0.2.4.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/commons-beanutils-1.8.3.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/logback-core-1.0.6.jar:/opt/storm/lib/httpcore-4.3.2.jar:/opt/storm/lib/curator-framework-2.4.0.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/servlet-api-2.5-20081211.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/javamail.jar:/opt/storm/lib/netty-3.2.2.Final.jar:/opt/storm/lib/json-lib-2.4-jdk15.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/slf4j-api-1.6.5.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/logback-classic-1.0.6.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/guava-13.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/activation.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/ezmorph-1.0.6.jar:/opt/storm/lib/commons-collections-3.2.1.jar:/opt/storm/lib/stanford-ner.jar:/opt/storm/lib/ring-devel-0.3.11.jar::/opt/storm/conf
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:java.io.tmpdir=/tmp
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:java.compiler=<NA>
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:os.name=Linux
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:os.arch=amd64
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:os.version=2.6.32-358.6.2.el6.x86_64
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:user.name=storm
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:user.home=/opt/storm
>> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> >> environment:user.dir=/opt/storm
>> >> 2014-08-07 22:47:25 b.s.d.supervisor [INFO] Starting Supervisor with
>> >> conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
>> >> "topology.tick.tuple.freq.secs" nil,
>> >> "topology.builtin.metrics.bucket.size.secs" 60,
>> >> "topology.fall.back.on.java.serialization" true,
>> >> "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000,
>> >> "topology.skip.missing.kryo.registrations" false,
>> >> "storm.messaging.netty.client_worker_threads" 1, "ui.childopts"
>> >> "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign"
>> >> true, "topology.trident.batch.emit.interval.millis" 500,
>> >> "drpc.servers" ["xxx.xxx.xxx.xxx"],
>> >> "storm.messaging.netty.flush.check.interval.ms" 10,
>> >> "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
>> >> "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib",
>> >> "topology.executor.send.buffer.size" 1024, "storm.local.dir"
>> >> "/opt/storm", "storm.messaging.netty.buffer_size" 5242880,
>> >> "supervisor.worker.start.timeout.secs" 120,
>> >> "topology.enable.message.timeouts" true,
>> >> "nimbus.cleanup.inbox.freq.secs" 600,
>> >> "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
>> >> "topology.worker.shared.thread.pool.size" 4, "nimbus.host"
>> >> "xxx.xxx.xxx.xxx", "storm.messaging.netty.min_wait_ms" 100,
>> >> "storm.zookeeper.port" 2181, "transactional.zookeeper.port" nil,
>> >> "topology.executor.receive.buffer.size" 1024,
>> >> "transactional.zookeeper.servers" nil, "storm.zookeeper.root"
>> >> "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
>> >> "supervisor.enable" true,
>> >> "storm.messaging.netty.server_worker_threads" 1,
>> >> "storm.zookeeper.servers" ["xxx.xxx.xxx.xxx"],
>> >> "transactional.zookeeper.root" "/transactional",
>> >> "topology.acker.executors" nil, "topology.transfer.buffer.size" 1024,
>> >> "topology.worker.childopts" nil, "drpc.queue.size" 128,
>> >> "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs"
>> >> 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0,
>> >> "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3,
>> >> "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8,
>> >> "task.heartbeat.frequency.secs" 3, "topology.tasks" nil,
>> >> "storm.messaging.netty.max_retries" 30, "topology.spout.wait.strategy"
>> >> "backtype.storm.spout.SleepSpoutWaitStrategy",
>> >> "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending"
>> >> nil, "storm.zookeeper.retry.interval" 1000,
>> >> "topology.sleep.spout.wait.strategy.time.ms" 1,
>> >> "nimbus.topology.validator"
>> >> "backtype.storm.nimbus.DefaultTopologyValidator",
>> >> "supervisor.slots.ports" [6700 6701 6702 6703 6704 6705 6706 6707 6708
>> >> 6709 6710 6711 6712 6713 6714], "topology.debug" false,
>> >> "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60,
>> >> "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10,
>> >> "topology.workers" 1, "supervisor.childopts" "-Xmx256m",
>> >> "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05,
>> >> "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer"
>> >> "backtype.storm.serialization.types.ListDelegateSerializer",
>> >> "topology.disruptor.wait.strategy"
>> >> "com.lmax.disruptor.BlockingWaitStrategy",
>> >> "topology.multilang.serializer"
>> >> "backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs"
>> >> 30, "storm.zookeeper.connection.timeout" 15000,
>> >> "topology.kryo.factory"
>> >> "backtype.storm.serialization.DefaultKryoFactory",
>> >> "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1,
>> >> "storm.zookeeper.retry.times" 5,
>> >> "topology.worker.receiver.thread.count" 1, "storm.thrift.transport"
>> >> "backtype.storm.security.auth.SimpleTransportPlugin",
>> >> "topology.state.synchronization.timeout.secs" 60,
>> >> "supervisor.worker.timeout.secs" 30,
>> >> "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport"
>> >> "backtype.storm.messaging.netty.Context", "logviewer.appender.name"
>> >> "A1", "storm.messaging.netty.max_wait_ms" 1000,
>> >> "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false,
>> >> "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode"
>> >> "distributed", "topology.max.task.parallelism" nil,
>> >> "storm.messaging.netty.transfer.batch.size" 262144,
>> >> "supervisor.scheduler.meta" {"name" "sarajevo-supervisor"}}
>> >> 2014-08-07 22:47:29 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> >> 2014-08-07 22:47:29 o.a.z.ZooKeeper [INFO] Initiating client
>> >> connection, connectString=xxx.xxx.xxx.xxx:2181 sessionTimeout=20000
>> >> watcher=org.apache.curator.ConnectionState@b4dadde
>> >> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Opening socket connection
>> >> to server server1/xxx.xxx.xxx.xxx:2181. Will not attempt to
>> >> authenticate using SASL (unknown error)
>> >> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Socket connection
>> >> established to server1/xxx.xxx.xxx.xxx:2181, initiating session
>> >> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Session establishment
>> >> complete on server server1/xxx.xxx.xxx.xxx:2181, sessionid =
>> >> 0x147391967370924, negotiated timeout = 20000
>> >> 2014-08-07 22:47:29 o.a.c.f.s.ConnectionStateManager [INFO] State
>> >> change: CONNECTED
>> >> 2014-08-07 22:47:29 o.a.c.f.s.ConnectionStateManager [WARN] There are
>> >> no ConnectionStateListeners registered.
>> >> 2014-08-07 22:47:29 b.s.zookeeper [INFO] Zookeeper state update:
>> >> :connected:none
>> >> 2014-08-07 22:47:30 o.a.z.ZooKeeper [INFO] Session: 0x147391967370924
>> >> closed
>> >> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] EventThread shut down
>> >> 2014-08-07 22:47:30 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> >> 2014-08-07 22:47:30 o.a.z.ZooKeeper [INFO] Initiating client
>> >> connection, connectString=xxx.xxx.xxx.xxx:2181/storm
>> >> sessionTimeout=20000
>> >> watcher=org.apache.curator.ConnectionState@4900269d
>> >> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Opening socket connection
>> >> to server server1/xxx.xxx.xxx.xxx:2181. Will not attempt to
>> >> authenticate using SASL (unknown error)
>> >> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Socket connection
>> >> established to server1/xxx.xxx.xxx.xxx:2181, initiating session
>> >> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Session establishment
>> >> complete on server server1/xxx.xxx.xxx.xxx:2181, sessionid =
>> >> 0x147391967370925, negotiated timeout = 20000
>> >> 2014-08-07 22:47:30 o.a.c.f.s.ConnectionStateManager [INFO] State
>> >> change: CONNECTED
>> >> 2014-08-07 22:47:30 o.a.c.f.s.ConnectionStateManager [WARN] There are
>> >> no ConnectionStateListeners registered.
>> >> 2014-08-07 22:47:30 b.s.d.supervisor [INFO] Starting supervisor with
>> >> id e7126247-5801-4337-9beb-c4e62ca3142b at host server4
>> >> 2014-08-07 22:47:32 b.s.event [ERROR] Error when processing event
>> >> java.lang.RuntimeException: java.io.InvalidClassException:
>> >> backtype.storm.daemon.common.WorkerHeartbeat; local class
>> >> incompatible: stream classdesc serialVersionUID =
>> >> -6996865048894131652, local class serialVersionUID =
>> >> 2074174925015471843
>> >> at backtype.storm.utils.Utils.deserialize(Utils.java:93)
>> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> >> at backtype.storm.utils.LocalState.snapshot(LocalState.java:45)
>> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> >> at backtype.storm.utils.LocalState.get(LocalState.java:56)
>> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> >> at
>> >> backtype.storm.daemon.supervisor$read_worker_heartbeat.invoke(supervisor.clj:77)
>> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> >> at
>> >> backtype.storm.daemon.supervisor$read_worker_heartbeats$iter__6167__6171$fn__6172.invoke(supervisor.clj:90)
>> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> >> at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na]
>> >> at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na]
>> >> at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na]
>> >> at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na]
>> >> at clojure.core$dorun.invoke(core.clj:2780) ~[clojure-1.5.1.jar:na]
>> >> at clojure.core$doall.invoke(core.clj:2796) ~[clojure-1.5.1.jar:na]
>> >> at
>> >> backtype.storm.daemon.supervisor$read_worker_heartbeats.invoke(supervisor.clj:89)
>> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> >> at
>> >> backtype.storm.daemon.supervisor$read_allocated_workers.invoke(supervisor.clj:106)
>> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> >> at
>> >> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:209)
>> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> >> at clojure.lang.AFn.applyToHelper(AFn.java:161) [clojure-1.5.1.jar:na]
>> >> at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
>> >> at clojure.core$apply.invoke(core.clj:619) ~[clojure-1.5.1.jar:na]
>> >> at clojure.core$partial$fn__445.doInvoke(core.clj:2396) ~[na:na]
>> >> at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.5.1.jar:na]
>> >> at backtype.storm.event$event_manager$fn__2378.invoke(event.clj:39)
>> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> >> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
>> >> at java.lang.Thread.run(Thread.java:679) [na:1.6.0_24]
>> >> Caused by: java.io.InvalidClassException:
>> >> backtype.storm.daemon.common.WorkerHeartbeat; local class
>> >> incompatible: stream classdesc serialVersionUID =
>> >> -6996865048894131652, local class serialVersionUID =
>> >> 2074174925015471843
>> >> at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:579)
>> >> ~[na:1.6.0_24]
>> >> at
>> >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1619)
>> >> ~[na:1.6.0_24]
>> >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)
>> >> ~[na:1.6.0_24]
>> >> at
>> >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1768)
>> >> ~[na:1.6.0_24]
>> >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
>> >> ~[na:1.6.0_24]
>> >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
>> >> ~[na:1.6.0_24]
>> >> at java.util.HashMap.readObject(HashMap.java:1047) ~[na:1.6.0_24]
>> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >> ~[na:1.6.0_24]
>> >> at
>> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >> ~[na:1.6.0_24]
>> >> at
>> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> ~[na:1.6.0_24]
>> >> at java.lang.reflect.Method.invoke(Method.java:616) ~[na:1.6.0_24]
>> >> at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:988)
>> >> ~[na:1.6.0_24]
>> >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1890)
>> >> ~[na:1.6.0_24]
>> >> at
>> >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1795)
>> >> ~[na:1.6.0_24]
>> >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
>> >> ~[na:1.6.0_24]
>> >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
>> >> ~[na:1.6.0_24]
>> >> at backtype.storm.utils.Utils.deserialize(Utils.java:89)
>> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> >> ... 21 common frames omitted
>> >> 2014-08-07 22:47:32 b.s.util [INFO] Halting process: ("Error when
>> >> processing an event")

Re: java.io.InvalidClassException: backtype.storm.daemon.common.WorkerHeartbeat

[Harsha <st...@harsha.io>]

Make sure you bring down your topologies and stop all the storm daemons
and zookeeper.
>From your config it looks like /opt/storm-local is your dir .Delete the
this dir contents and check your zookeeper config zoo.cfg look for
dataDir location and delete the content.  Restart your zookeeper and
storm
-Harsha

On Thu, Aug 7, 2014, at 09:42 PM, Shun KAWAHARA wrote:
> Thank you for answering my questions.
> 
> However, I don't know how to clear storm-local and zookeeper data.
> Sorry, please tell me how to clear theirs.
> 
> Shun.
> 
> 2014-08-08 13:17 GMT+09:00 Harsha <st...@harsha.io>:
> > "local class
> > incompatible: stream classdesc serialVersionUID =
> > -6996865048894131652, local class serialVersionUID =
> > 2074174925015471843"
> >
> > The above error usually happens when the storm versions (usually
> > dependent jars ) differ.
> > If its new installation make sure you've same version of storm on every
> > node.
> > If you are upgrading clear storm-local and zookeeper data and restart
> > the daemons.
> > If it persists I'll try clearing storm-local and zookeeper data.
> > -Harsha
> >
> > On Thu, Aug 7, 2014, at 08:58 PM, Shun KAWAHARA wrote:
> >> Hello.
> >>
> >> I started Storm by the following constitution.
> >>
> >> server1: nimbus, supervisor
> >> server2: supervisor
> >> server3: supervisor
> >> server4: supervisor
> >>
> >> However, An error has occurred in only server4.
> >> Supervisor's log of server4 is following.
> >> Please tell me the solution.
> >>
> >> Shun
> >>
> >>
> >> -----
> >>
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
> >> GMT
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:host.name=server4
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:java.version=1.6.0_24
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:java.vendor=Sun Microsystems Inc.
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:java.class.path=/opt/storm/storm-core-*.jar:/opt/storm/storm-netty-*.jar:/opt/storm/storm-console-logging-*.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.2-incubating.jar:/opt/storm/lib/netty-3.6.3.Final.jar:/opt/storm/lib/meat-locker-0.3.1.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/lib/javax.mail.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/httpclient-4.3.3.jar:/opt/storm/lib/curator-client-2.4.0.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/junit-3.8.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/disruptor-2.10.1.jar:/opt/storm/lib/zookeeper-3.4.5.jar:/opt/storm/lib/clj-stacktrace-0.2.4.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/commons-beanutils-1.8.3.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/logback-core-1.0.6.jar:/opt/storm/lib/httpcore-4.3.2.jar:/opt/storm/lib/curator-framework-2.4.0.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/servlet-api-2.5-20081211.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/javamail.jar:/opt/storm/lib/netty-3.2.2.Final.jar:/opt/storm/lib/json-lib-2.4-jdk15.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/slf4j-api-1.6.5.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/logback-classic-1.0.6.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/guava-13.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/activation.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/ezmorph-1.0.6.jar:/opt/storm/lib/commons-collections-3.2.1.jar:/opt/storm/lib/stanford-ner.jar:/opt/storm/lib/ring-devel-0.3.11.jar::/opt/storm/conf
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:java.io.tmpdir=/tmp
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:java.compiler=<NA>
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:os.name=Linux
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:os.arch=amd64
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:os.version=2.6.32-358.6.2.el6.x86_64
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:user.name=storm
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:user.home=/opt/storm
> >> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> >> environment:user.dir=/opt/storm
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
> >> GMT
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:host.name=server4
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:java.version=1.6.0_24
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:java.vendor=Sun Microsystems Inc.
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:java.class.path=/opt/storm/storm-core-*.jar:/opt/storm/storm-netty-*.jar:/opt/storm/storm-console-logging-*.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.2-incubating.jar:/opt/storm/lib/netty-3.6.3.Final.jar:/opt/storm/lib/meat-locker-0.3.1.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/lib/javax.mail.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/httpclient-4.3.3.jar:/opt/storm/lib/curator-client-2.4.0.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/junit-3.8.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/disruptor-2.10.1.jar:/opt/storm/lib/zookeeper-3.4.5.jar:/opt/storm/lib/clj-stacktrace-0.2.4.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/commons-beanutils-1.8.3.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/logback-core-1.0.6.jar:/opt/storm/lib/httpcore-4.3.2.jar:/opt/storm/lib/curator-framework-2.4.0.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/servlet-api-2.5-20081211.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/javamail.jar:/opt/storm/lib/netty-3.2.2.Final.jar:/opt/storm/lib/json-lib-2.4-jdk15.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/slf4j-api-1.6.5.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/logback-classic-1.0.6.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/guava-13.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/activation.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/ezmorph-1.0.6.jar:/opt/storm/lib/commons-collections-3.2.1.jar:/opt/storm/lib/stanford-ner.jar:/opt/storm/lib/ring-devel-0.3.11.jar::/opt/storm/conf
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:java.io.tmpdir=/tmp
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:java.compiler=<NA>
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:os.name=Linux
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:os.arch=amd64
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:os.version=2.6.32-358.6.2.el6.x86_64
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:user.name=storm
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:user.home=/opt/storm
> >> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> >> environment:user.dir=/opt/storm
> >> 2014-08-07 22:47:25 b.s.d.supervisor [INFO] Starting Supervisor with
> >> conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
> >> "topology.tick.tuple.freq.secs" nil,
> >> "topology.builtin.metrics.bucket.size.secs" 60,
> >> "topology.fall.back.on.java.serialization" true,
> >> "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000,
> >> "topology.skip.missing.kryo.registrations" false,
> >> "storm.messaging.netty.client_worker_threads" 1, "ui.childopts"
> >> "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign"
> >> true, "topology.trident.batch.emit.interval.millis" 500,
> >> "drpc.servers" ["xxx.xxx.xxx.xxx"],
> >> "storm.messaging.netty.flush.check.interval.ms" 10,
> >> "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
> >> "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib",
> >> "topology.executor.send.buffer.size" 1024, "storm.local.dir"
> >> "/opt/storm", "storm.messaging.netty.buffer_size" 5242880,
> >> "supervisor.worker.start.timeout.secs" 120,
> >> "topology.enable.message.timeouts" true,
> >> "nimbus.cleanup.inbox.freq.secs" 600,
> >> "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
> >> "topology.worker.shared.thread.pool.size" 4, "nimbus.host"
> >> "xxx.xxx.xxx.xxx", "storm.messaging.netty.min_wait_ms" 100,
> >> "storm.zookeeper.port" 2181, "transactional.zookeeper.port" nil,
> >> "topology.executor.receive.buffer.size" 1024,
> >> "transactional.zookeeper.servers" nil, "storm.zookeeper.root"
> >> "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
> >> "supervisor.enable" true,
> >> "storm.messaging.netty.server_worker_threads" 1,
> >> "storm.zookeeper.servers" ["xxx.xxx.xxx.xxx"],
> >> "transactional.zookeeper.root" "/transactional",
> >> "topology.acker.executors" nil, "topology.transfer.buffer.size" 1024,
> >> "topology.worker.childopts" nil, "drpc.queue.size" 128,
> >> "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs"
> >> 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0,
> >> "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3,
> >> "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8,
> >> "task.heartbeat.frequency.secs" 3, "topology.tasks" nil,
> >> "storm.messaging.netty.max_retries" 30, "topology.spout.wait.strategy"
> >> "backtype.storm.spout.SleepSpoutWaitStrategy",
> >> "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending"
> >> nil, "storm.zookeeper.retry.interval" 1000,
> >> "topology.sleep.spout.wait.strategy.time.ms" 1,
> >> "nimbus.topology.validator"
> >> "backtype.storm.nimbus.DefaultTopologyValidator",
> >> "supervisor.slots.ports" [6700 6701 6702 6703 6704 6705 6706 6707 6708
> >> 6709 6710 6711 6712 6713 6714], "topology.debug" false,
> >> "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60,
> >> "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10,
> >> "topology.workers" 1, "supervisor.childopts" "-Xmx256m",
> >> "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05,
> >> "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer"
> >> "backtype.storm.serialization.types.ListDelegateSerializer",
> >> "topology.disruptor.wait.strategy"
> >> "com.lmax.disruptor.BlockingWaitStrategy",
> >> "topology.multilang.serializer"
> >> "backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs"
> >> 30, "storm.zookeeper.connection.timeout" 15000,
> >> "topology.kryo.factory"
> >> "backtype.storm.serialization.DefaultKryoFactory",
> >> "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1,
> >> "storm.zookeeper.retry.times" 5,
> >> "topology.worker.receiver.thread.count" 1, "storm.thrift.transport"
> >> "backtype.storm.security.auth.SimpleTransportPlugin",
> >> "topology.state.synchronization.timeout.secs" 60,
> >> "supervisor.worker.timeout.secs" 30,
> >> "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport"
> >> "backtype.storm.messaging.netty.Context", "logviewer.appender.name"
> >> "A1", "storm.messaging.netty.max_wait_ms" 1000,
> >> "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false,
> >> "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode"
> >> "distributed", "topology.max.task.parallelism" nil,
> >> "storm.messaging.netty.transfer.batch.size" 262144,
> >> "supervisor.scheduler.meta" {"name" "sarajevo-supervisor"}}
> >> 2014-08-07 22:47:29 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
> >> 2014-08-07 22:47:29 o.a.z.ZooKeeper [INFO] Initiating client
> >> connection, connectString=xxx.xxx.xxx.xxx:2181 sessionTimeout=20000
> >> watcher=org.apache.curator.ConnectionState@b4dadde
> >> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Opening socket connection
> >> to server server1/xxx.xxx.xxx.xxx:2181. Will not attempt to
> >> authenticate using SASL (unknown error)
> >> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Socket connection
> >> established to server1/xxx.xxx.xxx.xxx:2181, initiating session
> >> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Session establishment
> >> complete on server server1/xxx.xxx.xxx.xxx:2181, sessionid =
> >> 0x147391967370924, negotiated timeout = 20000
> >> 2014-08-07 22:47:29 o.a.c.f.s.ConnectionStateManager [INFO] State
> >> change: CONNECTED
> >> 2014-08-07 22:47:29 o.a.c.f.s.ConnectionStateManager [WARN] There are
> >> no ConnectionStateListeners registered.
> >> 2014-08-07 22:47:29 b.s.zookeeper [INFO] Zookeeper state update:
> >> :connected:none
> >> 2014-08-07 22:47:30 o.a.z.ZooKeeper [INFO] Session: 0x147391967370924
> >> closed
> >> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] EventThread shut down
> >> 2014-08-07 22:47:30 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
> >> 2014-08-07 22:47:30 o.a.z.ZooKeeper [INFO] Initiating client
> >> connection, connectString=xxx.xxx.xxx.xxx:2181/storm
> >> sessionTimeout=20000
> >> watcher=org.apache.curator.ConnectionState@4900269d
> >> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Opening socket connection
> >> to server server1/xxx.xxx.xxx.xxx:2181. Will not attempt to
> >> authenticate using SASL (unknown error)
> >> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Socket connection
> >> established to server1/xxx.xxx.xxx.xxx:2181, initiating session
> >> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Session establishment
> >> complete on server server1/xxx.xxx.xxx.xxx:2181, sessionid =
> >> 0x147391967370925, negotiated timeout = 20000
> >> 2014-08-07 22:47:30 o.a.c.f.s.ConnectionStateManager [INFO] State
> >> change: CONNECTED
> >> 2014-08-07 22:47:30 o.a.c.f.s.ConnectionStateManager [WARN] There are
> >> no ConnectionStateListeners registered.
> >> 2014-08-07 22:47:30 b.s.d.supervisor [INFO] Starting supervisor with
> >> id e7126247-5801-4337-9beb-c4e62ca3142b at host server4
> >> 2014-08-07 22:47:32 b.s.event [ERROR] Error when processing event
> >> java.lang.RuntimeException: java.io.InvalidClassException:
> >> backtype.storm.daemon.common.WorkerHeartbeat; local class
> >> incompatible: stream classdesc serialVersionUID =
> >> -6996865048894131652, local class serialVersionUID =
> >> 2074174925015471843
> >> at backtype.storm.utils.Utils.deserialize(Utils.java:93)
> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> >> at backtype.storm.utils.LocalState.snapshot(LocalState.java:45)
> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> >> at backtype.storm.utils.LocalState.get(LocalState.java:56)
> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> >> at
> >> backtype.storm.daemon.supervisor$read_worker_heartbeat.invoke(supervisor.clj:77)
> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> >> at
> >> backtype.storm.daemon.supervisor$read_worker_heartbeats$iter__6167__6171$fn__6172.invoke(supervisor.clj:90)
> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> >> at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na]
> >> at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na]
> >> at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na]
> >> at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na]
> >> at clojure.core$dorun.invoke(core.clj:2780) ~[clojure-1.5.1.jar:na]
> >> at clojure.core$doall.invoke(core.clj:2796) ~[clojure-1.5.1.jar:na]
> >> at
> >> backtype.storm.daemon.supervisor$read_worker_heartbeats.invoke(supervisor.clj:89)
> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> >> at
> >> backtype.storm.daemon.supervisor$read_allocated_workers.invoke(supervisor.clj:106)
> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> >> at
> >> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:209)
> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> >> at clojure.lang.AFn.applyToHelper(AFn.java:161) [clojure-1.5.1.jar:na]
> >> at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
> >> at clojure.core$apply.invoke(core.clj:619) ~[clojure-1.5.1.jar:na]
> >> at clojure.core$partial$fn__445.doInvoke(core.clj:2396) ~[na:na]
> >> at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.5.1.jar:na]
> >> at backtype.storm.event$event_manager$fn__2378.invoke(event.clj:39)
> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> >> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
> >> at java.lang.Thread.run(Thread.java:679) [na:1.6.0_24]
> >> Caused by: java.io.InvalidClassException:
> >> backtype.storm.daemon.common.WorkerHeartbeat; local class
> >> incompatible: stream classdesc serialVersionUID =
> >> -6996865048894131652, local class serialVersionUID =
> >> 2074174925015471843
> >> at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:579)
> >> ~[na:1.6.0_24]
> >> at
> >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1619)
> >> ~[na:1.6.0_24]
> >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)
> >> ~[na:1.6.0_24]
> >> at
> >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1768)
> >> ~[na:1.6.0_24]
> >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
> >> ~[na:1.6.0_24]
> >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
> >> ~[na:1.6.0_24]
> >> at java.util.HashMap.readObject(HashMap.java:1047) ~[na:1.6.0_24]
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> ~[na:1.6.0_24]
> >> at
> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> ~[na:1.6.0_24]
> >> at
> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> ~[na:1.6.0_24]
> >> at java.lang.reflect.Method.invoke(Method.java:616) ~[na:1.6.0_24]
> >> at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:988)
> >> ~[na:1.6.0_24]
> >> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1890)
> >> ~[na:1.6.0_24]
> >> at
> >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1795)
> >> ~[na:1.6.0_24]
> >> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
> >> ~[na:1.6.0_24]
> >> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
> >> ~[na:1.6.0_24]
> >> at backtype.storm.utils.Utils.deserialize(Utils.java:89)
> >> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> >> ... 21 common frames omitted
> >> 2014-08-07 22:47:32 b.s.util [INFO] Halting process: ("Error when
> >> processing an event")

Re: java.io.InvalidClassException: backtype.storm.daemon.common.WorkerHeartbeat

[Shun KAWAHARA <sh...@gmail.com>]

Thank you for answering my questions.

However, I don't know how to clear storm-local and zookeeper data.
Sorry, please tell me how to clear theirs.

Shun.

2014-08-08 13:17 GMT+09:00 Harsha <st...@harsha.io>:
> "local class
> incompatible: stream classdesc serialVersionUID =
> -6996865048894131652, local class serialVersionUID =
> 2074174925015471843"
>
> The above error usually happens when the storm versions (usually
> dependent jars ) differ.
> If its new installation make sure you've same version of storm on every
> node.
> If you are upgrading clear storm-local and zookeeper data and restart
> the daemons.
> If it persists I'll try clearing storm-local and zookeeper data.
> -Harsha
>
> On Thu, Aug 7, 2014, at 08:58 PM, Shun KAWAHARA wrote:
>> Hello.
>>
>> I started Storm by the following constitution.
>>
>> server1: nimbus, supervisor
>> server2: supervisor
>> server3: supervisor
>> server4: supervisor
>>
>> However, An error has occurred in only server4.
>> Supervisor's log of server4 is following.
>> Please tell me the solution.
>>
>> Shun
>>
>>
>> -----
>>
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
>> GMT
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:host.name=server4
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:java.version=1.6.0_24
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:java.vendor=Sun Microsystems Inc.
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:java.class.path=/opt/storm/storm-core-*.jar:/opt/storm/storm-netty-*.jar:/opt/storm/storm-console-logging-*.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.2-incubating.jar:/opt/storm/lib/netty-3.6.3.Final.jar:/opt/storm/lib/meat-locker-0.3.1.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/lib/javax.mail.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/httpclient-4.3.3.jar:/opt/storm/lib/curator-client-2.4.0.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/junit-3.8.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/disruptor-2.10.1.jar:/opt/storm/lib/zookeeper-3.4.5.jar:/opt/storm/lib/clj-stacktrace-0.2.4.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/commons-beanutils-1.8.3.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/logback-core-1.0.6.jar:/opt/storm/lib/httpcore-4.3.2.jar:/opt/storm/lib/curator-framework-2.4.0.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/servlet-api-2.5-20081211.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/javamail.jar:/opt/storm/lib/netty-3.2.2.Final.jar:/opt/storm/lib/json-lib-2.4-jdk15.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/slf4j-api-1.6.5.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/logback-classic-1.0.6.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/guava-13.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/activation.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/ezmorph-1.0.6.jar:/opt/storm/lib/commons-collections-3.2.1.jar:/opt/storm/lib/stanford-ner.jar:/opt/storm/lib/ring-devel-0.3.11.jar::/opt/storm/conf
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:java.io.tmpdir=/tmp
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:java.compiler=<NA>
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:os.name=Linux
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:os.arch=amd64
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:os.version=2.6.32-358.6.2.el6.x86_64
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:user.name=storm
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:user.home=/opt/storm
>> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
>> environment:user.dir=/opt/storm
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
>> GMT
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:host.name=server4
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:java.version=1.6.0_24
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:java.vendor=Sun Microsystems Inc.
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:java.class.path=/opt/storm/storm-core-*.jar:/opt/storm/storm-netty-*.jar:/opt/storm/storm-console-logging-*.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.2-incubating.jar:/opt/storm/lib/netty-3.6.3.Final.jar:/opt/storm/lib/meat-locker-0.3.1.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/lib/javax.mail.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/httpclient-4.3.3.jar:/opt/storm/lib/curator-client-2.4.0.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/junit-3.8.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/disruptor-2.10.1.jar:/opt/storm/lib/zookeeper-3.4.5.jar:/opt/storm/lib/clj-stacktrace-0.2.4.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/commons-beanutils-1.8.3.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/logback-core-1.0.6.jar:/opt/storm/lib/httpcore-4.3.2.jar:/opt/storm/lib/curator-framework-2.4.0.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/servlet-api-2.5-20081211.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/javamail.jar:/opt/storm/lib/netty-3.2.2.Final.jar:/opt/storm/lib/json-lib-2.4-jdk15.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/slf4j-api-1.6.5.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/logback-classic-1.0.6.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/guava-13.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/activation.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/ezmorph-1.0.6.jar:/opt/storm/lib/commons-collections-3.2.1.jar:/opt/storm/lib/stanford-ner.jar:/opt/storm/lib/ring-devel-0.3.11.jar::/opt/storm/conf
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:java.io.tmpdir=/tmp
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:java.compiler=<NA>
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:os.name=Linux
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:os.arch=amd64
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:os.version=2.6.32-358.6.2.el6.x86_64
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:user.name=storm
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:user.home=/opt/storm
>> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
>> environment:user.dir=/opt/storm
>> 2014-08-07 22:47:25 b.s.d.supervisor [INFO] Starting Supervisor with
>> conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
>> "topology.tick.tuple.freq.secs" nil,
>> "topology.builtin.metrics.bucket.size.secs" 60,
>> "topology.fall.back.on.java.serialization" true,
>> "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000,
>> "topology.skip.missing.kryo.registrations" false,
>> "storm.messaging.netty.client_worker_threads" 1, "ui.childopts"
>> "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign"
>> true, "topology.trident.batch.emit.interval.millis" 500,
>> "drpc.servers" ["xxx.xxx.xxx.xxx"],
>> "storm.messaging.netty.flush.check.interval.ms" 10,
>> "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
>> "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib",
>> "topology.executor.send.buffer.size" 1024, "storm.local.dir"
>> "/opt/storm", "storm.messaging.netty.buffer_size" 5242880,
>> "supervisor.worker.start.timeout.secs" 120,
>> "topology.enable.message.timeouts" true,
>> "nimbus.cleanup.inbox.freq.secs" 600,
>> "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
>> "topology.worker.shared.thread.pool.size" 4, "nimbus.host"
>> "xxx.xxx.xxx.xxx", "storm.messaging.netty.min_wait_ms" 100,
>> "storm.zookeeper.port" 2181, "transactional.zookeeper.port" nil,
>> "topology.executor.receive.buffer.size" 1024,
>> "transactional.zookeeper.servers" nil, "storm.zookeeper.root"
>> "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
>> "supervisor.enable" true,
>> "storm.messaging.netty.server_worker_threads" 1,
>> "storm.zookeeper.servers" ["xxx.xxx.xxx.xxx"],
>> "transactional.zookeeper.root" "/transactional",
>> "topology.acker.executors" nil, "topology.transfer.buffer.size" 1024,
>> "topology.worker.childopts" nil, "drpc.queue.size" 128,
>> "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs"
>> 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0,
>> "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3,
>> "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8,
>> "task.heartbeat.frequency.secs" 3, "topology.tasks" nil,
>> "storm.messaging.netty.max_retries" 30, "topology.spout.wait.strategy"
>> "backtype.storm.spout.SleepSpoutWaitStrategy",
>> "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending"
>> nil, "storm.zookeeper.retry.interval" 1000,
>> "topology.sleep.spout.wait.strategy.time.ms" 1,
>> "nimbus.topology.validator"
>> "backtype.storm.nimbus.DefaultTopologyValidator",
>> "supervisor.slots.ports" [6700 6701 6702 6703 6704 6705 6706 6707 6708
>> 6709 6710 6711 6712 6713 6714], "topology.debug" false,
>> "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60,
>> "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10,
>> "topology.workers" 1, "supervisor.childopts" "-Xmx256m",
>> "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05,
>> "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer"
>> "backtype.storm.serialization.types.ListDelegateSerializer",
>> "topology.disruptor.wait.strategy"
>> "com.lmax.disruptor.BlockingWaitStrategy",
>> "topology.multilang.serializer"
>> "backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs"
>> 30, "storm.zookeeper.connection.timeout" 15000,
>> "topology.kryo.factory"
>> "backtype.storm.serialization.DefaultKryoFactory",
>> "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1,
>> "storm.zookeeper.retry.times" 5,
>> "topology.worker.receiver.thread.count" 1, "storm.thrift.transport"
>> "backtype.storm.security.auth.SimpleTransportPlugin",
>> "topology.state.synchronization.timeout.secs" 60,
>> "supervisor.worker.timeout.secs" 30,
>> "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport"
>> "backtype.storm.messaging.netty.Context", "logviewer.appender.name"
>> "A1", "storm.messaging.netty.max_wait_ms" 1000,
>> "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false,
>> "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode"
>> "distributed", "topology.max.task.parallelism" nil,
>> "storm.messaging.netty.transfer.batch.size" 262144,
>> "supervisor.scheduler.meta" {"name" "sarajevo-supervisor"}}
>> 2014-08-07 22:47:29 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> 2014-08-07 22:47:29 o.a.z.ZooKeeper [INFO] Initiating client
>> connection, connectString=xxx.xxx.xxx.xxx:2181 sessionTimeout=20000
>> watcher=org.apache.curator.ConnectionState@b4dadde
>> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Opening socket connection
>> to server server1/xxx.xxx.xxx.xxx:2181. Will not attempt to
>> authenticate using SASL (unknown error)
>> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Socket connection
>> established to server1/xxx.xxx.xxx.xxx:2181, initiating session
>> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Session establishment
>> complete on server server1/xxx.xxx.xxx.xxx:2181, sessionid =
>> 0x147391967370924, negotiated timeout = 20000
>> 2014-08-07 22:47:29 o.a.c.f.s.ConnectionStateManager [INFO] State
>> change: CONNECTED
>> 2014-08-07 22:47:29 o.a.c.f.s.ConnectionStateManager [WARN] There are
>> no ConnectionStateListeners registered.
>> 2014-08-07 22:47:29 b.s.zookeeper [INFO] Zookeeper state update:
>> :connected:none
>> 2014-08-07 22:47:30 o.a.z.ZooKeeper [INFO] Session: 0x147391967370924
>> closed
>> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] EventThread shut down
>> 2014-08-07 22:47:30 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
>> 2014-08-07 22:47:30 o.a.z.ZooKeeper [INFO] Initiating client
>> connection, connectString=xxx.xxx.xxx.xxx:2181/storm
>> sessionTimeout=20000
>> watcher=org.apache.curator.ConnectionState@4900269d
>> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Opening socket connection
>> to server server1/xxx.xxx.xxx.xxx:2181. Will not attempt to
>> authenticate using SASL (unknown error)
>> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Socket connection
>> established to server1/xxx.xxx.xxx.xxx:2181, initiating session
>> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Session establishment
>> complete on server server1/xxx.xxx.xxx.xxx:2181, sessionid =
>> 0x147391967370925, negotiated timeout = 20000
>> 2014-08-07 22:47:30 o.a.c.f.s.ConnectionStateManager [INFO] State
>> change: CONNECTED
>> 2014-08-07 22:47:30 o.a.c.f.s.ConnectionStateManager [WARN] There are
>> no ConnectionStateListeners registered.
>> 2014-08-07 22:47:30 b.s.d.supervisor [INFO] Starting supervisor with
>> id e7126247-5801-4337-9beb-c4e62ca3142b at host server4
>> 2014-08-07 22:47:32 b.s.event [ERROR] Error when processing event
>> java.lang.RuntimeException: java.io.InvalidClassException:
>> backtype.storm.daemon.common.WorkerHeartbeat; local class
>> incompatible: stream classdesc serialVersionUID =
>> -6996865048894131652, local class serialVersionUID =
>> 2074174925015471843
>> at backtype.storm.utils.Utils.deserialize(Utils.java:93)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> at backtype.storm.utils.LocalState.snapshot(LocalState.java:45)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> at backtype.storm.utils.LocalState.get(LocalState.java:56)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> at
>> backtype.storm.daemon.supervisor$read_worker_heartbeat.invoke(supervisor.clj:77)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> at
>> backtype.storm.daemon.supervisor$read_worker_heartbeats$iter__6167__6171$fn__6172.invoke(supervisor.clj:90)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na]
>> at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na]
>> at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na]
>> at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na]
>> at clojure.core$dorun.invoke(core.clj:2780) ~[clojure-1.5.1.jar:na]
>> at clojure.core$doall.invoke(core.clj:2796) ~[clojure-1.5.1.jar:na]
>> at
>> backtype.storm.daemon.supervisor$read_worker_heartbeats.invoke(supervisor.clj:89)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> at
>> backtype.storm.daemon.supervisor$read_allocated_workers.invoke(supervisor.clj:106)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> at
>> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:209)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> at clojure.lang.AFn.applyToHelper(AFn.java:161) [clojure-1.5.1.jar:na]
>> at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
>> at clojure.core$apply.invoke(core.clj:619) ~[clojure-1.5.1.jar:na]
>> at clojure.core$partial$fn__445.doInvoke(core.clj:2396) ~[na:na]
>> at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.5.1.jar:na]
>> at backtype.storm.event$event_manager$fn__2378.invoke(event.clj:39)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
>> at java.lang.Thread.run(Thread.java:679) [na:1.6.0_24]
>> Caused by: java.io.InvalidClassException:
>> backtype.storm.daemon.common.WorkerHeartbeat; local class
>> incompatible: stream classdesc serialVersionUID =
>> -6996865048894131652, local class serialVersionUID =
>> 2074174925015471843
>> at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:579)
>> ~[na:1.6.0_24]
>> at
>> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1619)
>> ~[na:1.6.0_24]
>> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)
>> ~[na:1.6.0_24]
>> at
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1768)
>> ~[na:1.6.0_24]
>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
>> ~[na:1.6.0_24]
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
>> ~[na:1.6.0_24]
>> at java.util.HashMap.readObject(HashMap.java:1047) ~[na:1.6.0_24]
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> ~[na:1.6.0_24]
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> ~[na:1.6.0_24]
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> ~[na:1.6.0_24]
>> at java.lang.reflect.Method.invoke(Method.java:616) ~[na:1.6.0_24]
>> at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:988)
>> ~[na:1.6.0_24]
>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1890)
>> ~[na:1.6.0_24]
>> at
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1795)
>> ~[na:1.6.0_24]
>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
>> ~[na:1.6.0_24]
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
>> ~[na:1.6.0_24]
>> at backtype.storm.utils.Utils.deserialize(Utils.java:89)
>> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
>> ... 21 common frames omitted
>> 2014-08-07 22:47:32 b.s.util [INFO] Halting process: ("Error when
>> processing an event")

Re: java.io.InvalidClassException: backtype.storm.daemon.common.WorkerHeartbeat

[Harsha <st...@harsha.io>]

"local class
incompatible: stream classdesc serialVersionUID =
-6996865048894131652, local class serialVersionUID =
2074174925015471843"

The above error usually happens when the storm versions (usually
dependent jars ) differ.
If its new installation make sure you've same version of storm on every
node.
If you are upgrading clear storm-local and zookeeper data and restart
the daemons.
If it persists I'll try clearing storm-local and zookeeper data.
-Harsha

On Thu, Aug 7, 2014, at 08:58 PM, Shun KAWAHARA wrote:
> Hello.
> 
> I started Storm by the following constitution.
> 
> server1: nimbus, supervisor
> server2: supervisor
> server3: supervisor
> server4: supervisor
> 
> However, An error has occurred in only server4.
> Supervisor's log of server4 is following.
> Please tell me the solution.
> 
> Shun
> 
> 
> -----
> 
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
> GMT
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:host.name=server4
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:java.version=1.6.0_24
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:java.vendor=Sun Microsystems Inc.
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:java.class.path=/opt/storm/storm-core-*.jar:/opt/storm/storm-netty-*.jar:/opt/storm/storm-console-logging-*.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.2-incubating.jar:/opt/storm/lib/netty-3.6.3.Final.jar:/opt/storm/lib/meat-locker-0.3.1.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/lib/javax.mail.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/httpclient-4.3.3.jar:/opt/storm/lib/curator-client-2.4.0.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/junit-3.8.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/disruptor-2.10.1.jar:/opt/storm/lib/zookeeper-3.4.5.jar:/opt/storm/lib/clj-stacktrace-0.2.4.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/commons-beanutils-1.8.3.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/logback-core-1.0.6.jar:/opt/storm/lib/httpcore-4.3.2.jar:/opt/storm/lib/curator-framework-2.4.0.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/servlet-api-2.5-20081211.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/javamail.jar:/opt/storm/lib/netty-3.2.2.Final.jar:/opt/storm/lib/json-lib-2.4-jdk15.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/slf4j-api-1.6.5.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/logback-classic-1.0.6.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/guava-13.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/activation.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/ezmorph-1.0.6.jar:/opt/storm/lib/commons-collections-3.2.1.jar:/opt/storm/lib/stanford-ner.jar:/opt/storm/lib/ring-devel-0.3.11.jar::/opt/storm/conf
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:java.io.tmpdir=/tmp
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:java.compiler=<NA>
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:os.name=Linux
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:os.arch=amd64
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:os.version=2.6.32-358.6.2.el6.x86_64
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:user.name=storm
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:user.home=/opt/storm
> 2014-08-07 22:47:22 o.a.z.ZooKeeper [INFO] Client
> environment:user.dir=/opt/storm
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
> GMT
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:host.name=server4
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.version=1.6.0_24
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.vendor=Sun Microsystems Inc.
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.class.path=/opt/storm/storm-core-*.jar:/opt/storm/storm-netty-*.jar:/opt/storm/storm-console-logging-*.jar:/opt/storm/lib/math.numeric-tower-0.0.1.jar:/opt/storm/lib/commons-io-2.4.jar:/opt/storm/lib/tools.logging-0.2.3.jar:/opt/storm/lib/objenesis-1.2.jar:/opt/storm/lib/reflectasm-1.07-shaded.jar:/opt/storm/lib/storm-core-0.9.2-incubating.jar:/opt/storm/lib/netty-3.6.3.Final.jar:/opt/storm/lib/meat-locker-0.3.1.jar:/opt/storm/lib/compojure-1.1.3.jar:/opt/storm/lib/ring-core-1.1.5.jar:/opt/storm/lib/javax.mail.jar:/opt/storm/lib/ring-servlet-0.3.11.jar:/opt/storm/lib/joda-time-2.0.jar:/opt/storm/lib/httpclient-4.3.3.jar:/opt/storm/lib/curator-client-2.4.0.jar:/opt/storm/lib/commons-logging-1.1.3.jar:/opt/storm/lib/junit-3.8.1.jar:/opt/storm/lib/minlog-1.2.jar:/opt/storm/lib/commons-lang-2.5.jar:/opt/storm/lib/disruptor-2.10.1.jar:/opt/storm/lib/zookeeper-3.4.5.jar:/opt/storm/lib/clj-stacktrace-0.2.4.jar:/opt/storm/lib/ring-jetty-adapter-0.3.11.jar:/opt/storm/lib/clout-1.0.1.jar:/opt/storm/lib/commons-beanutils-1.8.3.jar:/opt/storm/lib/tools.macro-0.1.0.jar:/opt/storm/lib/commons-exec-1.1.jar:/opt/storm/lib/kryo-2.21.jar:/opt/storm/lib/logback-core-1.0.6.jar:/opt/storm/lib/httpcore-4.3.2.jar:/opt/storm/lib/curator-framework-2.4.0.jar:/opt/storm/lib/servlet-api-2.5.jar:/opt/storm/lib/jgrapht-core-0.9.0.jar:/opt/storm/lib/clojure-1.5.1.jar:/opt/storm/lib/jetty-util-6.1.26.jar:/opt/storm/lib/servlet-api-2.5-20081211.jar:/opt/storm/lib/carbonite-1.4.0.jar:/opt/storm/lib/hiccup-0.3.6.jar:/opt/storm/lib/javamail.jar:/opt/storm/lib/netty-3.2.2.Final.jar:/opt/storm/lib/json-lib-2.4-jdk15.jar:/opt/storm/lib/jline-2.11.jar:/opt/storm/lib/chill-java-0.3.5.jar:/opt/storm/lib/clj-time-0.4.1.jar:/opt/storm/lib/commons-codec-1.6.jar:/opt/storm/lib/tools.cli-0.2.4.jar:/opt/storm/lib/slf4j-api-1.6.5.jar:/opt/storm/lib/core.incubator-0.1.0.jar:/opt/storm/lib/snakeyaml-1.11.jar:/opt/storm/lib/logback-classic-1.0.6.jar:/opt/storm/lib/log4j-over-slf4j-1.6.6.jar:/opt/storm/lib/guava-13.0.jar:/opt/storm/lib/jetty-6.1.26.jar:/opt/storm/lib/activation.jar:/opt/storm/lib/json-simple-1.1.jar:/opt/storm/lib/commons-fileupload-1.2.1.jar:/opt/storm/lib/asm-4.0.jar:/opt/storm/lib/ezmorph-1.0.6.jar:/opt/storm/lib/commons-collections-3.2.1.jar:/opt/storm/lib/stanford-ner.jar:/opt/storm/lib/ring-devel-0.3.11.jar::/opt/storm/conf
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.io.tmpdir=/tmp
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.compiler=<NA>
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:os.name=Linux
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:os.arch=amd64
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:os.version=2.6.32-358.6.2.el6.x86_64
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:user.name=storm
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:user.home=/opt/storm
> 2014-08-07 22:47:22 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:user.dir=/opt/storm
> 2014-08-07 22:47:25 b.s.d.supervisor [INFO] Starting Supervisor with
> conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
> "topology.tick.tuple.freq.secs" nil,
> "topology.builtin.metrics.bucket.size.secs" 60,
> "topology.fall.back.on.java.serialization" true,
> "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000,
> "topology.skip.missing.kryo.registrations" false,
> "storm.messaging.netty.client_worker_threads" 1, "ui.childopts"
> "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign"
> true, "topology.trident.batch.emit.interval.millis" 500,
> "drpc.servers" ["xxx.xxx.xxx.xxx"],
> "storm.messaging.netty.flush.check.interval.ms" 10,
> "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
> "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib",
> "topology.executor.send.buffer.size" 1024, "storm.local.dir"
> "/opt/storm", "storm.messaging.netty.buffer_size" 5242880,
> "supervisor.worker.start.timeout.secs" 120,
> "topology.enable.message.timeouts" true,
> "nimbus.cleanup.inbox.freq.secs" 600,
> "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
> "topology.worker.shared.thread.pool.size" 4, "nimbus.host"
> "xxx.xxx.xxx.xxx", "storm.messaging.netty.min_wait_ms" 100,
> "storm.zookeeper.port" 2181, "transactional.zookeeper.port" nil,
> "topology.executor.receive.buffer.size" 1024,
> "transactional.zookeeper.servers" nil, "storm.zookeeper.root"
> "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
> "supervisor.enable" true,
> "storm.messaging.netty.server_worker_threads" 1,
> "storm.zookeeper.servers" ["xxx.xxx.xxx.xxx"],
> "transactional.zookeeper.root" "/transactional",
> "topology.acker.executors" nil, "topology.transfer.buffer.size" 1024,
> "topology.worker.childopts" nil, "drpc.queue.size" 128,
> "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs"
> 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0,
> "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3,
> "drpc.childopts" "-Xmx768m", "topology.receiver.buffer.size" 8,
> "task.heartbeat.frequency.secs" 3, "topology.tasks" nil,
> "storm.messaging.netty.max_retries" 30, "topology.spout.wait.strategy"
> "backtype.storm.spout.SleepSpoutWaitStrategy",
> "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending"
> nil, "storm.zookeeper.retry.interval" 1000,
> "topology.sleep.spout.wait.strategy.time.ms" 1,
> "nimbus.topology.validator"
> "backtype.storm.nimbus.DefaultTopologyValidator",
> "supervisor.slots.ports" [6700 6701 6702 6703 6704 6705 6706 6707 6708
> 6709 6710 6711 6712 6713 6714], "topology.debug" false,
> "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60,
> "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10,
> "topology.workers" 1, "supervisor.childopts" "-Xmx256m",
> "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05,
> "worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer"
> "backtype.storm.serialization.types.ListDelegateSerializer",
> "topology.disruptor.wait.strategy"
> "com.lmax.disruptor.BlockingWaitStrategy",
> "topology.multilang.serializer"
> "backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs"
> 30, "storm.zookeeper.connection.timeout" 15000,
> "topology.kryo.factory"
> "backtype.storm.serialization.DefaultKryoFactory",
> "drpc.invocations.port" 3773, "logviewer.port" 8000, "zmq.threads" 1,
> "storm.zookeeper.retry.times" 5,
> "topology.worker.receiver.thread.count" 1, "storm.thrift.transport"
> "backtype.storm.security.auth.SimpleTransportPlugin",
> "topology.state.synchronization.timeout.secs" 60,
> "supervisor.worker.timeout.secs" 30,
> "nimbus.file.copy.expiration.secs" 600, "storm.messaging.transport"
> "backtype.storm.messaging.netty.Context", "logviewer.appender.name"
> "A1", "storm.messaging.netty.max_wait_ms" 1000,
> "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false,
> "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode"
> "distributed", "topology.max.task.parallelism" nil,
> "storm.messaging.netty.transfer.batch.size" 262144,
> "supervisor.scheduler.meta" {"name" "sarajevo-supervisor"}}
> 2014-08-07 22:47:29 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2014-08-07 22:47:29 o.a.z.ZooKeeper [INFO] Initiating client
> connection, connectString=xxx.xxx.xxx.xxx:2181 sessionTimeout=20000
> watcher=org.apache.curator.ConnectionState@b4dadde
> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Opening socket connection
> to server server1/xxx.xxx.xxx.xxx:2181. Will not attempt to
> authenticate using SASL (unknown error)
> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Socket connection
> established to server1/xxx.xxx.xxx.xxx:2181, initiating session
> 2014-08-07 22:47:29 o.a.z.ClientCnxn [INFO] Session establishment
> complete on server server1/xxx.xxx.xxx.xxx:2181, sessionid =
> 0x147391967370924, negotiated timeout = 20000
> 2014-08-07 22:47:29 o.a.c.f.s.ConnectionStateManager [INFO] State
> change: CONNECTED
> 2014-08-07 22:47:29 o.a.c.f.s.ConnectionStateManager [WARN] There are
> no ConnectionStateListeners registered.
> 2014-08-07 22:47:29 b.s.zookeeper [INFO] Zookeeper state update:
> :connected:none
> 2014-08-07 22:47:30 o.a.z.ZooKeeper [INFO] Session: 0x147391967370924
> closed
> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] EventThread shut down
> 2014-08-07 22:47:30 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2014-08-07 22:47:30 o.a.z.ZooKeeper [INFO] Initiating client
> connection, connectString=xxx.xxx.xxx.xxx:2181/storm
> sessionTimeout=20000
> watcher=org.apache.curator.ConnectionState@4900269d
> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Opening socket connection
> to server server1/xxx.xxx.xxx.xxx:2181. Will not attempt to
> authenticate using SASL (unknown error)
> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Socket connection
> established to server1/xxx.xxx.xxx.xxx:2181, initiating session
> 2014-08-07 22:47:30 o.a.z.ClientCnxn [INFO] Session establishment
> complete on server server1/xxx.xxx.xxx.xxx:2181, sessionid =
> 0x147391967370925, negotiated timeout = 20000
> 2014-08-07 22:47:30 o.a.c.f.s.ConnectionStateManager [INFO] State
> change: CONNECTED
> 2014-08-07 22:47:30 o.a.c.f.s.ConnectionStateManager [WARN] There are
> no ConnectionStateListeners registered.
> 2014-08-07 22:47:30 b.s.d.supervisor [INFO] Starting supervisor with
> id e7126247-5801-4337-9beb-c4e62ca3142b at host server4
> 2014-08-07 22:47:32 b.s.event [ERROR] Error when processing event
> java.lang.RuntimeException: java.io.InvalidClassException:
> backtype.storm.daemon.common.WorkerHeartbeat; local class
> incompatible: stream classdesc serialVersionUID =
> -6996865048894131652, local class serialVersionUID =
> 2074174925015471843
> at backtype.storm.utils.Utils.deserialize(Utils.java:93)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at backtype.storm.utils.LocalState.snapshot(LocalState.java:45)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at backtype.storm.utils.LocalState.get(LocalState.java:56)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.daemon.supervisor$read_worker_heartbeat.invoke(supervisor.clj:77)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.daemon.supervisor$read_worker_heartbeats$iter__6167__6171$fn__6172.invoke(supervisor.clj:90)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na]
> at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na]
> at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na]
> at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na]
> at clojure.core$dorun.invoke(core.clj:2780) ~[clojure-1.5.1.jar:na]
> at clojure.core$doall.invoke(core.clj:2796) ~[clojure-1.5.1.jar:na]
> at
> backtype.storm.daemon.supervisor$read_worker_heartbeats.invoke(supervisor.clj:89)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.daemon.supervisor$read_allocated_workers.invoke(supervisor.clj:106)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at
> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:209)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at clojure.lang.AFn.applyToHelper(AFn.java:161) [clojure-1.5.1.jar:na]
> at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na]
> at clojure.core$apply.invoke(core.clj:619) ~[clojure-1.5.1.jar:na]
> at clojure.core$partial$fn__445.doInvoke(core.clj:2396) ~[na:na]
> at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.5.1.jar:na]
> at backtype.storm.event$event_manager$fn__2378.invoke(event.clj:39)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
> at java.lang.Thread.run(Thread.java:679) [na:1.6.0_24]
> Caused by: java.io.InvalidClassException:
> backtype.storm.daemon.common.WorkerHeartbeat; local class
> incompatible: stream classdesc serialVersionUID =
> -6996865048894131652, local class serialVersionUID =
> 2074174925015471843
> at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:579)
> ~[na:1.6.0_24]
> at
> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1619)
> ~[na:1.6.0_24]
> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)
> ~[na:1.6.0_24]
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1768)
> ~[na:1.6.0_24]
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
> ~[na:1.6.0_24]
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
> ~[na:1.6.0_24]
> at java.util.HashMap.readObject(HashMap.java:1047) ~[na:1.6.0_24]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.6.0_24]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ~[na:1.6.0_24]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.6.0_24]
> at java.lang.reflect.Method.invoke(Method.java:616) ~[na:1.6.0_24]
> at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:988)
> ~[na:1.6.0_24]
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1890)
> ~[na:1.6.0_24]
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1795)
> ~[na:1.6.0_24]
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
> ~[na:1.6.0_24]
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
> ~[na:1.6.0_24]
> at backtype.storm.utils.Utils.deserialize(Utils.java:89)
> ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
> ... 21 common frames omitted
> 2014-08-07 22:47:32 b.s.util [INFO] Halting process: ("Error when
> processing an event")