You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Sa Li <sa...@gmail.com> on 2014/08/11 23:49:55 UTC
why storm supervisor automatically shutdown
Hi, All
I have a very basic question, when I start the storm cluster, I start nimbus, ui, supervisor (which is the same server with nimbus), I can see all of them in UI as below,
However, the supervisor is automatically shutdown, and is gone in UI as below
Why that happen?
Thanks
Alec
Re: why storm supervisor automatically shutdown
Posted by Sa Li <sa...@gmail.com>.
I use ./storm supervisor &, I didn't user supervisord to run it. Thanks
On Mon, Aug 11, 2014 at 4:35 PM, Telles Nobrega <te...@gmail.com>
wrote:
> Are you using supervisor deamon to run the supervisor or running in
> yourself?
>
> On 11 Aug 2014, at 18:49, Sa Li <sa...@gmail.com> wrote:
>
> > Hi, All
> >
> > I have a very basic question, when I start the storm cluster, I start
> nimbus, ui, supervisor (which is the same server with nimbus), I can see
> all of them in UI as below,
> >
> > <PastedGraphic-1.tiff>
> >
> >
> >
> >
> > However, the supervisor is automatically shutdown, and is gone in UI as
> below
> >
> > <PastedGraphic-2.tiff>
> >
> >
> > Why that happen?
> >
> >
> > Thanks
> >
> > Alec
> >
>
>
Re: why storm supervisor automatically shutdown
Posted by Kushan Maskey <ku...@mmillerassociates.com>.
We also had problem with supervisor shutting down itself and trying to
restart every now and then. We use the supervisodctl command to start the
nimbus, supervisor, ui etc. I looked into this post and looks like its a
persmission issue. So we ran those using this command and now it seem to
run better. su - storm -c "strom xxxx start".
It might be worth while to try that as well.
http://translate.google.com/translate?hl=en&sl=zh-CN&u=http://vcode.org/category/storm/&prev=/search%3Fq%3Db.s.event%2B%255BERROR%255D%2BError%2Bwhen%2Bprocessing%2Bevent%2Bhttp://java.io.IOException:%2B.%2Bdoesn%2527t%2Bexist.%26rlz%3D1C5CHFA_enUS539US539%26es_sm%3D91
Now debugging on a local mode should not be that bad in eclipse which I
use. I just set the breakpoints in my bolt and spout and publish message on
kafka on that particular topic and you should be good to go.
Hope this help.
Kushan
--
Kushan Maskey
817.403.7500
On Wed, Aug 13, 2014 at 11:36 AM, Sa Li <sa...@gmail.com> wrote:
> 2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client
> environment:java.io.tmpdir=/tmp
> 2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client
> environment:java.compiler=<NA>
> 2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client environment:os.name
> =Linux
> 2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd64
> 2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client
> environment:os.version=3.13.0-27-generic
> 2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client environment:user.name
> =root
> 2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client
> environment:user.home=/root
> 2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client
> environment:user.dir=/etc/apache-storm-0.9.2-incubating
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server environment:
> host.name=DO-mq-dev
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.version=1.7.0_55
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.vendor=Oracle Corporation
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.class.path=/etc/apache-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/etc/apache-storm-0.9.2-incubating/lib/joda-time-2.0.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/etc/apache-storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/etc/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/etc/apache-storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/etc/apache-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/etc/apache-storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/etc/apache-storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/etc/apache-storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/etc/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/etc/apache-storm-0.9.2-incubating/lib/asm-4.0.jar:/etc/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/etc/apache-storm-0.9.2-incubating/lib/jline-2.11.jar:/etc/apache-storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/etc/apache-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/etc/apache-storm-0.9.2-incubating/lib/clout-1.0.1.jar:/etc/apache-storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/etc/apache-storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-io-2.4.jar:/etc/apache-storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/etc/apache-storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/etc/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/etc/apache-storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/etc/apache-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/etc/apache-storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/etc/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/etc/apache-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/etc/apache-storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/etc/apache-storm-0.9.2-incubating/lib/json-simple-1.1.jar:/etc/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/etc/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/etc/apache-storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/etc/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/etc/apache-storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/etc/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/etc/apache-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/etc/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/etc/apache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/etc/apache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/etc/apache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/etc/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/etc/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/etc/apache-storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/etc/apache-storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/etc/apache-storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/etc/apache-storm-0.9.2-incubating/lib/kryo-2.21.jar:/etc/apache-storm-0.9.2-incubating/conf
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.library.path=/usr/lib/jvm/java-7-openjdk-amd64
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.io.tmpdir=/tmp
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:java.compiler=<NA>
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server environment:
> os.name=Linux
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:os.arch=amd64
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:os.version=3.13.0-27-generic
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server environment:
> user.name=root
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:user.home=/root
> 2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
> environment:user.dir=/etc/apache-storm-0.9.2-incubating
> 2014-08-11 13:39:57 b.s.d.supervisor [INFO] Starting Supervisor with conf
> {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
> "topology.tick.tuple.freq.secs" nil,
> "topology.builtin.metrics.bucket.size.secs" 60,
> "topology.fall.back.on.java.serialization" true,
> "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000,
> "topology.skip.missing.kryo.registrations" false,
> "storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
> "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
> "topology.trident.batch.emit.interval.millis" 500, "drpc.servers"
> ["10.100.70.128"], "storm.messaging.netty.flush.check.interval.ms" 10,
> "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
> "java.library.path" "/usr/lib/jvm/java-7-openjdk-amd64",
> "topology.executor.send.buffer.size" 1024, "storm.local.dir" "/app/storm",
> "storm.messaging.netty.buffer_size" 5242880,
> "supervisor.worker.start.timeout.secs" 120,
> "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs"
> 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
> "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "10.100.70.128",
> "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181,
> "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
> 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root"
> "/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
> "supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1,
> "storm.zookeeper.servers" ["10.100.70.128"], "transactional.zookeeper.root"
> "/transactional", "topology.acker.executors" nil,
> "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil,
> "drpc.queue.size" 128, "worker.childopts" "-Xmx768m",
> "supervisor.heartbeat.frequency.secs" 5,
> "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
> "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
> "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
> "topology.tasks" nil, "storm.messaging.netty.max_retries" 30,
> "topology.spout.wait.strategy"
> "backtype.storm.spout.SleepSpoutWaitStrategy",
> "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil,
> "storm.zookeeper.retry.interval" 1000, "
> topology.sleep.spout.wait.strategy.time.ms" 1,
> "nimbus.topology.validator"
> "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
> [6700 6701 6702 6703], "topology.debug" false, "nimbus.task.launch.secs"
> 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs"
> 30, "task.refresh.poll.secs" 10, "topology.workers" 1,
> "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627,
> "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1,
> "topology.tuple.serializer"
> "backtype.storm.serialization.types.ListDelegateSerializer",
> "topology.disruptor.wait.strategy"
> "com.lmax.disruptor.BlockingWaitStrategy", "topology.multilang.serializer"
> "backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs" 30,
> "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory"
> "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port"
> 3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times"
> 5, "topology.worker.receiver.thread.count" 1, "storm.thrift.transport"
> "backtype.storm.security.auth.SimpleTransportPlugin",
> "topology.state.synchronization.timeout.secs" 60,
> "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs"
> 600, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "
> logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
> "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port"
> 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "distributed",
> "topology.max.task.parallelism" nil,
> "storm.messaging.netty.transfer.batch.size" 262144}
> 2014-08-11 13:39:57 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2014-08-11 13:39:57 o.a.z.ZooKeeper [INFO] Initiating client connection,
> connectString=10.100.70.128:2181 sessionTimeout=20000
> watcher=org.apache.curator.ConnectionState@1a749b9c
> 2014-08-11 13:39:57 o.a.z.ClientCnxn [INFO] Opening socket connection to
> server 10.100.70.128/10.100.70.128:2181. Will not attempt to authenticate
> using SASL (unknown error)
> 2014-08-11 13:39:57 o.a.z.ClientCnxn [INFO] Socket connection established
> to 10.100.70.128/10.100.70.128:2181, initiating session
> 2014-08-11 13:39:57 o.a.z.ClientCnxn [INFO] Session establishment complete
> on server 10.100.70.128/10.100.70.128:2181, sessionid =
> 0x147ac88fa330039, negotiated timeout = 20000
> 2014-08-11 13:39:57 o.a.c.f.s.ConnectionStateManager [INFO] State change:
> CONNECTED
> 2014-08-11 13:39:57 o.a.c.f.s.ConnectionStateManager [WARN] There are no
> ConnectionStateListeners registered.
> 2014-08-11 13:39:57 b.s.zookeeper [INFO] Zookeeper state update:
> :connected:none
> 2014-08-11 13:39:58 o.a.z.ZooKeeper [INFO] Session: 0x147ac88fa330039
> closed
>
>
> I know these logs are kinda messy, and seems supervisor is started
> somehow, but I am running my code in localmode. I never be able to see the
> supervisor in UI.
>
> thanks
>
> Alec
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Tue, Aug 12, 2014 at 6:08 PM, Parth Brahmbhatt <
> pbrahmbhatt@hortonworks.com> wrote:
>
>> Can you please post some logs from your supervisor?
>>
>> Thanks
>> Parth
>>
>> On Aug 11, 2014, at 4:35 PM, Telles Nobrega <te...@gmail.com>
>> wrote:
>>
>> Are you using supervisor deamon to run the supervisor or running in
>> yourself?
>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>
>
>
Re: why storm supervisor automatically shutdown
Posted by Sa Li <sa...@gmail.com>.
2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client
environment:java.io.tmpdir=/tmp
2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client
environment:java.compiler=<NA>
2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client environment:os.name=Linux
2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd64
2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client
environment:os.version=3.13.0-27-generic
2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client environment:user.name=root
2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client
environment:user.home=/root
2014-08-11 13:39:56 o.a.z.ZooKeeper [INFO] Client
environment:user.dir=/etc/apache-storm-0.9.2-incubating
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server environment:
host.name=DO-mq-dev
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.version=1.7.0_55
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.vendor=Oracle Corporation
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.class.path=/etc/apache-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/etc/apache-storm-0.9.2-incubating/lib/joda-time-2.0.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/etc/apache-storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/etc/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/etc/apache-storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/etc/apache-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/etc/apache-storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/etc/apache-storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/etc/apache-storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/etc/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/etc/apache-storm-0.9.2-incubating/lib/asm-4.0.jar:/etc/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/etc/apache-storm-0.9.2-incubating/lib/jline-2.11.jar:/etc/apache-storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/etc/apache-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/etc/apache-storm-0.9.2-incubating/lib/clout-1.0.1.jar:/etc/apache-storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/etc/apache-storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-io-2.4.jar:/etc/apache-storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/etc/apache-storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/etc/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/etc/apache-storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/etc/apache-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/etc/apache-storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/etc/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/etc/apache-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/etc/apache-storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/etc/apache-storm-0.9.2-incubating/lib/json-simple-1.1.jar:/etc/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/etc/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/etc/apache-storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/etc/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/etc/apache-storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/etc/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/etc/apache-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/etc/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/etc/apache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/etc/apache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/etc/apache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/etc/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/etc/apache-storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/etc/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/etc/apache-storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/etc/apache-storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/etc/apache-storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/etc/apache-storm-0.9.2-incubating/lib/kryo-2.21.jar:/etc/apache-storm-0.9.2-incubating/conf
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.library.path=/usr/lib/jvm/java-7-openjdk-amd64
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.io.tmpdir=/tmp
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:java.compiler=<NA>
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server environment:
os.name=Linux
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:os.arch=amd64
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:os.version=3.13.0-27-generic
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server environment:
user.name=root
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:user.home=/root
2014-08-11 13:39:56 o.a.z.s.ZooKeeperServer [INFO] Server
environment:user.dir=/etc/apache-storm-0.9.2-incubating
2014-08-11 13:39:57 b.s.d.supervisor [INFO] Starting Supervisor with conf
{"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
"topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000,
"topology.skip.missing.kryo.registrations" false,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
"storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 500, "drpc.servers"
["10.100.70.128"], "storm.messaging.netty.flush.check.interval.ms" 10,
"nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
"java.library.path" "/usr/lib/jvm/java-7-openjdk-amd64",
"topology.executor.send.buffer.size" 1024, "storm.local.dir" "/app/storm",
"storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120,
"topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs"
600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
"topology.worker.shared.thread.pool.size" 4, "nimbus.host" "10.100.70.128",
"storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181,
"transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root"
"/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
"supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1,
"storm.zookeeper.servers" ["10.100.70.128"], "transactional.zookeeper.root"
"/transactional", "topology.acker.executors" nil,
"topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil,
"drpc.queue.size" 128, "worker.childopts" "-Xmx768m",
"supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
"supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
"topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
"topology.tasks" nil, "storm.messaging.netty.max_retries" 30,
"topology.spout.wait.strategy"
"backtype.storm.spout.SleepSpoutWaitStrategy",
"nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil,
"storm.zookeeper.retry.interval" 1000, "
topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
[6700 6701 6702 6703], "topology.debug" false, "nimbus.task.launch.secs"
120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs"
30, "task.refresh.poll.secs" 10, "topology.workers" 1,
"supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627,
"topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1,
"topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy"
"com.lmax.disruptor.BlockingWaitStrategy", "topology.multilang.serializer"
"backtype.storm.multilang.JsonSerializer", "nimbus.task.timeout.secs" 30,
"storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory"
"backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port"
3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times"
5, "topology.worker.receiver.thread.count" 1, "storm.thrift.transport"
"backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs"
600, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "
logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port"
8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "distributed",
"topology.max.task.parallelism" nil,
"storm.messaging.netty.transfer.batch.size" 262144}
2014-08-11 13:39:57 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2014-08-11 13:39:57 o.a.z.ZooKeeper [INFO] Initiating client connection,
connectString=10.100.70.128:2181 sessionTimeout=20000
watcher=org.apache.curator.ConnectionState@1a749b9c
2014-08-11 13:39:57 o.a.z.ClientCnxn [INFO] Opening socket connection to
server 10.100.70.128/10.100.70.128:2181. Will not attempt to authenticate
using SASL (unknown error)
2014-08-11 13:39:57 o.a.z.ClientCnxn [INFO] Socket connection established
to 10.100.70.128/10.100.70.128:2181, initiating session
2014-08-11 13:39:57 o.a.z.ClientCnxn [INFO] Session establishment complete
on server 10.100.70.128/10.100.70.128:2181, sessionid = 0x147ac88fa330039,
negotiated timeout = 20000
2014-08-11 13:39:57 o.a.c.f.s.ConnectionStateManager [INFO] State change:
CONNECTED
2014-08-11 13:39:57 o.a.c.f.s.ConnectionStateManager [WARN] There are no
ConnectionStateListeners registered.
2014-08-11 13:39:57 b.s.zookeeper [INFO] Zookeeper state update:
:connected:none
2014-08-11 13:39:58 o.a.z.ZooKeeper [INFO] Session: 0x147ac88fa330039 closed
I know these logs are kinda messy, and seems supervisor is started somehow,
but I am running my code in localmode. I never be able to see the
supervisor in UI.
thanks
Alec
On Tue, Aug 12, 2014 at 6:08 PM, Parth Brahmbhatt <
pbrahmbhatt@hortonworks.com> wrote:
> Can you please post some logs from your supervisor?
>
> Thanks
> Parth
>
> On Aug 11, 2014, at 4:35 PM, Telles Nobrega <te...@gmail.com>
> wrote:
>
> Are you using supervisor deamon to run the supervisor or running in
> yourself?
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
Re: why storm supervisor automatically shutdown
Posted by Parth Brahmbhatt <pb...@hortonworks.com>.
Can you please post some logs from your supervisor?
Thanks
Parth
On Aug 11, 2014, at 4:35 PM, Telles Nobrega <te...@gmail.com> wrote:
> Are you using supervisor deamon to run the supervisor or running in yourself?
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: why storm supervisor automatically shutdown
Posted by Telles Nobrega <te...@gmail.com>.
Are you using supervisor deamon to run the supervisor or running in yourself?
On 11 Aug 2014, at 18:49, Sa Li <sa...@gmail.com> wrote:
> Hi, All
>
> I have a very basic question, when I start the storm cluster, I start nimbus, ui, supervisor (which is the same server with nimbus), I can see all of them in UI as below,
>
> <PastedGraphic-1.tiff>
>
>
>
>
> However, the supervisor is automatically shutdown, and is gone in UI as below
>
> <PastedGraphic-2.tiff>
>
>
> Why that happen?
>
>
> Thanks
>
> Alec
>