You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by siddharth ubale <si...@gmail.com> on 2014/08/11 13:15:49 UTC
KafkaSpout
Hi guys,
I am facing an issue while using Kafka Spout in my topology to read from a
topic i created .
I can see the topic in the zookeper under /brokers. And also, i am
generating json objects and i want them to be displayed by the bolt.
however i am unable to get any success when i run the program. the producer
is producing the messages, the consumer is able to see the producer created
data , the kafkaspout is initialized however i can not get the topology to
be display what the kafka spout is reading.
Please see my error logs(which have no errrors only some info:
hi
log4j:WARN No appenders could be found for logger
(org.apache.zookeeper.ZooKeeper).
log4j:WARN Please initialize the log4j system properly.
2304 [main] INFO backtype.storm.zookeeper - Starting inprocess zookeeper
at port 2000 and dir /tmp/28a07471-6ce0-4dd9-b68c-e103d47d9a71
2552 [main] INFO backtype.storm.daemon.nimbus - Starting Nimbus with conf
{"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
"topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0,
"topology.skip.missing.kryo.registrations" true,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
"storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 50,
"nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
"java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib",
"topology.executor.send.buffer.size" 1024, "storm.local.dir"
"/tmp/9ab8714d-9391-4882-9be0-0c40b88ddff1",
"storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120,
"topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs"
600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
"topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost",
"storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2000,
"transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root"
"/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
"supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1,
"storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root"
"/transactional", "topology.acker.executors" nil,
"topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil,
"drpc.queue.size" 128, "worker.childopts" "-Xmx768m",
"supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
"supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
"topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
"topology.tasks" nil, "storm.messaging.netty.max_retries" 30,
"topology.spout.wait.strategy"
"backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending"
nil, "storm.zookeeper.retry.interval" 1000, "
topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
[6700 6701 6702 6703], "topology.debug" false, "nimbus.task.launch.secs"
120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs"
30, "task.refresh.poll.secs" 10, "topology.workers" 1,
"supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627,
"topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1,
"topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy"
"com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30,
"storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory"
"backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port"
3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times"
5, "storm.thrift.transport"
"backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs"
600, "storm.messaging.transport" "backtype.storm.messaging.zmq", "
logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port"
8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local",
"topology.optimize" true, "topology.max.task.parallelism" nil}
2564 [main] INFO backtype.storm.daemon.nimbus - Using default scheduler
2591 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl -
Starting
2640 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state
update: :connected:none
2666 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl -
Starting
2742 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl -
Starting
2754 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state
update: :connected:none
2759 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl -
Starting
2761 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl -
Starting
2774 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state
update: :connected:none
2777 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl -
Starting
2809 [main] INFO backtype.storm.daemon.supervisor - Starting Supervisor
with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
"topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0,
"topology.skip.missing.kryo.registrations" true,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
"storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 50,
"nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
"java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib",
"topology.executor.send.buffer.size" 1024, "storm.local.dir"
"/tmp/93a337fd-dceb-4306-a2b9-49747aa16fa7",
"storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120,
"topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs"
600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
"topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost",
"storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2000,
"transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root"
"/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
"supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1,
"storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root"
"/transactional", "topology.acker.executors" nil,
"topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil,
"drpc.queue.size" 128, "worker.childopts" "-Xmx768m",
"supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
"supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
"topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
"topology.tasks" nil, "storm.messaging.netty.max_retries" 30,
"topology.spout.wait.strategy"
"backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending"
nil, "storm.zookeeper.retry.interval" 1000, "
topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
(1 2 3), "topology.debug" false, "nimbus.task.launch.secs" 120,
"nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30,
"task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts"
"-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05,
"worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy"
"com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30,
"storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory"
"backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port"
3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times"
5, "storm.thrift.transport"
"backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs"
600, "storm.messaging.transport" "backtype.storm.messaging.zmq", "
logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port"
8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local",
"topology.optimize" true, "topology.max.task.parallelism" nil}
2826 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl -
Starting
2833 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state
update: :connected:none
2835 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl -
Starting
2906 [main] INFO backtype.storm.daemon.supervisor - Starting supervisor
with id 1b7ff988-e29b-433a-a489-e6512f03bf37 at host localhost
2924 [main] INFO backtype.storm.daemon.supervisor - Starting Supervisor
with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper",
"topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0,
"topology.skip.missing.kryo.registrations" true,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
"storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 50,
"nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
"java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib",
"topology.executor.send.buffer.size" 1024, "storm.local.dir"
"/tmp/69be37eb-f1ea-430f-b9d2-7cf5584885ca",
"storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120,
"topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs"
600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
"topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost",
"storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2000,
"transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root"
"/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
"supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1,
"storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root"
"/transactional", "topology.acker.executors" nil,
"topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil,
"drpc.queue.size" 128, "worker.childopts" "-Xmx768m",
"supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
"supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
"topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
"topology.tasks" nil, "storm.messaging.netty.max_retries" 30,
"topology.spout.wait.strategy"
"backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending"
nil, "storm.zookeeper.retry.interval" 1000, "
topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
(4 5 6), "topology.debug" false, "nimbus.task.launch.secs" 120,
"nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30,
"task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts"
"-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05,
"worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy"
"com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30,
"storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory"
"backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port"
3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times"
5, "storm.thrift.transport"
"backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs"
600, "storm.messaging.transport" "backtype.storm.messaging.zmq", "
logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port"
8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local",
"topology.optimize" true, "topology.max.task.parallelism" nil}
2953 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl -
Starting
2963 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state
update: :connected:none
2966 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl -
Starting
3001 [main] INFO backtype.storm.daemon.supervisor - Starting supervisor
with id 0bda2e40-b2fe-454a-840a-6b95715c24a1 at host localhost
1
connecting to localhost 2181
2
4
Properties Put
{broker.id=3, port=9099, zookeeper.connect=localhost:2181,
log.dir=localhost}
5
6
7
3
Created topic.sidd3
Error while executing topic command Topic "sidd3" already exists.
kafka.common.TopicExistsException: Topic "sidd3" already exists.
at
kafka.admin.AdminUtils$.createOrUpdateTopicPartitionAssignmentPathInZK(AdminUtils.scala:171)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:156)
at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:86)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:50)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
at first.trial.KafkaProducer.createTopic(KafkaProducer.java:162)
at first.trial.TestTopology.runTest(TestTopology.java:81)
at first.trial.TestTopology.main(TestTopology.java:68)
till here
till here 2
hi there
hi there, starting spout
hi there,connecting bolt
hi there
Kafka spout configured
20
jsonObject 0: {"index":1,"email":"clyde@o.bt","integer":30,"index_start_at":56,"name":"Mark","surname":"Shapiro","bool":false,"float":13.5577,"fullname":"Dan
Blackwell"}
4089 [main] INFO backtype.storm.daemon.nimbus - Received topology
submission for kafka-test with conf {"storm.id" "kafka-test-1-1407755256",
"kafka.zookeeper.connect" "localhost:2181", "kafka.consumer.timeout.ms"
10000, "topology.acker.executors" nil, "topology.kryo.decorators" (),
"kafka.spout.topic" "sidd3", "topology.name" "kafka-test", "topology.debug"
true, "topology.kryo.register" nil, "topology.max.task.parallelism" nil}
4179 [main] INFO backtype.storm.daemon.nimbus - Activating kafka-test:
kafka-test-1-1407755256
jsonObject 1: {"index":2,"email":"guy@robertson.pk","integer":33,"index_start_at":57,"name":"Maria","surname":"Monroe","bool":true,"float":14.7864,"fullname":"Lewis
Hawkins"}
jsonObject 2: {"index":3,"email":"shawn@creech.ye","integer":20,"index_start_at":58,"name":"Robyn","surname":"Fischer","bool":true,"float":10.4653,"fullname":"Melissa
Dillon"}
jsonObject 3: {"index":4,"email":"jordan@nixon.io","integer":40,"index_start_at":59,"name":"Peggy","surname":"Gibson","bool":true,"float":16.7735,"fullname":"Patrick
Cox"}
jsonObject 4: {"index":5,"email":"alison@wagner.cn","integer":47,"index_start_at":60,"name":"Calvin","surname":"Yates","bool":false,"float":14.7323,"fullname":"Geoffrey
Pate"}
jsonObject 5: {"index":6,"email":"neal@johnson.bf","integer":1,"index_start_at":61,"name":"Ernest","surname":"Love","bool":true,"float":12.0939,"fullname":"Linda
Gonzalez"}
jsonObject 6: {"index":7,"email":"tiffany@lamb.tz","integer":42,"index_start_at":62,"name":"Christine","surname":"Bowers","bool":false,"float":13.7826,"fullname":"Emma
Harris"}
jsonObject 7: {"index":8,"email":"barry@bray.bw","integer":17,"index_start_at":63,"name":"Ian","surname":"Abbott","bool":true,"float":10.1295,"fullname":"Arthur
Scarborough"}
jsonObject 8: {"index":9,"email":"roy@sawyer.my","integer":1,"index_start_at":64,"name":"Christian","surname":"Nixon","bool":true,"float":16.1275,"fullname":"Leroy
Singer"}
jsonObject 9: {"index":10,"email":"anna@currie.ao","integer":10,"index_start_at":65,"name":"Julian","surname":"Fisher","bool":false,"float":16.8829,"fullname":"Nelson
Levine"}
jsonObject 10: {"index":11,"email":"judith@stephens.is","integer":33,"index_start_at":66,"name":"Marsha","surname":"Maynard","bool":false,"float":16.0424,"fullname":"Pauline
Parsons"}
jsonObject 11: {"index":12,"email":"jon@lassiter.gy","integer":37,"index_start_at":67,"name":"Kimberly","surname":"McPherson","bool":true,"float":14.5787,"fullname":"Ethel
Quinn"}
jsonObject 12: {"index":13,"email":"donna@joseph.gh","integer":32,"index_start_at":68,"name":"Jordan","surname":"Cassidy","bool":true,"float":16.4224,"fullname":"Gary
Watts"}
jsonObject 13: {"index":14,"email":"patrick@fleming.cl","integer":15,"index_start_at":69,"name":"Bradley","surname":"Sellers","bool":false,"float":17.1929,"fullname":"Chris
Chan"}
jsonObject 14: {"index":15,"email":"kelly@reilly.om","integer":5,"index_start_at":70,"name":"Bruce","surname":"Smith","bool":false,"float":10.0177,"fullname":"Willie
Hanna"}
jsonObject 15: {"index":16,"email":"jerome@barr.fk","integer":6,"index_start_at":71,"name":"Ashley","surname":"Bunn","bool":true,"float":13.1174,"fullname":"Ricky
McFarland"}
jsonObject 16: {"index":17,"email":"juan@love.gd","integer":9,"index_start_at":72,"name":"Jack","surname":"Boyette","bool":false,"float":12.5841,"fullname":"Wade
Schultz"}
jsonObject 17: {"index":18,"email":"jason@proctor.st","integer":6,"index_start_at":73,"name":"Alfred","surname":"Olsen","bool":true,"float":18.3972,"fullname":"Billie
Rosenberg"}
jsonObject 18: {"index":19,"email":"dwight@freedman.fj","integer":26,"index_start_at":74,"name":"Ronald","surname":"Miles","bool":false,"float":15.1216,"fullname":"Sara
Bradley"}
jsonObject 19: {"index":20,"email":"luis@coleman.kw","integer":19,"index_start_at":75,"name":"Todd","surname":"Garrett","bool":true,"float":11.5306,"fullname":"Tim
Case"}
4494 [main] INFO backtype.storm.scheduler.EvenScheduler - Available slots:
(["1b7ff988-e29b-433a-a489-e6512f03bf37" 1]
["1b7ff988-e29b-433a-a489-e6512f03bf37" 2]
["1b7ff988-e29b-433a-a489-e6512f03bf37" 3]
["0bda2e40-b2fe-454a-840a-6b95715c24a1" 4]
["0bda2e40-b2fe-454a-840a-6b95715c24a1" 5]
["0bda2e40-b2fe-454a-840a-6b95715c24a1" 6])
4609 [main] INFO backtype.storm.daemon.nimbus - Setting new assignment for
topology id kafka-test-1-1407755256:
#backtype.storm.daemon.common.Assignment{:master-code-dir
"/tmp/9ab8714d-9391-4882-9be0-0c40b88ddff1/nimbus/stormdist/kafka-test-1-1407755256",
:node->host {"1b7ff988-e29b-433a-a489-e6512f03bf37" "localhost"},
:executor->node+port {[3 3] ["1b7ff988-e29b-433a-a489-e6512f03bf37" 1], [2
2] ["1b7ff988-e29b-433a-a489-e6512f03bf37" 1], [1 1]
["1b7ff988-e29b-433a-a489-e6512f03bf37" 1], [4 4]
["1b7ff988-e29b-433a-a489-e6512f03bf37" 1]}, :executor->start-time-secs {[4
4] 1407755256, [1 1] 1407755256, [2 2] 1407755256, [3 3] 1407755256}}
Started
4653 [Thread-5] INFO backtype.storm.daemon.supervisor - Downloading code
for storm id kafka-test-1-1407755256 from
/tmp/9ab8714d-9391-4882-9be0-0c40b88ddff1/nimbus/stormdist/kafka-test-1-1407755256
4784 [Thread-5] INFO backtype.storm.daemon.supervisor - Extracting
resources from jar at
/home/hduser/kafka_2.10-0.8.1.1/libs/kafka_2.10-0.8.1.1-javadoc.jar to
/tmp/93a337fd-dceb-4306-a2b9-49747aa16fa7/supervisor/stormdist/kafka-test-1-1407755256/resources
4797 [Thread-5] INFO backtype.storm.daemon.supervisor - Finished
downloading code for storm id kafka-test-1-1407755256 from
/tmp/9ab8714d-9391-4882-9be0-0c40b88ddff1/nimbus/stormdist/kafka-test-1-1407755256
4841 [Thread-6] INFO backtype.storm.daemon.supervisor - Launching worker
with assignment #backtype.storm.daemon.supervisor.LocalAssignment{:storm-id
"kafka-test-1-1407755256", :executors ([3 3] [2 2] [1 1] [4 4])} for this
supervisor 1b7ff988-e29b-433a-a489-e6512f03bf37 on port 1 with id
809d2557-d088-40fc-b58d-3efedcb969ca
4852 [Thread-6] INFO backtype.storm.daemon.worker - Launching worker for
kafka-test-1-1407755256 on 1b7ff988-e29b-433a-a489-e6512f03bf37:1 with id
809d2557-d088-40fc-b58d-3efedcb969ca and conf {"dev.zookeeper.path"
"/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true,
"topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0,
"topology.skip.missing.kryo.registrations" true,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
"storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 50,
"nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
"java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib",
"topology.executor.send.buffer.size" 1024, "storm.local.dir"
"/tmp/93a337fd-dceb-4306-a2b9-49747aa16fa7",
"storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120,
"topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs"
600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
"topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost",
"storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2000,
"transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root"
"/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
"supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1,
"storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root"
"/transactional", "topology.acker.executors" nil,
"topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil,
"drpc.queue.size" 128, "worker.childopts" "-Xmx768m",
"supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
"supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
"topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
"topology.tasks" nil, "storm.messaging.netty.max_retries" 30,
"topology.spout.wait.strategy"
"backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending"
nil, "storm.zookeeper.retry.interval" 1000, "
topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
(1 2 3), "topology.debug" false, "nimbus.task.launch.secs" 120,
"nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30,
"task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts"
"-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05,
"worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy"
"com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30,
"storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory"
"backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port"
3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times"
5, "storm.thrift.transport"
"backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs"
600, "storm.messaging.transport" "backtype.storm.messaging.zmq", "
logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port"
8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local",
"topology.optimize" true, "topology.max.task.parallelism" nil}
4853 [Thread-6] INFO
com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
4862 [Thread-6-EventThread] INFO backtype.storm.zookeeper - Zookeeper
state update: :connected:none
4865 [Thread-6] INFO
com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
5221 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
print:[2 2]
5231 [Thread-6] INFO backtype.storm.daemon.task - Emitting: print __system
["startup"]
5234 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor
tasks print:[2 2]
5248 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor print:[2 2]
5267 [Thread-21-print] INFO backtype.storm.daemon.executor - Preparing
bolt print:(2)
5277 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[3 3]
5278 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
5280 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor
tasks words:[3 3]
5283 [Thread-21-print] INFO backtype.storm.daemon.executor - Prepared bolt
print:(2)
5292 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[3 3]
5297 [Thread-23-words] INFO backtype.storm.daemon.executor - Opening spout
words:(3)
5308 [Thread-23-words] INFO
com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
5310 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
words:[4 4]
5315 [Thread-6] INFO backtype.storm.daemon.task - Emitting: words __system
["startup"]
5349 [Thread-23-words] INFO
com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
5353 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor
tasks words:[4 4]
5358 [Thread-23-words] INFO backtype.storm.daemon.executor - Opened spout
words:(3)
5370 [Thread-23-words] INFO backtype.storm.daemon.executor - Activating
spout words:(3)
5366 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor words:[4 4]
5378 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
5380 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
__system:[-1 -1]
5388 [Thread-6] INFO backtype.storm.daemon.task - Emitting: __system
__system ["startup"]
5385 [Thread-25-words] INFO backtype.storm.daemon.executor - Opening spout
words:(4)
5390 [Thread-25-words] INFO
com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
5385 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Deleted partition
managers: []
5391 [Thread-23-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
5391 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Finished refreshing
5391 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor
tasks __system:[-1 -1]
5399 [Thread-25-words] INFO
com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
5414 [Thread-25-words] INFO backtype.storm.daemon.executor - Opened spout
words:(4)
5415 [Thread-25-words] INFO backtype.storm.daemon.executor - Activating
spout words:(4)
5415 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
5415 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor __system:[-1 -1]
5424 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Deleted partition
managers: []
5424 [Thread-25-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
5424 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Finished refreshing
5438 [Thread-27-__system] INFO backtype.storm.daemon.executor - Preparing
bolt __system:(-1)
5438 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor
__acker:[1 1]
5439 [Thread-6] INFO backtype.storm.daemon.task - Emitting: __acker
__system ["startup"]
5439 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor
tasks __acker:[1 1]
5449 [Thread-27-__system] INFO backtype.storm.daemon.executor - Prepared
bolt __system:(-1)
5472 [Thread-6] INFO backtype.storm.daemon.executor - Timeouts disabled
for executor __acker:[1 1]
5472 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading
executor __acker:[1 1]
5472 [Thread-6] INFO backtype.storm.daemon.worker - Launching
receive-thread for 1b7ff988-e29b-433a-a489-e6512f03bf37:1
5474 [Thread-29-__acker] INFO backtype.storm.daemon.executor - Preparing
bolt __acker:(1)
5479 [Thread-29-__acker] INFO backtype.storm.daemon.executor - Prepared
bolt __acker:(1)
5489 [Thread-6] INFO backtype.storm.daemon.worker - Worker has topology
config {"storm.id" "kafka-test-1-1407755256", "dev.zookeeper.path"
"/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil,
"topology.builtin.metrics.bucket.size.secs" 60,
"topology.fall.back.on.java.serialization" true, "kafka.zookeeper.connect"
"localhost:2181", "topology.max.error.report.per.interval" 5,
"zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true,
"storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m",
"storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true,
"topology.trident.batch.emit.interval.millis" 50, "kafka.consumer.timeout.ms"
10000, "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m",
"java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib",
"topology.executor.send.buffer.size" 1024, "storm.local.dir"
"/tmp/93a337fd-dceb-4306-a2b9-49747aa16fa7",
"storm.messaging.netty.buffer_size" 5242880,
"supervisor.worker.start.timeout.secs" 120,
"topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs"
600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64,
"topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost",
"storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2000,
"transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size"
1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root"
"/storm", "storm.zookeeper.retry.intervalceiling.millis" 30000,
"supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1,
"storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root"
"/transactional", "topology.acker.executors" nil,
"topology.kryo.decorators" (), "kafka.spout.topic" "sidd3", "topology.name"
"kafka-test", "topology.transfer.buffer.size" 1024,
"topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts"
"-Xmx768m", "supervisor.heartbeat.frequency.secs" 5,
"topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772,
"supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m",
"topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3,
"topology.tasks" nil, "storm.messaging.netty.max_retries" 30,
"topology.spout.wait.strategy"
"backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending"
nil, "storm.zookeeper.retry.interval" 1000, "
topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator"
"backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports"
(1 2 3), "topology.debug" true, "nimbus.task.launch.secs" 120,
"nimbus.supervisor.timeout.secs" 60, "topology.kryo.register" nil,
"topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10,
"topology.workers" 1, "supervisor.childopts" "-Xmx256m",
"nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05,
"worker.heartbeat.frequency.secs" 1, "topology.tuple.serializer"
"backtype.storm.serialization.types.ListDelegateSerializer",
"topology.disruptor.wait.strategy"
"com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30,
"storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory"
"backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port"
3773, "logviewer.port" 8000, "zmq.threads" 1, "storm.zookeeper.retry.times"
5, "storm.thrift.transport"
"backtype.storm.security.auth.SimpleTransportPlugin",
"topology.state.synchronization.timeout.secs" 60,
"supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs"
600, "storm.messaging.transport" "backtype.storm.messaging.zmq", "
logviewer.appender.name" "A1", "storm.messaging.netty.max_wait_ms" 1000,
"drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port"
8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local",
"topology.optimize" true, "topology.max.task.parallelism" nil}
5489 [Thread-6] INFO backtype.storm.daemon.worker - Worker
809d2557-d088-40fc-b58d-3efedcb969ca for storm kafka-test-1-1407755256 on
1b7ff988-e29b-433a-a489-e6512f03bf37:1 has finished loading
20
jsonObject 0: {"index":1,"email":"clyde@o.bt","integer":30,"index_start_at":56,"name":"Mark","surname":"Shapiro","bool":false,"float":13.5577,"fullname":"Dan
Blackwell"}
jsonObject 1: {"index":2,"email":"guy@robertson.pk","integer":33,"index_start_at":57,"name":"Maria","surname":"Monroe","bool":true,"float":14.7864,"fullname":"Lewis
Hawkins"}
jsonObject 2: {"index":3,"email":"shawn@creech.ye","integer":20,"index_start_at":58,"name":"Robyn","surname":"Fischer","bool":true,"float":10.4653,"fullname":"Melissa
Dillon"}
jsonObject 3: {"index":4,"email":"jordan@nixon.io","integer":40,"index_start_at":59,"name":"Peggy","surname":"Gibson","bool":true,"float":16.7735,"fullname":"Patrick
Cox"}
jsonObject 4: {"index":5,"email":"alison@wagner.cn","integer":47,"index_start_at":60,"name":"Calvin","surname":"Yates","bool":false,"float":14.7323,"fullname":"Geoffrey
Pate"}
jsonObject 5: {"index":6,"email":"neal@johnson.bf","integer":1,"index_start_at":61,"name":"Ernest","surname":"Love","bool":true,"float":12.0939,"fullname":"Linda
Gonzalez"}
jsonObject 6: {"index":7,"email":"tiffany@lamb.tz","integer":42,"index_start_at":62,"name":"Christine","surname":"Bowers","bool":false,"float":13.7826,"fullname":"Emma
Harris"}
jsonObject 7: {"index":8,"email":"barry@bray.bw","integer":17,"index_start_at":63,"name":"Ian","surname":"Abbott","bool":true,"float":10.1295,"fullname":"Arthur
Scarborough"}
jsonObject 8: {"index":9,"email":"roy@sawyer.my","integer":1,"index_start_at":64,"name":"Christian","surname":"Nixon","bool":true,"float":16.1275,"fullname":"Leroy
Singer"}
jsonObject 9: {"index":10,"email":"anna@currie.ao","integer":10,"index_start_at":65,"name":"Julian","surname":"Fisher","bool":false,"float":16.8829,"fullname":"Nelson
Levine"}
jsonObject 10: {"index":11,"email":"judith@stephens.is","integer":33,"index_start_at":66,"name":"Marsha","surname":"Maynard","bool":false,"float":16.0424,"fullname":"Pauline
Parsons"}
jsonObject 11: {"index":12,"email":"jon@lassiter.gy","integer":37,"index_start_at":67,"name":"Kimberly","surname":"McPherson","bool":true,"float":14.5787,"fullname":"Ethel
Quinn"}
jsonObject 12: {"index":13,"email":"donna@joseph.gh","integer":32,"index_start_at":68,"name":"Jordan","surname":"Cassidy","bool":true,"float":16.4224,"fullname":"Gary
Watts"}
jsonObject 13: {"index":14,"email":"patrick@fleming.cl","integer":15,"index_start_at":69,"name":"Bradley","surname":"Sellers","bool":false,"float":17.1929,"fullname":"Chris
Chan"}
jsonObject 14: {"index":15,"email":"kelly@reilly.om","integer":5,"index_start_at":70,"name":"Bruce","surname":"Smith","bool":false,"float":10.0177,"fullname":"Willie
Hanna"}
jsonObject 15: {"index":16,"email":"jerome@barr.fk","integer":6,"index_start_at":71,"name":"Ashley","surname":"Bunn","bool":true,"float":13.1174,"fullname":"Ricky
McFarland"}
jsonObject 16: {"index":17,"email":"juan@love.gd","integer":9,"index_start_at":72,"name":"Jack","surname":"Boyette","bool":false,"float":12.5841,"fullname":"Wade
Schultz"}
jsonObject 17: {"index":18,"email":"jason@proctor.st","integer":6,"index_start_at":73,"name":"Alfred","surname":"Olsen","bool":true,"float":18.3972,"fullname":"Billie
Rosenberg"}
jsonObject 18: {"index":19,"email":"dwight@freedman.fj","integer":26,"index_start_at":74,"name":"Ronald","surname":"Miles","bool":false,"float":15.1216,"fullname":"Sara
Bradley"}
jsonObject 19: {"index":20,"email":"luis@coleman.kw","integer":19,"index_start_at":75,"name":"Todd","surname":"Garrett","bool":true,"float":11.5306,"fullname":"Tim
Case"}
13834 [Thread-2] INFO backtype.storm.daemon.nimbus - Setting new
assignment for topology id kafka-test-1-1407755256:
#backtype.storm.daemon.common.Assignment{:master-code-dir
"/tmp/9ab8714d-9391-4882-9be0-0c40b88ddff1/nimbus/stormdist/kafka-test-1-1407755256",
:node->host {"1b7ff988-e29b-433a-a489-e6512f03bf37" "localhost"},
:executor->node+port {[3 3] ["1b7ff988-e29b-433a-a489-e6512f03bf37" 1], [4
4] ["1b7ff988-e29b-433a-a489-e6512f03bf37" 1], [2 2]
["1b7ff988-e29b-433a-a489-e6512f03bf37" 1], [1 1]
["1b7ff988-e29b-433a-a489-e6512f03bf37" 1]}, :executor->start-time-secs {[4
4] 1407755265, [1 1] 1407755265, [2 2] 1407755265, [3 3] 1407755265}}
35291 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
35368 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
65284 [Thread-21-print] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
65291 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
65294 [Thread-21-print] INFO backtype.storm.daemon.task - Emitting: print
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@25fb025> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=1,
read_pos=0, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]>]]
65369 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
65370 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
65371 [Thread-23-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@58254e5a> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {}]> #<DataPoint [__fail-count = {}]> #<DataPoint
[__emit-count = {}]>]]
65394 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
65417 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
65420 [Thread-25-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@793b23eb> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=3, read_pos=2, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {__system=0}]> #<DataPoint [__fail-count = {}]>
#<DataPoint [__emit-count = {__system=20}]>]]
65422 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Deleted partition
managers: []
65422 [Thread-23-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
65422 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
65425 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
65427 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Deleted partition
managers: []
65427 [Thread-25-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
65427 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
65455 [Thread-27-__system] INFO backtype.storm.daemon.executor -
Processing received message source: __system:-1, stream: __metrics_tick,
id: {}, [60]
65456 [Thread-27-__system] INFO backtype.storm.daemon.task - Emitting:
__system __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@6346bbdc> [#<DataPoint
[__ack-count = {}]> #<DataPoint [memory/heap = {unusedBytes=39172800,
usedBytes=19350848, maxBytes=932577280, initBytes=60264576,
virtualFreeBytes=913226432, committedBytes=58523648}]> #<DataPoint
[__receive = {write_pos=1, read_pos=0, capacity=1024, population=1}]>
#<DataPoint [__fail-count = {}]> #<DataPoint [__execute-latency = {}]>
#<DataPoint [newWorkerEvent = 1]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]> #<DataPoint [__sendqueue =
{write_pos=-1, read_pos=-1, capacity=1024, population=0}]> #<DataPoint
[memory/nonHeap = {unusedBytes=594920, usedBytes=42069016,
maxBytes=224395264, initBytes=24313856, virtualFreeBytes=182326248,
committedBytes=42663936}]> #<DataPoint [uptimeSecs = 67.071]> #<DataPoint
[__transfer = {write_pos=0, read_pos=0, capacity=1024, population=0}]>
#<DataPoint [startTimeSecs = 1.407755250479E9]> #<DataPoint
[__process-latency = {}]> #<DataPoint [__transfer-count = {}]>]]
65481 [Thread-29-__acker] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
65482 [Thread-29-__acker] INFO backtype.storm.daemon.task - Emitting:
__acker __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@38d79b6f> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=1,
read_pos=0, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]>]]
95293 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
95370 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
125285 [Thread-21-print] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
125291 [Thread-21-print] INFO backtype.storm.daemon.task - Emitting: print
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@290efe9b> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=2,
read_pos=1, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]>]]
125293 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
125370 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
125370 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
125371 [Thread-23-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@176d872c> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {}]> #<DataPoint [__fail-count = {}]> #<DataPoint
[__emit-count = {}]>]]
125419 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
125420 [Thread-25-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@17c565b8> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=6, read_pos=5, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {__system=0}]> #<DataPoint [__fail-count = {}]>
#<DataPoint [__emit-count = {__system=0}]>]]
125423 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
125426 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Deleted
partition managers: []
125427 [Thread-23-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
125427 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
125428 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
125431 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Deleted
partition managers: []
125431 [Thread-25-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
125431 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
125455 [Thread-27-__system] INFO backtype.storm.daemon.executor -
Processing received message source: __system:-1, stream: __metrics_tick,
id: {}, [60]
125456 [Thread-27-__system] INFO backtype.storm.daemon.task - Emitting:
__system __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@20991f39> [#<DataPoint
[__ack-count = {}]> #<DataPoint [GC/Copy = {count=11, timeMs=33}]>
#<DataPoint [memory/heap = {unusedBytes=33221352, usedBytes=25302296,
maxBytes=932577280, initBytes=60264576, virtualFreeBytes=907274984,
committedBytes=58523648}]> #<DataPoint [__receive = {write_pos=2,
read_pos=1, capacity=1024, population=1}]> #<DataPoint [__fail-count = {}]>
#<DataPoint [__execute-latency = {}]> #<DataPoint [newWorkerEvent = 0]>
#<DataPoint [__emit-count = {}]> #<DataPoint [__execute-count = {}]>
#<DataPoint [GC/MarkSweepCompact = {count=0, timeMs=0}]> #<DataPoint
[__sendqueue = {write_pos=-1, read_pos=-1, capacity=1024, population=0}]>
#<DataPoint [memory/nonHeap = {unusedBytes=598568, usedBytes=42393048,
maxBytes=224395264, initBytes=24313856, virtualFreeBytes=182002216,
committedBytes=42991616}]> #<DataPoint [uptimeSecs = 127.071]> #<DataPoint
[__transfer = {write_pos=0, read_pos=0, capacity=1024, population=0}]>
#<DataPoint [startTimeSecs = 1.407755250479E9]> #<DataPoint
[__process-latency = {}]> #<DataPoint [__transfer-count = {}]>]]
125482 [Thread-29-__acker] INFO backtype.storm.daemon.executor -
Processing received message source: __system:-1, stream: __metrics_tick,
id: {}, [60]
125483 [Thread-29-__acker] INFO backtype.storm.daemon.task - Emitting:
__acker __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@24d0e7e7> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=2,
read_pos=1, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]>]]
155294 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
155370 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
185286 [Thread-21-print] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
185288 [Thread-21-print] INFO backtype.storm.daemon.task - Emitting: print
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@18d9fe61> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=3,
read_pos=2, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]>]]
185293 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
185370 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
185371 [Thread-23-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@fcb9f06> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=9, read_pos=8, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {}]> #<DataPoint [__fail-count = {}]> #<DataPoint
[__emit-count = {}]>]]
185371 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
185419 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
185420 [Thread-25-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@1354b04> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=9, read_pos=8, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {__system=0}]> #<DataPoint [__fail-count = {}]>
#<DataPoint [__emit-count = {__system=0}]>]]
185428 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
185432 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
185432 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Deleted
partition managers: []
185433 [Thread-23-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
185436 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
185436 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Deleted
partition managers: []
185437 [Thread-25-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
185437 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
185455 [Thread-27-__system] INFO backtype.storm.daemon.executor -
Processing received message source: __system:-1, stream: __metrics_tick,
id: {}, [60]
185456 [Thread-27-__system] INFO backtype.storm.daemon.task - Emitting:
__system __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@51cff43a> [#<DataPoint
[__ack-count = {}]> #<DataPoint [GC/Copy = {count=12, timeMs=33}]>
#<DataPoint [memory/heap = {unusedBytes=43092672, usedBytes=15430976,
maxBytes=932577280, initBytes=60264576, virtualFreeBytes=917146304,
committedBytes=58523648}]> #<DataPoint [__receive = {write_pos=3,
read_pos=2, capacity=1024, population=1}]> #<DataPoint [__fail-count = {}]>
#<DataPoint [__execute-latency = {}]> #<DataPoint [newWorkerEvent = 0]>
#<DataPoint [__emit-count = {}]> #<DataPoint [__execute-count = {}]>
#<DataPoint [GC/MarkSweepCompact = {count=0, timeMs=0}]> #<DataPoint
[__sendqueue = {write_pos=-1, read_pos=-1, capacity=1024, population=0}]>
#<DataPoint [memory/nonHeap = {unusedBytes=368360, usedBytes=42623256,
maxBytes=224395264, initBytes=24313856, virtualFreeBytes=181772008,
committedBytes=42991616}]> #<DataPoint [uptimeSecs = 187.071]> #<DataPoint
[__transfer = {write_pos=0, read_pos=0, capacity=1024, population=0}]>
#<DataPoint [startTimeSecs = 1.407755250479E9]> #<DataPoint
[__process-latency = {}]> #<DataPoint [__transfer-count = {}]>]]
185483 [Thread-29-__acker] INFO backtype.storm.daemon.executor -
Processing received message source: __system:-1, stream: __metrics_tick,
id: {}, [60]
185484 [Thread-29-__acker] INFO backtype.storm.daemon.task - Emitting:
__acker __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@2bab35cc> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=3,
read_pos=2, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]>]]
215298 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
215370 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
245287 [Thread-21-print] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
245289 [Thread-21-print] INFO backtype.storm.daemon.task - Emitting: print
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@5b9b2088> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=4,
read_pos=3, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]>]]
245296 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
245371 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
245371 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
245372 [Thread-23-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@4725931e> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=12, read_pos=11, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {}]> #<DataPoint [__fail-count = {}]> #<DataPoint
[__emit-count = {}]>]]
245420 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
245421 [Thread-25-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@73ddbb0f> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=12, read_pos=11, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {__system=0}]> #<DataPoint [__fail-count = {}]>
#<DataPoint [__emit-count = {__system=0}]>]]
245439 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
245442 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Deleted
partition managers: []
245442 [Thread-23-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
245442 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
245442 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
245444 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Deleted
partition managers: []
245444 [Thread-25-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
245444 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
245456 [Thread-27-__system] INFO backtype.storm.daemon.executor -
Processing received message source: __system:-1, stream: __metrics_tick,
id: {}, [60]
245457 [Thread-27-__system] INFO backtype.storm.daemon.task - Emitting:
__system __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@62241136> [#<DataPoint
[__ack-count = {}]> #<DataPoint [GC/Copy = {count=10, timeMs=26}]>
#<DataPoint [memory/heap = {unusedBytes=31192296, usedBytes=27331352,
maxBytes=932577280, initBytes=60264576, virtualFreeBytes=905245928,
committedBytes=58523648}]> #<DataPoint [__receive = {write_pos=4,
read_pos=3, capacity=1024, population=1}]> #<DataPoint [__fail-count = {}]>
#<DataPoint [__execute-latency = {}]> #<DataPoint [newWorkerEvent = 0]>
#<DataPoint [__emit-count = {}]> #<DataPoint [__execute-count = {}]>
#<DataPoint [GC/MarkSweepCompact = {count=0, timeMs=0}]> #<DataPoint
[__sendqueue = {write_pos=-1, read_pos=-1, capacity=1024, population=0}]>
#<DataPoint [memory/nonHeap = {unusedBytes=299176, usedBytes=42757976,
maxBytes=224395264, initBytes=24313856, virtualFreeBytes=181637288,
committedBytes=43057152}]> #<DataPoint [uptimeSecs = 247.072]> #<DataPoint
[__transfer = {write_pos=0, read_pos=0, capacity=1024, population=0}]>
#<DataPoint [startTimeSecs = 1.407755250479E9]> #<DataPoint
[__process-latency = {}]> #<DataPoint [__transfer-count = {}]>]]
245484 [Thread-29-__acker] INFO backtype.storm.daemon.executor -
Processing received message source: __system:-1, stream: __metrics_tick,
id: {}, [60]
245485 [Thread-29-__acker] INFO backtype.storm.daemon.task - Emitting:
__acker __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@6ea66d4c> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=4,
read_pos=3, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]>]]
275297 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
275371 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
305287 [Thread-21-print] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
305289 [Thread-21-print] INFO backtype.storm.daemon.task - Emitting: print
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@5f9aa8db> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=5,
read_pos=4, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {__metrics=0}]> #<DataPoint
[__execute-latency = {}]> #<DataPoint [__fail-count = {}]> #<DataPoint
[__emit-count = {__metrics=20}]> #<DataPoint [__execute-count = {}]>]]
305296 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
305371 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
305372 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
305373 [Thread-23-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@22c3f2e6> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=15, read_pos=14, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {}]> #<DataPoint [__fail-count = {}]> #<DataPoint
[__emit-count = {}]>]]
305420 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
305421 [Thread-25-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@6e7febde> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=15, read_pos=14, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {__system=0}]> #<DataPoint [__fail-count = {}]>
#<DataPoint [__emit-count = {__system=0}]>]]
305443 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
305446 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Deleted
partition managers: []
305446 [Thread-23-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
305446 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
305446 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
305449 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Deleted
partition managers: []
305450 [Thread-25-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
305450 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
305456 [Thread-27-__system] INFO backtype.storm.daemon.executor -
Processing received message source: __system:-1, stream: __metrics_tick,
id: {}, [60]
305457 [Thread-27-__system] INFO backtype.storm.daemon.task - Emitting:
__system __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@1056747f> [#<DataPoint
[__ack-count = {}]> #<DataPoint [GC/Copy = {count=12, timeMs=34}]>
#<DataPoint [memory/heap = {unusedBytes=42625928, usedBytes=15897720,
maxBytes=932577280, initBytes=60264576, virtualFreeBytes=916679560,
committedBytes=58523648}]> #<DataPoint [__receive = {write_pos=5,
read_pos=4, capacity=1024, population=1}]> #<DataPoint [__fail-count = {}]>
#<DataPoint [__execute-latency = {}]> #<DataPoint [newWorkerEvent = 0]>
#<DataPoint [__emit-count = {}]> #<DataPoint [__execute-count = {}]>
#<DataPoint [GC/MarkSweepCompact = {count=0, timeMs=0}]> #<DataPoint
[__sendqueue = {write_pos=-1, read_pos=-1, capacity=1024, population=0}]>
#<DataPoint [memory/nonHeap = {unusedBytes=253792, usedBytes=42868896,
maxBytes=224395264, initBytes=24313856, virtualFreeBytes=181526368,
committedBytes=43122688}]> #<DataPoint [uptimeSecs = 307.072]> #<DataPoint
[__transfer = {write_pos=0, read_pos=0, capacity=1024, population=0}]>
#<DataPoint [startTimeSecs = 1.407755250479E9]> #<DataPoint
[__process-latency = {}]> #<DataPoint [__transfer-count = {}]>]]
305484 [Thread-29-__acker] INFO backtype.storm.daemon.executor -
Processing received message source: __system:-1, stream: __metrics_tick,
id: {}, [60]
305485 [Thread-29-__acker] INFO backtype.storm.daemon.task - Emitting:
__acker __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@1bfa463a> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=5,
read_pos=4, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]>]]
335297 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
335373 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
365288 [Thread-21-print] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
365290 [Thread-21-print] INFO backtype.storm.daemon.task - Emitting: print
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@7222d7c2> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=6,
read_pos=5, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {__metrics=0}]> #<DataPoint
[__execute-latency = {}]> #<DataPoint [__fail-count = {}]> #<DataPoint
[__emit-count = {__metrics=0}]> #<DataPoint [__execute-count = {}]>]]
365298 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
365373 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
365373 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
365374 [Thread-23-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@4d833c18> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=18, read_pos=17, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {}]> #<DataPoint [__fail-count = {}]> #<DataPoint
[__emit-count = {}]>]]
365420 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __metrics_tick, id: {}, [60]
365421 [Thread-25-words] INFO backtype.storm.daemon.task - Emitting: words
__metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@62de7369> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__complete-latency = {}]>
#<DataPoint [__receive = {write_pos=18, read_pos=17, capacity=1024,
population=1}]> #<DataPoint [kafkaOffset = {totalLatestTime=0,
totalSpoutLag=0, totalLatestEmittedOffset=0}]> #<DataPoint
[__transfer-count = {__system=0}]> #<DataPoint [__fail-count = {}]>
#<DataPoint [__emit-count = {__system=0}]>]]
365447 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
365450 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Deleted
partition managers: []
365450 [Thread-23-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
365450 [Thread-23-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
365452 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Refreshing
partition manager connections
365454 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Deleted
partition managers: []
365454 [Thread-25-words] INFO storm.kafka.ZkCoordinator - New partition
managers: []
365454 [Thread-25-words] INFO storm.kafka.ZkCoordinator - Finished
refreshing
365457 [Thread-27-__system] INFO backtype.storm.daemon.executor -
Processing received message source: __system:-1, stream: __metrics_tick,
id: {}, [60]
365459 [Thread-27-__system] INFO backtype.storm.daemon.task - Emitting:
__system __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@7a6b2534> [#<DataPoint
[__ack-count = {}]> #<DataPoint [GC/Copy = {count=11, timeMs=35}]>
#<DataPoint [memory/heap = {unusedBytes=34780944, usedBytes=23742704,
maxBytes=932577280, initBytes=60264576, virtualFreeBytes=908834576,
committedBytes=58523648}]> #<DataPoint [__receive = {write_pos=6,
read_pos=5, capacity=1024, population=1}]> #<DataPoint [__fail-count = {}]>
#<DataPoint [__execute-latency = {}]> #<DataPoint [newWorkerEvent = 0]>
#<DataPoint [__emit-count = {}]> #<DataPoint [__execute-count = {}]>
#<DataPoint [GC/MarkSweepCompact = {count=0, timeMs=0}]> #<DataPoint
[__sendqueue = {write_pos=-1, read_pos=-1, capacity=1024, population=0}]>
#<DataPoint [memory/nonHeap = {unusedBytes=268072, usedBytes=42920152,
maxBytes=224395264, initBytes=24313856, virtualFreeBytes=181475112,
committedBytes=43188224}]> #<DataPoint [uptimeSecs = 367.073]> #<DataPoint
[__transfer = {write_pos=0, read_pos=0, capacity=1024, population=0}]>
#<DataPoint [startTimeSecs = 1.407755250479E9]> #<DataPoint
[__process-latency = {}]> #<DataPoint [__transfer-count = {}]>]]
365485 [Thread-29-__acker] INFO backtype.storm.daemon.executor -
Processing received message source: __system:-1, stream: __metrics_tick,
id: {}, [60]
365486 [Thread-29-__acker] INFO backtype.storm.daemon.task - Emitting:
__acker __metrics [#<TaskInfo
backtype.storm.metric.api.IMetricsConsumer$TaskInfo@1057b7b6> [#<DataPoint
[__ack-count = {}]> #<DataPoint [__sendqueue = {write_pos=-1, read_pos=-1,
capacity=1024, population=0}]> #<DataPoint [__receive = {write_pos=6,
read_pos=5, capacity=1024, population=1}]> #<DataPoint [__process-latency =
{}]> #<DataPoint [__transfer-count = {}]> #<DataPoint [__execute-latency =
{}]> #<DataPoint [__fail-count = {}]> #<DataPoint [__emit-count = {}]>
#<DataPoint [__execute-count = {}]>]]
395298 [Thread-23-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30]
395373 [Thread-25-words] INFO backtype.storm.daemon.executor - Processing
received message source: __system:-1, stream: __tick, id: {}, [30}
SOS guys....please find the code attached if required.
Thanks,
Siddharth Ubale