You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by ka...@apache.org on 2016/08/12 02:23:29 UTC
svn commit: r1756068 - in /storm/site/releases: 0.10.0/Concepts.md
0.10.1/Concepts.md 0.9.6/Concepts.md 1.0.0/Troubleshooting.md
1.0.1/Troubleshooting.md 1.0.2/Troubleshooting.md
2.0.0-SNAPSHOT/Troubleshooting.md
Author: kabhwan
Date: Fri Aug 12 02:23:29 2016
New Revision: 1756068
URL: http://svn.apache.org/viewvc?rev=1756068&view=rev
Log:
Apply STORM-841 and STORM-1999
Modified:
storm/site/releases/0.10.0/Concepts.md
storm/site/releases/0.10.1/Concepts.md
storm/site/releases/0.9.6/Concepts.md
storm/site/releases/1.0.0/Troubleshooting.md
storm/site/releases/1.0.1/Troubleshooting.md
storm/site/releases/1.0.2/Troubleshooting.md
storm/site/releases/2.0.0-SNAPSHOT/Troubleshooting.md
Modified: storm/site/releases/0.10.0/Concepts.md
URL: http://svn.apache.org/viewvc/storm/site/releases/0.10.0/Concepts.md?rev=1756068&r1=1756067&r2=1756068&view=diff
==============================================================================
--- storm/site/releases/0.10.0/Concepts.md (original)
+++ storm/site/releases/0.10.0/Concepts.md Fri Aug 12 02:23:29 2016
@@ -65,7 +65,8 @@ When you declare a bolt's input streams,
The main method in bolts is the `execute` method which takes in as input a new tuple. Bolts emit new tuples using the [OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) object. Bolts must call the `ack` method on the `OutputCollector` for every tuple they process so that Storm knows when tuples are completed (and can eventually determine that its safe to ack the original spout tuples). For the common case of processing an input tuple, emitting 0 or more tuples based on that tuple, and then acking the input tuple, Storm provides an [IBasicBolt](javadocs/backtype/storm/topology/IBasicBolt.html) interface which does the acking automatically.
-Its perfectly fine to launch new threads in bolts that do processing asynchronously. [OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) is thread-safe and can be called at any time.
+Please note that [OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) is not thread-safe, and all emits, acks, and fails must happen on the same thread. Please refer [Troubleshooting](troubleshooting.html) for more details.
+
**Resources:**
Modified: storm/site/releases/0.10.1/Concepts.md
URL: http://svn.apache.org/viewvc/storm/site/releases/0.10.1/Concepts.md?rev=1756068&r1=1756067&r2=1756068&view=diff
==============================================================================
--- storm/site/releases/0.10.1/Concepts.md (original)
+++ storm/site/releases/0.10.1/Concepts.md Fri Aug 12 02:23:29 2016
@@ -65,7 +65,8 @@ When you declare a bolt's input streams,
The main method in bolts is the `execute` method which takes in as input a new tuple. Bolts emit new tuples using the [OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) object. Bolts must call the `ack` method on the `OutputCollector` for every tuple they process so that Storm knows when tuples are completed (and can eventually determine that its safe to ack the original spout tuples). For the common case of processing an input tuple, emitting 0 or more tuples based on that tuple, and then acking the input tuple, Storm provides an [IBasicBolt](javadocs/backtype/storm/topology/IBasicBolt.html) interface which does the acking automatically.
-Its perfectly fine to launch new threads in bolts that do processing asynchronously. [OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) is thread-safe and can be called at any time.
+Please note that [OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) is not thread-safe, and all emits, acks, and fails must happen on the same thread. Please refer [Troubleshooting](troubleshooting.html) for more details.
+
**Resources:**
Modified: storm/site/releases/0.9.6/Concepts.md
URL: http://svn.apache.org/viewvc/storm/site/releases/0.9.6/Concepts.md?rev=1756068&r1=1756067&r2=1756068&view=diff
==============================================================================
--- storm/site/releases/0.9.6/Concepts.md (original)
+++ storm/site/releases/0.9.6/Concepts.md Fri Aug 12 02:23:29 2016
@@ -65,7 +65,8 @@ When you declare a bolt's input streams,
The main method in bolts is the `execute` method which takes in as input a new tuple. Bolts emit new tuples using the [OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) object. Bolts must call the `ack` method on the `OutputCollector` for every tuple they process so that Storm knows when tuples are completed (and can eventually determine that its safe to ack the original spout tuples). For the common case of processing an input tuple, emitting 0 or more tuples based on that tuple, and then acking the input tuple, Storm provides an [IBasicBolt](javadocs/backtype/storm/topology/IBasicBolt.html) interface which does the acking automatically.
-Its perfectly fine to launch new threads in bolts that do processing asynchronously. [OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) is thread-safe and can be called at any time.
+Please note that [OutputCollector](javadocs/backtype/storm/task/OutputCollector.html) is not thread-safe, and all emits, acks, and fails must happen on the same thread. Please refer [Troubleshooting](troubleshooting.html) for more details.
+
**Resources:**
Modified: storm/site/releases/1.0.0/Troubleshooting.md
URL: http://svn.apache.org/viewvc/storm/site/releases/1.0.0/Troubleshooting.md?rev=1756068&r1=1756067&r2=1756068&view=diff
==============================================================================
--- storm/site/releases/1.0.0/Troubleshooting.md (original)
+++ storm/site/releases/1.0.0/Troubleshooting.md Fri Aug 12 02:23:29 2016
@@ -112,71 +112,3 @@ Caused by: java.util.ConcurrentModificat
Solution:
* This means that you're emitting a mutable object as an output tuple. Everything you emit into the output collector must be immutable. What's happening is that your bolt is modifying the object while it is being serialized to be sent over the network.
-
-
-### NullPointerException from deep inside Storm
-
-Symptoms:
-
- * You get a NullPointerException that looks something like:
-
-```
-java.lang.RuntimeException: java.lang.NullPointerException
- at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84)
- at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:55)
- at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:56)
- at org.apache.storm.disruptor$consume_loop_STAR_$fn__1596.invoke(disruptor.clj:67)
- at org.apache.storm.util$async_loop$fn__465.invoke(util.clj:377)
- at clojure.lang.AFn.run(AFn.java:24)
- at java.lang.Thread.run(Thread.java:662)
-Caused by: java.lang.NullPointerException
- at org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:24)
- at org.apache.storm.daemon.worker$mk_transfer_fn$fn__4126$fn__4130.invoke(worker.clj:99)
- at org.apache.storm.util$fast_list_map.invoke(util.clj:771)
- at org.apache.storm.daemon.worker$mk_transfer_fn$fn__4126.invoke(worker.clj:99)
- at org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3904.invoke(executor.clj:205)
- at org.apache.storm.disruptor$clojure_handler$reify__1584.onEvent(disruptor.clj:43)
- at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:81)
- ... 6 more
-```
-
-or
-
-```
-java.lang.RuntimeException: java.lang.NullPointerException
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$consume_loop_STAR_$fn__759.invoke(disruptor.clj:94)
-~[storm-core-0.9.3.jar:0.9.3]
- at org.apache.storm.util$async_loop$fn__458.invoke(util.clj:463)
-~[storm-core-0.9.3.jar:0.9.3]
- at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
- at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
-Caused by: java.lang.NullPointerException: null
- at clojure.lang.RT.intCast(RT.java:1087) ~[clojure-1.5.1.jar:na]
- at
-org.apache.storm.daemon.worker$mk_transfer_fn$fn__3548.invoke(worker.clj:129)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3282.invoke(executor.clj:258)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$clojure_handler$reify__746.onEvent(disruptor.clj:58)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
-~[storm-core-0.9.3.jar:0.9.3]
- ... 6 common frames omitted
-```
-
-Solution:
-
- * This is caused by having multiple threads issue methods on the `OutputCollector`. All emits, acks, and fails must happen on the same thread. One subtle way this can happen is if you make a `IBasicBolt` that emits on a separate thread. `IBasicBolt`'s automatically ack after execute is called, so this would cause multiple threads to use the `OutputCollector` leading to this exception. When using a basic bolt, all emits must happen in the same thread that runs `execute`.
Modified: storm/site/releases/1.0.1/Troubleshooting.md
URL: http://svn.apache.org/viewvc/storm/site/releases/1.0.1/Troubleshooting.md?rev=1756068&r1=1756067&r2=1756068&view=diff
==============================================================================
--- storm/site/releases/1.0.1/Troubleshooting.md (original)
+++ storm/site/releases/1.0.1/Troubleshooting.md Fri Aug 12 02:23:29 2016
@@ -112,71 +112,3 @@ Caused by: java.util.ConcurrentModificat
Solution:
* This means that you're emitting a mutable object as an output tuple. Everything you emit into the output collector must be immutable. What's happening is that your bolt is modifying the object while it is being serialized to be sent over the network.
-
-
-### NullPointerException from deep inside Storm
-
-Symptoms:
-
- * You get a NullPointerException that looks something like:
-
-```
-java.lang.RuntimeException: java.lang.NullPointerException
- at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84)
- at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:55)
- at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:56)
- at org.apache.storm.disruptor$consume_loop_STAR_$fn__1596.invoke(disruptor.clj:67)
- at org.apache.storm.util$async_loop$fn__465.invoke(util.clj:377)
- at clojure.lang.AFn.run(AFn.java:24)
- at java.lang.Thread.run(Thread.java:662)
-Caused by: java.lang.NullPointerException
- at org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:24)
- at org.apache.storm.daemon.worker$mk_transfer_fn$fn__4126$fn__4130.invoke(worker.clj:99)
- at org.apache.storm.util$fast_list_map.invoke(util.clj:771)
- at org.apache.storm.daemon.worker$mk_transfer_fn$fn__4126.invoke(worker.clj:99)
- at org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3904.invoke(executor.clj:205)
- at org.apache.storm.disruptor$clojure_handler$reify__1584.onEvent(disruptor.clj:43)
- at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:81)
- ... 6 more
-```
-
-or
-
-```
-java.lang.RuntimeException: java.lang.NullPointerException
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$consume_loop_STAR_$fn__759.invoke(disruptor.clj:94)
-~[storm-core-0.9.3.jar:0.9.3]
- at org.apache.storm.util$async_loop$fn__458.invoke(util.clj:463)
-~[storm-core-0.9.3.jar:0.9.3]
- at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
- at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
-Caused by: java.lang.NullPointerException: null
- at clojure.lang.RT.intCast(RT.java:1087) ~[clojure-1.5.1.jar:na]
- at
-org.apache.storm.daemon.worker$mk_transfer_fn$fn__3548.invoke(worker.clj:129)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3282.invoke(executor.clj:258)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$clojure_handler$reify__746.onEvent(disruptor.clj:58)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
-~[storm-core-0.9.3.jar:0.9.3]
- ... 6 common frames omitted
-```
-
-Solution:
-
- * This is caused by having multiple threads issue methods on the `OutputCollector`. All emits, acks, and fails must happen on the same thread. One subtle way this can happen is if you make a `IBasicBolt` that emits on a separate thread. `IBasicBolt`'s automatically ack after execute is called, so this would cause multiple threads to use the `OutputCollector` leading to this exception. When using a basic bolt, all emits must happen in the same thread that runs `execute`.
Modified: storm/site/releases/1.0.2/Troubleshooting.md
URL: http://svn.apache.org/viewvc/storm/site/releases/1.0.2/Troubleshooting.md?rev=1756068&r1=1756067&r2=1756068&view=diff
==============================================================================
--- storm/site/releases/1.0.2/Troubleshooting.md (original)
+++ storm/site/releases/1.0.2/Troubleshooting.md Fri Aug 12 02:23:29 2016
@@ -112,71 +112,3 @@ Caused by: java.util.ConcurrentModificat
Solution:
* This means that you're emitting a mutable object as an output tuple. Everything you emit into the output collector must be immutable. What's happening is that your bolt is modifying the object while it is being serialized to be sent over the network.
-
-
-### NullPointerException from deep inside Storm
-
-Symptoms:
-
- * You get a NullPointerException that looks something like:
-
-```
-java.lang.RuntimeException: java.lang.NullPointerException
- at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84)
- at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:55)
- at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:56)
- at org.apache.storm.disruptor$consume_loop_STAR_$fn__1596.invoke(disruptor.clj:67)
- at org.apache.storm.util$async_loop$fn__465.invoke(util.clj:377)
- at clojure.lang.AFn.run(AFn.java:24)
- at java.lang.Thread.run(Thread.java:662)
-Caused by: java.lang.NullPointerException
- at org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:24)
- at org.apache.storm.daemon.worker$mk_transfer_fn$fn__4126$fn__4130.invoke(worker.clj:99)
- at org.apache.storm.util$fast_list_map.invoke(util.clj:771)
- at org.apache.storm.daemon.worker$mk_transfer_fn$fn__4126.invoke(worker.clj:99)
- at org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3904.invoke(executor.clj:205)
- at org.apache.storm.disruptor$clojure_handler$reify__1584.onEvent(disruptor.clj:43)
- at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:81)
- ... 6 more
-```
-
-or
-
-```
-java.lang.RuntimeException: java.lang.NullPointerException
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$consume_loop_STAR_$fn__759.invoke(disruptor.clj:94)
-~[storm-core-0.9.3.jar:0.9.3]
- at org.apache.storm.util$async_loop$fn__458.invoke(util.clj:463)
-~[storm-core-0.9.3.jar:0.9.3]
- at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
- at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
-Caused by: java.lang.NullPointerException: null
- at clojure.lang.RT.intCast(RT.java:1087) ~[clojure-1.5.1.jar:na]
- at
-org.apache.storm.daemon.worker$mk_transfer_fn$fn__3548.invoke(worker.clj:129)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3282.invoke(executor.clj:258)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$clojure_handler$reify__746.onEvent(disruptor.clj:58)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
-~[storm-core-0.9.3.jar:0.9.3]
- ... 6 common frames omitted
-```
-
-Solution:
-
- * This is caused by having multiple threads issue methods on the `OutputCollector`. All emits, acks, and fails must happen on the same thread. One subtle way this can happen is if you make a `IBasicBolt` that emits on a separate thread. `IBasicBolt`'s automatically ack after execute is called, so this would cause multiple threads to use the `OutputCollector` leading to this exception. When using a basic bolt, all emits must happen in the same thread that runs `execute`.
Modified: storm/site/releases/2.0.0-SNAPSHOT/Troubleshooting.md
URL: http://svn.apache.org/viewvc/storm/site/releases/2.0.0-SNAPSHOT/Troubleshooting.md?rev=1756068&r1=1756067&r2=1756068&view=diff
==============================================================================
--- storm/site/releases/2.0.0-SNAPSHOT/Troubleshooting.md (original)
+++ storm/site/releases/2.0.0-SNAPSHOT/Troubleshooting.md Fri Aug 12 02:23:29 2016
@@ -112,71 +112,3 @@ Caused by: java.util.ConcurrentModificat
Solution:
* This means that you're emitting a mutable object as an output tuple. Everything you emit into the output collector must be immutable. What's happening is that your bolt is modifying the object while it is being serialized to be sent over the network.
-
-
-### NullPointerException from deep inside Storm
-
-Symptoms:
-
- * You get a NullPointerException that looks something like:
-
-```
-java.lang.RuntimeException: java.lang.NullPointerException
- at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84)
- at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:55)
- at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:56)
- at org.apache.storm.disruptor$consume_loop_STAR_$fn__1596.invoke(disruptor.clj:67)
- at org.apache.storm.util$async_loop$fn__465.invoke(util.clj:377)
- at clojure.lang.AFn.run(AFn.java:24)
- at java.lang.Thread.run(Thread.java:662)
-Caused by: java.lang.NullPointerException
- at org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:24)
- at org.apache.storm.daemon.worker$mk_transfer_fn$fn__4126$fn__4130.invoke(worker.clj:99)
- at org.apache.storm.util$fast_list_map.invoke(util.clj:771)
- at org.apache.storm.daemon.worker$mk_transfer_fn$fn__4126.invoke(worker.clj:99)
- at org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3904.invoke(executor.clj:205)
- at org.apache.storm.disruptor$clojure_handler$reify__1584.onEvent(disruptor.clj:43)
- at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:81)
- ... 6 more
-```
-
-or
-
-```
-java.lang.RuntimeException: java.lang.NullPointerException
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$consume_loop_STAR_$fn__759.invoke(disruptor.clj:94)
-~[storm-core-0.9.3.jar:0.9.3]
- at org.apache.storm.util$async_loop$fn__458.invoke(util.clj:463)
-~[storm-core-0.9.3.jar:0.9.3]
- at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
- at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
-Caused by: java.lang.NullPointerException: null
- at clojure.lang.RT.intCast(RT.java:1087) ~[clojure-1.5.1.jar:na]
- at
-org.apache.storm.daemon.worker$mk_transfer_fn$fn__3548.invoke(worker.clj:129)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3282.invoke(executor.clj:258)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.disruptor$clojure_handler$reify__746.onEvent(disruptor.clj:58)
-~[storm-core-0.9.3.jar:0.9.3]
- at
-org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
-~[storm-core-0.9.3.jar:0.9.3]
- ... 6 common frames omitted
-```
-
-Solution:
-
- * This is caused by having multiple threads issue methods on the `OutputCollector`. All emits, acks, and fails must happen on the same thread. One subtle way this can happen is if you make a `IBasicBolt` that emits on a separate thread. `IBasicBolt`'s automatically ack after execute is called, so this would cause multiple threads to use the `OutputCollector` leading to this exception. When using a basic bolt, all emits must happen in the same thread that runs `execute`.