You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stormcrawler.apache.org by Tim Allison <ta...@apache.org> on 2025/04/01 20:52:23 UTC
FileSpout and "Local Server connection should not send BackPressure status"
All,
I recently upgraded to our latest release, and I also bumped OpenSearch
to the latest. When injecting several thousand seeds with the FileSpout, I
started getting the stacktrace below. I'm not confident that the upgrade
caused the problem...
The injector spins up a local zookeeper+storm instance.
If I modified the FileSpout to monitor the diff between emitted and
ack'd/error'd and kept that diff below the `topology.max.spout.pending`
with a very small Thread.sleep(), the problem went away.
The question: is this user error, or is this a bug that I should fix?
Thank you.
Best,
Tim
15:04:41.763 [Worker-Transfer] ERROR o.a.s.u.Utils - Async loop died!
│
│ java.lang.RuntimeException: Local Server connection should not send
BackPressure status
│
│ at
org.apache.storm.messaging.local.Context$LocalServer.sendBackPressureStatus(Context.java:111)
~[storm-client-2.8.0.jar:2.8.0]
│
│ at
org.apache.storm.daemon.worker.WorkerState.transferLocalBatch(WorkerState.java:586)
~[storm-client-2.8.0.jar:2.8.0]
│
│ at
org.apache.storm.messaging.DeserializingConnectionCallback.recv(DeserializingConnectionCallback.java:71)
~[storm-client-2.8.0.jar:2.8.0]
│
│ at
org.apache.storm.messaging.local.Context$LocalClient.send(Context.java:178)
~[storm-client-2.8.0.jar:2.8.0]
│
│ at
org.apache.storm.utils.TransferDrainer.send(TransferDrainer.java:53)
~[storm-client-2.8.0.jar:2.8.0]
│
│ at
org.apache.storm.daemon.worker.WorkerTransfer.flush(WorkerTransfer.java:103)
~[storm-client-2.8.0.jar:2.8.0]
│
│ at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:126)
~[storm-client-2.8.0.jar:2.8.0]
│
│ at org.apache.storm.utils.JCQueue.consume(JCQueue.java:89)
~[storm-client-2.8.0.jar:2.8.0]
│
│ at org.apache.storm.utils.JCQueue.consume(JCQueue.java:80)
~[storm-client-2.8.0.jar:2.8.0]
│
│ at
org.apache.storm.daemon.worker.WorkerTransfer.lambda$makeTransferThread$0(WorkerTransfer.java:85)
~[storm-client-2.8.0.jar:2.8.0]
│
│ at org.apache.storm.utils.Utils$1.run(Utils.java:398)
[storm-client-2.8.0.jar:2.8.0]
│
│ at java.base/java.lang.Thread.run(Unknown Source) [?:?]
Re: FileSpout and "Local Server connection should not send BackPressure status"
Posted by Richard Zowalla <rz...@apache.org>.
Hi Tim,
I think it is in the nature of things here. If we can avoid confusing the user, it would be beneficial, to fix it, imho.
I had the same thing a while back and simply increased the max pending limit.
Gruß
Richard
> Am 01.04.2025 um 22:52 schrieb Tim Allison <ta...@apache.org>:
>
> All,
>
> I recently upgraded to our latest release, and I also bumped OpenSearch
> to the latest. When injecting several thousand seeds with the FileSpout, I
> started getting the stacktrace below. I'm not confident that the upgrade
> caused the problem...
>
> The injector spins up a local zookeeper+storm instance.
>
> If I modified the FileSpout to monitor the diff between emitted and
> ack'd/error'd and kept that diff below the `topology.max.spout.pending`
> with a very small Thread.sleep(), the problem went away.
>
> The question: is this user error, or is this a bug that I should fix?
>
> Thank you.
>
> Best,
>
> Tim
>
> 15:04:41.763 [Worker-Transfer] ERROR o.a.s.u.Utils - Async loop died!
>
>
> │
> │ java.lang.RuntimeException: Local Server connection should not send
> BackPressure status
>
> │
> │ at
> org.apache.storm.messaging.local.Context$LocalServer.sendBackPressureStatus(Context.java:111)
> ~[storm-client-2.8.0.jar:2.8.0]
> │
> │ at
> org.apache.storm.daemon.worker.WorkerState.transferLocalBatch(WorkerState.java:586)
> ~[storm-client-2.8.0.jar:2.8.0]
> │
> │ at
> org.apache.storm.messaging.DeserializingConnectionCallback.recv(DeserializingConnectionCallback.java:71)
> ~[storm-client-2.8.0.jar:2.8.0]
> │
> │ at
> org.apache.storm.messaging.local.Context$LocalClient.send(Context.java:178)
> ~[storm-client-2.8.0.jar:2.8.0]
>
> │
> │ at
> org.apache.storm.utils.TransferDrainer.send(TransferDrainer.java:53)
> ~[storm-client-2.8.0.jar:2.8.0]
>
> │
> │ at
> org.apache.storm.daemon.worker.WorkerTransfer.flush(WorkerTransfer.java:103)
> ~[storm-client-2.8.0.jar:2.8.0]
>
> │
> │ at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:126)
> ~[storm-client-2.8.0.jar:2.8.0]
>
> │
> │ at org.apache.storm.utils.JCQueue.consume(JCQueue.java:89)
> ~[storm-client-2.8.0.jar:2.8.0]
>
> │
> │ at org.apache.storm.utils.JCQueue.consume(JCQueue.java:80)
> ~[storm-client-2.8.0.jar:2.8.0]
>
> │
> │ at
> org.apache.storm.daemon.worker.WorkerTransfer.lambda$makeTransferThread$0(WorkerTransfer.java:85)
> ~[storm-client-2.8.0.jar:2.8.0]
> │
> │ at org.apache.storm.utils.Utils$1.run(Utils.java:398)
> [storm-client-2.8.0.jar:2.8.0]
>
> │
> │ at java.base/java.lang.Thread.run(Unknown Source) [?:?]