You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Yang Nian <yd...@gmail.com> on 2020/11/13 10:00:47 UTC
Re: Storm 2.0.0 Kerberos Socket Ouput
someone who ever use online video compressor from PickFrom
<https://pickfrom.net>.
Online Video Compressor <https://pickfrom.net/compress-video>
On Thu, Oct 22, 2020 at 9:59 PM Kim, Peter <Pe...@blackrock.com> wrote:
> Hello,
>
>
>
> Recently, we’ve implemented storm 2.0.0 with Kerberos. Before integrating
> Kerberos, the logs were fine and no mention of socket output being closed
> but since we implemented Kerberos, we’re seeing the following in our logs :
>
> 2020-10-21 14:39:53.529 o.a.s.t.t.TIOStreamTransport
> refresh-connections-timer [WARN] Error closing output stream.
>
> java.net.SocketException: Socket closed
>
> at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116)
> ~[?:1.8.0_45]
>
> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> ~[?:1.8.0_45]
>
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> ~[?:1.8.0_45]
>
> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> ~[?:1.8.0_45]
>
> at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
> ~[?:1.8.0_45]
>
> at
> org.apache.storm.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
> [storm-shaded-deps-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.thrift.transport.TSocket.close(TSocket.java:235)
> [storm-shaded-deps-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:106)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:69)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.utils.SupervisorClient.<init>(SupervisorClient.java:45)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.utils.SupervisorClient.getConfiguredClientAs(SupervisorClient.java:73)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.utils.SupervisorClient.getConfiguredClient(SupervisorClient.java:61)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.daemon.worker.WorkerState.getLocalAssignment(WorkerState.java:647)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.daemon.worker.WorkerState.refreshConnections(WorkerState.java:367)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.daemon.worker.WorkerState.refreshConnections(WorkerState.java:356)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.daemon.worker.Worker$$Lambda$61/1922191129.run(Unknown
> Source) [storm-client-2.0.0.jar:2.0.0]
>
> at org.apache.storm.StormTimer$1.run(StormTimer.java:110)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:226)
> [storm-client-2.0.0.jar:2.0.0]
>
> 2020-10-21 14:43:37.827 o.a.s.t.t.TIOStreamTransport heartbeat-timer
> [WARN] Error closing output stream.
>
> java.net.SocketException: Socket closed
>
> at
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116)
> ~[?:1.8.0_45]
>
> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
> ~[?:1.8.0_45]
>
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> ~[?:1.8.0_45]
>
> at
> java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> ~[?:1.8.0_45]
>
> at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
> ~[?:1.8.0_45]
>
> at
> org.apache.storm.thrift.transport.TIOStreamTransport.close(TIOStreamTransport.java:110)
> [storm-shaded-deps-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.thrift.transport.TSocket.close(TSocket.java:235)
> [storm-shaded-deps-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:106)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:69)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.utils.SupervisorClient.<init>(SupervisorClient.java:45)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.utils.SupervisorClient.getConfiguredClientAs(SupervisorClient.java:73)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.utils.SupervisorClient.getConfiguredClient(SupervisorClient.java:61)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.daemon.worker.Worker.heartbeatToMasterIfLocalbeatFail(Worker.java:429)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.daemon.worker.Worker.doHeartBeat(Worker.java:344)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.daemon.worker.Worker.lambda$loadWorker$1(Worker.java:189)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.daemon.worker.Worker$$Lambda$30/435297131.run(Unknown
> Source) [storm-client-2.0.0.jar:2.0.0]
>
> at org.apache.storm.StormTimer$1.run(StormTimer.java:110)
> [storm-client-2.0.0.jar:2.0.0]
>
> at
> org.apache.storm.StormTimer$StormTimerTask.run(StormTimer.java:226)
> [storm-client-2.0.0.jar:2.0.0]
>
> 2020-10-21 14:43:37.827 o.a.s.d.w.Worker heartbeat-timer [WARN] Exception
> when send heartbeat to local supervisor
>
>
>
> I’ve done some searching and tried adjusting the worker heartbeat
> frequency sec but that did more harm than good. I’ve followed the trace and
> the only configuration that I see that is being pulled in is the
> worker.heartbeat.freqs.secs set at 1.
>
>
>
> Is this a known issue or has anyone come across this before? Any thoughts
> on what might be the issue and if it’s critical? Thanks for your help.
>
>
>
> Best,
>
> Peter
>
>
>
>
>
> This message may contain information that is confidential or privileged.
> If you are not the intended recipient, please advise the sender immediately
> and delete this message. See
> http://www.blackrock.com/corporate/compliance/email-disclaimers for
> further information. Please refer to
> http://www.blackrock.com/corporate/compliance/privacy-policy for more
> information about BlackRock’s Privacy Policy.
>
> For a list of BlackRock's office addresses worldwide, see
> http://www.blackrock.com/corporate/about-us/contacts-locations.
>
> © 2020 BlackRock, Inc. All rights reserved.
>
>