You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by John Lindwall <jl...@yahoo.com.INVALID> on 2021/01/14 18:55:35 UTC
3.4.6: Unable to read additional data from client sessionid 0xABC,
likely client has closed socket
We're seeing thousands of these a day in our zookeeper logs (zookeeper 3.4.6):
WARN [NIOServerCxn.Factory:X.X.X.X/X.X.X.X:Y:NIOServerCnxn@357] - caught end of stream exceptionEndOfStreamException: Unable to read additional data from client sessionid 0xABC, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)
Any ideas of the cause of this? Could an excessive number/size of znodes be a factor? We're not seeing any obvious client-side issues. We're not sure but we believe that these were not happening earlier.
-- John
Re: 3.4.6: Unable to read additional data from client sessionid
0xABC, likely client has closed socket
Posted by John Lindwall <jl...@yahoo.com.INVALID>.
Mate,
Thanks for the reply! We identified several thousand excess znodes lingering in the cluster and after deleting them we see no difference in the error rate.
We are exclusively using java clients in our code, no C or python.
We do indeed have plans to update our ensembles this year, thanks!
-- John
On Monday, January 18, 2021, 12:23:47 AM PST, Szalay-Bekő Máté <sz...@gmail.com> wrote:
resending my answer (it was sent originally to jlindwall@yahoo.com.invalid
- I wonder why)
On Mon, Jan 18, 2021 at 9:12 AM Szalay-Bekő Máté <sz...@gmail.com>
wrote:
> Hi John,
>
> > Could an excessive number/size of znodes be a factor?
> I don't think this would be a likely case... more like a client error I
> think.
>
> The stack-trace suggests that the client closed the TCP session
> unexpectedly. Maybe the clients are missing the zookeeper.close() calls?
>
> Also I wonder if it is caused by
> https://issues.apache.org/jira/browse/ZOOKEEPER-1105 - are you using C
> (or python) client?
> (there was this bug, that the C client didn't wait for the proper session
> close to finish)
>
>
> Btw: please consider upgrading your cluster, 3.4 is end-of-life now and it
> is not supported by the community anymore.
>
> Cheers,
> Mate
>
> On Thu, Jan 14, 2021 at 7:55 PM John Lindwall <jl...@yahoo.com.invalid>
> wrote:
>
>> We're seeing thousands of these a day in our zookeeper logs (zookeeper
>> 3.4.6):
>> WARN [NIOServerCxn.Factory:X.X.X.X/X.X.X.X:Y:NIOServerCnxn@357] -
>> caught end of stream exceptionEndOfStreamException: Unable to read
>> additional data from client sessionid 0xABC, likely client has closed socket
>> at
>> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
>> at
>> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> Any ideas of the cause of this? Could an excessive number/size of znodes
>> be a factor? We're not seeing any obvious client-side issues. We're not
>> sure but we believe that these were not happening earlier.
>>
>> -- John
>
>
Re: 3.4.6: Unable to read additional data from client sessionid
0xABC, likely client has closed socket
Posted by Szalay-Bekő Máté <sz...@gmail.com>.
resending my answer (it was sent originally to jlindwall@yahoo.com.invalid
- I wonder why)
On Mon, Jan 18, 2021 at 9:12 AM Szalay-Bekő Máté <sz...@gmail.com>
wrote:
> Hi John,
>
> > Could an excessive number/size of znodes be a factor?
> I don't think this would be a likely case... more like a client error I
> think.
>
> The stack-trace suggests that the client closed the TCP session
> unexpectedly. Maybe the clients are missing the zookeeper.close() calls?
>
> Also I wonder if it is caused by
> https://issues.apache.org/jira/browse/ZOOKEEPER-1105 - are you using C
> (or python) client?
> (there was this bug, that the C client didn't wait for the proper session
> close to finish)
>
>
> Btw: please consider upgrading your cluster, 3.4 is end-of-life now and it
> is not supported by the community anymore.
>
> Cheers,
> Mate
>
> On Thu, Jan 14, 2021 at 7:55 PM John Lindwall <jl...@yahoo.com.invalid>
> wrote:
>
>> We're seeing thousands of these a day in our zookeeper logs (zookeeper
>> 3.4.6):
>> WARN [NIOServerCxn.Factory:X.X.X.X/X.X.X.X:Y:NIOServerCnxn@357] -
>> caught end of stream exceptionEndOfStreamException: Unable to read
>> additional data from client sessionid 0xABC, likely client has closed socket
>> at
>> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
>> at
>> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> Any ideas of the cause of this? Could an excessive number/size of znodes
>> be a factor? We're not seeing any obvious client-side issues. We're not
>> sure but we believe that these were not happening earlier.
>>
>> -- John
>
>
Re: 3.4.6: Unable to read additional data from client sessionid
0xABC, likely client has closed socket
Posted by Szalay-Bekő Máté <sz...@gmail.com>.
Hi John,
> Could an excessive number/size of znodes be a factor?
I don't think this would be a likely case... more like a client error I
think.
The stack-trace suggests that the client closed the TCP session
unexpectedly. Maybe the clients are missing the zookeeper.close() calls?
Also I wonder if it is caused by
https://issues.apache.org/jira/browse/ZOOKEEPER-1105 - are you using C (or
python) client?
(there was this bug, that the C client didn't wait for the proper session
close to finish)
Btw: please consider upgrading your cluster, 3.4 is end-of-life now and it
is not supported by the community anymore.
Cheers,
Mate
On Thu, Jan 14, 2021 at 7:55 PM John Lindwall <jl...@yahoo.com.invalid>
wrote:
> We're seeing thousands of these a day in our zookeeper logs (zookeeper
> 3.4.6):
> WARN [NIOServerCxn.Factory:X.X.X.X/X.X.X.X:Y:NIOServerCnxn@357] - caught
> end of stream exceptionEndOfStreamException: Unable to read additional data
> from client sessionid 0xABC, likely client has closed socket
> at
> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> at
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> at java.lang.Thread.run(Thread.java:745)
>
> Any ideas of the cause of this? Could an excessive number/size of znodes
> be a factor? We're not seeing any obvious client-side issues. We're not
> sure but we believe that these were not happening earlier.
>
> -- John