You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@zookeeper.apache.org by "Kevin Wikant (Jira)" <ji...@apache.org> on 2024/06/18 16:47:00 UTC
[jira] [Created] (ZOOKEEPER-4840) Repeated SessionExpiredException after Zookeeper daemon restart
Kevin Wikant created ZOOKEEPER-4840:
---------------------------------------
Summary: Repeated SessionExpiredException after Zookeeper daemon restart
Key: ZOOKEEPER-4840
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-4840
Project: ZooKeeper
Issue Type: Bug
Reporter: Kevin Wikant
## Background
Application is using Zookeeper for leader election & metadata storage. The application runs on 3 hosts which each also have 1 Zookeeper daemon running.
Previously the application was running on Zookeeper version 3.5.10 & Curator version 4.3.0
After upgrading to Zookeeper version 3.9.1 & Curator version 5.2.0 a new edge case was observed where after Zookeeper daemons are restarted/failed the application (i.e. Zookeeper client) enters a 15+ minute loop of repeatedly logging "{{{}SessionExpiredException"{}}}
These repeated "{{{}SessionExpiredException"{}}} are not indicative of a full Zookeeper client outage because DEBUG logs show that other Zookeeper sessions are communicating just fine. The "{{{}SessionExpiredException"{}}} logs unfortunately do not show the associated Session ID
## Symptoms
When using Zookeeper version 3.9.1 & Curator version 5.2.0, after restarting/failing some of the Zookeeper daemons:
# All the 3 zookeeper clients experience some connections failures lasting a few seconds after the Zookeeper daemons were failed/restarted.
# These connection failure issues are resolved shortly without any action needed.
# Around 1 minute after the Zookeeper daemons were failed/restarted, all the 3 zookeeper clients start repeatedly logging "{{{}SessionExpiredException"{}}}
# The "{{{}SessionExpiredException" {}}}is repeatedly logged for 15+ minutes. During this time there are no connectivity issues. We can see from the Zookeeper server logs that all 3 Zookeeper servers are receiving regular traffic from the clients.
# Interestingly, each Zookeeper server is not receiving any requests from the local Zookeeper client for the duration of the period where "{{{}SessionExpiredException"{}}}is repeatedly logged. However, each Zookeeper server is receiving regular traffic from the 2 remote Zookeeper clients.
The evidence suggests that this is a client-side issue & the "{{{}SessionExpiredException" {}}}is being thrown before the request is even sent to the Zookeeper server.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)