[jira] [Commented] (ZOOKEEPER-1147) Add support for local sessions

["Patrick Hunt (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ZOOKEEPER-1147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13579626#comment-13579626 ] 

Patrick Hunt commented on ZOOKEEPER-1147:
-----------------------------------------

My initial thoughts after a quick first pass:

We should add docs as part of the patch.

What's the reason for having the config option to disable local session upgrade? Did you see a specific need for this?

Given some of the discussion above (more frequent expiry), it seems to me that we should make sessions global by default and provide an option to ZooKeeper(...) to enable local session usage. On the flip side making sessions local by default will result in everyone seeing an improvement w/o need for code changes (client side I'm saying). That said, it seems that for most people global vs local session is not an issue. For the small subset of users where it is an issue they could enable the local session support via a parameter, a very simple code change for users on the client. Thoughts?

The timeouts in the tests are too low in my experience. Use org.apache.zookeeper.test.ClientBase.CONNECTION_TIMEOUT unless there's some reason not to? (some of the hardware we run testing on is virtualized, can make things very slow)

The change to watchertest - move this to it's own jira. We can can get that reviewed/committed quickly. Likely we need this for more than just trunk.

                
> Add support for local sessions
> ------------------------------
>
>                 Key: ZOOKEEPER-1147
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1147
>             Project: ZooKeeper
>          Issue Type: Improvement
>          Components: server
>    Affects Versions: 3.3.3
>            Reporter: Vishal Kathuria
>            Assignee: Thawan Kooburat
>              Labels: api-change, scaling
>             Fix For: 3.5.0
>
>         Attachments: ZOOKEEPER-1147.patch
>
>   Original Estimate: 840h
>  Remaining Estimate: 840h
>
> This improvement is in the bucket of making ZooKeeper work at a large scale. We are planning on having about a 1 million clients connect to a ZooKeeper ensemble through a set of 50-100 observers. Majority of these clients are read only - ie they do not do any updates or create ephemeral nodes.
> In ZooKeeper today, the client creates a session and the session creation is handled like any other update. In the above use case, the session create/drop workload can easily overwhelm an ensemble. The following is a proposal for a "local session", to support a larger number of connections.
> 1.       The idea is to introduce a new type of session - "local" session. A "local" session doesn't have a full functionality of a normal session.
> 2.       Local sessions cannot create ephemeral nodes.
> 3.       Once a local session is lost, you cannot re-establish it using the session-id/password. The session and its watches are gone for good.
> 4.       When a local session connects, the session info is only maintained on the zookeeper server (in this case, an observer) that it is connected to. The leader is not aware of the creation of such a session and there is no state written to disk.
> 5.       The pings and expiration is handled by the server that the session is connected to.
> With the above changes, we can make ZooKeeper scale to a much larger number of clients without making the core ensemble a bottleneck.
> In terms of API, there are two options that are being considered
> 1. Let the client specify at the connect time which kind of session do they want.
> 2. All sessions connect as local sessions and automatically get promoted to global sessions when they do an operation that requires a global session (e.g. creating an ephemeral node)
> Chubby took the approach of lazily promoting all sessions to global, but I don't think that would work in our case, where we want to keep sessions which never create ephemeral nodes as always local. Option 2 would make it more broadly usable but option 1 would be easier to implement.
> We are thinking of implementing option 1 as the first cut. There would be a client flag, IsLocalSession (much like the current readOnly flag) that would be used to determine whether to create a local session or a global session.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira