You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by James Strachan <ja...@gmail.com> on 2006/07/12 15:16:53 UTC
Re: [wadi-dev] Session/clustering API and the web tier
On 7/12/06, Jules Gosnell <ju...@coredevelopers.net> wrote:
> Greg Wilkins wrote:
> > All,
> >
> >
> > Here are my comments on the Session API that were promised after apachecon dublin.
> > This is also CC'd to the wadi list and some of the points are relevant to them
> > as well.
> >
> > My own reason for focusing on the Session API when I think about clustering,
> > is that I like the idea of pluggable clustering implementations. Clustering
> > is not one size fits all and solutions can go from non-replicated nodes configured
> > in a flat file to auto discovered, self healing, redundant hierarchies.
Agreed. We should be focussed purely on what is the contract between a
container and the session API and making that contract as simple and
abstract as is possible while minimising leaky abstractions.
> > I think the previous discussions we had on this were making good progress, but
> > I think we ran out of steam before the API was improved etc. So I think it
> > worthwhile to re-read the original threads.
> >
> > But I will repeat my main unresolved concerns here:
> > While I appreciate the keep-it-simple-stupid approach adopted by the proposed
> > session API, I remain concerned that it may over simplify and may also mix concerns.
> >
> > However, I do think that the API is pitched at about the right level - namely
> > that it is below the specific concerns of things such as HTTP. As the implementor
> > of the web container, I would prefer to not delegate HttpSession management
> > or request proxying to a pluggable session implementation (I doubt that
> > a cluster impl wants to deal with non-blocking proxying of requests etc.)
>
> I think that our discussions about this have suffered from an ambiguity
> around the word 'delegate'...
>
> In one sense of the word, given WADI's current implementation, Jetty
> does delegate Session management and HTTP handling to WADI, in that WADI
> passes the WebApp/Jetty an object on which it calls a method and the
> work in question is done.
>
> However, in another sense, Jetty need not delegate this task, since the
> object returned in these cases is managed by WADI, but created by a
> Factory that is injected at startup time. This factory might be
> generating instances of a class that has very Jetty-specific knowledge
> or is even a part of the Jetty distro...
Thats certainly one approach. Another is for the container to just ask
the policy API what to do (i.e. is the request going to be serviced
locally or not) so that the container can take care of the rest.
I understand the cleanliness from the session API implementor's
perspective of using a factory and calling back the container when you
see fit - however I also understand the container developers
requirement to understand at all times what each thread is doing, to
tune things aggressively with full knowledge of threading models and
to generally be master of its own domain, so I can understand why a
container developer might prefer a non-callback related solution
(which could introduce all kinds of nasty thread related bugs into the
container). I don't see why both options can't be offered.
> I would wholeheartedly agree that the code for Http request relocation
> should be written by someone with expertise in that area - namely the
> container writer. I would just rather see it injected into the clustered
> manager, so that it can be called when required, without having to
> burden Jetty with the added task of making this decision itself.
I don't see that as mutually exclusive. Just have a way for Jetty to
ask the clustering solution if a request can be satisfied locally, if
not Jetty does the proxy/redirect thing.
> > I see that the webcontainer needs to interact with the cluster implementation
> > in 4 areas:
> >
> >
> > 1) Policy
> > ---------
> >
> > When a container receives a request, it needs to make a policy decision along
> > the lines of:
> >
> > 1) The request can be handled locally.
> > 2) The request can be handled locally, but only after some other actions
> > (eg session is moved to local)
> > 3) request cannot be handled locally, but can be redirected to another node
> > 4) request cannot be handled locally, but can be proxied to another node.
> >
> > This kind of corresponds to the Locator and SessionLocation APIs. However
> > these APIs give the power to enact a policy decision, but give no support to make
> > a policy decision.
> >
> > To implement a policy, you might want to use: the size of the cluster, the total
> > number of sessions, the number of session on the local node, the number of sessions
> > collocated with a remote session, how many requests for the session have recently
> > arrived on what nodes, etc. etc.
> >
> > The API does not give me this information and I think it would be
> > difficult to provide all that might be used. Potentially
> > we could get by with a mechanism to store/access cluster wide meta-data
> > attributes?
> >
> > However, it is very unlikely that one policy will fit all, so each consumer
> > of this Location API will have to implement a pluggable policy frame work of
> > some sorts.
> >
> > But as the session API is already a pluggable framework, why don't we
> > just delegate the policy decision to the API. The web container
> > should make the policy decision, but should call the session API to
> > make the decision. Something like:
> >
> > SessionLocation executeAt = locator.getSessionExecutionLocation(clientId);
> > if (executeAt.isLocal())
> > // handle request
> > else
> > // proxy or redirect to executeAt location.
> >
> > (Note the need for something like this has been discussed before and
> > generally agreed. I have seen the proposed RemoteSessionStrategy, but I am not
> > sure how you obtain a handle to one - nor do I think the policy should
> > decide between redirect and proxy - which is HTTP business).
Agreed. Just some way to ask the Session API if a request can be
processed locally might do the trick, then if not Jetty can do its
proxy/redirect thing. The trickier thing is what to pass into the
strategy to help it decide...
> By exposing the 'policy' api to the container and putting it in charge
> of when it used, you are exposing clustering details to it.
Also the container details may be required by this policy. e.g.
details about the previous http requests received at the current node,
their type and various metadata statitsics and so forth which only the
container is aware of.
> WADI's approach is to completely shield the container from having to
> know anything about clustering, whilst maintaining contracts with the
> container encapsulating various pieces of tier/domain-specific
> functionality that may be injected into the clustered session manager.
The issue is though, how invisible can clustering ever be? Information
from the container and from the clustering implementation will
typically be required for the policy decision.
> > 3) Life cycle
> >
> > Unfortunately the life and death of a session is not simple - specially
> > when cross context dispatch is considered. Session ID's may or may not
> > be reused, their uniqueness might need to be guarenteed and the decision
> > may depend on the existence of the same session ID in other contexts.
> >
> > I think this can be modeled with a structured name space - so perhaps
> > this is not an issue anymore?
> >
> >
> > 4) Configuration and Management
> > It would generally be good to be able to know how many nodes are in
> > the cluster (or to set what the nodes are). To be able to monitor node
> > status and give commands to gracefully or brutally shutdown a node, move
> > sessions etc.
> >
> > Clustering aware clients (JNDI stubs, EJB proxies or potentially fancy
> > Ajax web clients) might need to be passed a list of known nodes - but it
> > is not possible to obtain/set that from the API - thus every impl will need
> > to implement it's own cluster config/discover even if that information is
> > available in other implementations.
> >
> >
>
> This is the clustering API (in my mind) that was mooted in the meeting.
> A number of clustering substrates (JGroups, ActiveCluster, Tribes,
> etc...) have homesteaded this area (WADI maintains an abstraction layer
> that can map on to any of these three). All provide an API which
> provides membership notification/querying, 1->1 and 1->all messaging
> functionality. These are the basic building blocks of clustering and
> they will be required in every clustered service that is built for
> Geronimo. This is a natural candidate for encapsulation and sharing.
> Failing to do this will result in each different service having to build
> its own concepts about clustering from the ground up, which would be a
> disaster.
Agreed.
Things in the Java world have changed greatly since the introduction
of JGroups, JCluster, ActiveCluster, Tribes et al. Nowadays there is
no reason why we can't have a really simple POJO based model to
represent Nodes in a cluster with listeners to be notified when nodes
come and go. (Its really the main point of ActiveCluster - but we
could maybe refactor that API to be just a POJO model of a cluster
with no dependencies on external APIs or technologies and with the
ability maybe to cast a Node to some service interface to communicate
with the nodes).
Then using things like Spring Remoting we can add the remoting
technology as a deployment issue (rather than having lots of different
middleware specific APIs). e.g. see how Lingo allows you to invisibly
add JMS remoting to any POJO. (http://lingo.codehaus.org/)
--
James
-------
http://radio.weblogs.com/0112098/
Re: [wadi-dev] Session/clustering API and the web tier
Posted by Dain Sundstrom <da...@iq80.com>.
This reflects my sentiments as well.
-dain
On Jul 12, 2006, at 6:16 AM, James Strachan wrote:
> On 7/12/06, Jules Gosnell <ju...@coredevelopers.net> wrote:
>> Greg Wilkins wrote:
>> > All,
>> >
>> >
>> > Here are my comments on the Session API that were promised after
>> apachecon dublin.
>> > This is also CC'd to the wadi list and some of the points are
>> relevant to them
>> > as well.
>> >
>> > My own reason for focusing on the Session API when I think about
>> clustering,
>> > is that I like the idea of pluggable clustering
>> implementations. Clustering
>> > is not one size fits all and solutions can go from non-
>> replicated nodes configured
>> > in a flat file to auto discovered, self healing, redundant
>> hierarchies.
>
> Agreed. We should be focussed purely on what is the contract between a
> container and the session API and making that contract as simple and
> abstract as is possible while minimising leaky abstractions.
>
>
>> > I think the previous discussions we had on this were making good
>> progress, but
>> > I think we ran out of steam before the API was improved etc. So
>> I think it
>> > worthwhile to re-read the original threads.
>> >
>> > But I will repeat my main unresolved concerns here:
>> > While I appreciate the keep-it-simple-stupid approach adopted by
>> the proposed
>> > session API, I remain concerned that it may over simplify and
>> may also mix concerns.
>> >
>> > However, I do think that the API is pitched at about the right
>> level - namely
>> > that it is below the specific concerns of things such as HTTP.
>> As the implementor
>> > of the web container, I would prefer to not delegate HttpSession
>> management
>> > or request proxying to a pluggable session implementation (I
>> doubt that
>> > a cluster impl wants to deal with non-blocking proxying of
>> requests etc.)
>>
>> I think that our discussions about this have suffered from an
>> ambiguity
>> around the word 'delegate'...
>>
>> In one sense of the word, given WADI's current implementation, Jetty
>> does delegate Session management and HTTP handling to WADI, in
>> that WADI
>> passes the WebApp/Jetty an object on which it calls a method and the
>> work in question is done.
>>
>> However, in another sense, Jetty need not delegate this task,
>> since the
>> object returned in these cases is managed by WADI, but created by a
>> Factory that is injected at startup time. This factory might be
>> generating instances of a class that has very Jetty-specific
>> knowledge
>> or is even a part of the Jetty distro...
>
> Thats certainly one approach. Another is for the container to just ask
> the policy API what to do (i.e. is the request going to be serviced
> locally or not) so that the container can take care of the rest.
>
> I understand the cleanliness from the session API implementor's
> perspective of using a factory and calling back the container when you
> see fit - however I also understand the container developers
> requirement to understand at all times what each thread is doing, to
> tune things aggressively with full knowledge of threading models and
> to generally be master of its own domain, so I can understand why a
> container developer might prefer a non-callback related solution
> (which could introduce all kinds of nasty thread related bugs into the
> container). I don't see why both options can't be offered.
>
>
>> I would wholeheartedly agree that the code for Http request
>> relocation
>> should be written by someone with expertise in that area - namely the
>> container writer. I would just rather see it injected into the
>> clustered
>> manager, so that it can be called when required, without having to
>> burden Jetty with the added task of making this decision itself.
>
> I don't see that as mutually exclusive. Just have a way for Jetty to
> ask the clustering solution if a request can be satisfied locally, if
> not Jetty does the proxy/redirect thing.
>
>
>> > I see that the webcontainer needs to interact with the cluster
>> implementation
>> > in 4 areas:
>> >
>> >
>> > 1) Policy
>> > ---------
>> >
>> > When a container receives a request, it needs to make a policy
>> decision along
>> > the lines of:
>> >
>> > 1) The request can be handled locally.
>> > 2) The request can be handled locally, but only after some
>> other actions
>> > (eg session is moved to local)
>> > 3) request cannot be handled locally, but can be redirected
>> to another node
>> > 4) request cannot be handled locally, but can be proxied to
>> another node.
>> >
>> > This kind of corresponds to the Locator and SessionLocation
>> APIs. However
>> > these APIs give the power to enact a policy decision, but give
>> no support to make
>> > a policy decision.
>> >
>> > To implement a policy, you might want to use: the size of the
>> cluster, the total
>> > number of sessions, the number of session on the local node,
>> the number of sessions
>> > collocated with a remote session, how many requests for the
>> session have recently
>> > arrived on what nodes, etc. etc.
>> >
>> > The API does not give me this information and I think it would be
>> > difficult to provide all that might be used. Potentially
>> > we could get by with a mechanism to store/access cluster wide
>> meta-data
>> > attributes?
>> >
>> > However, it is very unlikely that one policy will fit all, so
>> each consumer
>> > of this Location API will have to implement a pluggable policy
>> frame work of
>> > some sorts.
>> >
>> > But as the session API is already a pluggable framework, why
>> don't we
>> > just delegate the policy decision to the API. The web container
>> > should make the policy decision, but should call the session API to
>> > make the decision. Something like:
>> >
>> > SessionLocation executeAt =
>> locator.getSessionExecutionLocation(clientId);
>> > if (executeAt.isLocal())
>> > // handle request
>> > else
>> > // proxy or redirect to executeAt location.
>> >
>> > (Note the need for something like this has been discussed before
>> and
>> > generally agreed. I have seen the proposed
>> RemoteSessionStrategy, but I am not
>> > sure how you obtain a handle to one - nor do I think the policy
>> should
>> > decide between redirect and proxy - which is HTTP business).
>
> Agreed. Just some way to ask the Session API if a request can be
> processed locally might do the trick, then if not Jetty can do its
> proxy/redirect thing. The trickier thing is what to pass into the
> strategy to help it decide...
>
>
>> By exposing the 'policy' api to the container and putting it in
>> charge
>> of when it used, you are exposing clustering details to it.
>
> Also the container details may be required by this policy. e.g.
> details about the previous http requests received at the current node,
> their type and various metadata statitsics and so forth which only the
> container is aware of.
>
>
>> WADI's approach is to completely shield the container from having to
>> know anything about clustering, whilst maintaining contracts with the
>> container encapsulating various pieces of tier/domain-specific
>> functionality that may be injected into the clustered session
>> manager.
>
> The issue is though, how invisible can clustering ever be? Information
> from the container and from the clustering implementation will
> typically be required for the policy decision.
>
>
>> > 3) Life cycle
>> >
>> > Unfortunately the life and death of a session is not simple -
>> specially
>> > when cross context dispatch is considered. Session ID's may or
>> may not
>> > be reused, their uniqueness might need to be guarenteed and the
>> decision
>> > may depend on the existence of the same session ID in other
>> contexts.
>> >
>> > I think this can be modeled with a structured name space - so
>> perhaps
>> > this is not an issue anymore?
>> >
>> >
>> > 4) Configuration and Management
>> > It would generally be good to be able to know how many nodes are in
>> > the cluster (or to set what the nodes are). To be able to
>> monitor node
>> > status and give commands to gracefully or brutally shutdown a
>> node, move
>> > sessions etc.
>> >
>> > Clustering aware clients (JNDI stubs, EJB proxies or potentially
>> fancy
>> > Ajax web clients) might need to be passed a list of known nodes
>> - but it
>> > is not possible to obtain/set that from the API - thus every
>> impl will need
>> > to implement it's own cluster config/discover even if that
>> information is
>> > available in other implementations.
>> >
>> >
>>
>> This is the clustering API (in my mind) that was mooted in the
>> meeting.
>> A number of clustering substrates (JGroups, ActiveCluster, Tribes,
>> etc...) have homesteaded this area (WADI maintains an abstraction
>> layer
>> that can map on to any of these three). All provide an API which
>> provides membership notification/querying, 1->1 and 1->all messaging
>> functionality. These are the basic building blocks of clustering and
>> they will be required in every clustered service that is built for
>> Geronimo. This is a natural candidate for encapsulation and sharing.
>> Failing to do this will result in each different service having to
>> build
>> its own concepts about clustering from the ground up, which would
>> be a
>> disaster.
>
> Agreed.
>
> Things in the Java world have changed greatly since the introduction
> of JGroups, JCluster, ActiveCluster, Tribes et al. Nowadays there is
> no reason why we can't have a really simple POJO based model to
> represent Nodes in a cluster with listeners to be notified when nodes
> come and go. (Its really the main point of ActiveCluster - but we
> could maybe refactor that API to be just a POJO model of a cluster
> with no dependencies on external APIs or technologies and with the
> ability maybe to cast a Node to some service interface to communicate
> with the nodes).
>
> Then using things like Spring Remoting we can add the remoting
> technology as a deployment issue (rather than having lots of different
> middleware specific APIs). e.g. see how Lingo allows you to invisibly
> add JMS remoting to any POJO. (http://lingo.codehaus.org/)
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
Re: [wadi-dev] Session/clustering API and the web tier
Posted by Jules Gosnell <ju...@coredevelopers.net>.
James Strachan wrote:
> On 7/12/06, Jules Gosnell <ju...@coredevelopers.net> wrote:
>
>> Greg Wilkins wrote:
>> > All,
>> >
>> >
>> > Here are my comments on the Session API that were promised after
>> apachecon dublin.
>> > This is also CC'd to the wadi list and some of the points are
>> relevant to them
>> > as well.
>> >
>> > My own reason for focusing on the Session API when I think about
>> clustering,
>> > is that I like the idea of pluggable clustering implementations.
>> Clustering
>> > is not one size fits all and solutions can go from non-replicated
>> nodes configured
>> > in a flat file to auto discovered, self healing, redundant hierarchies.
>
>
> Agreed. We should be focussed purely on what is the contract between a
> container and the session API and making that contract as simple and
> abstract as is possible while minimising leaky abstractions.
>
>
>> > I think the previous discussions we had on this were making good
>> progress, but
>> > I think we ran out of steam before the API was improved etc. So I
>> think it
>> > worthwhile to re-read the original threads.
>> >
>> > But I will repeat my main unresolved concerns here:
>> > While I appreciate the keep-it-simple-stupid approach adopted by the
>> proposed
>> > session API, I remain concerned that it may over simplify and may
>> also mix concerns.
>> >
>> > However, I do think that the API is pitched at about the right level
>> - namely
>> > that it is below the specific concerns of things such as HTTP. As
>> the implementor
>> > of the web container, I would prefer to not delegate HttpSession
>> management
>> > or request proxying to a pluggable session implementation (I doubt that
>> > a cluster impl wants to deal with non-blocking proxying of requests
>> etc.)
>>
>> I think that our discussions about this have suffered from an ambiguity
>> around the word 'delegate'...
>>
>> In one sense of the word, given WADI's current implementation, Jetty
>> does delegate Session management and HTTP handling to WADI, in that WADI
>> passes the WebApp/Jetty an object on which it calls a method and the
>> work in question is done.
>>
>> However, in another sense, Jetty need not delegate this task, since the
>> object returned in these cases is managed by WADI, but created by a
>> Factory that is injected at startup time. This factory might be
>> generating instances of a class that has very Jetty-specific knowledge
>> or is even a part of the Jetty distro...
>
>
> Thats certainly one approach. Another is for the container to just ask
> the policy API what to do (i.e. is the request going to be serviced
> locally or not) so that the container can take care of the rest.
This leaks clustering concerns into the container's space.
>
> I understand the cleanliness from the session API implementor's
> perspective of using a factory and calling back the container when you
> see fit - however I also understand the container developers
> requirement to understand at all times what each thread is doing, to
> tune things aggressively with full knowledge of threading models and
> to generally be master of its own domain, so I can understand why a
> container developer might prefer a non-callback related solution
> (which could introduce all kinds of nasty thread related bugs into the
> container).
Any clustering solution will use threads underneath its API. If this is
a concern you should simply make explicit where they may be used.
> I don't see why both options can't be offered.
>
>
>> I would wholeheartedly agree that the code for Http request relocation
>> should be written by someone with expertise in that area - namely the
>> container writer. I would just rather see it injected into the clustered
>> manager, so that it can be called when required, without having to
>> burden Jetty with the added task of making this decision itself.
>
>
> I don't see that as mutually exclusive. Just have a way for Jetty to
> ask the clustering solution if a request can be satisfied locally, if
> not Jetty does the proxy/redirect thing.
>
>
>> > I see that the webcontainer needs to interact with the cluster
>> implementation
>> > in 4 areas:
>> >
>> >
>> > 1) Policy
>> > ---------
>> >
>> > When a container receives a request, it needs to make a policy
>> decision along
>> > the lines of:
>> >
>> > 1) The request can be handled locally.
>> > 2) The request can be handled locally, but only after some other
>> actions
>> > (eg session is moved to local)
>> > 3) request cannot be handled locally, but can be redirected to
>> another node
>> > 4) request cannot be handled locally, but can be proxied to
>> another node.
>> >
>> > This kind of corresponds to the Locator and SessionLocation APIs.
>> However
>> > these APIs give the power to enact a policy decision, but give no
>> support to make
>> > a policy decision.
>> >
>> > To implement a policy, you might want to use: the size of the
>> cluster, the total
>> > number of sessions, the number of session on the local node, the
>> number of sessions
>> > collocated with a remote session, how many requests for the session
>> have recently
>> > arrived on what nodes, etc. etc.
>> >
>> > The API does not give me this information and I think it would be
>> > difficult to provide all that might be used. Potentially
>> > we could get by with a mechanism to store/access cluster wide meta-data
>> > attributes?
>> >
>> > However, it is very unlikely that one policy will fit all, so each
>> consumer
>> > of this Location API will have to implement a pluggable policy frame
>> work of
>> > some sorts.
>> >
>> > But as the session API is already a pluggable framework, why don't we
>> > just delegate the policy decision to the API. The web container
>> > should make the policy decision, but should call the session API to
>> > make the decision. Something like:
>> >
>> > SessionLocation executeAt =
>> locator.getSessionExecutionLocation(clientId);
>> > if (executeAt.isLocal())
>> > // handle request
>> > else
>> > // proxy or redirect to executeAt location.
>> >
>> > (Note the need for something like this has been discussed before and
>> > generally agreed. I have seen the proposed RemoteSessionStrategy,
>> but I am not
>> > sure how you obtain a handle to one - nor do I think the policy should
>> > decide between redirect and proxy - which is HTTP business).
>
>
> Agreed. Just some way to ask the Session API if a request can be
> processed locally might do the trick, then if not Jetty can do its
> proxy/redirect thing. The trickier thing is what to pass into the
> strategy to help it decide...
>
>
by having Jetty make the decision:
- you leak clustering concerns into the web tier
- you have to duplicate similar code in every clustered tier
>> By exposing the 'policy' api to the container and putting it in charge
>> of when it used, you are exposing clustering details to it.
>
>
> Also the container details may be required by this policy. e.g.
> details about the previous http requests received at the current node,
> their type and various metadata statitsics and so forth which only the
> container is aware of.
>
>
sophisticated policies require access to both container and clustered
session manager details on which to make an informed decision.
there are two places that you can abstract -
a) an api over the necessary details in the container
b) an api over the necessary details in the clustered session manager
if you go with (a), then each session manager can provide a single
policy which will run on any container.
if you go with (b), then you each container will have to implement its
own policy code that will run on any session manager.
since the number of containers in the equation is always likely to
outnumber the number of clustered session manager implementations (a)
will allow for the most code reuse - WADI takes this approach, to get
something going with minimum code, whilst not shutting the door on the
plugging in of policies which use native APIs on both sides, thus
allowing maximum sophistication.
taking the (b) route, will allow different tiers to use different logic
to decide where to locate their session. This is a bad idea because :
1) tier owners are not clustering architects - once again we have the
leakage of concerns.
2) this opens us to the possibility of different tiers making
contradictory decisions and session-groups (e.g. a web and ejb session)
being ping-ponged back and forth within the cluster because (e.g. the
web and ejb) containers are using different logic to decide the best
place to keep their session.
Ultimately you could abstract on both sides of the coin - but I think
that you would over constrain the policy's input and don't see much
value in a policy that would port between different clustered session
managers as their implementations are likely to be very different.
>> WADI's approach is to completely shield the container from having to
>> know anything about clustering, whilst maintaining contracts with the
>> container encapsulating various pieces of tier/domain-specific
>> functionality that may be injected into the clustered session manager.
>
>
> The issue is though, how invisible can clustering ever be? Information
> from the container and from the clustering implementation will
> typically be required for the policy decision.
>
>
>> > 3) Life cycle
>> >
>> > Unfortunately the life and death of a session is not simple - specially
>> > when cross context dispatch is considered. Session ID's may or may not
>> > be reused, their uniqueness might need to be guarenteed and the
>> decision
>> > may depend on the existence of the same session ID in other contexts.
>> >
>> > I think this can be modeled with a structured name space - so perhaps
>> > this is not an issue anymore?
>> >
>> >
>> > 4) Configuration and Management
>> > It would generally be good to be able to know how many nodes are in
>> > the cluster (or to set what the nodes are). To be able to monitor node
>> > status and give commands to gracefully or brutally shutdown a node,
>> move
>> > sessions etc.
>> >
>> > Clustering aware clients (JNDI stubs, EJB proxies or potentially fancy
>> > Ajax web clients) might need to be passed a list of known nodes -
>> but it
>> > is not possible to obtain/set that from the API - thus every impl
>> will need
>> > to implement it's own cluster config/discover even if that
>> information is
>> > available in other implementations.
>> >
>> >
>>
>> This is the clustering API (in my mind) that was mooted in the meeting.
>> A number of clustering substrates (JGroups, ActiveCluster, Tribes,
>> etc...) have homesteaded this area (WADI maintains an abstraction layer
>> that can map on to any of these three). All provide an API which
>> provides membership notification/querying, 1->1 and 1->all messaging
>> functionality. These are the basic building blocks of clustering and
>> they will be required in every clustered service that is built for
>> Geronimo. This is a natural candidate for encapsulation and sharing.
>> Failing to do this will result in each different service having to build
>> its own concepts about clustering from the ground up, which would be a
>> disaster.
>
>
> Agreed.
>
> Things in the Java world have changed greatly since the introduction
> of JGroups, JCluster, ActiveCluster, Tribes et al. Nowadays there is
> no reason why we can't have a really simple POJO based model to
> represent Nodes in a cluster with listeners to be notified when nodes
> come and go. (Its really the main point of ActiveCluster - but we
> could maybe refactor that API to be just a POJO model of a cluster
> with no dependencies on external APIs or technologies and with the
> ability maybe to cast a Node to some service interface to communicate
> with the nodes).
>
> Then using things like Spring Remoting we can add the remoting
> technology as a deployment issue (rather than having lots of different
> middleware specific APIs). e.g. see how Lingo allows you to invisibly
> add JMS remoting to any POJO. (http://lingo.codehaus.org/)
>
This is an interesting thought. I'll let it soak in for a while.
Jules
--
"Open Source is a self-assembling organism. You dangle a piece of
string into a super-saturated solution and a whole operating-system
crystallises out around it."
/**********************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
*
* www.coredevelopers.net
*
* Open Source Training & Support.
**********************************/
Re: [wadi-dev] Session/clustering API and the web tier
Posted by Jules Gosnell <ju...@coredevelopers.net>.
Jules Gosnell wrote:
>
> I pressed 'send' without counting to 10 first on the last response. Sorry.
>
> The thread seems to be getting nowhere fast - probably my fault - let me
> try an alternate tack.
>
> here is how I would like to see it work.
>
>
> The Interception Contract :
>
> The container provides a generic interception point with which the
> geronimo session manager (GSM) MAY register an interceptor.
>
> The Policy Contract:
!!! This is the Relocation Contract
>
> IFF the container chooses to provide invocation/request relocation
> fnality it can encapsulate this in an api that knowns nothing about
> clusters and everything about relocation and provide this to the GSM layer.
>
> The Relocation Contract :
!!! and this is the Policy Contract
>
> IFF the container wishes to be involved in decisions around whether to
> relocate session or invocation (ultimately about where sessions should
> be located) it may do so, via another api (to be discussed when the need
> arises). This api will know nothing about clustering, but everything
> about the resource usage of a given web container.
>
> ...
>
> A request arrives at the boundary of the container which passes it to
> the GSM interceptor.
>
> GSM takes control, uses information garnered from its own internals and
> potentially from policy contract with the container to decide what to
> do. If it wishes to relocate the invocation/request it does so using the
> relocation contract and terminates the interception chain. If it wishes
> to run the invocation/request locally, it returns flow-control to the
> container, allowing the request to continue into it.
>
>
> That is it.
>
> The container writer need know nothing about custering or session
> location. He need not even provide any of the contracts ! - if he
> doesn't, the GSM just forgets about invocation relocation and reverts to
> shifting state around all the time.
>
>
> By giving the container any more control over this you will variously:
>
> - duplicate code in every container
> - allow different tiers to make conflicting decisions about relocation
> - leak clustering issues into container-space.
>
>
> I've obviously done a bad job of explaining myself so far. Is this
> clearer ?
>
>
> Jules
>
>
> Greg Wilkins wrote:
>
>> Jules Gosnell wrote:
>>
>>> James Strachan wrote:
>>>
>>>> On 7/12/06, Jules Gosnell
>>>>
>>>>> Greg Wilkins wrote:
>>
>>
>>
>>>>> However, in another sense, Jetty need not delegate this task, since
>>>>> the
>>>>> object returned in these cases is managed by WADI, but created by a
>>>>> Factory that is injected at startup time. This factory might be
>>>>> generating instances of a class that has very Jetty-specific knowledge
>>>>> or is even a part of the Jetty distro...
>>>>
>>>>
>>>>
>>>> Thats certainly one approach. Another is for the container to just ask
>>>> the policy API what to do (i.e. is the request going to be serviced
>>>> locally or not) so that the container can take care of the rest.
>>>
>>>
>>> This leaks clustering concerns into the container's space.
>>
>>
>>
>> If the container is going to do the redirecting/proxying (either
>> directly or as a call back), then it has to be concerned with clustering.
>>
>> I understand that WADI has an implementation of this, but I want to
>> be able to use clustering mechanisms other than WADI. I think it is
>> raising the bar for clustering implementations if they must support
>> this.
>>
>>
>>
>>>> I understand the cleanliness from the session API implementor's
>>>> perspective of using a factory and calling back the container when you
>>>> see fit - however I also understand the container developers
>>>> requirement to understand at all times what each thread is doing, to
>>>> tune things aggressively with full knowledge of threading models and
>>>> to generally be master of its own domain, so I can understand why a
>>>> container developer might prefer a non-callback related solution
>>>> (which could introduce all kinds of nasty thread related bugs into the
>>>> container).
>>>
>>>
>>> Any clustering solution will use threads underneath its API. If this is
>>> a concern you should simply make explicit where they may be used.
>>>
>>>
>>>> I don't see why both options can't be offered.
>>
>>
>>
>> Exactly - why can we not have the simple policy API that we call to
>> understand what to do.
>>
>> Jetty can have a session interceptor to call this API and do it's
>> proxying or redirection and then Jetty can use lots of different
>> clustering implementations.
>>
>> But the Jetty developers may be lazy, and instead of implementing their
>> own interceptor they may choose to use the WADI implementation of it
>> that wraps up the whole invocation. Tomcat or openEJB may not be so
>> lazy or have other concerns - thus they might not use the WADI
>> interceptor but code directly to the policy API.
>>
>>
>>
>>>> Agreed. Just some way to ask the Session API if a request can be
>>>> processed locally might do the trick, then if not Jetty can do its
>>>> proxy/redirect thing. The trickier thing is what to pass into the
>>>> strategy to help it decide...
>>>
>>>
>>> by having Jetty make the decision:
>>>
>>> - you leak clustering concerns into the web tier
>>> - you have to duplicate similar code in every clustered tier
>>
>>
>>
>> Jetty is not making the decision, it is calling an API
>> to make the decision. Jetty simply implements the execution of the
>> decision.
>>
>> The code to do this has to be written for every clustering tier.
>> Now it is great that WADI wants to write it for every clustering tier
>> - but that is a big work load and I am not sure that WADI developers
>> will be able to keep every tier happy and deal with
>> all of their concerns.
>>
>> Also, it kind of implies that G will only use WADI. Now I know
>> that wadi is itself extensible/pluggable/etc. so this might not
>> be a bad idea. But wadi either has be the G cluster API (and allow
>> other technologies to be plugged into it), or it has to give up
>> some of the space (eg implementation of interceptors) to the tier
>> implementations.
>>
>> That is a tough call to make at this stage because people just do not
>> have enough exposure to WADI.
>>
>> If it was at all possible to have an approach of a low level
>> API which can either be directly used or used via the WADI
>> interceptors, then that would give us a chance to evaluate
>> WADI. If the wadi interceptors are indeed the best, then
>> great - WADI because the cluster API for G.
>>
>> So the question is - is the session API good enough to form the
>> interface between WADI interceptors and WADI
>> clustering implementations. Originally the answer was no, but
>> there have been some improvements proposed to the API.
>> Is that still the case? Eg. with the getExecutionLocation style API,
>> their is no need for the inefficiencies of getSessionLocation calls.
>>
>> Alternately (if wadi understandably does not want to be cleft in two
>> by the G session API), if somebody writes a clustering
>> impl behind the G session API (eg the distributed map currently
>> being worked on), could that be plugged into Wadi as a pluggable
>> store implementation?
>>
>> If so, Jetty could just use Wadi and we would get G integration
>> for free!
>>
>>
>> cheers
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list please visit:
>>
>> http://xircles.codehaus.org/manage_email
>>
>
>
--
"Open Source is a self-assembling organism. You dangle a piece of
string into a super-saturated solution and a whole operating-system
crystallises out around it."
/**********************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
*
* www.coredevelopers.net
*
* Open Source Training & Support.
**********************************/
Re: [wadi-dev] Session/clustering API and the web tier
Posted by Jules Gosnell <ju...@coredevelopers.net>.
I pressed 'send' without counting to 10 first on the last response. Sorry.
The thread seems to be getting nowhere fast - probably my fault - let me
try an alternate tack.
here is how I would like to see it work.
The Interception Contract :
The container provides a generic interception point with which the
geronimo session manager (GSM) MAY register an interceptor.
The Policy Contract:
IFF the container chooses to provide invocation/request relocation
fnality it can encapsulate this in an api that knowns nothing about
clusters and everything about relocation and provide this to the GSM layer.
The Relocation Contract :
IFF the container wishes to be involved in decisions around whether to
relocate session or invocation (ultimately about where sessions should
be located) it may do so, via another api (to be discussed when the need
arises). This api will know nothing about clustering, but everything
about the resource usage of a given web container.
...
A request arrives at the boundary of the container which passes it to
the GSM interceptor.
GSM takes control, uses information garnered from its own internals and
potentially from policy contract with the container to decide what to
do. If it wishes to relocate the invocation/request it does so using the
relocation contract and terminates the interception chain. If it wishes
to run the invocation/request locally, it returns flow-control to the
container, allowing the request to continue into it.
That is it.
The container writer need know nothing about custering or session
location. He need not even provide any of the contracts ! - if he
doesn't, the GSM just forgets about invocation relocation and reverts to
shifting state around all the time.
By giving the container any more control over this you will variously:
- duplicate code in every container
- allow different tiers to make conflicting decisions about relocation
- leak clustering issues into container-space.
I've obviously done a bad job of explaining myself so far. Is this clearer ?
Jules
Greg Wilkins wrote:
> Jules Gosnell wrote:
>
>>James Strachan wrote:
>>
>>>On 7/12/06, Jules Gosnell
>>>
>>>>Greg Wilkins wrote:
>
>
>>>>However, in another sense, Jetty need not delegate this task, since the
>>>>object returned in these cases is managed by WADI, but created by a
>>>>Factory that is injected at startup time. This factory might be
>>>>generating instances of a class that has very Jetty-specific knowledge
>>>>or is even a part of the Jetty distro...
>>>
>>>
>>>Thats certainly one approach. Another is for the container to just ask
>>>the policy API what to do (i.e. is the request going to be serviced
>>>locally or not) so that the container can take care of the rest.
>>
>>This leaks clustering concerns into the container's space.
>
>
> If the container is going to do the redirecting/proxying (either
> directly or as a call back), then it has to be concerned with clustering.
>
> I understand that WADI has an implementation of this, but I want to
> be able to use clustering mechanisms other than WADI. I think it is
> raising the bar for clustering implementations if they must support
> this.
>
>
>
>>>I understand the cleanliness from the session API implementor's
>>>perspective of using a factory and calling back the container when you
>>>see fit - however I also understand the container developers
>>>requirement to understand at all times what each thread is doing, to
>>>tune things aggressively with full knowledge of threading models and
>>>to generally be master of its own domain, so I can understand why a
>>>container developer might prefer a non-callback related solution
>>>(which could introduce all kinds of nasty thread related bugs into the
>>>container).
>>
>>Any clustering solution will use threads underneath its API. If this is
>>a concern you should simply make explicit where they may be used.
>>
>>
>>>I don't see why both options can't be offered.
>
>
> Exactly - why can we not have the simple policy API that we call to
> understand what to do.
>
> Jetty can have a session interceptor to call this API and do it's
> proxying or redirection and then Jetty can use lots of different
> clustering implementations.
>
> But the Jetty developers may be lazy, and instead of implementing their
> own interceptor they may choose to use the WADI implementation of it
> that wraps up the whole invocation. Tomcat or openEJB may not
> be so lazy or have other concerns - thus they might not use the WADI
> interceptor but code directly to the policy API.
>
>
>
>>>Agreed. Just some way to ask the Session API if a request can be
>>>processed locally might do the trick, then if not Jetty can do its
>>>proxy/redirect thing. The trickier thing is what to pass into the
>>>strategy to help it decide...
>>
>>by having Jetty make the decision:
>>
>>- you leak clustering concerns into the web tier
>>- you have to duplicate similar code in every clustered tier
>
>
> Jetty is not making the decision, it is calling an API
> to make the decision. Jetty simply implements the execution of
> the decision.
>
> The code to do this has to be written for every clustering tier.
> Now it is great that WADI wants to write it for every clustering
> tier - but that is a big work load and I am not sure that WADI
> developers will be able to keep every tier happy and deal with
> all of their concerns.
>
> Also, it kind of implies that G will only use WADI. Now I know
> that wadi is itself extensible/pluggable/etc. so this might not
> be a bad idea. But wadi either has be the G cluster API (and
> allow other technologies to be plugged into it), or it has to give up
> some of the space (eg implementation of interceptors) to the
> tier implementations.
>
> That is a tough call to make at this stage because people just
> do not have enough exposure to WADI.
>
> If it was at all possible to have an approach of a low level
> API which can either be directly used or used via the WADI
> interceptors, then that would give us a chance to evaluate
> WADI. If the wadi interceptors are indeed the best, then
> great - WADI because the cluster API for G.
>
> So the question is - is the session API good enough to
> form the interface between WADI interceptors and WADI
> clustering implementations. Originally the answer was no, but
> there have been some improvements proposed to the API.
> Is that still the case? Eg. with the getExecutionLocation
> style API, their is no need for the inefficiencies of
> getSessionLocation calls.
>
> Alternately (if wadi understandably does not want to be cleft
> in two by the G session API), if somebody writes a clustering
> impl behind the G session API (eg the distributed map currently
> being worked on), could that be plugged into Wadi as a
> pluggable store implementation?
>
> If so, Jetty could just use Wadi and we would get G integration
> for free!
>
>
> cheers
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
> http://xircles.codehaus.org/manage_email
>
--
"Open Source is a self-assembling organism. You dangle a piece of
string into a super-saturated solution and a whole operating-system
crystallises out around it."
/**********************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
*
* www.coredevelopers.net
*
* Open Source Training & Support.
**********************************/
Re: [wadi-dev] Session/clustering API and the web tier
Posted by Jules Gosnell <ju...@coredevelopers.net>.
Greg Wilkins wrote:
> Jules Gosnell wrote:
>
>>James Strachan wrote:
>>
>>>On 7/12/06, Jules Gosnell
>>>
>>>>Greg Wilkins wrote:
>
>
>>>>However, in another sense, Jetty need not delegate this task, since the
>>>>object returned in these cases is managed by WADI, but created by a
>>>>Factory that is injected at startup time. This factory might be
>>>>generating instances of a class that has very Jetty-specific knowledge
>>>>or is even a part of the Jetty distro...
>>>
>>>
>>>Thats certainly one approach. Another is for the container to just ask
>>>the policy API what to do (i.e. is the request going to be serviced
>>>locally or not) so that the container can take care of the rest.
>>
>>This leaks clustering concerns into the container's space.
>
>
> If the container is going to do the redirecting/proxying (either
> directly or as a call back), then it has to be concerned with clustering.
No - the redirection/proxying is not concerned with clustering - we have
divided and conquered the idea down to a much smaller concern, that of
relocating a request to another container (endpoint provided by
clustering layer):
boolean isRelocatable();
void relocate(EndPoint endPoint); // endpoint is an ip:port
The implementation of this concern is tier-specific and should be
implemented by the tier owner.
>
> I understand that WADI has an implementation of this, but I want to
> be able to use clustering mechanisms other than WADI. I think it is
> raising the bar for clustering implementations if they must support
> this.
I would far rather WADI didn't have to implement this and that it was
implemented by Jetty as described. We are getting muddled between what
WADI currently provides and where I would like to see WADI and the
g-session-api go.
If the Jetty6 continuations fn-ality complicates the situation, then
this can be addressed in the API above.
>
>
>
>>>I understand the cleanliness from the session API implementor's
>>>perspective of using a factory and calling back the container when you
>>>see fit - however I also understand the container developers
>>>requirement to understand at all times what each thread is doing, to
>>>tune things aggressively with full knowledge of threading models and
>>>to generally be master of its own domain, so I can understand why a
>>>container developer might prefer a non-callback related solution
>>>(which could introduce all kinds of nasty thread related bugs into the
>>>container).
>>
>>Any clustering solution will use threads underneath its API. If this is
>>a concern you should simply make explicit where they may be used.
>>
>>
>>>I don't see why both options can't be offered.
>
>
> Exactly - why can we not have the simple policy API that we call to
> understand what to do.
>
> Jetty can have a session interceptor to call this API and do it's
> proxying or redirection and then Jetty can use lots of different
> clustering implementations.
>
> But the Jetty developers may be lazy, and instead of implementing their
> own interceptor they may choose to use the WADI implementation of it
> that wraps up the whole invocation. Tomcat or openEJB may not
> be so lazy or have other concerns - thus they might not use the WADI
> interceptor but code directly to the policy API.
>
>
>
>>>Agreed. Just some way to ask the Session API if a request can be
>>>processed locally might do the trick, then if not Jetty can do its
>>>proxy/redirect thing. The trickier thing is what to pass into the
>>>strategy to help it decide...
>>
>>by having Jetty make the decision:
>>
>>- you leak clustering concerns into the web tier
>>- you have to duplicate similar code in every clustered tier
>
>
> Jetty is not making the decision, it is calling an API
> to make the decision.
This means that it is aware that it is clustered.
The code to call this API and react accordingly will be duplicated in
every tier.
Jetty simply implements the execution of
> the decision.
I agree that the tier-ower should provide an implementation of this
piece of fn-ality. This will not need access to any clustering detail.
Simply the ip:port of the peer to which you are relocating the
invocation/request.
>
> The code to do this has to be written for every clustering tier.
> Now it is great that WADI wants to write it for every clustering
> tier
No - I don't - I want the clustered session manager to call the policy,
and, based on the result of that call, call some tier-specific behaviour
- see above - very good reasons for this were given in my last mail.
I don't want the container calling the policy, because then you need to
know that you are clustered - see those reasons.
- but that is a big work load and I am not sure that WADI
> developers will be able to keep every tier happy and deal with
> all of their concerns.
N/A
>
> Also, it kind of implies that G will only use WADI.
wrong.
We are talking about a common API here and I am drawing on WADI for
examples of how I believe it should be done. No specific implementation
is being suggested.
Now I know
> that wadi is itself extensible/pluggable/etc. so this might not
> be a bad idea. But wadi either has be the G cluster API (and
> allow other technologies to be plugged into it), or it has to give up
> some of the space (eg implementation of interceptors) to the
> tier implementations.
wrong.
and I'm not sure whether you are talking about the session or clustering
api here.
Both session and clustering apis are completely in the domain of
Geronimo. If either or both apis are pitched flexibly enough for WADI to
use them, it will implement them. If implementing them involves
significant sacrifice of fn-ality, simplicity or performance, then it won't.
As for giving up space, I have no problem with it, provided that there
is no leakage of concerns as a result of transfer of ownership of said
space.
>
> That is a tough call to make at this stage because people just
> do not have enough exposure to WADI.
N/A - I'm not making that call - just giving the benefit of my
experience to a design discussion.
>
> If it was at all possible to have an approach of a low level
> API which can either be directly used or used via the WADI
> interceptors, then that would give us a chance to evaluate
> WADI. If the wadi interceptors are indeed the best, then
> great - WADI because the cluster API for G.
I'm not asking you to evaluate WADI. I am explaining what I see as
mistakes in the existing session API, in the hope that change may be made.
Anyone who is interested in WADI is welcome to join our lists and ask
questions that I will do my best to answer.
>
> So the question is - is the session API good enough to
> form the interface between WADI interceptors and WADI
> clustering implementations. Originally the answer was no, but
> there have been some improvements proposed to the API.
> Is that still the case? Eg. with the getExecutionLocation
> style API, their is no need for the inefficiencies of
> getSessionLocation calls.
I still believe not.
My sticking points are as follows (OTTOMH):
1) no exposure of any clustering detail should be made to the client
container - the existing API will do this, even if it just requires you
to call a boolean method, test the result and relocate an invocation if
it it true - reasons given in last mail.
2) no detail of any colocation policy should be exposed to the client
container - the existing API does this, by exposing implementational
structure (a map of web session attributes mixed with sessions from
other tiers) - reasons given in last mail.
3) I would like to see the api provide a cross-tier abstraction with
which the invocation/request can be encapsulated and manipulated, and
encourage the provision of optimal, tier-specific implementations of
invocation relocation fn-ality encapsulated by this interface.
4) I haven't studied the current API closely enough to ascertain how
apportions responsibility for a number of important issues - session
lifecycle, notifications, eviction, destruction, ttl, etc... - and I
don't think that they have been discussed in depth yet - so there may be
issues here as well.
I concur that some further code (e.g. policy and session impl) will need
access to expertise from both container and clustering tiers. This
knowledge should be confined entirely to these code sections.
>
> Alternately (if wadi understandably does not want to be cleft
> in two by the G session API), if somebody writes a clustering
> impl behind the G session API (eg the distributed map currently
> being worked on), could that be plugged into Wadi as a
> pluggable store implementation?
Yes
>
> If so, Jetty could just use Wadi and we would get G integration
> for free!
>
Yes - if someone wrote a WADI/Geronimo adapter. I haven't given it any
thought, but I do't think that this would pose a problem.
Jules
>
> cheers
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
> http://xircles.codehaus.org/manage_email
>
--
"Open Source is a self-assembling organism. You dangle a piece of
string into a super-saturated solution and a whole operating-system
crystallises out around it."
/**********************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
*
* www.coredevelopers.net
*
* Open Source Training & Support.
**********************************/
Re: [wadi-dev] Session/clustering API and the web tier
Posted by Jules Gosnell <ju...@coredevelopers.net>.
Greg Wilkins wrote:
> Jules Gosnell wrote:
>
>>I pressed 'send' without counting to 10 first on the last response. Sorry.
>
>
> No probs - no offence taken.
>
>
>
>>I've obviously done a bad job of explaining myself so far. Is this
>>clearer ?
>
>
> Actually no - I preferred the slightly pissed off version for clarity :-)
>
>
> I actually think we are in agreement.. but don´t realize it.
>
> You say that the getExcecutionLocation() style API I suggest leaks
> clustering concerns into the webtier, but I am not suggesting that general
> container code calls this API.
>
> I am suggesting a that a cluster aware interceptor calls it.
> This is the GSM intercpetion contract of your last email.
>
> But given that interceptors are going to be very tier implementation
> specific and that Jetty has to write geronimo specific interceptors for
> security, transactions, .... then it appears natural to me that the
> G-jetty module is the natural place for that interceptor to live and
> we don't need to standardize it's contract.
>
>
> Or is it the existence of the standardized API between interceptor
> and Policy that you object to? Are your concerns such that
> you do not think this API can be implemented efficiently?
>
>
> Give me 30minutes and I will post my latest mental picture of
> how I would like to see the Cluster/Session API.
>
> cheers
>
>
>
>
I see the GSM interceptor as a 'peninsula' of the GSM impl, because it
deals with clustering concerns. The point at which it meets Jetty is at
the generic interceptor API... - I guess this is one of the bones of
contention.
Seen this way, the same interceptor code can be shared across all
containers (via different interceptor adaptors), rather than duplicated
by each container.... and less of the GSM's API needs to be exposed to
the container...
Jules
--
"Open Source is a self-assembling organism. You dangle a piece of
string into a super-saturated solution and a whole operating-system
crystallises out around it."
/**********************************
* Jules Gosnell
* Partner
* Core Developers Network (Europe)
*
* www.coredevelopers.net
*
* Open Source Training & Support.
**********************************/
Re: Cluster API proposal?
Posted by Greg Wilkins <gr...@mortbay.com>.
Dain Sundstrom wrote:
> I think we are starting to agree.
agreed :-)
I think that a Cluster interface with the stateless methods as proposed by filip would be
good, plus a SessionManager interface with the stateful stuff.
Actually, I don't like SessionManager as a name... maybe StateManager?
If the same bean implements both APIs, great! But I see no need for
a common root.
I am still keen on a context ID, but I'll advocate that in a separate email.
cheers
Re: Cluster API proposal?
Posted by Dain Sundstrom <da...@iq80.com>.
I think we are starting to agree. I see the session manager as a
high level component that *can* be implemented on top of a cluster or
just be a local hash map. The idea was to create an abstraction such
that, the people writing clustering code can easily plugin into
geronimo services that have session data (which is about the only
pice of cluster data that we have right now).
As for low level Clustering apis, I don't have much of a preference
unless I have to write to them, and the stuff I'm working on can be
completely satisfied by the session api.
-dain
On Jul 14, 2006, at 6:41 AM, Filip Hanik - Dev Lists wrote:
> Cool, thanks for posting this.
> While I do believe everything in this API is very useful, I see it
> as an extension to the one I created.
> My API is only about the cluster, and its meta data, while the API
> below is very session oriented.
>
> In a cluster without state replication, most of the methods below
> would return null or throw UnsupportedOperationException, hence it
> would make it harder to implement, and less useful.
> The API below is essentially meta data about session state. I
> personally consider this an extension to the Cluster API, or a
> higher level component, and bada bim, we are back at SessionAPI :),
> our favorite topics.
>
> Does this make sense? I was looking for the lowest common
> denominator, for what a "cluster" is, essentially, nothing but a
> group of VMs.
> So what I try to do, is that the group wont be forced to expose
> session state, cause if there is no state replication, you couldn't
> implement that API.
>
> Because I haven't thought much of the session API, as I do consider
> that a higher level component, I haven't yet though of a good way,
> if there is one, how that would sit on top of a cluster API. But I
> do believe they shouldn't be morphed together, instead of the
> SessionAPI must know about nodes and clusters, it would get that
> from the cluster api, <ot>although i personally believe the session
> api should only know about sessions and nothing else, but that is
> why I am staying out of that topic :)</ot>
>
> Filip
>
>
> Greg Wilkins wrote:
>> This is my idea of how we could morph the currently proposed
>> session APIs
>> into a cluster API
>> I have created a spot for Cluster meta data - but I have not
>> filled it out much.
>>
>> The key difference is that the state Map is now indexed by session
>> ID and context ID.
>> This allows the state for different contexts within the same
>> session to be on different
>> nodes (this is a real requirement) and also means that locking is
>> at context rather
>> than meta session level. Note that some implementations may not
>> fully support this and may just do sessionId+contextId behind the
>> scenes and colocate all context
>> states for the same session (and move them as one).
>> I have also added an async aspect to the API for potentially long
>> operations
>> such as moving state about the place - again this can be
>> optionally supported.
>>
>> Also I support the idea of multiple Nodes per server (really
>> useful for testing
>> and heterogeneous clusters).
>>
>>
>>
>> // The top level Cluster API - this was the Locator... but let's
>> call a spade a spade.
>>
>> interface Cluster
>> {
>> // methods to get/set meta data about the cluster
>> // these signatures here are just a guess... but you get the
>> idea.
>> int getMaxNodes();
>> Set<Node> getKnownNodes();
>> void setKnownNodes(Set<Node> nodes);
>> Node getLocalNode();
>>
>> // Access sessions in cluster. MetaSession
>> getMetaSession(String clientID);
>> Session createMetaSession(String sessionId);
>>
>> }
>>
>>
>> // Node API
>> // was Server - but may have multiple Nodes per server
>> interface Node
>> {
>> String getName(); String[] getAddresses(String
>> protocol); void setAddresses(String string, String[]
>> strings); boolean isLocalServer();
>> boolean isActive();
>>
>> int getPort(String protocol); // one way to handle the multi
>> nodes per server
>> int getPortOffset(); // or this one (add to standard
>> port)
>> }
>>
>> // Meta Session - was SessionLocation
>> interface MetaSession
>> {
>> String getSessionId();
>> void invalidate();
>>
>> void addEventListener(MetaSessionListener listener);
>> void removeEventListener(MetaSessionListener listener);
>>
>> // State API has map per context ID , where a context
>> // ID might be "web:/context" or "ejb:" or random
>> boolean isStateLocal(String contextId);
>>
>>
>> Map getState(String contextId); // implies a move local!
>> void getStateAsync(Object key, String contextId); // async
>> version
>> Map createState(String contextId);
>> void releaseState(String contextId); // don't lock whole meta
>> session!
>> void invalidate(String contextId);
>>
>> // Locaton / Policy API.
>> Node getNode(String contextId); Node getExecutionNode
>> (String contextId); void getExecutionNodeAsync(Object key,
>> String contextId);
>>
>> // Don't know if these are too HTTP specific... but we need
>> them void setPassivationTimeout(long ms, String contextId);
>> void setInvalidationTimeout(long ms, String contextId);
>> }
>>
>>
>> interface MetaSessionListener
>> {
>> // callbacks to allow session manager to inspect contents
>> for // tier specific handling (eg servlet listeners etc.)
>> void activateState(String sessionId, String contextId, Map
>> state);
>> void passivateState(String sessionId, String contextId, Map
>> state);
>> void invalidateState(String sessionId, String contextId, Map
>> state);
>>
>> // callbacks for async operations
>> void gotState(Object key, String sessionId, String contextId,
>> Map state);
>> void executionNode(Object key, String sessionId, String
>> contextId, Node location);
>>
>> }
>>
>>
>>
Re: Cluster API proposal?
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
Cool, thanks for posting this.
While I do believe everything in this API is very useful, I see it as an
extension to the one I created.
My API is only about the cluster, and its meta data, while the API below
is very session oriented.
In a cluster without state replication, most of the methods below would
return null or throw UnsupportedOperationException, hence it would make
it harder to implement, and less useful.
The API below is essentially meta data about session state. I personally
consider this an extension to the Cluster API, or a higher level
component, and bada bim, we are back at SessionAPI :), our favorite topics.
Does this make sense? I was looking for the lowest common denominator,
for what a "cluster" is, essentially, nothing but a group of VMs.
So what I try to do, is that the group wont be forced to expose session
state, cause if there is no state replication, you couldn't implement
that API.
Because I haven't thought much of the session API, as I do consider that
a higher level component, I haven't yet though of a good way, if there
is one, how that would sit on top of a cluster API. But I do believe
they shouldn't be morphed together, instead of the SessionAPI must know
about nodes and clusters, it would get that from the cluster api,
<ot>although i personally believe the session api should only know about
sessions and nothing else, but that is why I am staying out of that
topic :)</ot>
Filip
Greg Wilkins wrote:
> This is my idea of how we could morph the currently proposed session APIs
> into a cluster API
>
> I have created a spot for Cluster meta data - but I have not filled it out much.
>
> The key difference is that the state Map is now indexed by session ID and context ID.
> This allows the state for different contexts within the same session to be on different
> nodes (this is a real requirement) and also means that locking is at context rather
> than meta session level. Note that some implementations may not fully support
> this and may just do sessionId+contextId behind the scenes and colocate all context
> states for the same session (and move them as one).
> I have also added an async aspect to the API for potentially long operations
> such as moving state about the place - again this can be optionally supported.
>
> Also I support the idea of multiple Nodes per server (really useful for testing
> and heterogeneous clusters).
>
>
>
> // The top level Cluster API - this was the Locator... but let's call a spade a spade.
>
> interface Cluster
> {
> // methods to get/set meta data about the cluster
> // these signatures here are just a guess... but you get the idea.
> int getMaxNodes();
> Set<Node> getKnownNodes();
> void setKnownNodes(Set<Node> nodes);
> Node getLocalNode();
>
> // Access sessions in cluster.
> MetaSession getMetaSession(String clientID);
> Session createMetaSession(String sessionId);
>
> }
>
>
> // Node API
> // was Server - but may have multiple Nodes per server
> interface Node
> {
> String getName();
> String[] getAddresses(String protocol);
> void setAddresses(String string, String[] strings);
> boolean isLocalServer();
> boolean isActive();
>
> int getPort(String protocol); // one way to handle the multi nodes per server
> int getPortOffset(); // or this one (add to standard port)
> }
>
> // Meta Session - was SessionLocation
> interface MetaSession
> {
> String getSessionId();
> void invalidate();
>
> void addEventListener(MetaSessionListener listener);
> void removeEventListener(MetaSessionListener listener);
>
> // State API has map per context ID , where a context
> // ID might be "web:/context" or "ejb:" or random
> boolean isStateLocal(String contextId);
>
>
> Map getState(String contextId); // implies a move local!
> void getStateAsync(Object key, String contextId); // async version
>
> Map createState(String contextId);
> void releaseState(String contextId); // don't lock whole meta session!
> void invalidate(String contextId);
>
> // Locaton / Policy API.
> Node getNode(String contextId);
> Node getExecutionNode(String contextId);
> void getExecutionNodeAsync(Object key, String contextId);
>
>
> // Don't know if these are too HTTP specific... but we need them
> void setPassivationTimeout(long ms, String contextId);
> void setInvalidationTimeout(long ms, String contextId);
> }
>
>
> interface MetaSessionListener
> {
> // callbacks to allow session manager to inspect contents for
> // tier specific handling (eg servlet listeners etc.)
> void activateState(String sessionId, String contextId, Map state);
> void passivateState(String sessionId, String contextId, Map state);
> void invalidateState(String sessionId, String contextId, Map state);
>
> // callbacks for async operations
> void gotState(Object key, String sessionId, String contextId, Map state);
> void executionNode(Object key, String sessionId, String contextId, Node location);
>
> }
>
>
>
Re: Cluster API proposal?
Posted by Greg Wilkins <gr...@mortbay.com>.
Dain Sundstrom wrote:
> On Jul 14, 2006, at 1:47 AM, Greg Wilkins wrote:
>
>> This is my idea of how we could morph the currently proposed session APIs
>> into a cluster API.
>>
>> I have created a spot for Cluster meta data - but I have not filled it
>> out much.
>>
>> The key difference is that the state Map is now indexed by session ID
>> and context ID.
>> This allows the state for different contexts within the same session
>> to be on different
>> nodes (this is a real requirement)
>
> I think I missed some of the discussion. Why is this a requirement? I
> know some deployments like to split applications into multiple tiers,
> but why would the servlet or ejb containers need to know about that?
I can see many reasons for having a session be a map of maps rather
than just a flat map:
The actual thing we are modelling is a map of maps. We know that
beneath a session ID there can be multiple non-intersecting state
spaces for EJBs, webcontexts, POJOs, whatever. If we know this
is the model why not model it in out data structure?
If the underlying implementation is just a single flat space, then
it is trivial to store a map of maps into a single map by adding name
prefixes.
But if the underlying implementation is capable of modelling a map
of maps, then we avoid the inefficiencies of adding prefixes for
every lookup, excessive iteration, coarse locking etc.
Conversely, if our API is a flat space - all the state for
all contexts must be collocated and a request to one context will
lock the entire state. Note that with mashups and portlets, you can
expect that multiple contexts for the same session could be accessed
in parallel.
Heterogeneous clusters are needed. As well as clusters that are
partitioned by tier (which if often a bad idea), we need to support
clusters partitioned by context (some web contexts to a subset of
nodes and others to another subset). This includes some single sign-on
solutions which are actually multiple clusters sharing a session ID space.
If a session state space is a single flat space, then it will be very
difficult (if not impossible) to support heterogeneous clusters as
the entire state for a session ID will need to be co-located.
It may also be that individual context will need different
locking strategies and their individual context maps can be
wrapped accordingly. For example, some contexts may release
their state map when a request has completed, while others
may release after every setAttribute call (because they may be
part of a long held Ajax/Comet request!)
So I see a map of maps gives us much better targeted semantics
with little extra complexity.
cheers
Re: Cluster API proposal?
Posted by Dain Sundstrom <da...@iq80.com>.
On Jul 14, 2006, at 1:47 AM, Greg Wilkins wrote:
> This is my idea of how we could morph the currently proposed
> session APIs
> into a cluster API.
>
> I have created a spot for Cluster meta data - but I have not filled
> it out much.
>
> The key difference is that the state Map is now indexed by session
> ID and context ID.
> This allows the state for different contexts within the same
> session to be on different
> nodes (this is a real requirement)
I think I missed some of the discussion. Why is this a requirement?
I know some deployments like to split applications into multiple
tiers, but why would the servlet or ejb containers need to know about
that?
> and also means that locking is at context rather
> than meta session level. Note that some implementations may not
> fully support
> this and may just do sessionId+contextId behind the scenes and
> colocate all context
> states for the same session (and move them as one).
If we assume we are working in an IoC environment, I don't see why
the api would need to explicitly call out support for a context.
Lets say Jetty has a setter for SessionManager, if you want to have a
private context for your jetty data, you can simply have the IoC
layer inject a wrapped version of SessionManager that does the string
append, or if your clustering framework supports the concept of a
context directly, the IoC layer could inject
fooCluster.getPrivateSessionContext(contextId).
This is what I like about the session API. It only contains the apis
needed for someone to get their session data. All complex deployment
decisions or configuration can be pushed to the IoC layer.
I have specific comments about the apis below.
-dain
> // The top level Cluster API - this was the Locator... but let's
> call a spade a spade.
>
> interface Cluster
> {
> // methods to get/set meta data about the cluster
> // these signatures here are just a guess... but you get the
> idea.
> int getMaxNodes();
> Set<Node> getKnownNodes();
> void setKnownNodes(Set<Node> nodes);
> Node getLocalNode();
>
> // Access sessions in cluster.
> MetaSession getMetaSession(String clientID);
> Session createMetaSession(String sessionId);
> }
I'd prefer we don't call it cluster as it could then concievely
contain anything. The job of this interface is to manage session and
I think we should call it a SessionManager.
> // Node API
> // was Server - but may have multiple Nodes per server
> interface Node
> {
> String getName();
> String[] getAddresses(String protocol);
> void setAddresses(String string, String[] strings);
> boolean isLocalServer();
> boolean isActive();
>
> int getPort(String protocol); // one way to handle the multi
> nodes per server
> int getPortOffset(); // or this one (add to standard
> port)
> }
Calling it a Node is cool with me.
I'd drop the port stuff. Ports imply IP and protocols could be P2P
and wouldn't have a port. When James and I wrote this, we expected
addresses to contain a URI, but didn't want to force java.net.URI on
everyone, so we went with String.
> // Meta Session - was SessionLocation
> interface MetaSession
> {
> String getSessionId();
> void invalidate();
>
> void addEventListener(MetaSessionListener listener);
> void removeEventListener(MetaSessionListener listener);
>
> // State API has map per context ID , where a context
> // ID might be "web:/context" or "ejb:" or random
> boolean isStateLocal(String contextId);
>
>
> Map getState(String contextId); // implies a move local!
> void getStateAsync(Object key, String contextId); // async
> version
>
> Map createState(String contextId);
> void releaseState(String contextId); // don't lock whole meta
> session!
> void invalidate(String contextId);
>
> // Locaton / Policy API.
> Node getNode(String contextId);
> Node getExecutionNode(String contextId);
> void getExecutionNodeAsync(Object key, String contextId);
>
>
> // Don't know if these are too HTTP specific... but we need them
> void setPassivationTimeout(long ms, String contextId);
> void setInvalidationTimeout(long ms, String contextId);
> }
As I mentioned above, I don't like the context stuff, but the rest is
good.
I think we should drop the async method until we have some
implementations. Specially, I think it will need an optional
listener parameter so you can be notified when the transfer is complete.
I think the passivation methods are good general purpose items. Do
we need a last modified, or last accessed items also?
> interface MetaSessionListener
> {
> // callbacks to allow session manager to inspect contents for
> // tier specific handling (eg servlet listeners etc.)
> void activateState(String sessionId, String contextId, Map state);
> void passivateState(String sessionId, String contextId, Map
> state);
> void invalidateState(String sessionId, String contextId, Map
> state);
>
> // callbacks for async operations
> void gotState(Object key, String sessionId, String contextId,
> Map state);
> void executionNode(Object key, String sessionId, String
> contextId, Node location);
>
> }
I don't think we need (or want) the async listener methods part of
this interface. I think we need a separate one if we add the method.
Cluster API proposal?
Posted by Greg Wilkins <gr...@mortbay.com>.
This is my idea of how we could morph the currently proposed session APIs
into a cluster API.
I have created a spot for Cluster meta data - but I have not filled it out much.
The key difference is that the state Map is now indexed by session ID and context ID.
This allows the state for different contexts within the same session to be on different
nodes (this is a real requirement) and also means that locking is at context rather
than meta session level. Note that some implementations may not fully support
this and may just do sessionId+contextId behind the scenes and colocate all context
states for the same session (and move them as one).
I have also added an async aspect to the API for potentially long operations
such as moving state about the place - again this can be optionally supported.
Also I support the idea of multiple Nodes per server (really useful for testing
and heterogeneous clusters).
// The top level Cluster API - this was the Locator... but let's call a spade a spade.
interface Cluster
{
// methods to get/set meta data about the cluster
// these signatures here are just a guess... but you get the idea.
int getMaxNodes();
Set<Node> getKnownNodes();
void setKnownNodes(Set<Node> nodes);
Node getLocalNode();
// Access sessions in cluster.
MetaSession getMetaSession(String clientID);
Session createMetaSession(String sessionId);
}
// Node API
// was Server - but may have multiple Nodes per server
interface Node
{
String getName();
String[] getAddresses(String protocol);
void setAddresses(String string, String[] strings);
boolean isLocalServer();
boolean isActive();
int getPort(String protocol); // one way to handle the multi nodes per server
int getPortOffset(); // or this one (add to standard port)
}
// Meta Session - was SessionLocation
interface MetaSession
{
String getSessionId();
void invalidate();
void addEventListener(MetaSessionListener listener);
void removeEventListener(MetaSessionListener listener);
// State API has map per context ID , where a context
// ID might be "web:/context" or "ejb:" or random
boolean isStateLocal(String contextId);
Map getState(String contextId); // implies a move local!
void getStateAsync(Object key, String contextId); // async version
Map createState(String contextId);
void releaseState(String contextId); // don't lock whole meta session!
void invalidate(String contextId);
// Locaton / Policy API.
Node getNode(String contextId);
Node getExecutionNode(String contextId);
void getExecutionNodeAsync(Object key, String contextId);
// Don't know if these are too HTTP specific... but we need them
void setPassivationTimeout(long ms, String contextId);
void setInvalidationTimeout(long ms, String contextId);
}
interface MetaSessionListener
{
// callbacks to allow session manager to inspect contents for
// tier specific handling (eg servlet listeners etc.)
void activateState(String sessionId, String contextId, Map state);
void passivateState(String sessionId, String contextId, Map state);
void invalidateState(String sessionId, String contextId, Map state);
// callbacks for async operations
void gotState(Object key, String sessionId, String contextId, Map state);
void executionNode(Object key, String sessionId, String contextId, Node location);
}
Re: [wadi-dev] Session/clustering API and the web tier
Posted by Greg Wilkins <gr...@mortbay.com>.
Jules Gosnell wrote:
> I see the GSM interceptor as a 'peninsula' of the GSM impl, because it
> deals with clustering concerns. The point at which it meets Jetty is at
> the generic interceptor API... - I guess this is one of the bones of
> contention.
>
> Seen this way, the same interceptor code can be shared across all
> containers (via different interceptor adaptors), rather than duplicated
> by each container.... and less of the GSM's API needs to be exposed to
> the container...
OK, this is the point that I don't get.
What of the pseudo code below can be shared with other tiers or containers:
public class JettyGeronimoClusterInterceptor extends AbstractJETTYHandler
{
void handle(JETTYRequest request, JETTYResponse response)
{
String sessionId = request.getRequestedSessionId();
if (sessionId!=null)
{
MetaSession meta = getCluster().getMetaSession(sessionId);
if (meta!=null)
{
Node node = meta.getExecutionContext(getContextId());
if (!node.isLocal())
{
relocateRequestInJETTYSpecificWay(node,request,response);
return;
}
}
}
super.handle(request,response);
}
}
I guess the 3 getXxx calls and the 3 ifs are common, but they
are in a Jetty specific method and call jetty specific code and
use jetty specific params.
In reality, this code will be even more Jetty specific has it
will have to handle continuations and async proxying etc.
I just don't see that there is enough here to make a huge
sticking point. But if you want to wrap this up into a common
ServletRequestClusterInterceptor, then sure! But I still
think that does not effect the underlying Cluster API.
cheers
Re: [wadi-dev] Session/clustering API and the web tier
Posted by Greg Wilkins <gr...@mortbay.com>.
Jules Gosnell wrote:
>
> I pressed 'send' without counting to 10 first on the last response. Sorry.
No probs - no offence taken.
> I've obviously done a bad job of explaining myself so far. Is this
> clearer ?
Actually no - I preferred the slightly pissed off version for clarity :-)
I actually think we are in agreement.. but don´t realize it.
You say that the getExcecutionLocation() style API I suggest leaks
clustering concerns into the webtier, but I am not suggesting that general
container code calls this API.
I am suggesting a that a cluster aware interceptor calls it.
This is the GSM intercpetion contract of your last email.
But given that interceptors are going to be very tier implementation
specific and that Jetty has to write geronimo specific interceptors for
security, transactions, .... then it appears natural to me that the
G-jetty module is the natural place for that interceptor to live and
we don't need to standardize it's contract.
Or is it the existence of the standardized API between interceptor
and Policy that you object to? Are your concerns such that
you do not think this API can be implemented efficiently?
Give me 30minutes and I will post my latest mental picture of
how I would like to see the Cluster/Session API.
cheers