You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by Gianny Damour <gi...@optusnet.com.au> on 2007/10/17 14:55:05 UTC
Effectiveness of WADI's Design and Implementation Comforted
Hi,
Over the last two weeks, I have been working on various
experimentations in order to validate and demonstrate the
effectiveness of WADI's distributed session engine and replication
engine. I wrote a WIKI page to capture the findings:
http://docs.codehaus.org/pages/viewpage.action?pageId=18186271
Excerpt of the conclusion:
"
The effectiveness of the design and implementation of WADI's
distributed session lookup engine and replication engine is further
comforted by the observed average response times and scalability
characteristics.
For the considered scenarios, WADI performs better than Terracotta,
which is not really surprising as...
"
If people are interested by clustering development, then please
respond as I will resume some Geronimo clustering work and could
really use a hand.
Thanks,
Gianny
Re: Effectiveness of WADI's Design and Implementation Comforted
Posted by Gianny Damour <gi...@optusnet.com.au>.
Dain, Jeff,
Thanks for your feedback on the weakness of the annotation approach.
It should be easy to have a less obtrusive configuration mechanism as
the logic is encapsulated within a single (AspectJ) aspect. I will
see what I can do.
In more details, an aspect adds the interface ClusteredStateMarker to
any classes having @ClusteredState:
declare parents:
(@ClusteredState *) implements ClusteredStateMarker;
It is easy to have a pattern driven mechanism instead, e.g.:
declare parents:
org.apache.xyz..* implements ClusteredStateMarker;
I now need to find a way to specify patterns outside of the aspect.
Regarding the WADI vs Tribes on Tomcat suggestion, there isn't yet a
Tomcat session manager integrated with WADI. BTW, WADI actually
relies on Tribes for its group communications thanks to Filip Hanik's
work on wadi-tribes.
Regarding a test with a bigger session size, I also would like to
know how wadi-aop performs. However, I would like to first resume
some Geronimo clustering work. Once done, I will touch base with
Jason as suggested to know if he can grant me access to GBuild in
order to conduct a real distributed test.
Thanks,
Gianny
On 19/10/2007, at 10:19 AM, Dain Sundstrom wrote:
> I agree about the annotation thing. Would it be difficult change
> the design to have some reasonable defaults, and only use
> annotations (or xml) for overrides?
>
> -dain
>
> On Oct 18, 2007, at 3:49 PM, Jeff Genender wrote:
>
>> Gianni,
>>
>> What you have done is very cool. I guess my only comment is that
>> what I
>> am reading is that the annotations force a lock to the clustering
>> engine, as opposed to being somewhat transparent by swapping out the
>> clustering manager.
>>
>> Therefore, my application code needs these annotations coded as a
>> part
>> of it. In otherwords, in order for me to leverage the fine grained
>> capabilities of WADI, my application needs to be coded with the WADI
>> annotations. Did I read that correctly?
>>
>> Regardless...its pretty cool stuff. We should talk about the
>> contract/interface for openejb...I look forward to working with
>> you ;-)
>>
>> Jeff
>>
Re: Effectiveness of WADI's Design and Implementation Comforted
Posted by Dain Sundstrom <da...@iq80.com>.
I agree about the annotation thing. Would it be difficult change the
design to have some reasonable defaults, and only use annotations (or
xml) for overrides?
-dain
On Oct 18, 2007, at 3:49 PM, Jeff Genender wrote:
> Gianni,
>
> What you have done is very cool. I guess my only comment is that
> what I
> am reading is that the annotations force a lock to the clustering
> engine, as opposed to being somewhat transparent by swapping out the
> clustering manager.
>
> Therefore, my application code needs these annotations coded as a part
> of it. In otherwords, in order for me to leverage the fine grained
> capabilities of WADI, my application needs to be coded with the WADI
> annotations. Did I read that correctly?
>
> Regardless...its pretty cool stuff. We should talk about the
> contract/interface for openejb...I look forward to working with
> you ;-)
>
> Jeff
>
>
>
> Gianny Damour wrote:
>> On 17/10/2007, at 11:53 PM, Jeff Genender wrote:
>>
>>>
>>>
>>> Gianny Damour wrote:
>>>>
>>>> Excerpt of the conclusion:
>>>>
>>>> "
>>>> The effectiveness of the design and implementation of WADI's
>>>> distributed
>>>> session lookup engine and replication engine is further
>>>> comforted by the
>>>> observed average response times and scalability characteristics.
>>>> For the considered scenarios, WADI performs better than Terracotta,
>>>> which is not really surprising as...
>>>>
>>>
>>> If I may comment here...Without fine-grained clustering
>>> capabilities, I
>>> have a hard time believing that WADI can outperform Terracotta.
>>> Especially with large objects...WADI would push over the entire
>>> object
>>> each time, where Terracotta would only ship the changed members.
>>> If you
>>> are going to publish the numbers you did, you probably should
>>> explain
>>> what is getting pushed across.
>>
>> Hi,
>>
>> I believe you simply skimmed through what I have been writing:
>> firstly,
>> I did provide a description of the state stored in session. Secondly,
>> your comment about large objects and the inadequacy of WADI's
>> design to
>> handle them is clearly and explicitly discussed in the second
>> paragraph
>> of the conclusion. As redundancy of information is not much of a
>> problem
>> for theoretical clustering discussions, I am reiterating: WADI does
>> provide a fine grained replication mechanism, which is able to track
>> field updates or method executions and replay them against
>> replicas. If
>> you are interested by this kind of stuff, then please feel free to
>> have
>> a look to this WIKI page -
>> http://docs.codehaus.org/display/WADI/5.+Delta+Session+Replication.
>>
>> At this stage, the delta replication stuff is under performing and
>> resource intensive: to give you an idea, its average response time is
>> 6.10ms for the third test scenario (average response time for default
>> replication mechanism is 4.77ms for WADI and 6.36ms for Terracotta).
>> Once again, as a preliminary comment to Ari's response, this is a
>> scenario with HTTP session stickiness on. After this week-end, the
>> performance for fine-grained replication should improve
>> significantly. I
>> intend to: get ride of native reflection; index constructor, field
>> and
>> methods description for really fast packaging on wire and resolution
>> when reading back from a byte stream; and re-use reflective invokers
>> (CGLIB FastMethod/Constructor and ASM customed field updaters).
>>
>> FWIW, I do not see support of large objects as crucial as you for Web
>> solutions. Even if I spent a couple of months working on wadi-aop to
>> provide this feature, it was more with the ultimate goal to
>> leverage it
>> for efficient distributed caching than HTTP session replication. I
>> truly
>> believe that large scale Web solutions requiring HA have to be
>> designed
>> with this constraint in mind; with this constraint in mind, keeping
>> session size small sounds reasonable.
>>
>>
>>>> "
>>>>
>>>> If people are interested by clustering development, then please
>>>> respond
>>>> as I will resume some Geronimo clustering work and could really
>>>> use a
>>>> hand.
>>>>
>>>
>>> I am in the midst of getting OpenEJB clusterable. I would be
>>> happy to
>>> combine forces and get clustering finished up for G ;-)
>>
>> Great! Will post back to the dev@ list as soon as I resume Geronimo
>> clustering work.
>>
>> Thanks,
>> Gianny
>>
>>
>>
>>>
>>>> Thanks,
>>>> Gianny
Re: Effectiveness of WADI's Design and Implementation Comforted
Posted by Jeff Genender <jg...@apache.org>.
Gianni,
What you have done is very cool. I guess my only comment is that what I
am reading is that the annotations force a lock to the clustering
engine, as opposed to being somewhat transparent by swapping out the
clustering manager.
Therefore, my application code needs these annotations coded as a part
of it. In otherwords, in order for me to leverage the fine grained
capabilities of WADI, my application needs to be coded with the WADI
annotations. Did I read that correctly?
Regardless...its pretty cool stuff. We should talk about the
contract/interface for openejb...I look forward to working with you ;-)
Jeff
Gianny Damour wrote:
> On 17/10/2007, at 11:53 PM, Jeff Genender wrote:
>
>>
>>
>> Gianny Damour wrote:
>>>
>>> Excerpt of the conclusion:
>>>
>>> "
>>> The effectiveness of the design and implementation of WADI's distributed
>>> session lookup engine and replication engine is further comforted by the
>>> observed average response times and scalability characteristics.
>>> For the considered scenarios, WADI performs better than Terracotta,
>>> which is not really surprising as...
>>>
>>
>> If I may comment here...Without fine-grained clustering capabilities, I
>> have a hard time believing that WADI can outperform Terracotta.
>> Especially with large objects...WADI would push over the entire object
>> each time, where Terracotta would only ship the changed members. If you
>> are going to publish the numbers you did, you probably should explain
>> what is getting pushed across.
>
> Hi,
>
> I believe you simply skimmed through what I have been writing: firstly,
> I did provide a description of the state stored in session. Secondly,
> your comment about large objects and the inadequacy of WADI's design to
> handle them is clearly and explicitly discussed in the second paragraph
> of the conclusion. As redundancy of information is not much of a problem
> for theoretical clustering discussions, I am reiterating: WADI does
> provide a fine grained replication mechanism, which is able to track
> field updates or method executions and replay them against replicas. If
> you are interested by this kind of stuff, then please feel free to have
> a look to this WIKI page -
> http://docs.codehaus.org/display/WADI/5.+Delta+Session+Replication.
>
> At this stage, the delta replication stuff is under performing and
> resource intensive: to give you an idea, its average response time is
> 6.10ms for the third test scenario (average response time for default
> replication mechanism is 4.77ms for WADI and 6.36ms for Terracotta).
> Once again, as a preliminary comment to Ari's response, this is a
> scenario with HTTP session stickiness on. After this week-end, the
> performance for fine-grained replication should improve significantly. I
> intend to: get ride of native reflection; index constructor, field and
> methods description for really fast packaging on wire and resolution
> when reading back from a byte stream; and re-use reflective invokers
> (CGLIB FastMethod/Constructor and ASM customed field updaters).
>
> FWIW, I do not see support of large objects as crucial as you for Web
> solutions. Even if I spent a couple of months working on wadi-aop to
> provide this feature, it was more with the ultimate goal to leverage it
> for efficient distributed caching than HTTP session replication. I truly
> believe that large scale Web solutions requiring HA have to be designed
> with this constraint in mind; with this constraint in mind, keeping
> session size small sounds reasonable.
>
>
>>> "
>>>
>>> If people are interested by clustering development, then please respond
>>> as I will resume some Geronimo clustering work and could really use a
>>> hand.
>>>
>>
>> I am in the midst of getting OpenEJB clusterable. I would be happy to
>> combine forces and get clustering finished up for G ;-)
>
> Great! Will post back to the dev@ list as soon as I resume Geronimo
> clustering work.
>
> Thanks,
> Gianny
>
>
>
>>
>>> Thanks,
>>> Gianny
Re: Effectiveness of WADI's Design and Implementation Comforted
Posted by Gianny Damour <gi...@optusnet.com.au>.
On 17/10/2007, at 11:53 PM, Jeff Genender wrote:
>
>
> Gianny Damour wrote:
>>
>> Excerpt of the conclusion:
>>
>> "
>> The effectiveness of the design and implementation of WADI's
>> distributed
>> session lookup engine and replication engine is further comforted
>> by the
>> observed average response times and scalability characteristics.
>> For the considered scenarios, WADI performs better than Terracotta,
>> which is not really surprising as...
>>
>
> If I may comment here...Without fine-grained clustering
> capabilities, I
> have a hard time believing that WADI can outperform Terracotta.
> Especially with large objects...WADI would push over the entire object
> each time, where Terracotta would only ship the changed members.
> If you
> are going to publish the numbers you did, you probably should explain
> what is getting pushed across.
Hi,
I believe you simply skimmed through what I have been writing:
firstly, I did provide a description of the state stored in session.
Secondly, your comment about large objects and the inadequacy of
WADI's design to handle them is clearly and explicitly discussed in
the second paragraph of the conclusion. As redundancy of information
is not much of a problem for theoretical clustering discussions, I am
reiterating: WADI does provide a fine grained replication mechanism,
which is able to track field updates or method executions and replay
them against replicas. If you are interested by this kind of stuff,
then please feel free to have a look to this WIKI page - http://
docs.codehaus.org/display/WADI/5.+Delta+Session+Replication.
At this stage, the delta replication stuff is under performing and
resource intensive: to give you an idea, its average response time is
6.10ms for the third test scenario (average response time for default
replication mechanism is 4.77ms for WADI and 6.36ms for Terracotta).
Once again, as a preliminary comment to Ari's response, this is a
scenario with HTTP session stickiness on. After this week-end, the
performance for fine-grained replication should improve
significantly. I intend to: get ride of native reflection; index
constructor, field and methods description for really fast packaging
on wire and resolution when reading back from a byte stream; and re-
use reflective invokers (CGLIB FastMethod/Constructor and ASM
customed field updaters).
FWIW, I do not see support of large objects as crucial as you for Web
solutions. Even if I spent a couple of months working on wadi-aop to
provide this feature, it was more with the ultimate goal to leverage
it for efficient distributed caching than HTTP session replication. I
truly believe that large scale Web solutions requiring HA have to be
designed with this constraint in mind; with this constraint in mind,
keeping session size small sounds reasonable.
>> "
>>
>> If people are interested by clustering development, then please
>> respond
>> as I will resume some Geronimo clustering work and could really
>> use a hand.
>>
>
> I am in the midst of getting OpenEJB clusterable. I would be happy to
> combine forces and get clustering finished up for G ;-)
Great! Will post back to the dev@ list as soon as I resume Geronimo
clustering work.
Thanks,
Gianny
>
>> Thanks,
>> Gianny
Re: Effectiveness of WADI's Design and Implementation Comforted
Posted by Dain Sundstrom <da...@iq80.com>.
I'd also like to see a sticky test as most of the clusters I have
worked with use a high-end hardware load balancer (although I do find
this test very interesting). I'd also like to see the test use a
larger and mutating session. One of the things that will effect
latency is the time (and effort) it takes to move the session data.
With people sticking large caches of detached JPA objects in sessions
now days, I'd like to know how that effects my application performance.
Finally, how about a wadi vs tribes on Tomcat?
BTW you should talk to Jason about testing this on GBuild, so you can
test in a true distributed state.
-dain
On Oct 18, 2007, at 12:02 AM, ikarzali wrote:
>
>
>
> jgenender wrote:
>>
>>
>>> Excerpt of the conclusion:
>>>
>>> "
>>> The effectiveness of the design and implementation of WADI's
>>> distributed
>>> session lookup engine and replication engine is further comforted
>>> by the
>>> observed average response times and scalability characteristics.
>>> For the considered scenarios, WADI performs better than Terracotta,
>>> which is not really surprising as...
>>>
>>
>> If I may comment here...Without fine-grained clustering
>> capabilities, I
>> have a hard time believing that WADI can outperform Terracotta.
>> Especially with large objects...WADI would push over the entire
>> object
>> each time, where Terracotta would only ship the changed members.
>> If you
>> are going to publish the numbers you did, you probably should explain
>> what is getting pushed across.
>>
>
> Interesting test of Terracotta. I wouldn't trust any test that
> pegs the CPU
> at 100%. May I suggest the following potential changes:
>
> 1. Running Jetty, Grinder, and Terracotta on a single laptop should
> change.
> Run Terracotta on its own server. It will run faster even though
> it won't
> be over loopback.
>
> 2. Run sticky and see what happens. See, the test is not testing
> the same
> thing with WADI and Terracotta. With WADI, the clustering
> implementation is
> configured to keep data on a finite number of nodes. With
> Terracotta, you
> have a consistent clustered view of sessions. Since you are round-
> robin,
> with Terracotta every node is holding a reference to every session
> and as
> the sessions change, all Jetty nodes are updated with the change.
> So, round
> robin WITH WADI replication off is actually pretty much cheating
> because TC
> has the sessions in all nodes and WADI has them in one. Run sticky
> sessions
> in your load balancer. Then Terracotta will have the session in
> one node
> just like WADI. _Then_ you will have apples-to-apples and maybe
> find TC
> latency to be lower and throughput higher.
>
> I would be happy to help explain more but this use of WADI and
> Terracotta
> seem like you are getting opposite behaviors out of the products
> (full n-way
> replication with no SPoF under Terracotta versus zero replication
> under
> WADI) and a different test will more accurately reflect the relative
> performance of the systems.
>
> --Ari
>
> --
> View this message in context: http://www.nabble.com/Effectiveness-
> of-WADI%27s-Design-and-Implementation-Comforted-
> tf4640401s134.html#a13269164
> Sent from the Apache Geronimo - Dev mailing list archive at
> Nabble.com.
>
Re: Effectiveness of WADI's Design and Implementation Comforted
Posted by ikarzali <ik...@hotmail.com>.
Jules Gosnell-2 wrote:
>
>
>
> So, WADI offers a delta-based replication route as well - but this was
> not the one under test.
>
>
I am not sure what the goal is here. is WADI trying to compete directly
with Terracotta? I don't think so. Terracotta is general-purpose
clustering and can share almost any POJO (without the replay concept),
preserves object identity, and clusters thread coordination as well. Sounds
like WADI field-level is dependent on an HTTPSession lifecycle. Not that
such an idea is bad, just different. Field-level replication for Terracotta
is different that a conceptual "replay" in that it can push only the deltas
to complex fields and object hierarchies, anywhere in the hierarchy. It
differs in that it doesn't present a risk like "replay" does that there
might be side-effects outside update of the session attributes when running
methods more than once. And, Terracotta, has thread coordination so
consistency and coordination across JVMs is possible just like across
threads. Now, for webapps, there should be no threads, but the container is
still multi-threaded and race conditions can easily occur if concurrency is
not properly isolated.
I go into more detail below as to how our approach to field-level
replication differs from "replay."
Jules Gosnell-2 wrote:
>
>
> Ari,
>
> I know nothing about Terracotta, so would you mind spending a little
> more time comparing and contrasting architectures...
>
> I agree that the best case scenario is fully functional stickiness - but
> then you would simply be testing session replication - see Gianny's
> "Session Replication Test" explanation and results.
>
I think I failed to explain Terracotta that well in my previous email. So I
will try with more detail (apologies for the rambling email at this point).
We have a library that sits inside your JVM allowing us access to your reads
and writes to heap. We then have a central server cluster who's job it is
to keep all the JVMs in sync. We keep track of which JVM has what
references resident and active...and then our server only pushes field-level
deltas and only to JVMs that need the deltas. This is not a session
replication product. It is cross-JVM POJO coordination at the heap-level...
In a session context, this means many things (not good or bad, just reality
of Terracotta):
1. If any node needs a session it goes to look it up from the
SessionManager. Terracotta is inside the SessionManager looking for
heap-reads and when a session'ed request comes in (a cookie already set in
the browser and passed in the request), if that session is not local
already, the SessionManager nor the JVM never realize this. Terracotta
pulls the appropriate session object (without pulling all its attributes)
into the JVM that needs it. In effect, any session is available ANYWHERE at
any time without ALL sessions being everwhere all the time. What's more,
attributes will now fault in lazily as attributes are requested and
references traversed. Thus Jeff G's assertion that TC can handle large
session objects (I know of one TC user with 15 meg session graphs).
2. Since Terracotta is heap-level in nature, and it knows what objects are
where, it has the ability to push session updates made in one JVM to other
JVMs. If sessions are only resident in a single JVM, Terracotta will only
store the change in our own server just in case of JVM-level failure but TC
will not push the change to any other node. If session has been balanced to
more than one server, those JVMs are automatically connected in a
conversation about the aforementioned session. Its neither Peer-to-Peer nor
client-server. It is, instead, locality of reference: trying to keep
objects (sessions in this case) consistent where they are needed and
simultaneously keep them from leaking to where they are not.
3. Since Terracotta is heap-level in nature, it gets a big performance boost
on read. In WADI's case, this might be true as well but Terracotta allows
the SessionManager to trust its own session map and all reads that are for
sticky session data come from local heap w/o a network call of any sort.
Some clustering solutions can't allow this at scale and in a round-robin
scenario. They will store session in a single node w/ n-replicants but
every node that is not the home node of that session must read/write to the
home node and cannot cache the session data.
This information in theory explains some of the test results:
1. round-robin puts the session in every JVM so the TC cluster is
maintaining consistency and WADI might be async or may be going to the same
node for session storage? Note that round-robin is not how TC is tuned with
regard to session. If you want to cluster and share every object on all
nodes, TC tends to outrun most solutions in such highly contentious write
scenarios. But webapps should not be configured such that session is
necessary across requests and yet you do round-robin such that nodes are
constantly faulting in session data in order to handle the request.
2. Your Terracotta server is on the same machine as your test cluster and,
thus, is being throttled by the test itself. Once TC is throttled, the app
above it becomes throttled simply because TC is blocking heap-level reads
and writes while it fights for CPU to push updates around. So, if you run
TC on the same laptop as your test, our latency comes mostly from the CPU
context switching and not from our software. Its not a valid test. Its
like running Oracle on the same node as your web app and then running a
request that maxes out CPU due to your container chewing on something and in
the middle, asking Oracle to serve up a query. Oracle's latency would then
be measured to be HUGE, when in fact Oracle on another box may return that
answer sub-millisecond if it had enough RAM to keep some data in cache, etc.
Starvation is just that...
Jules Gosnell-2 wrote:
>
>
> !! - I would hope that Terracotta doesn't always do one-to-all
> replication and should point out that my reading of "One again, the cost
> of keeping one session replica, i.e. the cost of keeping a copy of a
> session on an another node than the one currently owning the session,
> remains constant while the number of nodes in the cluster increases for
> WADI." (see the Session Migration & Replication Test" observations),
> indicates that WADI is running in a 1+1 configuration, that means it IS
> replicating - each session has 1 (configurable) backup session off-node.
> and therefore no SPoF either.
>
> and a different test will more accurately reflect the relative
>> performance of the systems.
>
> This is an interesting thread - but I think that there is a lot of
> misunderstanding/miscommunication occurring that it would be useful to
> clear up :-)
>
> Jules
>
>
I must have misread that WADI had replication disabled since sesions had a
home in the test and any replication would be 2-stage replication versus
Terracotta's 1-stage. My misunderstanding.
I am not being defensive here. After all, I know we have a feature on the
roadmap to make round-robin work fast. Such a feature would basically cause
your JVM running your web container to forget it saw a session right after
it responded to a request. Since TC can always give a particular JVM a
session reference just in time on heap-read, why not drop the session and
avoid the broadcast storm, right? We don't have that feature at the moment,
however, so any test of us at round robin will work but might not be fast.
Sticky with a TC server on its own box is the way to go.
I want to make sure everyone who wants to know what Terracotta is good at
and how fast it can go is aware of all the content at
http://www.terracotta.org/. I recommend reading it before passing judgment
based on these graphs. Also, I don't know if I extended the offer but I
would love to get on a Skype or concall with anyone who wants to dig in more
on the two approaches. I am happy to help WADI learn from what we do and
vice versa. I think there is a lot of value in sharing.
Kewl stuff you guys are working on. Keep it up!
--Ari
BTW, I wasn't suggesting that anyone was cheating. I meant that the test is
an unfair test based on assumptions about the product. No harm no foul.
--
View this message in context: http://www.nabble.com/Effectiveness-of-WADI%27s-Design-and-Implementation-Comforted-tf4640401s134.html#a13286716
Sent from the Apache Geronimo - Dev mailing list archive at Nabble.com.
Re: Effectiveness of WADI's Design and Implementation Comforted
Posted by Jules Gosnell <ju...@mortbay.com>.
ikarzali wrote:
>
>
> jgenender wrote:
>>
>>> Excerpt of the conclusion:
>>>
>>> "
>>> The effectiveness of the design and implementation of WADI's distributed
>>> session lookup engine and replication engine is further comforted by the
>>> observed average response times and scalability characteristics.
>>> For the considered scenarios, WADI performs better than Terracotta,
>>> which is not really surprising as...
>>>
>> If I may comment here...Without fine-grained clustering capabilities, I
>> have a hard time believing that WADI can outperform Terracotta.
>> Especially with large objects...WADI would push over the entire object
>> each time, where Terracotta would only ship the changed members. If you
>> are going to publish the numbers you did, you probably should explain
>> what is getting pushed across.
Jeff,
from Gianny's article:
"In order to efficiently cluster the few online solutions having a
rather big session size, WADI offers an alternative replication
mechanism, which only replicates field updates or method executions
against replicas."
So, WADI offers a delta-based replication route as well - but this was
not the one under test.
>>
>
> Interesting test of Terracotta. I wouldn't trust any test that pegs the CPU
> at 100%. May I suggest the following potential changes:
>
> 1. Running Jetty, Grinder, and Terracotta on a single laptop should change.
> Run Terracotta on its own server. It will run faster even though it won't
> be over loopback.
>
> 2. Run sticky and see what happens. See, the test is not testing the same
> thing with WADI and Terracotta. With WADI, the clustering implementation is
> configured to keep data on a finite number of nodes. With Terracotta, you
> have a consistent clustered view of sessions. Since you are round-robin,
> with Terracotta every node is holding a reference to every session and as
> the sessions change, all Jetty nodes are updated with the change.
Ari,
I know nothing about Terracotta, so would you mind spending a little
more time comparing and contrasting architectures...
I agree that the best case scenario is fully functional stickiness - but
then you would simply be testing session replication - see Gianny's
"Session Replication Test" explanation and results.
Every now and then, stickiness is not enough - for instance when you
take down or lose a node (unless Terracotta has some form of integrated
load-balancer which negotiates a session's new location). WADI is
designed to work with the LCD - a loadbalancer which will just dump the
subsequent request for a dead node anywhere in the cluster and then restick.
Thus the worst case scenario, often an interesting thing to examine, is
not temporary, as above, but prolonged lack of stickiness in the load
balancer.
WADI will maintain 1+numReplicas copies of the session, no matter how
many different nodes may have to shelter the session during its lifetime.
Are you saying that each Terracotta node that shelters a session becomes
part of its replication group (i.e. has every change to that session
synchronously replicated to it) ? Can you expand ? Can you not place a
limit on group size ? If I churn nodes regularly, am I not going to end
up with a lot of unecessary replication ?
> So, round
> robin WITH WADI replication off is actually pretty much cheating because TC
> has the sessions in all nodes and WADI has them in one.
"cheating" is a bit harsh...
> Run sticky sessions
> in your load balancer. Then Terracotta will have the session in one node
> just like WADI. _Then_ you will have apples-to-apples and maybe find TC
> latency to be lower and throughput higher.
As I said - see the "apples-to-apples" straight replication test results
in the article.
>
> I would be happy to help explain more but this use of WADI and Terracotta
> seem like you are getting opposite behaviors out of the products (full n-way
> replication with no SPoF under Terracotta versus zero replication under
> WADI)
!! - I would hope that Terracotta doesn't always do one-to-all
replication and should point out that my reading of "One again, the cost
of keeping one session replica, i.e. the cost of keeping a copy of a
session on an another node than the one currently owning the session,
remains constant while the number of nodes in the cluster increases for
WADI." (see the Session Migration & Replication Test" observations),
indicates that WADI is running in a 1+1 configuration, that means it IS
replicating - each session has 1 (configurable) backup session off-node.
and therefore no SPoF either.
and a different test will more accurately reflect the relative
> performance of the systems.
This is an interesting thread - but I think that there is a lot of
misunderstanding/miscommunication occurring that it would be useful to
clear up :-)
Jules
>
> --Ari
>
Re: Effectiveness of WADI's Design and Implementation Comforted
Posted by ikarzali <ik...@hotmail.com>.
jgenender wrote:
>
>
>> Excerpt of the conclusion:
>>
>> "
>> The effectiveness of the design and implementation of WADI's distributed
>> session lookup engine and replication engine is further comforted by the
>> observed average response times and scalability characteristics.
>> For the considered scenarios, WADI performs better than Terracotta,
>> which is not really surprising as...
>>
>
> If I may comment here...Without fine-grained clustering capabilities, I
> have a hard time believing that WADI can outperform Terracotta.
> Especially with large objects...WADI would push over the entire object
> each time, where Terracotta would only ship the changed members. If you
> are going to publish the numbers you did, you probably should explain
> what is getting pushed across.
>
Interesting test of Terracotta. I wouldn't trust any test that pegs the CPU
at 100%. May I suggest the following potential changes:
1. Running Jetty, Grinder, and Terracotta on a single laptop should change.
Run Terracotta on its own server. It will run faster even though it won't
be over loopback.
2. Run sticky and see what happens. See, the test is not testing the same
thing with WADI and Terracotta. With WADI, the clustering implementation is
configured to keep data on a finite number of nodes. With Terracotta, you
have a consistent clustered view of sessions. Since you are round-robin,
with Terracotta every node is holding a reference to every session and as
the sessions change, all Jetty nodes are updated with the change. So, round
robin WITH WADI replication off is actually pretty much cheating because TC
has the sessions in all nodes and WADI has them in one. Run sticky sessions
in your load balancer. Then Terracotta will have the session in one node
just like WADI. _Then_ you will have apples-to-apples and maybe find TC
latency to be lower and throughput higher.
I would be happy to help explain more but this use of WADI and Terracotta
seem like you are getting opposite behaviors out of the products (full n-way
replication with no SPoF under Terracotta versus zero replication under
WADI) and a different test will more accurately reflect the relative
performance of the systems.
--Ari
--
View this message in context: http://www.nabble.com/Effectiveness-of-WADI%27s-Design-and-Implementation-Comforted-tf4640401s134.html#a13269164
Sent from the Apache Geronimo - Dev mailing list archive at Nabble.com.
Re: Effectiveness of WADI's Design and Implementation Comforted
Posted by Jeff Genender <jg...@apache.org>.
Gianny Damour wrote:
> Hi,
>
>
> Over the last two weeks, I have been working on various experimentations
> in order to validate and demonstrate the effectiveness of WADI's
> distributed session engine and replication engine. I wrote a WIKI page
> to capture the findings:
> http://docs.codehaus.org/pages/viewpage.action?pageId=18186271
>
You have been busy ;-)
> Excerpt of the conclusion:
>
> "
> The effectiveness of the design and implementation of WADI's distributed
> session lookup engine and replication engine is further comforted by the
> observed average response times and scalability characteristics.
> For the considered scenarios, WADI performs better than Terracotta,
> which is not really surprising as...
>
If I may comment here...Without fine-grained clustering capabilities, I
have a hard time believing that WADI can outperform Terracotta.
Especially with large objects...WADI would push over the entire object
each time, where Terracotta would only ship the changed members. If you
are going to publish the numbers you did, you probably should explain
what is getting pushed across.
> "
>
> If people are interested by clustering development, then please respond
> as I will resume some Geronimo clustering work and could really use a hand.
>
I am in the midst of getting OpenEJB clusterable. I would be happy to
combine forces and get clustering finished up for G ;-)
> Thanks,
> Gianny