You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@wicket.apache.org by Robin Shine <ro...@pmease.com> on 2013/06/11 12:23:29 UTC
Possible optimization of page store manager
Hi All,
We are using Wicket to develop our web application product (QuickBuild) for six years, and this makes our life a lot easier when comes to maintain our code base due to Wicket's component approach and type safe nature of Java.
As our product gets more used in companies, it is not uncommon that thousands of users are accessing the application concurrently, and at peak time the server sometimes gets stressed to have slow response time. Our benchmark shows that offen the server is busy serializing pages. This is not strange as our application uses ajax heavily and every ajax call to the server triggers the page store to persist (serialize and write) the whole page on disk.
To my understanding, Wicket serializes/saves stateful pages for purpose of back button support. At commit stage of every request, all touched pages will be put into session cache, and also get persisted via page store. This mechanism works but results in a lot of unnecessary page persistence. To explain it, assume below workflow:
1. User loads the stateful page and wicket creates a page instance to put in session cache, as well as persist to disk.
2. User clicks some action links (either ajax or non-ajax) subsequently to update parts of the page, but still remains in the same page instance, and the url displayed in browser remains unchanged. For every request, Wicket updates the page instance, put it into session cache, and finally persist it to disk.
3. User clicks some other links to cause new page instance being created, and Wicket does the same to put new instance in session cache and persist the new instance.
Here in step 2, page persistence seems unnecessary to me except for the last request. That is to say, if a page instance is touched by many requests before switching to a different page instance, only the last touch has to persist the instance. This is because when user goes back to previous page instance, only the last saved state of that instance will be used.
Based on this assumption, I modified method "storeTouchedPages" of PageStoreManager.java to compare ids of previous page instances (stored in session cache) and touched page instances. If they are different, persist the previous page instances. I tested the modification with several cases of page refreshing/backing and it seems that they all work correctly.
Although I used Wicket for some time, I seldomly digged into Wicket internals. So probably I have missed some important factors when assuming above. Can someone here take a look at attached modification and kindly let me know if this is meaningful?
Thanks
Robin
Re: Possible optimization of page store manager
Posted by Robin Shine <ro...@pmease.com>.
Hi Martin,
Thanks for your comment. My previous test of non-ajax action link only involves model change so version is not changed. If I replace parts of the page or call page.dirty() directly, the version does changed, and this makes my modfication no longer valid as I in this case Wicket directly uses instances cached in session to make the change and this makes it impossible to know version being changed by comparing cached instance and touched instance (as they always point to same object).
Since ajax requests can still be optimized to only save last state of the instance before switching to new instance (although my previous modification does not work), I will further investigate this to see if this can be implemented.
Regards
Robin
--- On Tue, 6/11/13, Martin Grigorov <mg...@apache.org> wrote:
From: Martin Grigorov <mg...@apache.org>
Subject: Re: Possible optimization of page store manager
To: "users@wicket.apache.org" <us...@wicket.apache.org>
Date: Tuesday, June 11, 2013, 7:05 PM
Hi,
On Tue, Jun 11, 2013 at 1:23 PM, Robin Shine <ro...@pmease.com> wrote:
> Hi All,
>
> We are using Wicket to develop our web application product (QuickBuild)
> for six years, and this makes our life a lot easier when comes to maintain
> our code base due to Wicket's component approach and type safe nature of
> Java.
>
> As our product gets more used in companies, it is not uncommon that
> thousands of users are accessing the application concurrently, and at peak
> time the server sometimes gets stressed to have slow response time. Our
> benchmark shows that offen the server is busy serializing pages. This is
> not strange as our application uses ajax heavily and every ajax call to the
> server triggers the page store to persist (serialize and write) the whole
> page on disk.
>
> To my understanding, Wicket serializes/saves stateful pages for purpose of
> back button support. At commit stage of every request, all touched pages
> will be put into session cache, and also get persisted via page store. This
> mechanism works but results in a lot of unnecessary page persistence. To
> explain it, assume below workflow:
> 1. User loads the stateful page and wicket creates a page instance to put
> in session cache, as well as persist to disk.
> 2. User clicks some action links (either ajax or non-ajax) subsequently to
> update parts of the page, but still remains in the same page instance, and
> the url displayed in browser remains unchanged. For every request, Wicket
> updates the page instance, put it into session cache, and finally persist
> it to disk.
>
There is a difference in the behavior between Ajax and non-Ajax request.
When non-Ajax - Wicket creates a new version of the page (unless
#isVersioned() returns false) and stores it. So going back in the history
will go over all versions/states of the page instance.
When Ajax - the version (the pageId) is not incremented and the last state
*overrides* the state in the page store. Later when/if the user goes back
there will be only one display of that page.
> 3. User clicks some other links to cause new page instance being created,
> and Wicket does the same to put new instance in session cache and persist
> the new instance.
>
> Here in step 2, page persistence seems unnecessary to me except for the
> last request. That is to say, if a page instance is touched by many
> requests before switching to a different page instance, only the last touch
> has to persist the instance. This is because when user goes back to
> previous page instance, only the last saved state of that instance will be
> used.
>
> Based on this assumption, I modified method "storeTouchedPages" of
> PageStoreManager.java to compare ids of previous page instances (stored in
> session cache) and touched page instances. If they are different, persist
> the previous page instances. I tested the modification with several cases
> of page refreshing/backing and it seems that they all work correctly.
>
> Although I used Wicket for some time, I seldomly digged into Wicket
> internals. So probably I have missed some important factors when assuming
> above. Can someone here take a look at attached modification and kindly let
> me know if this is meaningful?
>
At the moment serializing of the page (via JavaSerializer) is synchronous,
i.e. it is executed in the http worker thread. The write to the disk is
asynchronous (see AsynchronousDataStore) and is done in a different thread,
so the http thread doesn't have to wait for the actual write to the disk.
Maybe we can make the serialization in a separate thread too.
>
> Thanks
> Robin
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
Re: Possible optimization of page store manager
Posted by Martin Grigorov <mg...@apache.org>.
Hi,
On Tue, Jun 11, 2013 at 1:23 PM, Robin Shine <ro...@pmease.com> wrote:
> Hi All,
>
> We are using Wicket to develop our web application product (QuickBuild)
> for six years, and this makes our life a lot easier when comes to maintain
> our code base due to Wicket's component approach and type safe nature of
> Java.
>
> As our product gets more used in companies, it is not uncommon that
> thousands of users are accessing the application concurrently, and at peak
> time the server sometimes gets stressed to have slow response time. Our
> benchmark shows that offen the server is busy serializing pages. This is
> not strange as our application uses ajax heavily and every ajax call to the
> server triggers the page store to persist (serialize and write) the whole
> page on disk.
>
> To my understanding, Wicket serializes/saves stateful pages for purpose of
> back button support. At commit stage of every request, all touched pages
> will be put into session cache, and also get persisted via page store. This
> mechanism works but results in a lot of unnecessary page persistence. To
> explain it, assume below workflow:
> 1. User loads the stateful page and wicket creates a page instance to put
> in session cache, as well as persist to disk.
> 2. User clicks some action links (either ajax or non-ajax) subsequently to
> update parts of the page, but still remains in the same page instance, and
> the url displayed in browser remains unchanged. For every request, Wicket
> updates the page instance, put it into session cache, and finally persist
> it to disk.
>
There is a difference in the behavior between Ajax and non-Ajax request.
When non-Ajax - Wicket creates a new version of the page (unless
#isVersioned() returns false) and stores it. So going back in the history
will go over all versions/states of the page instance.
When Ajax - the version (the pageId) is not incremented and the last state
*overrides* the state in the page store. Later when/if the user goes back
there will be only one display of that page.
> 3. User clicks some other links to cause new page instance being created,
> and Wicket does the same to put new instance in session cache and persist
> the new instance.
>
> Here in step 2, page persistence seems unnecessary to me except for the
> last request. That is to say, if a page instance is touched by many
> requests before switching to a different page instance, only the last touch
> has to persist the instance. This is because when user goes back to
> previous page instance, only the last saved state of that instance will be
> used.
>
> Based on this assumption, I modified method "storeTouchedPages" of
> PageStoreManager.java to compare ids of previous page instances (stored in
> session cache) and touched page instances. If they are different, persist
> the previous page instances. I tested the modification with several cases
> of page refreshing/backing and it seems that they all work correctly.
>
> Although I used Wicket for some time, I seldomly digged into Wicket
> internals. So probably I have missed some important factors when assuming
> above. Can someone here take a look at attached modification and kindly let
> me know if this is meaningful?
>
At the moment serializing of the page (via JavaSerializer) is synchronous,
i.e. it is executed in the http worker thread. The write to the disk is
asynchronous (see AsynchronousDataStore) and is done in a different thread,
so the http thread doesn't have to wait for the actual write to the disk.
Maybe we can make the serialization in a separate thread too.
>
> Thanks
> Robin
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
Re: Possible optimization of page store manager
Posted by Robin Shine <ro...@pmease.com>.
Hi Martijn,
Yes I will if the modification is valid. Right now Martin pointed to me that non-ajax action link also cause version changes, and my previous test does not cover this as I only updated model in my action link so version is not changed. I will further investigate this.
Regards
Robin
--- On Tue, 6/11/13, Martijn Dashorst <ma...@gmail.com> wrote:
From: Martijn Dashorst <ma...@gmail.com>
Subject: Re: Possible optimization of page store manager
To: users@wicket.apache.org
Date: Tuesday, June 11, 2013, 6:48 PM
Hi Robin,
Thanks for your thoughts! Can you create a JIRA ticket and attach the path
to it? The patch didn't make it through to the mailing list, and things
tend to get lost anyway due to the amount of traffic on this list.
Martijn
On Tue, Jun 11, 2013 at 12:23 PM, Robin Shine <ro...@pmease.com> wrote:
> Hi All,
>
> We are using Wicket to develop our web application product (QuickBuild)
> for six years, and this makes our life a lot easier when comes to maintain
> our code base due to Wicket's component approach and type safe nature of
> Java.
>
> As our product gets more used in companies, it is not uncommon that
> thousands of users are accessing the application concurrently, and at peak
> time the server sometimes gets stressed to have slow response time. Our
> benchmark shows that offen the server is busy serializing pages. This is
> not strange as our application uses ajax heavily and every ajax call to the
> server triggers the page store to persist (serialize and write) the whole
> page on disk.
>
> To my understanding, Wicket serializes/saves stateful pages for purpose of
> back button support. At commit stage of every request, all touched pages
> will be put into session cache, and also get persisted via page store. This
> mechanism works but results in a lot of unnecessary page persistence. To
> explain it, assume below workflow:
> 1. User loads the stateful page and wicket creates a page instance to put
> in session cache, as well as persist to disk.
> 2. User clicks some action links (either ajax or non-ajax) subsequently to
> update parts of the page, but still remains in the same page instance, and
> the url displayed in browser remains unchanged. For every request, Wicket
> updates the page instance, put it into session cache, and finally persist
> it to disk.
> 3. User clicks some other links to cause new page instance being created,
> and Wicket does the same to put new instance in session cache and persist
> the new instance.
>
> Here in step 2, page persistence seems unnecessary to me except for the
> last request. That is to say, if a page instance is touched by many
> requests before switching to a different page instance, only the last touch
> has to persist the instance. This is because when user goes back to
> previous page instance, only the last saved state of that instance will be
> used.
>
> Based on this assumption, I modified method "storeTouchedPages" of
> PageStoreManager.java to compare ids of previous page instances (stored in
> session cache) and touched page instances. If they are different, persist
> the previous page instances. I tested the modification with several cases
> of page refreshing/backing and it seems that they all work correctly.
>
> Although I used Wicket for some time, I seldomly digged into Wicket
> internals. So probably I have missed some important factors when assuming
> above. Can someone here take a look at attached modification and kindly let
> me know if this is meaningful?
>
> Thanks
> Robin
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com
Re: Possible optimization of page store manager
Posted by Martijn Dashorst <ma...@gmail.com>.
Hi Robin,
Thanks for your thoughts! Can you create a JIRA ticket and attach the path
to it? The patch didn't make it through to the mailing list, and things
tend to get lost anyway due to the amount of traffic on this list.
Martijn
On Tue, Jun 11, 2013 at 12:23 PM, Robin Shine <ro...@pmease.com> wrote:
> Hi All,
>
> We are using Wicket to develop our web application product (QuickBuild)
> for six years, and this makes our life a lot easier when comes to maintain
> our code base due to Wicket's component approach and type safe nature of
> Java.
>
> As our product gets more used in companies, it is not uncommon that
> thousands of users are accessing the application concurrently, and at peak
> time the server sometimes gets stressed to have slow response time. Our
> benchmark shows that offen the server is busy serializing pages. This is
> not strange as our application uses ajax heavily and every ajax call to the
> server triggers the page store to persist (serialize and write) the whole
> page on disk.
>
> To my understanding, Wicket serializes/saves stateful pages for purpose of
> back button support. At commit stage of every request, all touched pages
> will be put into session cache, and also get persisted via page store. This
> mechanism works but results in a lot of unnecessary page persistence. To
> explain it, assume below workflow:
> 1. User loads the stateful page and wicket creates a page instance to put
> in session cache, as well as persist to disk.
> 2. User clicks some action links (either ajax or non-ajax) subsequently to
> update parts of the page, but still remains in the same page instance, and
> the url displayed in browser remains unchanged. For every request, Wicket
> updates the page instance, put it into session cache, and finally persist
> it to disk.
> 3. User clicks some other links to cause new page instance being created,
> and Wicket does the same to put new instance in session cache and persist
> the new instance.
>
> Here in step 2, page persistence seems unnecessary to me except for the
> last request. That is to say, if a page instance is touched by many
> requests before switching to a different page instance, only the last touch
> has to persist the instance. This is because when user goes back to
> previous page instance, only the last saved state of that instance will be
> used.
>
> Based on this assumption, I modified method "storeTouchedPages" of
> PageStoreManager.java to compare ids of previous page instances (stored in
> session cache) and touched page instances. If they are different, persist
> the previous page instances. I tested the modification with several cases
> of page refreshing/backing and it seems that they all work correctly.
>
> Although I used Wicket for some time, I seldomly digged into Wicket
> internals. So probably I have missed some important factors when assuming
> above. Can someone here take a look at attached modification and kindly let
> me know if this is meaningful?
>
> Thanks
> Robin
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com