You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@wicket.apache.org by Ard Schrijvers <a....@onehippo.com> on 2008/10/03 15:21:24 UTC
AbstractPageStore memory implementation instead of disk
Hello everybody,
Did anybody perhaps ever implement a memory version of the
AbstractPageStore. Currently, I only see a DiskPageStore, which happens
to be quite a large cpu bottleneck for Windows users AFAICS.
So, before starting to implement one, just wondering whether somebody
has experience on a memory page store version,
Thx for any pointers,
Regards Ard
a.schrijvers@onehippo.com - ard@apache.org - www.onehippo.com
-------------------------------------------------------------
Amsterdam - Hippo B.V. Oosteinde 11 1017 WT Amsterdam +31(0)20-5224466
San Francisco - Hippo USA Inc. 101 H Street, suite Q Petaluma CA
94952-3329 +1 (707) 773-4646
-------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: AbstractPageStore memory implementation instead of disk
Posted by Eelco Hillenius <ee...@gmail.com>.
> Ok, I think I get it :-) One more questionL if I would have sticky
> sessions, and no page back functionality, would I still need wicket to
> serialize?
Technically not, if you really don't need page back functionality (are
you sure you can enforce that?), than it would suffice to just have
the current page of any page maps you use in RAM. And you don't need a
background saving thread in that case either, so I think if you go
about it smart, you can store that page in your session, so that it
gets send over the cluster if you want with not a lot of
configuration. Anyway, make sure that your assumptions about not
having to support the back button are correct.
Eelco
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
RE: AbstractPageStore memory implementation instead of disk
Posted by Ard Schrijvers <a....@onehippo.com>.
> > The overhead of serializing is only there when running
> clustered? The
> > 'page back' functionality also uses it, isn't, so I thought
> it wasn't
> > only for clustered environments (and then specifically for
> clustering
> > without sticky sessions, right?)
> That's not quite what I meant. First it is strongly
> recommended to use sticky sessions when using wicket on
> cluster. Without sticky sessions you have to use redirect to
> render (instead of redirect to buffer) and that certainly
> won't help your performance.
>
> Also on cluster you probably want to have failover (must have
> if you don't use sticky sessions). In that case the changes
> needs to be distributed across cluster. Thus the page must be
> serialized. Wicket reuses the serialized data from the
> serializaton is diskpagestore so on cluster the page is only
> serialized once (instead of twice - once in diskpagestore and
> for session replication).
Ok, I think I get it :-) One more questionL if I would have sticky
sessions, and no page back functionality, would I still need wicket to
serialize?
Regards Ard and thanks for the explanations
>
> -Matej
>
> >
> > -Ard
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: AbstractPageStore memory implementation instead of disk
Posted by Matej Knopp <ma...@gmail.com>.
On Wed, Oct 8, 2008 at 10:30 AM, Ard Schrijvers
<a....@onehippo.com> wrote:
> Hello,
>
>> Again, serialization and writing to filesystem are two
>> completely different things.
>> Are you *really* sure that the writing (which is done in
>> separate thread btw) is really the bottleneck? I have
>
> No, I am not really sure. But, I suspect(ed...i am in doubt now :-) )
> the writing mainly because it seems to be only happening on Windows
> machines. While profiling, I see a lot of cpu time in
>
> org.apache.wicket.protocol.http.SecondLevelCacheSessionStore$SecondLevel
> CachePageMap.put(Page)
>
> See http://people.apache.org/~ard/cpu-profile.html
>
> There I have 2 times
>
> org.apache.wicket.protocol.http.SecondLevelCacheSessionStore$SecondLevel
> CachePageMap.put(Page)
>
> One of 47% and one of 42%. But, as indicated, this very well might all
> just be the serialization part (I now also see some time spend in
> logging for undetached models, oooppsss). Perhaps I was to early with my
> conclusion, but still do not get why Linux machines do not seem the
> suffer from this behavior. Anyway, if it would be the serialization
> only, and not the writing, I would be seeing the (almost) same
> statistics for the Terracotta in memory page store, right?
>
>> profiled this a lot and the serialization takes
>> signifficantly more time than writing the stuff to disk. But
>> the serialization is necessary, plus the overhead is there
>> only until you run your application in clustered environment.
>
> The overhead of serializing is only there when running clustered? The
> 'page back' functionality also uses it, isn't, so I thought it wasn't
> only for clustered environments (and then specifically for clustering
> without sticky sessions, right?)
That's not quite what I meant. First it is strongly recommended to use
sticky sessions when using wicket on cluster. Without sticky sessions
you have to use redirect to render (instead of redirect to buffer) and
that certainly won't help your performance.
Also on cluster you probably want to have failover (must have if you
don't use sticky sessions). In that case the changes needs to be
distributed across cluster. Thus the page must be serialized. Wicket
reuses the serialized data from the serializaton is diskpagestore so
on cluster the page is only serialized once (instead of twice - once
in diskpagestore and for session replication).
-Matej
>
> -Ard
>
>>
>> -Matej
>>
>> > Regards Ard
>> >
>> >> Serialization takes a significant part of request processing, but
>> >> that is necessary.
>> >>
>> >> -Matej
>> >>
>> >> On Fri, Oct 3, 2008 at 3:21 PM, Ard Schrijvers
>> >> <a....@onehippo.com> wrote:
>> >> > Hello everybody,
>> >> >
>> >> > Did anybody perhaps ever implement a memory version of the
>> >> > AbstractPageStore. Currently, I only see a DiskPageStore, which
>> >> > happens to be quite a large cpu bottleneck for Windows
>> users AFAICS.
>> >> >
>> >> > So, before starting to implement one, just wondering
>> >> whether somebody
>> >> > has experience on a memory page store version,
>> >> >
>> >> > Thx for any pointers,
>> >> >
>> >> > Regards Ard
>> >> >
>> >> > a.schrijvers@onehippo.com - ard@apache.org - www.onehippo.com
>> >> > -------------------------------------------------------------
>> >> > Amsterdam - Hippo B.V. Oosteinde 11 1017 WT Amsterdam
>> >> +31(0)20-5224466
>> >> > San Francisco - Hippo USA Inc. 101 H Street, suite Q Petaluma CA
>> >> > 94952-3329 +1 (707) 773-4646
>> >> > -------------------------------------------------------------
>> >> >
>> >> >
>> >>
>> ---------------------------------------------------------------------
>> >> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> >> > For additional commands, e-mail: users-help@wicket.apache.org
>> >> >
>> >> >
>> >>
>> >>
>> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> >> For additional commands, e-mail: users-help@wicket.apache.org
>> >>
>> >>
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> > For additional commands, e-mail: users-help@wicket.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> For additional commands, e-mail: users-help@wicket.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
RE: AbstractPageStore memory implementation instead of disk
Posted by Ard Schrijvers <a....@onehippo.com>.
Hello,
> Again, serialization and writing to filesystem are two
> completely different things.
> Are you *really* sure that the writing (which is done in
> separate thread btw) is really the bottleneck? I have
No, I am not really sure. But, I suspect(ed...i am in doubt now :-) )
the writing mainly because it seems to be only happening on Windows
machines. While profiling, I see a lot of cpu time in
org.apache.wicket.protocol.http.SecondLevelCacheSessionStore$SecondLevel
CachePageMap.put(Page)
See http://people.apache.org/~ard/cpu-profile.html
There I have 2 times
org.apache.wicket.protocol.http.SecondLevelCacheSessionStore$SecondLevel
CachePageMap.put(Page)
One of 47% and one of 42%. But, as indicated, this very well might all
just be the serialization part (I now also see some time spend in
logging for undetached models, oooppsss). Perhaps I was to early with my
conclusion, but still do not get why Linux machines do not seem the
suffer from this behavior. Anyway, if it would be the serialization
only, and not the writing, I would be seeing the (almost) same
statistics for the Terracotta in memory page store, right?
> profiled this a lot and the serialization takes
> signifficantly more time than writing the stuff to disk. But
> the serialization is necessary, plus the overhead is there
> only until you run your application in clustered environment.
The overhead of serializing is only there when running clustered? The
'page back' functionality also uses it, isn't, so I thought it wasn't
only for clustered environments (and then specifically for clustering
without sticky sessions, right?)
-Ard
>
> -Matej
>
> > Regards Ard
> >
> >> Serialization takes a significant part of request processing, but
> >> that is necessary.
> >>
> >> -Matej
> >>
> >> On Fri, Oct 3, 2008 at 3:21 PM, Ard Schrijvers
> >> <a....@onehippo.com> wrote:
> >> > Hello everybody,
> >> >
> >> > Did anybody perhaps ever implement a memory version of the
> >> > AbstractPageStore. Currently, I only see a DiskPageStore, which
> >> > happens to be quite a large cpu bottleneck for Windows
> users AFAICS.
> >> >
> >> > So, before starting to implement one, just wondering
> >> whether somebody
> >> > has experience on a memory page store version,
> >> >
> >> > Thx for any pointers,
> >> >
> >> > Regards Ard
> >> >
> >> > a.schrijvers@onehippo.com - ard@apache.org - www.onehippo.com
> >> > -------------------------------------------------------------
> >> > Amsterdam - Hippo B.V. Oosteinde 11 1017 WT Amsterdam
> >> +31(0)20-5224466
> >> > San Francisco - Hippo USA Inc. 101 H Street, suite Q Petaluma CA
> >> > 94952-3329 +1 (707) 773-4646
> >> > -------------------------------------------------------------
> >> >
> >> >
> >>
> ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> >> > For additional commands, e-mail: users-help@wicket.apache.org
> >> >
> >> >
> >>
> >>
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> >> For additional commands, e-mail: users-help@wicket.apache.org
> >>
> >>
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> > For additional commands, e-mail: users-help@wicket.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: AbstractPageStore memory implementation instead of disk
Posted by Matej Knopp <ma...@gmail.com>.
On Tue, Oct 7, 2008 at 4:24 PM, Ard Schrijvers
<a....@onehippo.com> wrote:
> Hello Matej,
>
>> I'd really be interested in how exactly did you come to
>> conclusion that DiskPageStore writing to disk is the
>> bottleneck. Unless your filesystem caching is broken :)
>
> Hmmm, don't think the filesystem cache is broken. I am working with
> filesystem caches and lucene a lot and don't experience really bad
> performance. Through when profiling our wicket application with yourkit,
> I happened to see a lot of cpu being absorbed in the serialisation /
> deserialisation to filesytem
>
Again, serialization and writing to filesystem are two completely
different things.
Are you *really* sure that the writing (which is done in separate
thread btw) is really the bottleneck? I have profiled this a lot and
the serialization takes signifficantly more time than writing the
stuff to disk. But the serialization is necessary, plus the overhead
is there only until you run your application in clustered environment.
-Matej
> Regards Ard
>
>> Serialization takes a significant part of request processing,
>> but that is necessary.
>>
>> -Matej
>>
>> On Fri, Oct 3, 2008 at 3:21 PM, Ard Schrijvers
>> <a....@onehippo.com> wrote:
>> > Hello everybody,
>> >
>> > Did anybody perhaps ever implement a memory version of the
>> > AbstractPageStore. Currently, I only see a DiskPageStore, which
>> > happens to be quite a large cpu bottleneck for Windows users AFAICS.
>> >
>> > So, before starting to implement one, just wondering
>> whether somebody
>> > has experience on a memory page store version,
>> >
>> > Thx for any pointers,
>> >
>> > Regards Ard
>> >
>> > a.schrijvers@onehippo.com - ard@apache.org - www.onehippo.com
>> > -------------------------------------------------------------
>> > Amsterdam - Hippo B.V. Oosteinde 11 1017 WT Amsterdam
>> +31(0)20-5224466
>> > San Francisco - Hippo USA Inc. 101 H Street, suite Q Petaluma CA
>> > 94952-3329 +1 (707) 773-4646
>> > -------------------------------------------------------------
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> > For additional commands, e-mail: users-help@wicket.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> For additional commands, e-mail: users-help@wicket.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
RE: AbstractPageStore memory implementation instead of disk
Posted by Ard Schrijvers <a....@onehippo.com>.
Hello Matej,
> I'd really be interested in how exactly did you come to
> conclusion that DiskPageStore writing to disk is the
> bottleneck. Unless your filesystem caching is broken :)
Hmmm, don't think the filesystem cache is broken. I am working with
filesystem caches and lucene a lot and don't experience really bad
performance. Through when profiling our wicket application with yourkit,
I happened to see a lot of cpu being absorbed in the serialisation /
deserialisation to filesytem
Regards Ard
> Serialization takes a significant part of request processing,
> but that is necessary.
>
> -Matej
>
> On Fri, Oct 3, 2008 at 3:21 PM, Ard Schrijvers
> <a....@onehippo.com> wrote:
> > Hello everybody,
> >
> > Did anybody perhaps ever implement a memory version of the
> > AbstractPageStore. Currently, I only see a DiskPageStore, which
> > happens to be quite a large cpu bottleneck for Windows users AFAICS.
> >
> > So, before starting to implement one, just wondering
> whether somebody
> > has experience on a memory page store version,
> >
> > Thx for any pointers,
> >
> > Regards Ard
> >
> > a.schrijvers@onehippo.com - ard@apache.org - www.onehippo.com
> > -------------------------------------------------------------
> > Amsterdam - Hippo B.V. Oosteinde 11 1017 WT Amsterdam
> +31(0)20-5224466
> > San Francisco - Hippo USA Inc. 101 H Street, suite Q Petaluma CA
> > 94952-3329 +1 (707) 773-4646
> > -------------------------------------------------------------
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> > For additional commands, e-mail: users-help@wicket.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: AbstractPageStore memory implementation instead of disk
Posted by Matej Knopp <ma...@gmail.com>.
I'd really be interested in how exactly did you come to conclusion
that DiskPageStore writing to disk is the bottleneck. Unless your
filesystem caching is
broken :)
Serialization takes a significant part of request processing, but that
is necessary.
-Matej
On Fri, Oct 3, 2008 at 3:21 PM, Ard Schrijvers
<a....@onehippo.com> wrote:
> Hello everybody,
>
> Did anybody perhaps ever implement a memory version of the
> AbstractPageStore. Currently, I only see a DiskPageStore, which happens
> to be quite a large cpu bottleneck for Windows users AFAICS.
>
> So, before starting to implement one, just wondering whether somebody
> has experience on a memory page store version,
>
> Thx for any pointers,
>
> Regards Ard
>
> a.schrijvers@onehippo.com - ard@apache.org - www.onehippo.com
> -------------------------------------------------------------
> Amsterdam - Hippo B.V. Oosteinde 11 1017 WT Amsterdam +31(0)20-5224466
> San Francisco - Hippo USA Inc. 101 H Street, suite Q Petaluma CA
> 94952-3329 +1 (707) 773-4646
> -------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
RE: AbstractPageStore memory implementation instead of disk
Posted by Ard Schrijvers <a....@onehippo.com>.
Great, I'll check out the code for the TerracottaPageStore,
Thanks a lot Richard,
Regards Ard
>
> The HttpSessionStore is an in memory session store, so it has
> no disk writing bottleneck. It does however have some other
> problems, specifically with the back button i think.
>
> the terracotta page store works like this:
>
> protected ISessionStore newSessionStore()
> {
> return new SecondLevelCacheSessionStore(this,
> new TerracottaPageStore(100));
> }
>
> as it is just an implementation of the IPageStore interface
> as is the DiskPageStore, but where the DiskPageStore writes
> to disk the TerracottaPageStore writes to http session.
>
> The 100 is number of pages to keep in history as far as i
> remember, check the source to be sure.
>
>
> Ard Schrijvers-3 wrote:
> >
> >
> >> but serialization and writing to disk are 2 different things.
> >
> > Yes sorry for the confusion. I intertwine the serialization
> and disk
> > performance (hence asking whether a memory page store
> exists) because
> > AFAIU, it is the serialized pagemaps that are being written to
> > filesystem. Indeed, when using a memory page store, still
> > serialization to memory is needed, so you are indeed right
> to say that
> > they are different. I should talk about the cpu bottleneck
> of writing
> > the serialized pagemaps to disk and whether there is a
> pagestore using
> > memory. As indicated by Richard, this one seems to exist in
> Terracotta.
> > I will try and see whether this avoids my experienced cpu
> bottleneck.
> > Also the virus scanner option I will look into
> >
> > By the way, using
> >
> > protected ISessionStore newSessionStore()
> > {
> > return new HttpSessionStore(this);
> > }
> >
> > Instead of
> >
> > protected ISessionStore newSessionStore()
> > {
> > return new SecondLevelCacheSessionStore(this,
> new DiskPageStore());
> > }
> >
> > Also avoids the disk writing bottleneck. Now, I am not sure whether
> > HttpSessionStore can be used instead of
> SecondLevelCacheSessionStore.
> > It does not seem to write to diks.
> >
> > Thanks for all the pointers and fast responses everybody
> >
> > Regards Ard
> >
> >>
> >>
> >>
> >> On Tue, Oct 7, 2008 at 4:21 PM, Ard Schrijvers
> >> <a....@onehippo.com>wrote:
> >>
> >> > Hello Igor and Timo,
> >> >
> >> > Sorry for my really way to late response, I was caught up
> >> entirely by
> >> > some stressful tasks which needed to be finished.
> >> >
> >> > I currently do not have statistics, but I could have a
> cpu yourkit
> >> > snapshot. I have seen up to 75% cpu in the serialization and
> >> > deserialization. Recently I saw 25%.
> >> >
> >> > People working with Linux do not experience this cpu bottleneck
> >> > though, so it might very well be an issue with how windows
> >> handles the
> >> > filesystem. I will sort out the suggestion by Timo, would
> >> be great if
> >> > that solved the issue.
> >> >
> >> > Thanks a lot, and again, sorry for my late response!
> >> >
> >> > -Ard
> >> >
> >> > >
> >> > > On Fri, 03 Oct 2008, Igor Vaynberg wrote:
> >> > > > do you have some statistics for us? or some profiler timing
> >> > > screenshots?
> >> > >
> >> > > While waiting for the reply from Ard... I remember
> >> hearing of a case
> >> > > where the pagemaps were stored to a directory that the
> >> virus scanner
> >> > > was monitoring, and the scanner hogged all cpu during
> >> load testing
> >> > > :) I suppose that this kind of stuff happens more easily
> >> on Windows
> >> > > than on unices.
> >> > >
> >> > > Best wishes,
> >> > > Timo
> >> > >
> >> > > --
> >> > > Timo Rantalaiho
> >> > > Reaktor Innovations Oy <URL: http://www.ri.fi/ >
> >> > >
> >> > >
> >>
> --------------------------------------------------------------------
> >> > > - To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> >> > > For additional commands, e-mail: users-help@wicket.apache.org
> >> > >
> >> > >
> >> >
> >> >
> >>
> ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> >> > For additional commands, e-mail: users-help@wicket.apache.org
> >> >
> >> >
> >>
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> > For additional commands, e-mail: users-help@wicket.apache.org
> >
> >
> >
>
>
> -----
> http://www.richard-wilkinson.co.uk My blog:
> http://www.richard-wilkinson.co.uk
> --
> View this message in context:
> http://www.nabble.com/AbstractPageStore-memory-implementation-
> instead-of-disk-tp19797628p19860012.html
> Sent from the Wicket - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
RE: AbstractPageStore memory implementation instead of disk
Posted by richardwilko <ri...@gmail.com>.
The HttpSessionStore is an in memory session store, so it has no disk writing
bottleneck. It does however have some other problems, specifically with the
back button i think.
the terracotta page store works like this:
protected ISessionStore newSessionStore()
{
return new SecondLevelCacheSessionStore(this, new
TerracottaPageStore(100));
}
as it is just an implementation of the IPageStore interface as is the
DiskPageStore, but where the DiskPageStore writes to disk the
TerracottaPageStore writes to http session.
The 100 is number of pages to keep in history as far as i remember, check
the source to be sure.
Ard Schrijvers-3 wrote:
>
>
>> but serialization and writing to disk are 2 different things.
>
> Yes sorry for the confusion. I intertwine the serialization and disk
> performance (hence asking whether a memory page store exists) because
> AFAIU, it is the serialized pagemaps that are being written to
> filesystem. Indeed, when using a memory page store, still serialization
> to memory is needed, so you are indeed right to say that they are
> different. I should talk about the cpu bottleneck of writing the
> serialized pagemaps to disk and whether there is a pagestore using
> memory. As indicated by Richard, this one seems to exist in Terracotta.
> I will try and see whether this avoids my experienced cpu bottleneck.
> Also the virus scanner option I will look into
>
> By the way, using
>
> protected ISessionStore newSessionStore()
> {
> return new HttpSessionStore(this);
> }
>
> Instead of
>
> protected ISessionStore newSessionStore()
> {
> return new SecondLevelCacheSessionStore(this, new
> DiskPageStore());
> }
>
> Also avoids the disk writing bottleneck. Now, I am not sure whether
> HttpSessionStore can be used instead of SecondLevelCacheSessionStore. It
> does not seem to write to diks.
>
> Thanks for all the pointers and fast responses everybody
>
> Regards Ard
>
>>
>>
>>
>> On Tue, Oct 7, 2008 at 4:21 PM, Ard Schrijvers
>> <a....@onehippo.com>wrote:
>>
>> > Hello Igor and Timo,
>> >
>> > Sorry for my really way to late response, I was caught up
>> entirely by
>> > some stressful tasks which needed to be finished.
>> >
>> > I currently do not have statistics, but I could have a cpu yourkit
>> > snapshot. I have seen up to 75% cpu in the serialization and
>> > deserialization. Recently I saw 25%.
>> >
>> > People working with Linux do not experience this cpu bottleneck
>> > though, so it might very well be an issue with how windows
>> handles the
>> > filesystem. I will sort out the suggestion by Timo, would
>> be great if
>> > that solved the issue.
>> >
>> > Thanks a lot, and again, sorry for my late response!
>> >
>> > -Ard
>> >
>> > >
>> > > On Fri, 03 Oct 2008, Igor Vaynberg wrote:
>> > > > do you have some statistics for us? or some profiler timing
>> > > screenshots?
>> > >
>> > > While waiting for the reply from Ard... I remember
>> hearing of a case
>> > > where the pagemaps were stored to a directory that the
>> virus scanner
>> > > was monitoring, and the scanner hogged all cpu during
>> load testing
>> > > :) I suppose that this kind of stuff happens more easily
>> on Windows
>> > > than on unices.
>> > >
>> > > Best wishes,
>> > > Timo
>> > >
>> > > --
>> > > Timo Rantalaiho
>> > > Reaktor Innovations Oy <URL: http://www.ri.fi/ >
>> > >
>> > >
>> --------------------------------------------------------------------
>> > > - To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> > > For additional commands, e-mail: users-help@wicket.apache.org
>> > >
>> > >
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> > For additional commands, e-mail: users-help@wicket.apache.org
>> >
>> >
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
>
-----
http://www.richard-wilkinson.co.uk My blog:
http://www.richard-wilkinson.co.uk
--
View this message in context: http://www.nabble.com/AbstractPageStore-memory-implementation-instead-of-disk-tp19797628p19860012.html
Sent from the Wicket - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
RE: AbstractPageStore memory implementation instead of disk
Posted by Ard Schrijvers <a....@onehippo.com>.
> but serialization and writing to disk are 2 different things.
Yes sorry for the confusion. I intertwine the serialization and disk
performance (hence asking whether a memory page store exists) because
AFAIU, it is the serialized pagemaps that are being written to
filesystem. Indeed, when using a memory page store, still serialization
to memory is needed, so you are indeed right to say that they are
different. I should talk about the cpu bottleneck of writing the
serialized pagemaps to disk and whether there is a pagestore using
memory. As indicated by Richard, this one seems to exist in Terracotta.
I will try and see whether this avoids my experienced cpu bottleneck.
Also the virus scanner option I will look into
By the way, using
protected ISessionStore newSessionStore()
{
return new HttpSessionStore(this);
}
Instead of
protected ISessionStore newSessionStore()
{
return new SecondLevelCacheSessionStore(this, new
DiskPageStore());
}
Also avoids the disk writing bottleneck. Now, I am not sure whether
HttpSessionStore can be used instead of SecondLevelCacheSessionStore. It
does not seem to write to diks.
Thanks for all the pointers and fast responses everybody
Regards Ard
>
>
>
> On Tue, Oct 7, 2008 at 4:21 PM, Ard Schrijvers
> <a....@onehippo.com>wrote:
>
> > Hello Igor and Timo,
> >
> > Sorry for my really way to late response, I was caught up
> entirely by
> > some stressful tasks which needed to be finished.
> >
> > I currently do not have statistics, but I could have a cpu yourkit
> > snapshot. I have seen up to 75% cpu in the serialization and
> > deserialization. Recently I saw 25%.
> >
> > People working with Linux do not experience this cpu bottleneck
> > though, so it might very well be an issue with how windows
> handles the
> > filesystem. I will sort out the suggestion by Timo, would
> be great if
> > that solved the issue.
> >
> > Thanks a lot, and again, sorry for my late response!
> >
> > -Ard
> >
> > >
> > > On Fri, 03 Oct 2008, Igor Vaynberg wrote:
> > > > do you have some statistics for us? or some profiler timing
> > > screenshots?
> > >
> > > While waiting for the reply from Ard... I remember
> hearing of a case
> > > where the pagemaps were stored to a directory that the
> virus scanner
> > > was monitoring, and the scanner hogged all cpu during
> load testing
> > > :) I suppose that this kind of stuff happens more easily
> on Windows
> > > than on unices.
> > >
> > > Best wishes,
> > > Timo
> > >
> > > --
> > > Timo Rantalaiho
> > > Reaktor Innovations Oy <URL: http://www.ri.fi/ >
> > >
> > >
> --------------------------------------------------------------------
> > > - To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> > > For additional commands, e-mail: users-help@wicket.apache.org
> > >
> > >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> > For additional commands, e-mail: users-help@wicket.apache.org
> >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: AbstractPageStore memory implementation instead of disk
Posted by Johan Compagner <jc...@gmail.com>.
but serialization and writing to disk are 2 different things.
On Tue, Oct 7, 2008 at 4:21 PM, Ard Schrijvers <a....@onehippo.com>wrote:
> Hello Igor and Timo,
>
> Sorry for my really way to late response, I was caught up entirely by
> some stressful tasks which needed to be finished.
>
> I currently do not have statistics, but I could have a cpu yourkit
> snapshot. I have seen up to 75% cpu in the serialization and
> deserialization. Recently I saw 25%.
>
> People working with Linux do not experience this cpu bottleneck though,
> so it might very well be an issue with how windows handles the
> filesystem. I will sort out the suggestion by Timo, would be great if
> that solved the issue.
>
> Thanks a lot, and again, sorry for my late response!
>
> -Ard
>
> >
> > On Fri, 03 Oct 2008, Igor Vaynberg wrote:
> > > do you have some statistics for us? or some profiler timing
> > screenshots?
> >
> > While waiting for the reply from Ard... I remember hearing of
> > a case where the pagemaps were stored to a directory that the
> > virus scanner was monitoring, and the scanner hogged all cpu
> > during load testing :) I suppose that this kind of stuff
> > happens more easily on Windows than on unices.
> >
> > Best wishes,
> > Timo
> >
> > --
> > Timo Rantalaiho
> > Reaktor Innovations Oy <URL: http://www.ri.fi/ >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> > For additional commands, e-mail: users-help@wicket.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
RE: AbstractPageStore memory implementation instead of disk
Posted by Ard Schrijvers <a....@onehippo.com>.
Hello Igor and Timo,
Sorry for my really way to late response, I was caught up entirely by
some stressful tasks which needed to be finished.
I currently do not have statistics, but I could have a cpu yourkit
snapshot. I have seen up to 75% cpu in the serialization and
deserialization. Recently I saw 25%.
People working with Linux do not experience this cpu bottleneck though,
so it might very well be an issue with how windows handles the
filesystem. I will sort out the suggestion by Timo, would be great if
that solved the issue.
Thanks a lot, and again, sorry for my late response!
-Ard
>
> On Fri, 03 Oct 2008, Igor Vaynberg wrote:
> > do you have some statistics for us? or some profiler timing
> screenshots?
>
> While waiting for the reply from Ard... I remember hearing of
> a case where the pagemaps were stored to a directory that the
> virus scanner was monitoring, and the scanner hogged all cpu
> during load testing :) I suppose that this kind of stuff
> happens more easily on Windows than on unices.
>
> Best wishes,
> Timo
>
> --
> Timo Rantalaiho
> Reaktor Innovations Oy <URL: http://www.ri.fi/ >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
RE: AbstractPageStore memory implementation instead of disk
Posted by richardwilko <ri...@gmail.com>.
I should say that the terracotta page map also does the page serialisation,
but it does it all in memory.
You could probably write out the serialisation bit quite easily, but it is
required for terracotta.
Ard Schrijvers-3 wrote:
>
> Thanks a lot Richard,
>
> I will take a look!
>
> Regards Ard
>
>>
>> The wicket integration with terracotta uses an in-memory page map.
>>
>> The code is here, but will soon be part of the terracotta wicket-tim.
>>
>> http://www.nabble.com/file/p19826206/TerracottaPageStore.java
>> TerracottaPageStore.java
>>
>>
>>
>> Timo Rantalaiho wrote:
>> >
>> > On Fri, 03 Oct 2008, Igor Vaynberg wrote:
>> >> do you have some statistics for us? or some profiler
>> timing screenshots?
>> >
>> > While waiting for the reply from Ard... I remember hearing
>> of a case
>> > where the pagemaps were stored to a directory that the
>> virus scanner
>> > was monitoring, and the scanner hogged all cpu during load
>> testing :)
>> > I suppose that this kind of stuff happens more easily on
>> Windows than
>> > on unices.
>> >
>> > Best wishes,
>> > Timo
>> >
>> > --
>> > Timo Rantalaiho
>> > Reaktor Innovations Oy <URL: http://www.ri.fi/ >
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> > For additional commands, e-mail: users-help@wicket.apache.org
>> >
>> >
>> >
>>
>>
>> -----
>> http://www.richard-wilkinson.co.uk My blog:
>> http://www.richard-wilkinson.co.uk
>> --
>> View this message in context:
>> http://www.nabble.com/AbstractPageStore-memory-implementation-
>> instead-of-disk-tp19797628p19826206.html
>> Sent from the Wicket - User mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> For additional commands, e-mail: users-help@wicket.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
>
-----
http://www.richard-wilkinson.co.uk My blog:
http://www.richard-wilkinson.co.uk
--
View this message in context: http://www.nabble.com/AbstractPageStore-memory-implementation-instead-of-disk-tp19797628p19859483.html
Sent from the Wicket - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
RE: AbstractPageStore memory implementation instead of disk
Posted by Ard Schrijvers <a....@onehippo.com>.
Thanks a lot Richard,
I will take a look!
Regards Ard
>
> The wicket integration with terracotta uses an in-memory page map.
>
> The code is here, but will soon be part of the terracotta wicket-tim.
>
> http://www.nabble.com/file/p19826206/TerracottaPageStore.java
> TerracottaPageStore.java
>
>
>
> Timo Rantalaiho wrote:
> >
> > On Fri, 03 Oct 2008, Igor Vaynberg wrote:
> >> do you have some statistics for us? or some profiler
> timing screenshots?
> >
> > While waiting for the reply from Ard... I remember hearing
> of a case
> > where the pagemaps were stored to a directory that the
> virus scanner
> > was monitoring, and the scanner hogged all cpu during load
> testing :)
> > I suppose that this kind of stuff happens more easily on
> Windows than
> > on unices.
> >
> > Best wishes,
> > Timo
> >
> > --
> > Timo Rantalaiho
> > Reaktor Innovations Oy <URL: http://www.ri.fi/ >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> > For additional commands, e-mail: users-help@wicket.apache.org
> >
> >
> >
>
>
> -----
> http://www.richard-wilkinson.co.uk My blog:
> http://www.richard-wilkinson.co.uk
> --
> View this message in context:
> http://www.nabble.com/AbstractPageStore-memory-implementation-
> instead-of-disk-tp19797628p19826206.html
> Sent from the Wicket - User mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: AbstractPageStore memory implementation instead of disk
Posted by richardwilko <ri...@gmail.com>.
The wicket integration with terracotta uses an in-memory page map.
The code is here, but will soon be part of the terracotta wicket-tim.
http://www.nabble.com/file/p19826206/TerracottaPageStore.java
TerracottaPageStore.java
Timo Rantalaiho wrote:
>
> On Fri, 03 Oct 2008, Igor Vaynberg wrote:
>> do you have some statistics for us? or some profiler timing screenshots?
>
> While waiting for the reply from Ard... I remember hearing
> of a case where the pagemaps were stored to a directory that
> the virus scanner was monitoring, and the scanner hogged all
> cpu during load testing :) I suppose that this kind of
> stuff happens more easily on Windows than on unices.
>
> Best wishes,
> Timo
>
> --
> Timo Rantalaiho
> Reaktor Innovations Oy <URL: http://www.ri.fi/ >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
>
-----
http://www.richard-wilkinson.co.uk My blog:
http://www.richard-wilkinson.co.uk
--
View this message in context: http://www.nabble.com/AbstractPageStore-memory-implementation-instead-of-disk-tp19797628p19826206.html
Sent from the Wicket - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: AbstractPageStore memory implementation instead of disk
Posted by Timo Rantalaiho <Ti...@ri.fi>.
On Fri, 03 Oct 2008, Igor Vaynberg wrote:
> do you have some statistics for us? or some profiler timing screenshots?
While waiting for the reply from Ard... I remember hearing
of a case where the pagemaps were stored to a directory that
the virus scanner was monitoring, and the scanner hogged all
cpu during load testing :) I suppose that this kind of
stuff happens more easily on Windows than on unices.
Best wishes,
Timo
--
Timo Rantalaiho
Reaktor Innovations Oy <URL: http://www.ri.fi/ >
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: AbstractPageStore memory implementation instead of disk
Posted by Igor Vaynberg <ig...@gmail.com>.
do you have some statistics for us? or some profiler timing screenshots?
-igor
On Fri, Oct 3, 2008 at 6:21 AM, Ard Schrijvers
<a....@onehippo.com> wrote:
> Hello everybody,
>
> Did anybody perhaps ever implement a memory version of the
> AbstractPageStore. Currently, I only see a DiskPageStore, which happens
> to be quite a large cpu bottleneck for Windows users AFAICS.
>
> So, before starting to implement one, just wondering whether somebody
> has experience on a memory page store version,
>
> Thx for any pointers,
>
> Regards Ard
>
> a.schrijvers@onehippo.com - ard@apache.org - www.onehippo.com
> -------------------------------------------------------------
> Amsterdam - Hippo B.V. Oosteinde 11 1017 WT Amsterdam +31(0)20-5224466
> San Francisco - Hippo USA Inc. 101 H Street, suite Q Petaluma CA
> 94952-3329 +1 (707) 773-4646
> -------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org