You are viewing a plain text version of this content. The canonical link for it is here.
Posted to asp@perl.apache.org by raystormphd <st...@iparadigms.com> on 2003/06/06 03:31:59 UTC
session data slowdown
Hello,
In an attempt to offload SELECT queries from our database and decrease
the time spent gathering data, we have begun to aggressively cache
data in users' sessions (StateDB=MLDBM::Sync::SDBM_File,
StateSerializer=Data::Dumper). Unfortunately, when the data set is
too large the process of reading/thawing the data becomes painfully
slow. Switching to DB_File didn't seem to help. We haven't tried
Storable but my inclination is that it wouldn't make that much of a
difference. The session files are on the order of a 100k.
1) Is the thawing of the data to blame?
2) How might we speed this process up or are we heading down a blind
alley?
3) Lastly, I have heard good things about IPC:MM (fastest gun in the
west when it comes to using Shared memory for a cache). Could
this be used instead of MLDBM::Sync::SDBM_File?
Any hints or help would be appreciated!
Christian
---------------------------------------------------------------------
To unsubscribe, e-mail: asp-unsubscribe@perl.apache.org
For additional commands, e-mail: asp-help@perl.apache.org
Re: session data slowdown
Posted by Josh Chamas <jo...@chamas.com>.
raystormphd wrote:
>
> data1=$Session->{'key1'}
> data2=$Session->{'key2'}
> data3=$Session->{'key3'}
>
> Does each statement cause a freeze/thaw of the whole Session data
> structure to/from disk?
>
No, each one does its own freeze/thaw just for that key / value pair.
> In your code, you make one read only copy. Does this cause only one
> thaw? This is why your proposed trick would help, right?
>
The idea behind that is if you are just using it to look up lots of
global values & db data, which is not recommended anyway, you might
just init an in memory hash to reference in your code instead of
going to the $Session each time, so perhaps better to...
$Session->Lock;
$SessionCopy = { %$Session };
$Session->UnLock;
...
$SessionCopy->{KEY};
$SessionCopy->{KEY};
$SessionCopy->{KEY};
than to:
$Session->{KEY}
$Session->{KEY}
$Session->{KEY}
but really better is to do this...
my $value = $Session->{KEY};
$value;
$value;
Like I said, a lot of it depends on how you are programming. But overall,
you want to make as few calls getting as little data from $Session as possible.
Regards,
Josh
________________________________________________________________
Josh Chamas, Founder phone:925-552-0128
Chamas Enterprises Inc. http://www.chamas.com
NodeWorks Link Checking http://www.nodeworks.com
---------------------------------------------------------------------
To unsubscribe, e-mail: asp-unsubscribe@perl.apache.org
For additional commands, e-mail: asp-help@perl.apache.org
Re: session data slowdown
Posted by raystormphd <st...@iparadigms.com>.
Josh,
Thank you for your insite into our problem. We have some more
ammunition to play with now.
> > 2) How might we speed this process up or are we heading down
blind
> > alley?
> >
>
> Try SessionSerialize setting, similar to calling $Session->Lock in
the
> Script_OnStart. Try not accessing keys too often. You might try to
> make a read only copy of data like this:
>
> use vars qw($SessionReadOnly);
> sub Script_OnStart {
> $SessionReadOnly = { %$Session };
> }
>
> so if you are accessing same data multiple times, this might be
faster.
This brings up another set of questions I have had regarding Session
data. I have tried finding the answer in posts/docs/etc. but haven't
found a definitive answer.
How does a series of reads or writes to a Session object hit the disk
and the StateSerializer?
For instance, let's say I did this:
$Session->{'key1'}=data1
$Session->{'key2'}=data2
$Session->{'key3'}=data3
or
data1=$Session->{'key1'}
data2=$Session->{'key2'}
data3=$Session->{'key3'}
Does each statement cause a freeze/thaw of the whole Session data
structure to/from disk?
In your code, you make one read only copy. Does this cause only one
thaw? This is why your proposed trick would help, right?
Cheers,
Christian
---------------------------------------------------------------------
To unsubscribe, e-mail: asp-unsubscribe@perl.apache.org
For additional commands, e-mail: asp-help@perl.apache.org
Re: session data slowdown
Posted by Josh Chamas <jo...@chamas.com>.
raystormphd wrote:
> Hello,
>
> In an attempt to offload SELECT queries from our database and decrease
> the time spent gathering data, we have begun to aggressively cache
> data in users' sessions (StateDB=MLDBM::Sync::SDBM_File,
> StateSerializer=Data::Dumper). Unfortunately, when the data set is
> too large the process of reading/thawing the data becomes painfully
> slow. Switching to DB_File didn't seem to help. We haven't tried
> Storable but my inclination is that it wouldn't make that much of a
> difference. The session files are on the order of a 100k.
>
Storable might help. It might be twice as fast as Dumper for large
data sets like this.
> 1) Is the thawing of the data to blame?
>
Yes. This really is too much data I would say. You should really
just try to store small bits in $Session. Any global data might be
better to cache in a global data structure, or even use the CacheDB
and includes caching.
> 2) How might we speed this process up or are we heading down a blind
> alley?
>
Try SessionSerialize setting, similar to calling $Session->Lock in the
Script_OnStart. Try not accessing keys too often. You might try to
make a read only copy of data like this:
use vars qw($SessionReadOnly);
sub Script_OnStart {
$SessionReadOnly = { %$Session };
}
so if you are accessing same data multiple times, this might be faster.
> 3) Lastly, I have heard good things about IPC:MM (fastest gun in the
> west when it comes to using Shared memory for a cache). Could
> this be used instead of MLDBM::Sync::SDBM_File?
>
No, but you could have your own cache layer reference per user
by $Session->SessionID. You might also try using Tie::TextDir either
for StateDB or for CacheDB ( if you start includes caching ).
Note that if you have a really fast local database like MySQL, you
might be better off not caching at all & lettings most queries hit
the database.
> Any hints or help would be appreciated!
>
My hints were very generic. A lot of it has to do with exactly how you are
using $Session programmatically.
Regards,
Josh
________________________________________________________________
Josh Chamas, Founder phone:925-552-0128
Chamas Enterprises Inc. http://www.chamas.com
NodeWorks Link Checking http://www.nodeworks.com
---------------------------------------------------------------------
To unsubscribe, e-mail: asp-unsubscribe@perl.apache.org
For additional commands, e-mail: asp-help@perl.apache.org