You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@velocity.apache.org by dave <da...@miceda-data.com> on 2000/08/26 00:38:09 UTC
Re: Velocity Service- Caching
On Fri, 25 Aug 2000, you wrote:
> On Fri, 25 Aug 2000 daveb@miceda-data.com wrote:
> I'm going to work on caching mechanism in Velocity
> when I finish with Torque today. So if you think
> about how you would ideally like to interact with
> Velocity then I can try to accommodate your needs/wishes.
I 'm working on the FileTemplateLoader for Velocity tonight. I planned on
using a caching mechanism within the loader that works similar to the way a
web server loads files. The caching mechanism will keep a last modified
stamp on the file and reload as necessary. I need this before I start the
service stuff.
How's this sound?
--
dave
daveb@miceda-data.com
----------------------
Re: Velocity Service- Caching
Posted by "Daniel L. Rall" <dl...@finemaltcoding.com>.
dave wrote:
>
> On Fri, 25 Aug 2000, you wrote:
> > On Fri, 25 Aug 2000 daveb@miceda-data.com wrote:
> > I'm going to work on caching mechanism in Velocity
> > when I finish with Torque today. So if you think
> > about how you would ideally like to interact with
> > Velocity then I can try to accommodate your needs/wishes.
>
> I 'm working on the FileTemplateLoader for Velocity tonight. I planned on
> using a caching mechanism within the loader that works similar to the way a
> web server loads files. The caching mechanism will keep a last modified
> stamp on the file and reload as necessary. I need this before I start the
> service stuff.
>
> How's this sound?
Hmmm, depends on how often you touch the filesystem to determine whether
a template needs to be refreshed. Avoid disk I/O as much as possible.
Perhaps time between checks could be a preference?
--
Daniel Rall <dl...@finemaltcoding.com>
Re: Velocity Service- Caching
Posted by "Daniel L. Rall" <dl...@finemaltcoding.com>.
dave wrote:
> [snip]
> > I would prefer that. Would serialization to disk be useful for server
> > restarts?
>
> Yes the checkpoint dumps to disk and the caching mechanism reloads on
> server start. I'm thinking along the lines of the way a main-memory
> database works.
>
> I think this would be especially powerful for pre-compiled templates. With
> the proper data stucture, you could hold a butt load of template objects
> in memory.
+1, sounds perfect.
--
Daniel Rall <dl...@finemaltcoding.com>
Re: Velocity Service- Caching
Posted by dave <da...@miceda-data.com>.
On Fri, 25 Aug 2000, you wrote:
> dave wrote:
> >
> > On Fri, 25 Aug 2000, you wrote:
> > > Of course...it's the AST node structure that needs to be serialized.
> > > But how expensive will the serialization/deserialization be? It might
> > > be overkill for only shallowly nested templates, or templates with only
> > > a single token in them. Perhaps the AST node data (i.e. the "compiled"
> > > template) should only be written to disk if it hasn't been used for a
> > > given period of time. This would reduce disk I/O by keeping the
> > > compiled template around in memory for awhile.
> >
> > Here's a crazy thought - what's the real overhead of keeping the templates
> > in memory all the time? You could use a checkingpointing mechanism
> > to periodically write to disk to save any physical changes.
>
> I would prefer that. Would serialization to disk be useful for server
> restarts?
Yes the checkpoint dumps to disk and the caching mechanism reloads on
server start. I'm thinking along the lines of the way a main-memory
database works.
I think this would be especially powerful for pre-compiled templates. With
the proper data stucture, you could hold a butt load of template objects
in memory.
--
dave
daveb@miceda-data.com
----------------------
Re: Velocity Service- Caching
Posted by "Daniel L. Rall" <dl...@finemaltcoding.com>.
dave wrote:
>
> On Fri, 25 Aug 2000, you wrote:
> > Of course...it's the AST node structure that needs to be serialized.
> > But how expensive will the serialization/deserialization be? It might
> > be overkill for only shallowly nested templates, or templates with only
> > a single token in them. Perhaps the AST node data (i.e. the "compiled"
> > template) should only be written to disk if it hasn't been used for a
> > given period of time. This would reduce disk I/O by keeping the
> > compiled template around in memory for awhile.
>
> Here's a crazy thought - what's the real overhead of keeping the templates
> in memory all the time? You could use a checkingpointing mechanism
> to periodically write to disk to save any physical changes.
I would prefer that. Would serialization to disk be useful for server
restarts?
--
Daniel Rall <dl...@finemaltcoding.com>
Re: Velocity Service- Caching
Posted by dave <da...@miceda-data.com>.
On Fri, 25 Aug 2000, you wrote:
> Of course...it's the AST node structure that needs to be serialized.
> But how expensive will the serialization/deserialization be? It might
> be overkill for only shallowly nested templates, or templates with only
> a single token in them. Perhaps the AST node data (i.e. the "compiled"
> template) should only be written to disk if it hasn't been used for a
> given period of time. This would reduce disk I/O by keeping the
> compiled template around in memory for awhile.
Here's a crazy thought - what's the real overhead of keeping the templates
in memory all the time? You could use a checkingpointing mechanism
to periodically write to disk to save any physical changes.
--
dave
daveb@miceda-data.com
----------------------
Re: Velocity Service- Caching
Posted by Jon Stevens <jo...@latchkey.com>.
lets not CC the turbine list. lets keep the discussion here. ok? thanks.
-jon
--
Scarab -
Java Servlet Based - Open Source
Bug/Issue Tracking System
<http://scarab.tigris.org/>
Re: Velocity Service- Caching
Posted by "Daniel L. Rall" <dl...@finemaltcoding.com>.
Jon Stevens wrote:
>
> on 8/25/2000 3:38 PM, "dave" <da...@miceda-data.com> wrote:
>
> > I 'm working on the FileTemplateLoader for Velocity tonight. I planned on
> > using a caching mechanism within the loader that works similar to the way a
> > web server loads files. The caching mechanism will keep a last modified
> > stamp on the file and reload as necessary. I need this before I start the
> > service stuff.
> >
> > How's this sound?
>
> 0.
>
> We want to be able to expand the whole process to include compiled templates
> that have been pre-parsed (somehow, you need to get a reference to the AST
> tree). The way that I was envisioning this to work is essentially a
> serialized byte[] of an object to disk as being the compiled version. So,
> what I would like is for this caching service to wrap each template into an
> object that can be serialized and then the internal code within Velocity
> should read this object from memory or get it from disk.
>
> did that make any sense? :-)
Of course...it's the AST node structure that needs to be serialized.
But how expensive will the serialization/deserialization be? It might
be overkill for only shallowly nested templates, or templates with only
a single token in them. Perhaps the AST node data (i.e. the "compiled"
template) should only be written to disk if it hasn't been used for a
given period of time. This would reduce disk I/O by keeping the
compiled template around in memory for awhile.
--
Daniel Rall <dl...@finemaltcoding.com>
Re: Velocity Service- Caching
Posted by Jon Stevens <jo...@latchkey.com>.
on 8/25/2000 4:46 PM, "Daniel L. Rall" <dl...@finemaltcoding.com> wrote:
> Is the GlobalCache high enough performance? Jon had some--good--reason
> for why I shouldn't use the GlobalCache as the cache for the
> ConnectionPool.
In the case where you don't know the size of the cache, then you need to use
the GlobalCache. In the case of the ConnectionPool, you know the max size
because it is defined as a property. It is essentially the difference
between using a Stack and a Vector.
-jon
--
Scarab -
Java Servlet Based - Open Source
Bug/Issue Tracking System
<http://scarab.tigris.org/>
Re: Velocity Service- Caching
Posted by "Daniel L. Rall" <dl...@finemaltcoding.com>.
Jason van Zyl wrote:
> [snip]
> After the initial parsing what you are left with is
> a Template object that has a body of text with a
> series of dynamic content insertion points.
>
> For each one of those points there will be
> a corresponding Object to insert the content
> at that point. All these object might implement
> an insertContent() or something like that.
>
> So there is the content body, and a series of
> content inserters.
>
> And I have plans to try and "flatten" as much
> as possible, for example if I see something like
>
> #set $foo = "bar"
>
> Then I know that because the RHS is a string literal
> it will never change. The same goes with methods
> with string literal parameters:
>
> $object.method("param1", "param2");
Yum, a real compiler!
> Somewhere in the content insertion object I will
> have a container for a Method object with the
> parameters "param1" and "param2" stored.
>
> I'd like to try and detect as much staticity as
> possible, and what remains will always have to
> refreshed.
>
> So stick the Template object in the GlobalCache,
> then nab the content out of Template object and
> pass through content insertion points plugging
> in the required content.
Is the GlobalCache high enough performance? Jon had some--good--reason
for why I shouldn't use the GlobalCache as the cache for the
ConnectionPool.
> I would like to add options to Velocity where
> you can mark things as static/cacheable so that
> it would be possible to have near static content
> serving if desired.
+1, I already have applications for this. =)
--
Daniel Rall <dl...@finemaltcoding.com>
Re: Velocity Service- Caching
Posted by Jason van Zyl <jv...@periapt.com>.
On Fri, 25 Aug 2000, Jon Stevens wrote:
> on 8/25/2000 3:38 PM, "dave" <da...@miceda-data.com> wrote:
>
> > I 'm working on the FileTemplateLoader for Velocity tonight. I planned on
> > using a caching mechanism within the loader that works similar to the way a
> > web server loads files. The caching mechanism will keep a last modified
> > stamp on the file and reload as necessary. I need this before I start the
> > service stuff.
> >
> > How's this sound?
>
> 0.
>
> We want to be able to expand the whole process to include compiled templates
> that have been pre-parsed (somehow, you need to get a reference to the AST
> tree). The way that I was envisioning this to work is essentially a
> serialized byte[] of an object to disk as being the compiled version. So,
> what I would like is for this caching service to wrap each template into an
> object that can be serialized and then the internal code within Velocity
> should read this object from memory or get it from disk.
We'll hash this out ... this is what I was planning for
caching/compiling
After the initial parsing what you are left with is
a Template object that has a body of text with a
series of dynamic content insertion points.
For each one of those points there will be
a corresponding Object to insert the content
at that point. All these object might implement
an insertContent() or something like that.
So there is the content body, and a series of
content inserters.
And I have plans to try and "flatten" as much
as possible, for example if I see something like
#set $foo = "bar"
Then I know that because the RHS is a string literal
it will never change. The same goes with methods
with string literal parameters:
$object.method("param1", "param2");
Somewhere in the content insertion object I will
have a container for a Method object with the
parameters "param1" and "param2" stored.
I'd like to try and detect as much staticity as
possible, and what remains will always have to
refreshed.
So stick the Template object in the GlobalCache,
then nab the content out of Template object and
pass through content insertion points plugging
in the required content.
I would like to add options to Velocity where
you can mark things as static/cacheable so that
it would be possible to have near static content
serving if desired.
jvz.
--
Jason van Zyl
jvanzyl@periapt.com
Re: Velocity Service- Caching
Posted by Jon Stevens <jo...@latchkey.com>.
on 8/25/2000 3:38 PM, "dave" <da...@miceda-data.com> wrote:
> I 'm working on the FileTemplateLoader for Velocity tonight. I planned on
> using a caching mechanism within the loader that works similar to the way a
> web server loads files. The caching mechanism will keep a last modified
> stamp on the file and reload as necessary. I need this before I start the
> service stuff.
>
> How's this sound?
0.
We want to be able to expand the whole process to include compiled templates
that have been pre-parsed (somehow, you need to get a reference to the AST
tree). The way that I was envisioning this to work is essentially a
serialized byte[] of an object to disk as being the compiled version. So,
what I would like is for this caching service to wrap each template into an
object that can be serialized and then the internal code within Velocity
should read this object from memory or get it from disk.
did that make any sense? :-)
In other words, what you have proposed is a good start, but don't forget to
include storing the compiled templates which are just pre-parsed templates
serialized to disk.
-jon
--
Scarab -
Java Servlet Based - Open Source
Bug/Issue Tracking System
<http://scarab.tigris.org/>
Re: Velocity Service- Caching
Posted by Jason van Zyl <jv...@periapt.com>.
On Fri, 25 Aug 2000, dave wrote:
> On Fri, 25 Aug 2000, you wrote:
> > On Fri, 25 Aug 2000 daveb@miceda-data.com wrote:
> > I'm going to work on caching mechanism in Velocity
> > when I finish with Torque today. So if you think
> > about how you would ideally like to interact with
> > Velocity then I can try to accommodate your needs/wishes.
>
> I 'm working on the FileTemplateLoader for Velocity tonight. I planned on
> using a caching mechanism within the loader that works similar to the way a
> web server loads files. The caching mechanism will keep a last modified
> stamp on the file and reload as necessary. I need this before I start the
> service stuff.
>
> How's this sound?
The actual template might not change, but the context might.
So there might always be a portion of the template gets
generated dynamically.
For now I think what you are suggesting will work, but I
don't think it will detect changes in the context. Right
now you would probably have to rebuild it every time for
it to be accurate. But that would be fantastic for now!
I going to be working on the caching/compiled template
thing this weekend. So we'll get this sorted out
quickly.
jvz.
--
Jason van Zyl
jvanzyl@periapt.com