You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@velocity.apache.org by Ståle Undheim <st...@coretrek.com> on 2006/04/04 13:44:10 UTC
Velocity vs. WebMacro performance
I am currently working on a project where we are evaluating wheter to
use Velocity or WebMacro. I did some performance test in relation to how
we intend to use either of the parsing engines.
We are making a Web UI framework, and its very much based on nesting
components. IE. you have a frame, and inside it you have various
sections, which again contains components. So you end up with nesting
alot of components and their associated template files.
In order to test things the way we used it, I made a simple Class that
would have an associated VM/WM file, and recurse several levels deep.
Full code can be found here:
http://apollo.coretrek.com/~staaleu/VelocityPerformance.tar.gz
Dependent packages are here:
http://apollo.coretrek.com/~staaleu/
There is a slight difference in code from WM 1.1 to 2.0, commented on
line 91 of NestedObject.java
The basic constructor of the class is as follows:
________________________________
public NestedObject() {
this(0);
}
private NestedObject(int level) {
_level = level;
for (int i=0; i<level; i++) {
_indentation+=" ";
}
if (_level == LEVELS) {
_nested = Collections.EMPTY_LIST;
}
else {
_nested = new ArrayList(CHILD_COUNT);
for (int i = 0; i < CHILD_COUNT; i++) {
_nested.add(new NestedObject(_level + 1));
}
}
}
________________________________
So creating a NestedObject, creates a tree of child objects. The Class
has an associated VM/WM file that looks like this:
________________________________
$this.getIndentation()START LEVEL $this.Level
#foreach ($obj in $this.Nested)
$obj
#end
$this.getIndentation()END LEVEL $this.Level
________________________________
Basically recursing several levels deep.
For the WebMacro part, I use the Macro interface, which means I get an
outputstream in to write each object to. For Velocity I had to use the
toString() method, so a seperate String would be created for each level.
Here are the test results as I got them:
WebMacro 1.1 init memoryConsumption: 1346552
Run: 0, memoryConsumption: 926968, time: 439ms, size: 67179
Run: 10, memoryConsumption: 467648, time: 54ms, size: 67179
Run: 20, memoryConsumption: 468128, time: 54ms, size: 67179
Run: 30, memoryConsumption: 468320, time: 54ms, size: 67179
Run: 40, memoryConsumption: 468320, time: 64ms, size: 67179
Run: 50, memoryConsumption: 468128, time: 56ms, size: 67179
Run: 60, memoryConsumption: 468320, time: 66ms, size: 67179
Run: 70, memoryConsumption: 468320, time: 57ms, size: 67179
Run: 80, memoryConsumption: 468128, time: 55ms, size: 67179
Run: 90, memoryConsumption: 468320, time: 54ms, size: 67179
Run: 100, memoryConsumption: 468320, time: 54ms, size: 67179
Total time for WebMacro: 7416
WebMacro 2.0 init memoryConsumption: 456808
Run: 0, memoryConsumption: 713456, time: 1668ms, size: 67179
Run: 10, memoryConsumption: 608160, time: 145ms, size: 67179
Run: 20, memoryConsumption: 608208, time: 346ms, size: 67179
Run: 30, memoryConsumption: 608160, time: 157ms, size: 67179
Run: 40, memoryConsumption: 580208, time: 154ms, size: 67179
Run: 50, memoryConsumption: 608160, time: 146ms, size: 67179
Run: 60, memoryConsumption: 608208, time: 1534ms, size: 67179
Run: 70, memoryConsumption: 608400, time: 144ms, size: 67179
Run: 80, memoryConsumption: 580208, time: 149ms, size: 67179
Run: 90, memoryConsumption: 608160, time: 143ms, size: 67179
Run: 100, memoryConsumption: 608208, time: 146ms, size: 67179
Total time for WebMacro2: 20406
Velocity 1.4 init memoryConsumption: 2280416
Run: 0, memoryConsumption: 1872776, time: 2063ms, size: 67179
Run: 10, memoryConsumption: 1729616, time: 1074ms, size: 67179
Run: 20, memoryConsumption: 1707664, time: 1037ms, size: 67179
Run: 30, memoryConsumption: 1707624, time: 1076ms, size: 67179
Run: 40, memoryConsumption: 1707624, time: 1414ms, size: 67179
Run: 50, memoryConsumption: 1707624, time: 1018ms, size: 67179
Run: 60, memoryConsumption: 1707624, time: 1037ms, size: 67179
Run: 70, memoryConsumption: 1707576, time: 1152ms, size: 67179
Run: 80, memoryConsumption: 1707624, time: 1090ms, size: 67179
Run: 90, memoryConsumption: 1707624, time: 1501ms, size: 67179
Run: 100, memoryConsumption: 1707640, time: 1032ms, size: 67179
Total time for Velocity: 130396
Please note that WebMacro 2.0 produces errornous output, but thats not
for this list.
Am I using Velocity totally wrong? Is there interfaces that allows me to
nest objects and have good performance?
Here is the full code for the NestedObject, which handles the output:
http://apollo.coretrek.com/~staaleu/NestedObject.java.txt
--
Ståle Undheim
Systemutvikler
CoreTrek
Tlf.: +47 51 97 85 97
E-post: staaleu@coretrek.no
WEB: www.coretrek.no
"Simple is beautiful"
Re: How to use DateTool
Posted by Dominik Bruhn <do...@dbruhn.de>.
Ah, ok, now I understand, I thought these functions would be applied to the
Date-Object automatically.
Thanks a lot
Dominik
Am Dienstag 04 April 2006 21:33 schrieb Nathan Bubna:
> Well, you have to actually use the tool in your templates, not just
> add it to the context. It doesn't magically format your dates for
> you. You have to tell it to format them (e.g. $date.format('long',
> $bu.LastVisited) ). The documentation provides examples of how to do
> this and a thorough explanation of the methods available to you.
>
> On 4/4/06, Dominik Bruhn <do...@dbruhn.de> wrote:
> > Hy,
> > thanks to your response. I added the proposed code to my servlet but
> > nothing happens. I use the following code to output the timestamp:
> > $bu.LastVisited
> >
> > The function "getLastVisited" of the Class of $bu returns a
> > "Date"-Object. This outputs the date in "normal" format, no matter
> > whether DateTool is embedded or not. With embedded DateTool I tried the
> > "$bu.LastVisited.long" but it simply outputs the string
> > "$bu.LastVisited.long" and not the formated date.
> >
> > So how do I use this tool?
> >
> > Thanks
> > TO
> >
> > Am Dienstag 04 April 2006 19:47 schrieb Nathan Bubna:
> > > You don't have to use a toolbox to use any of the Generic Tools. Just
> > > create an instance and add it to your context before the context is
> > > merged with the template:
> > >
> > > context.put("date", new DateTool());
> > >
> > > pretty simple.
> > >
> > > On 4/4/06, Dominik Bruhn <do...@dbruhn.de> wrote:
> > > > Hy,
> > > > I want to use the DateTool (
> > > > http://jakarta.apache.org/velocity/tools/javadoc/org/apache/velocity/
> > > >tool s/generic/DateTool.html ) in my VelocityApplet. I not using
> > > > Struts. How do I tell Velocity to use this Tool when outputting
> > > > Dates. Some tell me about a toolbox.xml, but I don't know hwere to
> > > > place it and with which content. A simple copy of the jar-file to the
> > > > classpath didn't help.
> > > >
> > > > Who can help?
> > > >
> > > > Thanks
> > > > Dominik
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail:
> > > > velocity-user-help@jakarta.apache.org
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: How to use DateTool
Posted by Nathan Bubna <nb...@gmail.com>.
Well, you have to actually use the tool in your templates, not just
add it to the context. It doesn't magically format your dates for
you. You have to tell it to format them (e.g. $date.format('long',
$bu.LastVisited) ). The documentation provides examples of how to do
this and a thorough explanation of the methods available to you.
On 4/4/06, Dominik Bruhn <do...@dbruhn.de> wrote:
> Hy,
> thanks to your response. I added the proposed code to my servlet but nothing
> happens. I use the following code to output the timestamp:
> $bu.LastVisited
>
> The function "getLastVisited" of the Class of $bu returns a "Date"-Object.
> This outputs the date in "normal" format, no matter whether DateTool is
> embedded or not. With embedded DateTool I tried the "$bu.LastVisited.long"
> but it simply outputs the string "$bu.LastVisited.long" and not the formated
> date.
>
> So how do I use this tool?
>
> Thanks
> TO
>
>
>
> Am Dienstag 04 April 2006 19:47 schrieb Nathan Bubna:
> > You don't have to use a toolbox to use any of the Generic Tools. Just
> > create an instance and add it to your context before the context is
> > merged with the template:
> >
> > context.put("date", new DateTool());
> >
> > pretty simple.
> >
> > On 4/4/06, Dominik Bruhn <do...@dbruhn.de> wrote:
> > > Hy,
> > > I want to use the DateTool (
> > > http://jakarta.apache.org/velocity/tools/javadoc/org/apache/velocity/tool
> > >s/generic/DateTool.html ) in my VelocityApplet. I not using Struts. How do
> > > I tell Velocity to use this Tool when outputting Dates. Some tell me
> > > about a toolbox.xml, but I don't know hwere to place it and with which
> > > content. A simple copy of the jar-file to the classpath didn't help.
> > >
> > > Who can help?
> > >
> > > Thanks
> > > Dominik
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: How to use DateTool
Posted by Mike Kienenberger <mk...@gmail.com>.
On 4/4/06, Dominik Bruhn <do...@dbruhn.de> wrote:
> So how do I use this tool?
http://jakarta.apache.org/velocity/tools/javadoc/org/apache/velocity/tools/generic/DateTool.html
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: How to use DateTool
Posted by Dominik Bruhn <do...@dbruhn.de>.
Hy,
thanks to your response. I added the proposed code to my servlet but nothing
happens. I use the following code to output the timestamp:
$bu.LastVisited
The function "getLastVisited" of the Class of $bu returns a "Date"-Object.
This outputs the date in "normal" format, no matter whether DateTool is
embedded or not. With embedded DateTool I tried the "$bu.LastVisited.long"
but it simply outputs the string "$bu.LastVisited.long" and not the formated
date.
So how do I use this tool?
Thanks
TO
Am Dienstag 04 April 2006 19:47 schrieb Nathan Bubna:
> You don't have to use a toolbox to use any of the Generic Tools. Just
> create an instance and add it to your context before the context is
> merged with the template:
>
> context.put("date", new DateTool());
>
> pretty simple.
>
> On 4/4/06, Dominik Bruhn <do...@dbruhn.de> wrote:
> > Hy,
> > I want to use the DateTool (
> > http://jakarta.apache.org/velocity/tools/javadoc/org/apache/velocity/tool
> >s/generic/DateTool.html ) in my VelocityApplet. I not using Struts. How do
> > I tell Velocity to use this Tool when outputting Dates. Some tell me
> > about a toolbox.xml, but I don't know hwere to place it and with which
> > content. A simple copy of the jar-file to the classpath didn't help.
> >
> > Who can help?
> >
> > Thanks
> > Dominik
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Ben <be...@pcguy.org>.
thanks, didn't see the maxloops one in there though, is it included in
version 1.4 or do i need to get the latest source code or what?
----- Original Message -----
From: "Will Glass-Husain" <wg...@forio.com>
To: "Velocity Users List" <ve...@jakarta.apache.org>
Sent: Friday, April 07, 2006 2:36 PM
Subject: Re: using Velocity in an untrusted environment
Have you seen this?
http://jakarta.apache.org/velocity/docs/developer-guide.html#Velocity%20Configuration%20Keys%20and%20Values
Most of the properties should be documented. If anyone finds missing
ones they should file JIRA issues.
In addition to:
directive.foreach.maxloops
you can also limit #parse with
directive.parse.maxdepth
Nathan - Congrats on the upcoming baby, by the way. We just had our
second two months ago.
WILL
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Will Glass-Husain <wg...@forio.com>.
Have you seen this?
http://jakarta.apache.org/velocity/docs/developer-guide.html#Velocity%20Configuration%20Keys%20and%20Values
Most of the properties should be documented. If anyone finds missing
ones they should file JIRA issues.
In addition to:
directive.foreach.maxloops
you can also limit #parse with
directive.parse.maxdepth
Nathan - Congrats on the upcoming baby, by the way. We just had our
second two months ago.
WILL
On 4/7/06, Nathan Bubna <nb...@gmail.com> wrote:
> On 4/7/06, Ben <be...@pcguy.org> wrote:
> > I just don't want to have to create a new thread every time a request comes
> > in, since that would double the number of threads on the server from 1 per
> > page access to 2. If i were to create a new thread it would be easy to
> > interrupt it, by doing thread.join(number of milliseconds); and then
> > thread.interrupt();
> >
> > Thanks about pointing out foreach.maxloops property, is there a list
> > somewhere of all properties i can set for velocity? I browsed through the
> > docs, but didn't find them.
>
> unfortunately, the config properties are not all documented well.
> it's something on my long wish-i-had-time-to-do-this list, but with a
> new house, pregnant wife, and pressure from my main paying job, i
> haven't had time. :)
>
> however, the latest version of the RuntimeConstants class will give a
> pretty good of the properties available, as it has deprecated the
> meaningless ones and include most of the newer ones.
>
> > ----- Original Message -----
> > From: "Nathan Bubna" <nb...@gmail.com>
> > To: "Velocity Users List" <ve...@jakarta.apache.org>
> > Sent: Friday, April 07, 2006 12:18 PM
> > Subject: Re: using Velocity in an untrusted environment
> >
> >
> >
> >
> > i don't see why there should be any problem with using a separate
> > thread. servlet requests are all about threads.
> >
> > if it is just #foreach looping that you are concerned about, then
> > there is already a directive.foreach.maxloops property that you can
> > set.
> >
> > > If people are interested I can paste the results of my experiment when I
> > > am
> > > done, as well as any modifications I make.
> > >
> > >
> > >
> > > Hmm. To be honest, I'm not interested in having this be an
> > > out-of-the-box piece of Velocity. Adding this "maximum cost" option
> > > for "every operation it does" would mean a performance hit, a big rise
> > > in complexity, or both. I would want to see a lot of interest in this
> > > from others before i would let this change go through without vetoing
> > > it. I really don't think this is something most of our users want or
> > > need. No one else has asked for it (to my memory) in the five years
> > > i've been around.
> > >
> > > For you, however, it ought to be fairly easy straightforward to create
> > > a VelocityRunnable that you can start in a new Thread to do the
> > > template merge/render and then have the request thread check up on it
> > > (sleeping in between checks, of course) periodically.
> > >
> > > The tricky part is stopping the rendering thread when it goes over
> > > time. It's not really safe to use the deprecated Thread.stop()
> > > method. The recommended replacement
> > > (http://java.sun.com/j2se/1.4.2/docs/guide/misc/threadPrimitiveDeprecation.html)
> > > is to create velocityRunnable.stop() method that can flip a flag to
> > > interrupt the rendering. But Velocity doesn't have any built in way
> > > to *interrupt* the rendering. The only thing i'm aware of is the
> > > #stop directives ability to make Velocity stop sending output to the
> > > writer. So far as i know, it doesn't actually stop the template
> > > processing (personally, i think it'd be better if it did).
> > >
> > > To actually stop template processing, you will probably have to alter
> > > some of the internals yourself. The driver of the rendering process
> > > is a simple for() loop in the render(context, writer) method of the
> > > SimpleNode class. all the nodes extend this class, so this method is
> > > how the AST is traversed. I would imagine that the "real way" to do
> > > this would be to somehow put the flag in that for() loop's conditional
> > > that would be shared by all nodes in that template. That probably
> > > means you need a flag that resides in the context that's being passed
> > > around. When the flag is tripped, no further nodes should be
> > > rendered.
> > >
> > > Of course, i'm not 100% sure that that is all you'll need to change,
> > > and it also might not catch all possible problems. For instance, if
> > > it is the rendering of a particular leaf on the AST that is taking
> > > forever, then this won't stop that node's rendering; it would only
> > > stop further traversal of the tree. Still, that is hopefully enough
> > > info to get you started...
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
--
Forio Business Simulations
Will Glass-Husain
wglass@forio.com
www.forio.com
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Nathan Bubna <nb...@gmail.com>.
On 4/7/06, Ben <be...@pcguy.org> wrote:
> I just don't want to have to create a new thread every time a request comes
> in, since that would double the number of threads on the server from 1 per
> page access to 2. If i were to create a new thread it would be easy to
> interrupt it, by doing thread.join(number of milliseconds); and then
> thread.interrupt();
>
> Thanks about pointing out foreach.maxloops property, is there a list
> somewhere of all properties i can set for velocity? I browsed through the
> docs, but didn't find them.
unfortunately, the config properties are not all documented well.
it's something on my long wish-i-had-time-to-do-this list, but with a
new house, pregnant wife, and pressure from my main paying job, i
haven't had time. :)
however, the latest version of the RuntimeConstants class will give a
pretty good of the properties available, as it has deprecated the
meaningless ones and include most of the newer ones.
> ----- Original Message -----
> From: "Nathan Bubna" <nb...@gmail.com>
> To: "Velocity Users List" <ve...@jakarta.apache.org>
> Sent: Friday, April 07, 2006 12:18 PM
> Subject: Re: using Velocity in an untrusted environment
>
>
>
>
> i don't see why there should be any problem with using a separate
> thread. servlet requests are all about threads.
>
> if it is just #foreach looping that you are concerned about, then
> there is already a directive.foreach.maxloops property that you can
> set.
>
> > If people are interested I can paste the results of my experiment when I
> > am
> > done, as well as any modifications I make.
> >
> >
> >
> > Hmm. To be honest, I'm not interested in having this be an
> > out-of-the-box piece of Velocity. Adding this "maximum cost" option
> > for "every operation it does" would mean a performance hit, a big rise
> > in complexity, or both. I would want to see a lot of interest in this
> > from others before i would let this change go through without vetoing
> > it. I really don't think this is something most of our users want or
> > need. No one else has asked for it (to my memory) in the five years
> > i've been around.
> >
> > For you, however, it ought to be fairly easy straightforward to create
> > a VelocityRunnable that you can start in a new Thread to do the
> > template merge/render and then have the request thread check up on it
> > (sleeping in between checks, of course) periodically.
> >
> > The tricky part is stopping the rendering thread when it goes over
> > time. It's not really safe to use the deprecated Thread.stop()
> > method. The recommended replacement
> > (http://java.sun.com/j2se/1.4.2/docs/guide/misc/threadPrimitiveDeprecation.html)
> > is to create velocityRunnable.stop() method that can flip a flag to
> > interrupt the rendering. But Velocity doesn't have any built in way
> > to *interrupt* the rendering. The only thing i'm aware of is the
> > #stop directives ability to make Velocity stop sending output to the
> > writer. So far as i know, it doesn't actually stop the template
> > processing (personally, i think it'd be better if it did).
> >
> > To actually stop template processing, you will probably have to alter
> > some of the internals yourself. The driver of the rendering process
> > is a simple for() loop in the render(context, writer) method of the
> > SimpleNode class. all the nodes extend this class, so this method is
> > how the AST is traversed. I would imagine that the "real way" to do
> > this would be to somehow put the flag in that for() loop's conditional
> > that would be shared by all nodes in that template. That probably
> > means you need a flag that resides in the context that's being passed
> > around. When the flag is tripped, no further nodes should be
> > rendered.
> >
> > Of course, i'm not 100% sure that that is all you'll need to change,
> > and it also might not catch all possible problems. For instance, if
> > it is the rendering of a particular leaf on the AST that is taking
> > forever, then this won't stop that node's rendering; it would only
> > stop further traversal of the tree. Still, that is hopefully enough
> > info to get you started...
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Ben <be...@pcguy.org>.
I just don't want to have to create a new thread every time a request comes
in, since that would double the number of threads on the server from 1 per
page access to 2. If i were to create a new thread it would be easy to
interrupt it, by doing thread.join(number of milliseconds); and then
thread.interrupt();
Thanks about pointing out foreach.maxloops property, is there a list
somewhere of all properties i can set for velocity? I browsed through the
docs, but didn't find them.
----- Original Message -----
From: "Nathan Bubna" <nb...@gmail.com>
To: "Velocity Users List" <ve...@jakarta.apache.org>
Sent: Friday, April 07, 2006 12:18 PM
Subject: Re: using Velocity in an untrusted environment
i don't see why there should be any problem with using a separate
thread. servlet requests are all about threads.
if it is just #foreach looping that you are concerned about, then
there is already a directive.foreach.maxloops property that you can
set.
> If people are interested I can paste the results of my experiment when I
> am
> done, as well as any modifications I make.
>
>
>
> Hmm. To be honest, I'm not interested in having this be an
> out-of-the-box piece of Velocity. Adding this "maximum cost" option
> for "every operation it does" would mean a performance hit, a big rise
> in complexity, or both. I would want to see a lot of interest in this
> from others before i would let this change go through without vetoing
> it. I really don't think this is something most of our users want or
> need. No one else has asked for it (to my memory) in the five years
> i've been around.
>
> For you, however, it ought to be fairly easy straightforward to create
> a VelocityRunnable that you can start in a new Thread to do the
> template merge/render and then have the request thread check up on it
> (sleeping in between checks, of course) periodically.
>
> The tricky part is stopping the rendering thread when it goes over
> time. It's not really safe to use the deprecated Thread.stop()
> method. The recommended replacement
> (http://java.sun.com/j2se/1.4.2/docs/guide/misc/threadPrimitiveDeprecation.html)
> is to create velocityRunnable.stop() method that can flip a flag to
> interrupt the rendering. But Velocity doesn't have any built in way
> to *interrupt* the rendering. The only thing i'm aware of is the
> #stop directives ability to make Velocity stop sending output to the
> writer. So far as i know, it doesn't actually stop the template
> processing (personally, i think it'd be better if it did).
>
> To actually stop template processing, you will probably have to alter
> some of the internals yourself. The driver of the rendering process
> is a simple for() loop in the render(context, writer) method of the
> SimpleNode class. all the nodes extend this class, so this method is
> how the AST is traversed. I would imagine that the "real way" to do
> this would be to somehow put the flag in that for() loop's conditional
> that would be shared by all nodes in that template. That probably
> means you need a flag that resides in the context that's being passed
> around. When the flag is tripped, no further nodes should be
> rendered.
>
> Of course, i'm not 100% sure that that is all you'll need to change,
> and it also might not catch all possible problems. For instance, if
> it is the rendering of a particular leaf on the AST that is taking
> forever, then this won't stop that node's rendering; it would only
> stop further traversal of the tree. Still, that is hopefully enough
> info to get you started...
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Nathan Bubna <nb...@gmail.com>.
On 4/6/06, Ben <be...@pcguy.org> wrote:
> Thanks for pointing the way, I'll definitely take a look at the SimpleNode
> class and see if it can be used for my purpose. I don't want to use a
> separate thread since by me the render function is called from within a
> servlet running inside resin, so what i'll probably do is have it check the
> time, let's say every 10,000/100,000/1,000,000 loops or so, depending on
> how many loops it does per second, and if it's past the time limit stop the
> render process, or maybe just limit the rendering process to a certain
> amount of loops, to prevent people from writing templates with a near
> infinite loop. I do some work with Yahoo's RTML scripting language, which is
> being interpreted by a perl script running on Yahoo's servers, and I think
> that's what Yahoo does, limit the loops to 10,000,000 iterations or so.
i don't see why there should be any problem with using a separate
thread. servlet requests are all about threads.
if it is just #foreach looping that you are concerned about, then
there is already a directive.foreach.maxloops property that you can
set.
> If people are interested I can paste the results of my experiment when I am
> done, as well as any modifications I make.
>
>
>
> Hmm. To be honest, I'm not interested in having this be an
> out-of-the-box piece of Velocity. Adding this "maximum cost" option
> for "every operation it does" would mean a performance hit, a big rise
> in complexity, or both. I would want to see a lot of interest in this
> from others before i would let this change go through without vetoing
> it. I really don't think this is something most of our users want or
> need. No one else has asked for it (to my memory) in the five years
> i've been around.
>
> For you, however, it ought to be fairly easy straightforward to create
> a VelocityRunnable that you can start in a new Thread to do the
> template merge/render and then have the request thread check up on it
> (sleeping in between checks, of course) periodically.
>
> The tricky part is stopping the rendering thread when it goes over
> time. It's not really safe to use the deprecated Thread.stop()
> method. The recommended replacement
> (http://java.sun.com/j2se/1.4.2/docs/guide/misc/threadPrimitiveDeprecation.html)
> is to create velocityRunnable.stop() method that can flip a flag to
> interrupt the rendering. But Velocity doesn't have any built in way
> to *interrupt* the rendering. The only thing i'm aware of is the
> #stop directives ability to make Velocity stop sending output to the
> writer. So far as i know, it doesn't actually stop the template
> processing (personally, i think it'd be better if it did).
>
> To actually stop template processing, you will probably have to alter
> some of the internals yourself. The driver of the rendering process
> is a simple for() loop in the render(context, writer) method of the
> SimpleNode class. all the nodes extend this class, so this method is
> how the AST is traversed. I would imagine that the "real way" to do
> this would be to somehow put the flag in that for() loop's conditional
> that would be shared by all nodes in that template. That probably
> means you need a flag that resides in the context that's being passed
> around. When the flag is tripped, no further nodes should be
> rendered.
>
> Of course, i'm not 100% sure that that is all you'll need to change,
> and it also might not catch all possible problems. For instance, if
> it is the rendering of a particular leaf on the AST that is taking
> forever, then this won't stop that node's rendering; it would only
> stop further traversal of the tree. Still, that is hopefully enough
> info to get you started...
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Will Glass-Husain <wg...@forio.com>.
Thanks - let us know what you find...
On 4/6/06, Ben <be...@pcguy.org> wrote:
> Thanks for pointing the way, I'll definitely take a look at the SimpleNode
> class and see if it can be used for my purpose. I don't want to use a
> separate thread since by me the render function is called from within a
> servlet running inside resin, so what i'll probably do is have it check the
> time, let's say every 10,000/100,000/1,000,000 loops or so, depending on
> how many loops it does per second, and if it's past the time limit stop the
> render process, or maybe just limit the rendering process to a certain
> amount of loops, to prevent people from writing templates with a near
> infinite loop. I do some work with Yahoo's RTML scripting language, which is
> being interpreted by a perl script running on Yahoo's servers, and I think
> that's what Yahoo does, limit the loops to 10,000,000 iterations or so.
>
> If people are interested I can paste the results of my experiment when I am
> done, as well as any modifications I make.
>
>
>
> Hmm. To be honest, I'm not interested in having this be an
> out-of-the-box piece of Velocity. Adding this "maximum cost" option
> for "every operation it does" would mean a performance hit, a big rise
> in complexity, or both. I would want to see a lot of interest in this
> from others before i would let this change go through without vetoing
> it. I really don't think this is something most of our users want or
> need. No one else has asked for it (to my memory) in the five years
> i've been around.
>
> For you, however, it ought to be fairly easy straightforward to create
> a VelocityRunnable that you can start in a new Thread to do the
> template merge/render and then have the request thread check up on it
> (sleeping in between checks, of course) periodically.
>
> The tricky part is stopping the rendering thread when it goes over
> time. It's not really safe to use the deprecated Thread.stop()
> method. The recommended replacement
> (http://java.sun.com/j2se/1.4.2/docs/guide/misc/threadPrimitiveDeprecation.html)
> is to create velocityRunnable.stop() method that can flip a flag to
> interrupt the rendering. But Velocity doesn't have any built in way
> to *interrupt* the rendering. The only thing i'm aware of is the
> #stop directives ability to make Velocity stop sending output to the
> writer. So far as i know, it doesn't actually stop the template
> processing (personally, i think it'd be better if it did).
>
> To actually stop template processing, you will probably have to alter
> some of the internals yourself. The driver of the rendering process
> is a simple for() loop in the render(context, writer) method of the
> SimpleNode class. all the nodes extend this class, so this method is
> how the AST is traversed. I would imagine that the "real way" to do
> this would be to somehow put the flag in that for() loop's conditional
> that would be shared by all nodes in that template. That probably
> means you need a flag that resides in the context that's being passed
> around. When the flag is tripped, no further nodes should be
> rendered.
>
> Of course, i'm not 100% sure that that is all you'll need to change,
> and it also might not catch all possible problems. For instance, if
> it is the rendering of a particular leaf on the AST that is taking
> forever, then this won't stop that node's rendering; it would only
> stop further traversal of the tree. Still, that is hopefully enough
> info to get you started...
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
--
Forio Business Simulations
Will Glass-Husain
wglass@forio.com
www.forio.com
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Ben <be...@pcguy.org>.
Thanks for pointing the way, I'll definitely take a look at the SimpleNode
class and see if it can be used for my purpose. I don't want to use a
separate thread since by me the render function is called from within a
servlet running inside resin, so what i'll probably do is have it check the
time, let's say every 10,000/100,000/1,000,000 loops or so, depending on
how many loops it does per second, and if it's past the time limit stop the
render process, or maybe just limit the rendering process to a certain
amount of loops, to prevent people from writing templates with a near
infinite loop. I do some work with Yahoo's RTML scripting language, which is
being interpreted by a perl script running on Yahoo's servers, and I think
that's what Yahoo does, limit the loops to 10,000,000 iterations or so.
If people are interested I can paste the results of my experiment when I am
done, as well as any modifications I make.
Hmm. To be honest, I'm not interested in having this be an
out-of-the-box piece of Velocity. Adding this "maximum cost" option
for "every operation it does" would mean a performance hit, a big rise
in complexity, or both. I would want to see a lot of interest in this
from others before i would let this change go through without vetoing
it. I really don't think this is something most of our users want or
need. No one else has asked for it (to my memory) in the five years
i've been around.
For you, however, it ought to be fairly easy straightforward to create
a VelocityRunnable that you can start in a new Thread to do the
template merge/render and then have the request thread check up on it
(sleeping in between checks, of course) periodically.
The tricky part is stopping the rendering thread when it goes over
time. It's not really safe to use the deprecated Thread.stop()
method. The recommended replacement
(http://java.sun.com/j2se/1.4.2/docs/guide/misc/threadPrimitiveDeprecation.html)
is to create velocityRunnable.stop() method that can flip a flag to
interrupt the rendering. But Velocity doesn't have any built in way
to *interrupt* the rendering. The only thing i'm aware of is the
#stop directives ability to make Velocity stop sending output to the
writer. So far as i know, it doesn't actually stop the template
processing (personally, i think it'd be better if it did).
To actually stop template processing, you will probably have to alter
some of the internals yourself. The driver of the rendering process
is a simple for() loop in the render(context, writer) method of the
SimpleNode class. all the nodes extend this class, so this method is
how the AST is traversed. I would imagine that the "real way" to do
this would be to somehow put the flag in that for() loop's conditional
that would be shared by all nodes in that template. That probably
means you need a flag that resides in the context that's being passed
around. When the flag is tripped, no further nodes should be
rendered.
Of course, i'm not 100% sure that that is all you'll need to change,
and it also might not catch all possible problems. For instance, if
it is the rendering of a particular leaf on the AST that is taking
forever, then this won't stop that node's rendering; it would only
stop further traversal of the tree. Still, that is hopefully enough
info to get you started...
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Nathan Bubna <nb...@gmail.com>.
On 4/6/06, Ben <be...@pcguy.org> wrote:
> Thanks Will, I appreciate your response.
<snip/>
> I'll be happy to share my solution about interrupting the rendering process,
> if I come up with a generic way which can be applied to other projects
> besides mine. Ideally I would like this to be a function of Velocity itself,
> where for every operation it does it would assign some number, which would
> be the cost of that operation, and then the developer would have the ability
> to set the maximum cost, after which velocity would automatically interrupt
> the rendering process. Another option would be for it to save the starting
> date/time of the rendering process, and then every now and then to check how
> much time has passes, and if the time limit has expired to interrupt the
> rendering process. I am not sure I have sufficient understanding of the
> Velocity source code to do this though, can anybody who knows the Velocity
> code well tell me if any of these two options are feasable, and if yes, what
> class/classes would I need to modify for this?
Hmm. To be honest, I'm not interested in having this be an
out-of-the-box piece of Velocity. Adding this "maximum cost" option
for "every operation it does" would mean a performance hit, a big rise
in complexity, or both. I would want to see a lot of interest in this
from others before i would let this change go through without vetoing
it. I really don't think this is something most of our users want or
need. No one else has asked for it (to my memory) in the five years
i've been around.
For you, however, it ought to be fairly easy straightforward to create
a VelocityRunnable that you can start in a new Thread to do the
template merge/render and then have the request thread check up on it
(sleeping in between checks, of course) periodically.
The tricky part is stopping the rendering thread when it goes over
time. It's not really safe to use the deprecated Thread.stop()
method. The recommended replacement
(http://java.sun.com/j2se/1.4.2/docs/guide/misc/threadPrimitiveDeprecation.html)
is to create velocityRunnable.stop() method that can flip a flag to
interrupt the rendering. But Velocity doesn't have any built in way
to *interrupt* the rendering. The only thing i'm aware of is the
#stop directives ability to make Velocity stop sending output to the
writer. So far as i know, it doesn't actually stop the template
processing (personally, i think it'd be better if it did).
To actually stop template processing, you will probably have to alter
some of the internals yourself. The driver of the rendering process
is a simple for() loop in the render(context, writer) method of the
SimpleNode class. all the nodes extend this class, so this method is
how the AST is traversed. I would imagine that the "real way" to do
this would be to somehow put the flag in that for() loop's conditional
that would be shared by all nodes in that template. That probably
means you need a flag that resides in the context that's being passed
around. When the flag is tripped, no further nodes should be
rendered.
Of course, i'm not 100% sure that that is all you'll need to change,
and it also might not catch all possible problems. For instance, if
it is the rendering of a particular leaf on the AST that is taking
forever, then this won't stop that node's rendering; it would only
stop further traversal of the tree. Still, that is hopefully enough
info to get you started...
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Nathan Bubna <nb...@gmail.com>.
On 4/6/06, Will Glass-Husain <wg...@forio.com> wrote:
> Hi Ben,
>
> You probably don't have an issue with #include though it depends how
> you organize the templates.
>
> In our case, I load templates specific to a given user via part of the
> path, e.g. "/will/abc.htm". Hence we needed an event handler to
> decide if you have permission to include the files or not. If you
> don't indicate user specific info on the path you're likely ok. It's
> probably pretty obvious whether this is the case or not.
>
> In regards to "unescaped HTML", do a Google search on "Cross-Site
> Scripting". Esentially, the issue is that if user A can insert text
> that appears on User B's screen, you need to prevent that text from
> including arbitrary HTML or JavaScript. The easiest way to do this is
> to convert all entities (e.g. "<" into <).
>
> I'll have to think about the rendering cost issue. I agree with
> Nathan in that I'm reluctant to add a lot of complexity to the core to
> address this. I suggest either putting such code in a custom plugin
> (e.g. the ReferenceInsertEventHandler) that checks for resource
> activity and throws a RuntimeException if appropriate, or adding such
> a routine externally to Velocity in some type of thread handler.
> (e.g. it launches Velocity in a thread and kills the thread if it
> takes too long -- is that possible? I'm not a thread guru).
Killing threads with the Thread.stop() and such methods are all
deprecated and not good ways to do it. You have to implement your own
stop() methods. That's the tricky part, since Velocity doesn't have
any real stop-rendering hooks.
> Best, WILL
>
>
>
> On 4/6/06, Ben <be...@pcguy.org> wrote:
> > Thanks Will, I appreciate your response.
> >
> > Two more questions though, in my project the templates are kept inside a
> > MySQL database and are loaded by a custom class from there, does this mean I
> > don't have to worry about the #include and the #parse loading another user's
> > templates, since they only load them from a directory on the hard-drive?
> >
> > Also I dont' fully understand the problem of unescaped HTML inside the
> > templates, what kind of issues can be expected from that in my case, where
> > users are allowed to upload their own templates anyway, or it only applies
> > in a case where they don't have permission to do this?
> >
> > I'll be happy to share my solution about interrupting the rendering process,
> > if I come up with a generic way which can be applied to other projects
> > besides mine. Ideally I would like this to be a function of Velocity itself,
> > where for every operation it does it would assign some number, which would
> > be the cost of that operation, and then the developer would have the ability
> > to set the maximum cost, after which velocity would automatically interrupt
> > the rendering process. Another option would be for it to save the starting
> > date/time of the rendering process, and then every now and then to check how
> > much time has passes, and if the time limit has expired to interrupt the
> > rendering process. I am not sure I have sufficient understanding of the
> > Velocity source code to do this though, can anybody who knows the Velocity
> > code well tell me if any of these two options are feasable, and if yes, what
> > class/classes would I need to modify for this?
> >
> > Thanks,
> > Ben
> >
> > ----- Original Message -----
> > From: "Will Glass-Husain" <wg...@forio.com>
> > To: "Velocity Users List" <ve...@jakarta.apache.org>
> > Sent: Thursday, April 06, 2006 4:39 AM
> > Subject: Re: using Velocity in an untrusted environment
> >
> >
> > Hi Ben,
> >
> > I run a web site with some hundreds of users who upload their own
> > Velocity templates. It works pretty well. We've tried to address
> > most security-related issues, but so far we've just ignored the
> > potential DenialOfService issue and haven't had any trouble.
> >
> > As far as the classloader patch goes, you can put it into a custom
> > Uberspector and use it right now. A little complicated, but it works.
> >
> > Some of the other items in the article have been fixed. The current
> > source control head contains an Event Handler which can automatically
> > escape HTML. There's also a new event handler that can be used to
> > modify #include and #parse (say, to prevent one user from including
> > files of another user).
> >
> > In Tomcat you can set up "catalina.policy" to restrict access to files
> > outside the webapp using the Java security manager. It takes a little
> > trial and error to get this right. Most third-party libraries (e.g.
> > Velocity, Hibernate, etc.) require certain permissions and these are
> > almost never documented. I can share mine if that's helpful although
> > it's pretty specific to my apps.
> >
> > Finally, you need to be very careful as to what you put in the
> > context. Any methods that allow access to files, databases and other
> > resources are actively dangerous.
> >
> > WILL
> >
> > On 4/5/06, Nathan Bubna <nb...@gmail.com> wrote:
> > > Well, this is really more Will's area of expertise. I have the luxury
> > > of not letting users of my apps define their own templates. So, i've
> > > not had any need to use a JavaSecurityManager.
> > >
> > > The language in the article is a little unclear. Version 1.6 has not
> > > started development yet. We are still tweaking 1.5 in our
> > > collectively scarce free time. It's more that the patch in question
> > > has been put on the roadmap for 1.6.
> > >
> > > In the meantime, the info and references in
> > > http://issues.apache.org/jira/browse/VELOCITY-179 should provide a
> > > number of options for restricting classloader use in your user's
> > > templates.
> > >
> > > As far as interrupting template processing... i've been in this
> > > community for about five years and i can't recall anyone else ever
> > > asking for or needing this. so, no, this is supported nor is it
> > > scheduled to be. i can't imagine that it would be difficult to
> > > implement using Threads. if you come up with something useful here,
> > > you might consider sharing it with the community. :)
> > >
> > > On 4/5/06, Ben <be...@pcguy.org> wrote:
> > > > Thanks, that is the page i read about this problem from before. It has a
> > > > link to http://issues.apache.org/jira/browse/VELOCITY-179 which has the
> > > > classloader patch. It aslo says the proposed patch has been accepted for
> > > > velocity version 1.6, hence my question about the scheduled release date
> > > > of
> > > > that version.
> > > >
> > > > Also didn't see anywhere in that article about the ability to interrupt
> > > > the
> > > > rendering process after let's say 5 seconds so that a single user
> > > > doesn't
> > > > take up all resources on the server. Is that something which velocity
> > > > currently supports/is cheduled to be supported in some future version,
> > > > or do
> > > > i have to built in that feature in my application, and if yes, can you
> > > > please suggest the best route to do this? Is there an interrupt method
> > > > in
> > > > velocity which i can call after 5 sec, let's say, which will interrupt
> > > > the
> > > > rendering process?
> > > >
> > > > Thanks,
> > > > Ben
> > > >
> > > > ----- Original Message -----
> > > > From: "Nathan Bubna" <nb...@gmail.com>
> > > > To: "Velocity Users List" <ve...@jakarta.apache.org>
> > > > Sent: Tuesday, April 04, 2006 2:59 PM
> > > > Subject: Re: using Velocity in an untrusted environment
> > > >
> > > >
> > > > http://wiki.apache.org/jakarta-velocity/BuildingSecureWebApplications
> > > >
> > > > On 4/4/06, Ben <be...@pcguy.org> wrote:
> > > > > Hi all,
> > > > >
> > > > > I am thinking of using Velocity engine in an e-commerce platform,
> > > > > where
> > > > > the
> > > > > users will be able to upload their own templates to customize the
> > > > > layout
> > > > > of
> > > > > their store. I've read somewhere that Velocity has a built in security
> > > > > flaw,
> > > > > where peole could do things like AnyClass.getClassLoader() and use
> > > > > that to
> > > > > load any java class and basically do anything they want. I've also
> > > > > read
> > > > > about a patch being developed to address this issue which is scheduled
> > > > > to
> > > > > be
> > > > > integrated into Velocity version 1.6
> > > > >
> > > > > I'm wondering, when is that version of velocity scheduled to come out,
> > > > > and
> > > > > are there any other security related issues i should watch out for in
> > > > > my
> > > > > scenario, where basically people who upload templates are untrusted
> > > > > users.
> > > > >
> > > > > Also, does velocity have a built in timeout feature, where for example
> > > > > if
> > > > > any template takes more than 5 seconds to render, I'll be able to
> > > > > interrupt
> > > > > the rendering process? This feature is also important to me, as I
> > > > > don't
> > > > > want
> > > > > any single user to tie up all system resources.
> > > > >
> > > > > Thanks,
> > > > > Ben
> > > > >
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > > >
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > >
> > >
> >
> >
> > --
> > Forio Business Simulations
> >
> > Will Glass-Husain
> > wglass@forio.com
> > www.forio.com
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
>
>
> --
> Forio Business Simulations
>
> Will Glass-Husain
> wglass@forio.com
> www.forio.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Will Glass-Husain <wg...@forio.com>.
Hi Ben,
You probably don't have an issue with #include though it depends how
you organize the templates.
In our case, I load templates specific to a given user via part of the
path, e.g. "/will/abc.htm". Hence we needed an event handler to
decide if you have permission to include the files or not. If you
don't indicate user specific info on the path you're likely ok. It's
probably pretty obvious whether this is the case or not.
In regards to "unescaped HTML", do a Google search on "Cross-Site
Scripting". Esentially, the issue is that if user A can insert text
that appears on User B's screen, you need to prevent that text from
including arbitrary HTML or JavaScript. The easiest way to do this is
to convert all entities (e.g. "<" into <).
I'll have to think about the rendering cost issue. I agree with
Nathan in that I'm reluctant to add a lot of complexity to the core to
address this. I suggest either putting such code in a custom plugin
(e.g. the ReferenceInsertEventHandler) that checks for resource
activity and throws a RuntimeException if appropriate, or adding such
a routine externally to Velocity in some type of thread handler.
(e.g. it launches Velocity in a thread and kills the thread if it
takes too long -- is that possible? I'm not a thread guru).
Best, WILL
On 4/6/06, Ben <be...@pcguy.org> wrote:
> Thanks Will, I appreciate your response.
>
> Two more questions though, in my project the templates are kept inside a
> MySQL database and are loaded by a custom class from there, does this mean I
> don't have to worry about the #include and the #parse loading another user's
> templates, since they only load them from a directory on the hard-drive?
>
> Also I dont' fully understand the problem of unescaped HTML inside the
> templates, what kind of issues can be expected from that in my case, where
> users are allowed to upload their own templates anyway, or it only applies
> in a case where they don't have permission to do this?
>
> I'll be happy to share my solution about interrupting the rendering process,
> if I come up with a generic way which can be applied to other projects
> besides mine. Ideally I would like this to be a function of Velocity itself,
> where for every operation it does it would assign some number, which would
> be the cost of that operation, and then the developer would have the ability
> to set the maximum cost, after which velocity would automatically interrupt
> the rendering process. Another option would be for it to save the starting
> date/time of the rendering process, and then every now and then to check how
> much time has passes, and if the time limit has expired to interrupt the
> rendering process. I am not sure I have sufficient understanding of the
> Velocity source code to do this though, can anybody who knows the Velocity
> code well tell me if any of these two options are feasable, and if yes, what
> class/classes would I need to modify for this?
>
> Thanks,
> Ben
>
> ----- Original Message -----
> From: "Will Glass-Husain" <wg...@forio.com>
> To: "Velocity Users List" <ve...@jakarta.apache.org>
> Sent: Thursday, April 06, 2006 4:39 AM
> Subject: Re: using Velocity in an untrusted environment
>
>
> Hi Ben,
>
> I run a web site with some hundreds of users who upload their own
> Velocity templates. It works pretty well. We've tried to address
> most security-related issues, but so far we've just ignored the
> potential DenialOfService issue and haven't had any trouble.
>
> As far as the classloader patch goes, you can put it into a custom
> Uberspector and use it right now. A little complicated, but it works.
>
> Some of the other items in the article have been fixed. The current
> source control head contains an Event Handler which can automatically
> escape HTML. There's also a new event handler that can be used to
> modify #include and #parse (say, to prevent one user from including
> files of another user).
>
> In Tomcat you can set up "catalina.policy" to restrict access to files
> outside the webapp using the Java security manager. It takes a little
> trial and error to get this right. Most third-party libraries (e.g.
> Velocity, Hibernate, etc.) require certain permissions and these are
> almost never documented. I can share mine if that's helpful although
> it's pretty specific to my apps.
>
> Finally, you need to be very careful as to what you put in the
> context. Any methods that allow access to files, databases and other
> resources are actively dangerous.
>
> WILL
>
> On 4/5/06, Nathan Bubna <nb...@gmail.com> wrote:
> > Well, this is really more Will's area of expertise. I have the luxury
> > of not letting users of my apps define their own templates. So, i've
> > not had any need to use a JavaSecurityManager.
> >
> > The language in the article is a little unclear. Version 1.6 has not
> > started development yet. We are still tweaking 1.5 in our
> > collectively scarce free time. It's more that the patch in question
> > has been put on the roadmap for 1.6.
> >
> > In the meantime, the info and references in
> > http://issues.apache.org/jira/browse/VELOCITY-179 should provide a
> > number of options for restricting classloader use in your user's
> > templates.
> >
> > As far as interrupting template processing... i've been in this
> > community for about five years and i can't recall anyone else ever
> > asking for or needing this. so, no, this is supported nor is it
> > scheduled to be. i can't imagine that it would be difficult to
> > implement using Threads. if you come up with something useful here,
> > you might consider sharing it with the community. :)
> >
> > On 4/5/06, Ben <be...@pcguy.org> wrote:
> > > Thanks, that is the page i read about this problem from before. It has a
> > > link to http://issues.apache.org/jira/browse/VELOCITY-179 which has the
> > > classloader patch. It aslo says the proposed patch has been accepted for
> > > velocity version 1.6, hence my question about the scheduled release date
> > > of
> > > that version.
> > >
> > > Also didn't see anywhere in that article about the ability to interrupt
> > > the
> > > rendering process after let's say 5 seconds so that a single user
> > > doesn't
> > > take up all resources on the server. Is that something which velocity
> > > currently supports/is cheduled to be supported in some future version,
> > > or do
> > > i have to built in that feature in my application, and if yes, can you
> > > please suggest the best route to do this? Is there an interrupt method
> > > in
> > > velocity which i can call after 5 sec, let's say, which will interrupt
> > > the
> > > rendering process?
> > >
> > > Thanks,
> > > Ben
> > >
> > > ----- Original Message -----
> > > From: "Nathan Bubna" <nb...@gmail.com>
> > > To: "Velocity Users List" <ve...@jakarta.apache.org>
> > > Sent: Tuesday, April 04, 2006 2:59 PM
> > > Subject: Re: using Velocity in an untrusted environment
> > >
> > >
> > > http://wiki.apache.org/jakarta-velocity/BuildingSecureWebApplications
> > >
> > > On 4/4/06, Ben <be...@pcguy.org> wrote:
> > > > Hi all,
> > > >
> > > > I am thinking of using Velocity engine in an e-commerce platform,
> > > > where
> > > > the
> > > > users will be able to upload their own templates to customize the
> > > > layout
> > > > of
> > > > their store. I've read somewhere that Velocity has a built in security
> > > > flaw,
> > > > where peole could do things like AnyClass.getClassLoader() and use
> > > > that to
> > > > load any java class and basically do anything they want. I've also
> > > > read
> > > > about a patch being developed to address this issue which is scheduled
> > > > to
> > > > be
> > > > integrated into Velocity version 1.6
> > > >
> > > > I'm wondering, when is that version of velocity scheduled to come out,
> > > > and
> > > > are there any other security related issues i should watch out for in
> > > > my
> > > > scenario, where basically people who upload templates are untrusted
> > > > users.
> > > >
> > > > Also, does velocity have a built in timeout feature, where for example
> > > > if
> > > > any template takes more than 5 seconds to render, I'll be able to
> > > > interrupt
> > > > the rendering process? This feature is also important to me, as I
> > > > don't
> > > > want
> > > > any single user to tie up all system resources.
> > > >
> > > > Thanks,
> > > > Ben
> > > >
> > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
>
>
> --
> Forio Business Simulations
>
> Will Glass-Husain
> wglass@forio.com
> www.forio.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
--
Forio Business Simulations
Will Glass-Husain
wglass@forio.com
www.forio.com
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Ben <be...@pcguy.org>.
Thanks Will, I appreciate your response.
Two more questions though, in my project the templates are kept inside a
MySQL database and are loaded by a custom class from there, does this mean I
don't have to worry about the #include and the #parse loading another user's
templates, since they only load them from a directory on the hard-drive?
Also I dont' fully understand the problem of unescaped HTML inside the
templates, what kind of issues can be expected from that in my case, where
users are allowed to upload their own templates anyway, or it only applies
in a case where they don't have permission to do this?
I'll be happy to share my solution about interrupting the rendering process,
if I come up with a generic way which can be applied to other projects
besides mine. Ideally I would like this to be a function of Velocity itself,
where for every operation it does it would assign some number, which would
be the cost of that operation, and then the developer would have the ability
to set the maximum cost, after which velocity would automatically interrupt
the rendering process. Another option would be for it to save the starting
date/time of the rendering process, and then every now and then to check how
much time has passes, and if the time limit has expired to interrupt the
rendering process. I am not sure I have sufficient understanding of the
Velocity source code to do this though, can anybody who knows the Velocity
code well tell me if any of these two options are feasable, and if yes, what
class/classes would I need to modify for this?
Thanks,
Ben
----- Original Message -----
From: "Will Glass-Husain" <wg...@forio.com>
To: "Velocity Users List" <ve...@jakarta.apache.org>
Sent: Thursday, April 06, 2006 4:39 AM
Subject: Re: using Velocity in an untrusted environment
Hi Ben,
I run a web site with some hundreds of users who upload their own
Velocity templates. It works pretty well. We've tried to address
most security-related issues, but so far we've just ignored the
potential DenialOfService issue and haven't had any trouble.
As far as the classloader patch goes, you can put it into a custom
Uberspector and use it right now. A little complicated, but it works.
Some of the other items in the article have been fixed. The current
source control head contains an Event Handler which can automatically
escape HTML. There's also a new event handler that can be used to
modify #include and #parse (say, to prevent one user from including
files of another user).
In Tomcat you can set up "catalina.policy" to restrict access to files
outside the webapp using the Java security manager. It takes a little
trial and error to get this right. Most third-party libraries (e.g.
Velocity, Hibernate, etc.) require certain permissions and these are
almost never documented. I can share mine if that's helpful although
it's pretty specific to my apps.
Finally, you need to be very careful as to what you put in the
context. Any methods that allow access to files, databases and other
resources are actively dangerous.
WILL
On 4/5/06, Nathan Bubna <nb...@gmail.com> wrote:
> Well, this is really more Will's area of expertise. I have the luxury
> of not letting users of my apps define their own templates. So, i've
> not had any need to use a JavaSecurityManager.
>
> The language in the article is a little unclear. Version 1.6 has not
> started development yet. We are still tweaking 1.5 in our
> collectively scarce free time. It's more that the patch in question
> has been put on the roadmap for 1.6.
>
> In the meantime, the info and references in
> http://issues.apache.org/jira/browse/VELOCITY-179 should provide a
> number of options for restricting classloader use in your user's
> templates.
>
> As far as interrupting template processing... i've been in this
> community for about five years and i can't recall anyone else ever
> asking for or needing this. so, no, this is supported nor is it
> scheduled to be. i can't imagine that it would be difficult to
> implement using Threads. if you come up with something useful here,
> you might consider sharing it with the community. :)
>
> On 4/5/06, Ben <be...@pcguy.org> wrote:
> > Thanks, that is the page i read about this problem from before. It has a
> > link to http://issues.apache.org/jira/browse/VELOCITY-179 which has the
> > classloader patch. It aslo says the proposed patch has been accepted for
> > velocity version 1.6, hence my question about the scheduled release date
> > of
> > that version.
> >
> > Also didn't see anywhere in that article about the ability to interrupt
> > the
> > rendering process after let's say 5 seconds so that a single user
> > doesn't
> > take up all resources on the server. Is that something which velocity
> > currently supports/is cheduled to be supported in some future version,
> > or do
> > i have to built in that feature in my application, and if yes, can you
> > please suggest the best route to do this? Is there an interrupt method
> > in
> > velocity which i can call after 5 sec, let's say, which will interrupt
> > the
> > rendering process?
> >
> > Thanks,
> > Ben
> >
> > ----- Original Message -----
> > From: "Nathan Bubna" <nb...@gmail.com>
> > To: "Velocity Users List" <ve...@jakarta.apache.org>
> > Sent: Tuesday, April 04, 2006 2:59 PM
> > Subject: Re: using Velocity in an untrusted environment
> >
> >
> > http://wiki.apache.org/jakarta-velocity/BuildingSecureWebApplications
> >
> > On 4/4/06, Ben <be...@pcguy.org> wrote:
> > > Hi all,
> > >
> > > I am thinking of using Velocity engine in an e-commerce platform,
> > > where
> > > the
> > > users will be able to upload their own templates to customize the
> > > layout
> > > of
> > > their store. I've read somewhere that Velocity has a built in security
> > > flaw,
> > > where peole could do things like AnyClass.getClassLoader() and use
> > > that to
> > > load any java class and basically do anything they want. I've also
> > > read
> > > about a patch being developed to address this issue which is scheduled
> > > to
> > > be
> > > integrated into Velocity version 1.6
> > >
> > > I'm wondering, when is that version of velocity scheduled to come out,
> > > and
> > > are there any other security related issues i should watch out for in
> > > my
> > > scenario, where basically people who upload templates are untrusted
> > > users.
> > >
> > > Also, does velocity have a built in timeout feature, where for example
> > > if
> > > any template takes more than 5 seconds to render, I'll be able to
> > > interrupt
> > > the rendering process? This feature is also important to me, as I
> > > don't
> > > want
> > > any single user to tie up all system resources.
> > >
> > > Thanks,
> > > Ben
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
--
Forio Business Simulations
Will Glass-Husain
wglass@forio.com
www.forio.com
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Will Glass-Husain <wg...@forio.com>.
Hi Ben,
I run a web site with some hundreds of users who upload their own
Velocity templates. It works pretty well. We've tried to address
most security-related issues, but so far we've just ignored the
potential DenialOfService issue and haven't had any trouble.
As far as the classloader patch goes, you can put it into a custom
Uberspector and use it right now. A little complicated, but it works.
Some of the other items in the article have been fixed. The current
source control head contains an Event Handler which can automatically
escape HTML. There's also a new event handler that can be used to
modify #include and #parse (say, to prevent one user from including
files of another user).
In Tomcat you can set up "catalina.policy" to restrict access to files
outside the webapp using the Java security manager. It takes a little
trial and error to get this right. Most third-party libraries (e.g.
Velocity, Hibernate, etc.) require certain permissions and these are
almost never documented. I can share mine if that's helpful although
it's pretty specific to my apps.
Finally, you need to be very careful as to what you put in the
context. Any methods that allow access to files, databases and other
resources are actively dangerous.
WILL
On 4/5/06, Nathan Bubna <nb...@gmail.com> wrote:
> Well, this is really more Will's area of expertise. I have the luxury
> of not letting users of my apps define their own templates. So, i've
> not had any need to use a JavaSecurityManager.
>
> The language in the article is a little unclear. Version 1.6 has not
> started development yet. We are still tweaking 1.5 in our
> collectively scarce free time. It's more that the patch in question
> has been put on the roadmap for 1.6.
>
> In the meantime, the info and references in
> http://issues.apache.org/jira/browse/VELOCITY-179 should provide a
> number of options for restricting classloader use in your user's
> templates.
>
> As far as interrupting template processing... i've been in this
> community for about five years and i can't recall anyone else ever
> asking for or needing this. so, no, this is supported nor is it
> scheduled to be. i can't imagine that it would be difficult to
> implement using Threads. if you come up with something useful here,
> you might consider sharing it with the community. :)
>
> On 4/5/06, Ben <be...@pcguy.org> wrote:
> > Thanks, that is the page i read about this problem from before. It has a
> > link to http://issues.apache.org/jira/browse/VELOCITY-179 which has the
> > classloader patch. It aslo says the proposed patch has been accepted for
> > velocity version 1.6, hence my question about the scheduled release date of
> > that version.
> >
> > Also didn't see anywhere in that article about the ability to interrupt the
> > rendering process after let's say 5 seconds so that a single user doesn't
> > take up all resources on the server. Is that something which velocity
> > currently supports/is cheduled to be supported in some future version, or do
> > i have to built in that feature in my application, and if yes, can you
> > please suggest the best route to do this? Is there an interrupt method in
> > velocity which i can call after 5 sec, let's say, which will interrupt the
> > rendering process?
> >
> > Thanks,
> > Ben
> >
> > ----- Original Message -----
> > From: "Nathan Bubna" <nb...@gmail.com>
> > To: "Velocity Users List" <ve...@jakarta.apache.org>
> > Sent: Tuesday, April 04, 2006 2:59 PM
> > Subject: Re: using Velocity in an untrusted environment
> >
> >
> > http://wiki.apache.org/jakarta-velocity/BuildingSecureWebApplications
> >
> > On 4/4/06, Ben <be...@pcguy.org> wrote:
> > > Hi all,
> > >
> > > I am thinking of using Velocity engine in an e-commerce platform, where
> > > the
> > > users will be able to upload their own templates to customize the layout
> > > of
> > > their store. I've read somewhere that Velocity has a built in security
> > > flaw,
> > > where peole could do things like AnyClass.getClassLoader() and use that to
> > > load any java class and basically do anything they want. I've also read
> > > about a patch being developed to address this issue which is scheduled to
> > > be
> > > integrated into Velocity version 1.6
> > >
> > > I'm wondering, when is that version of velocity scheduled to come out, and
> > > are there any other security related issues i should watch out for in my
> > > scenario, where basically people who upload templates are untrusted users.
> > >
> > > Also, does velocity have a built in timeout feature, where for example if
> > > any template takes more than 5 seconds to render, I'll be able to
> > > interrupt
> > > the rendering process? This feature is also important to me, as I don't
> > > want
> > > any single user to tie up all system resources.
> > >
> > > Thanks,
> > > Ben
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
--
Forio Business Simulations
Will Glass-Husain
wglass@forio.com
www.forio.com
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Nathan Bubna <nb...@gmail.com>.
Well, this is really more Will's area of expertise. I have the luxury
of not letting users of my apps define their own templates. So, i've
not had any need to use a JavaSecurityManager.
The language in the article is a little unclear. Version 1.6 has not
started development yet. We are still tweaking 1.5 in our
collectively scarce free time. It's more that the patch in question
has been put on the roadmap for 1.6.
In the meantime, the info and references in
http://issues.apache.org/jira/browse/VELOCITY-179 should provide a
number of options for restricting classloader use in your user's
templates.
As far as interrupting template processing... i've been in this
community for about five years and i can't recall anyone else ever
asking for or needing this. so, no, this is supported nor is it
scheduled to be. i can't imagine that it would be difficult to
implement using Threads. if you come up with something useful here,
you might consider sharing it with the community. :)
On 4/5/06, Ben <be...@pcguy.org> wrote:
> Thanks, that is the page i read about this problem from before. It has a
> link to http://issues.apache.org/jira/browse/VELOCITY-179 which has the
> classloader patch. It aslo says the proposed patch has been accepted for
> velocity version 1.6, hence my question about the scheduled release date of
> that version.
>
> Also didn't see anywhere in that article about the ability to interrupt the
> rendering process after let's say 5 seconds so that a single user doesn't
> take up all resources on the server. Is that something which velocity
> currently supports/is cheduled to be supported in some future version, or do
> i have to built in that feature in my application, and if yes, can you
> please suggest the best route to do this? Is there an interrupt method in
> velocity which i can call after 5 sec, let's say, which will interrupt the
> rendering process?
>
> Thanks,
> Ben
>
> ----- Original Message -----
> From: "Nathan Bubna" <nb...@gmail.com>
> To: "Velocity Users List" <ve...@jakarta.apache.org>
> Sent: Tuesday, April 04, 2006 2:59 PM
> Subject: Re: using Velocity in an untrusted environment
>
>
> http://wiki.apache.org/jakarta-velocity/BuildingSecureWebApplications
>
> On 4/4/06, Ben <be...@pcguy.org> wrote:
> > Hi all,
> >
> > I am thinking of using Velocity engine in an e-commerce platform, where
> > the
> > users will be able to upload their own templates to customize the layout
> > of
> > their store. I've read somewhere that Velocity has a built in security
> > flaw,
> > where peole could do things like AnyClass.getClassLoader() and use that to
> > load any java class and basically do anything they want. I've also read
> > about a patch being developed to address this issue which is scheduled to
> > be
> > integrated into Velocity version 1.6
> >
> > I'm wondering, when is that version of velocity scheduled to come out, and
> > are there any other security related issues i should watch out for in my
> > scenario, where basically people who upload templates are untrusted users.
> >
> > Also, does velocity have a built in timeout feature, where for example if
> > any template takes more than 5 seconds to render, I'll be able to
> > interrupt
> > the rendering process? This feature is also important to me, as I don't
> > want
> > any single user to tie up all system resources.
> >
> > Thanks,
> > Ben
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: velocity-user-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Ben <be...@pcguy.org>.
Thanks, that is the page i read about this problem from before. It has a
link to http://issues.apache.org/jira/browse/VELOCITY-179 which has the
classloader patch. It aslo says the proposed patch has been accepted for
velocity version 1.6, hence my question about the scheduled release date of
that version.
Also didn't see anywhere in that article about the ability to interrupt the
rendering process after let's say 5 seconds so that a single user doesn't
take up all resources on the server. Is that something which velocity
currently supports/is cheduled to be supported in some future version, or do
i have to built in that feature in my application, and if yes, can you
please suggest the best route to do this? Is there an interrupt method in
velocity which i can call after 5 sec, let's say, which will interrupt the
rendering process?
Thanks,
Ben
----- Original Message -----
From: "Nathan Bubna" <nb...@gmail.com>
To: "Velocity Users List" <ve...@jakarta.apache.org>
Sent: Tuesday, April 04, 2006 2:59 PM
Subject: Re: using Velocity in an untrusted environment
http://wiki.apache.org/jakarta-velocity/BuildingSecureWebApplications
On 4/4/06, Ben <be...@pcguy.org> wrote:
> Hi all,
>
> I am thinking of using Velocity engine in an e-commerce platform, where
> the
> users will be able to upload their own templates to customize the layout
> of
> their store. I've read somewhere that Velocity has a built in security
> flaw,
> where peole could do things like AnyClass.getClassLoader() and use that to
> load any java class and basically do anything they want. I've also read
> about a patch being developed to address this issue which is scheduled to
> be
> integrated into Velocity version 1.6
>
> I'm wondering, when is that version of velocity scheduled to come out, and
> are there any other security related issues i should watch out for in my
> scenario, where basically people who upload templates are untrusted users.
>
> Also, does velocity have a built in timeout feature, where for example if
> any template takes more than 5 seconds to render, I'll be able to
> interrupt
> the rendering process? This feature is also important to me, as I don't
> want
> any single user to tie up all system resources.
>
> Thanks,
> Ben
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: using Velocity in an untrusted environment
Posted by Nathan Bubna <nb...@gmail.com>.
http://wiki.apache.org/jakarta-velocity/BuildingSecureWebApplications
On 4/4/06, Ben <be...@pcguy.org> wrote:
> Hi all,
>
> I am thinking of using Velocity engine in an e-commerce platform, where the
> users will be able to upload their own templates to customize the layout of
> their store. I've read somewhere that Velocity has a built in security flaw,
> where peole could do things like AnyClass.getClassLoader() and use that to
> load any java class and basically do anything they want. I've also read
> about a patch being developed to address this issue which is scheduled to be
> integrated into Velocity version 1.6
>
> I'm wondering, when is that version of velocity scheduled to come out, and
> are there any other security related issues i should watch out for in my
> scenario, where basically people who upload templates are untrusted users.
>
> Also, does velocity have a built in timeout feature, where for example if
> any template takes more than 5 seconds to render, I'll be able to interrupt
> the rendering process? This feature is also important to me, as I don't want
> any single user to tie up all system resources.
>
> Thanks,
> Ben
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
using Velocity in an untrusted environment
Posted by Ben <be...@pcguy.org>.
Hi all,
I am thinking of using Velocity engine in an e-commerce platform, where the
users will be able to upload their own templates to customize the layout of
their store. I've read somewhere that Velocity has a built in security flaw,
where peole could do things like AnyClass.getClassLoader() and use that to
load any java class and basically do anything they want. I've also read
about a patch being developed to address this issue which is scheduled to be
integrated into Velocity version 1.6
I'm wondering, when is that version of velocity scheduled to come out, and
are there any other security related issues i should watch out for in my
scenario, where basically people who upload templates are untrusted users.
Also, does velocity have a built in timeout feature, where for example if
any template takes more than 5 seconds to render, I'll be able to interrupt
the rendering process? This feature is also important to me, as I don't want
any single user to tie up all system resources.
Thanks,
Ben
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: How to use DateTool
Posted by Nathan Bubna <nb...@gmail.com>.
You don't have to use a toolbox to use any of the Generic Tools. Just
create an instance and add it to your context before the context is
merged with the template:
context.put("date", new DateTool());
pretty simple.
On 4/4/06, Dominik Bruhn <do...@dbruhn.de> wrote:
> Hy,
> I want to use the DateTool (
> http://jakarta.apache.org/velocity/tools/javadoc/org/apache/velocity/tools/generic/DateTool.html )
> in my VelocityApplet. I not using Struts. How do I tell Velocity to use this
> Tool when outputting Dates. Some tell me about a toolbox.xml, but I don't
> know hwere to place it and with which content. A simple copy of the jar-file
> to the classpath didn't help.
>
> Who can help?
>
> Thanks
> Dominik
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
How to use DateTool
Posted by Dominik Bruhn <do...@dbruhn.de>.
Hy,
I want to use the DateTool (
http://jakarta.apache.org/velocity/tools/javadoc/org/apache/velocity/tools/generic/DateTool.html )
in my VelocityApplet. I not using Struts. How do I tell Velocity to use this
Tool when outputting Dates. Some tell me about a toolbox.xml, but I don't
know hwere to place it and with which content. A simple copy of the jar-file
to the classpath didn't help.
Who can help?
Thanks
Dominik
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: Velocity vs. WebMacro performance
Posted by Ståle Undheim <st...@coretrek.com>.
Will Glass-Husain wrote:
> Did you turn on caching and use getTemplate? Makes a big difference.
>
I changed the init code to read:
Velocity.init();
Velocity.setProperty(Velocity.RUNTIME_LOG_LOGSYSTEM, this);
Velocity.setProperty(Velocity.FILE_RESOURCE_LOADER_CACHE, Boolean.TRUE);
I also changed the code from using Velocity.mergeTemplate to:
StringWriter w = new StringWriter();
Template vt = Velocity.getTemplate(getVMTemplate());
vt.merge(context, w);
The performance was marginally better.
The main problem is of course that there are so many String objects
created on each level of nesting instead of having a common context and
OutputStream to write to. Is there any way of getting this information?
Should I perhaps use thread locals?
> On 4/4/06, Ståle Undheim <st...@coretrek.com> wrote:
>
>>I am currently working on a project where we are evaluating wheter to
>>use Velocity or WebMacro. I did some performance test in relation to how
>>we intend to use either of the parsing engines.
>>
>>We are making a Web UI framework, and its very much based on nesting
>>components. IE. you have a frame, and inside it you have various
>>sections, which again contains components. So you end up with nesting
>>alot of components and their associated template files.
>>
>>In order to test things the way we used it, I made a simple Class that
>>would have an associated VM/WM file, and recurse several levels deep.
>>
>>Full code can be found here:
>>http://apollo.coretrek.com/~staaleu/VelocityPerformance.tar.gz
>>
>>Dependent packages are here:
>>http://apollo.coretrek.com/~staaleu/
>>
>>There is a slight difference in code from WM 1.1 to 2.0, commented on
>>line 91 of NestedObject.java
>>
>>The basic constructor of the class is as follows:
>>
>>________________________________
>> public NestedObject() {
>> this(0);
>> }
>>
>> private NestedObject(int level) {
>> _level = level;
>> for (int i=0; i<level; i++) {
>> _indentation+=" ";
>> }
>> if (_level == LEVELS) {
>> _nested = Collections.EMPTY_LIST;
>> }
>> else {
>> _nested = new ArrayList(CHILD_COUNT);
>> for (int i = 0; i < CHILD_COUNT; i++) {
>> _nested.add(new NestedObject(_level + 1));
>> }
>> }
>> }
>>________________________________
>>
>>So creating a NestedObject, creates a tree of child objects. The Class
>>has an associated VM/WM file that looks like this:
>>________________________________
>>$this.getIndentation()START LEVEL $this.Level
>>#foreach ($obj in $this.Nested)
>>$obj
>>#end
>>$this.getIndentation()END LEVEL $this.Level
>>________________________________
>>
>>Basically recursing several levels deep.
>>
>>For the WebMacro part, I use the Macro interface, which means I get an
>>outputstream in to write each object to. For Velocity I had to use the
>>toString() method, so a seperate String would be created for each level.
>>
>>Here are the test results as I got them:
>>WebMacro 1.1 init memoryConsumption: 1346552
>>Run: 0, memoryConsumption: 926968, time: 439ms, size: 67179
>>Run: 10, memoryConsumption: 467648, time: 54ms, size: 67179
>>Run: 20, memoryConsumption: 468128, time: 54ms, size: 67179
>>Run: 30, memoryConsumption: 468320, time: 54ms, size: 67179
>>Run: 40, memoryConsumption: 468320, time: 64ms, size: 67179
>>Run: 50, memoryConsumption: 468128, time: 56ms, size: 67179
>>Run: 60, memoryConsumption: 468320, time: 66ms, size: 67179
>>Run: 70, memoryConsumption: 468320, time: 57ms, size: 67179
>>Run: 80, memoryConsumption: 468128, time: 55ms, size: 67179
>>Run: 90, memoryConsumption: 468320, time: 54ms, size: 67179
>>Run: 100, memoryConsumption: 468320, time: 54ms, size: 67179
>>Total time for WebMacro: 7416
>>
>>WebMacro 2.0 init memoryConsumption: 456808
>>Run: 0, memoryConsumption: 713456, time: 1668ms, size: 67179
>>Run: 10, memoryConsumption: 608160, time: 145ms, size: 67179
>>Run: 20, memoryConsumption: 608208, time: 346ms, size: 67179
>>Run: 30, memoryConsumption: 608160, time: 157ms, size: 67179
>>Run: 40, memoryConsumption: 580208, time: 154ms, size: 67179
>>Run: 50, memoryConsumption: 608160, time: 146ms, size: 67179
>>Run: 60, memoryConsumption: 608208, time: 1534ms, size: 67179
>>Run: 70, memoryConsumption: 608400, time: 144ms, size: 67179
>>Run: 80, memoryConsumption: 580208, time: 149ms, size: 67179
>>Run: 90, memoryConsumption: 608160, time: 143ms, size: 67179
>>Run: 100, memoryConsumption: 608208, time: 146ms, size: 67179
>>Total time for WebMacro2: 20406
>>
>>Velocity 1.4 init memoryConsumption: 2280416
>>Run: 0, memoryConsumption: 1872776, time: 2063ms, size: 67179
>>Run: 10, memoryConsumption: 1729616, time: 1074ms, size: 67179
>>Run: 20, memoryConsumption: 1707664, time: 1037ms, size: 67179
>>Run: 30, memoryConsumption: 1707624, time: 1076ms, size: 67179
>>Run: 40, memoryConsumption: 1707624, time: 1414ms, size: 67179
>>Run: 50, memoryConsumption: 1707624, time: 1018ms, size: 67179
>>Run: 60, memoryConsumption: 1707624, time: 1037ms, size: 67179
>>Run: 70, memoryConsumption: 1707576, time: 1152ms, size: 67179
>>Run: 80, memoryConsumption: 1707624, time: 1090ms, size: 67179
>>Run: 90, memoryConsumption: 1707624, time: 1501ms, size: 67179
>>Run: 100, memoryConsumption: 1707640, time: 1032ms, size: 67179
>>Total time for Velocity: 130396
>>
>>Please note that WebMacro 2.0 produces errornous output, but thats not
>>for this list.
>>
>>Am I using Velocity totally wrong? Is there interfaces that allows me to
>>nest objects and have good performance?
>>
>>Here is the full code for the NestedObject, which handles the output:
>>http://apollo.coretrek.com/~staaleu/NestedObject.java.txt
>>
>>--
>>Ståle Undheim
>>Systemutvikler
>>
>>CoreTrek
>>Tlf.: +47 51 97 85 97
>>E-post: staaleu@coretrek.no
>>WEB: www.coretrek.no
>>
>>"Simple is beautiful"
>>
>>
>>
>>
>
>
>
> --
> Forio Business Simulations
>
> Will Glass-Husain
> wglass@forio.com
> www.forio.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: velocity-user-help@jakarta.apache.org
>
--
Ståle Undheim
Systemutvikler
CoreTrek
Tlf.: +47 51 97 85 97
E-post: staaleu@coretrek.no
WEB: www.coretrek.no
"Simple is beautiful"
Re: Velocity vs. WebMacro performance
Posted by Will Glass-Husain <wg...@forio.com>.
Did you turn on caching and use getTemplate? Makes a big difference.
WILL
On 4/4/06, Ståle Undheim <st...@coretrek.com> wrote:
> I am currently working on a project where we are evaluating wheter to
> use Velocity or WebMacro. I did some performance test in relation to how
> we intend to use either of the parsing engines.
>
> We are making a Web UI framework, and its very much based on nesting
> components. IE. you have a frame, and inside it you have various
> sections, which again contains components. So you end up with nesting
> alot of components and their associated template files.
>
> In order to test things the way we used it, I made a simple Class that
> would have an associated VM/WM file, and recurse several levels deep.
>
> Full code can be found here:
> http://apollo.coretrek.com/~staaleu/VelocityPerformance.tar.gz
>
> Dependent packages are here:
> http://apollo.coretrek.com/~staaleu/
>
> There is a slight difference in code from WM 1.1 to 2.0, commented on
> line 91 of NestedObject.java
>
> The basic constructor of the class is as follows:
>
> ________________________________
> public NestedObject() {
> this(0);
> }
>
> private NestedObject(int level) {
> _level = level;
> for (int i=0; i<level; i++) {
> _indentation+=" ";
> }
> if (_level == LEVELS) {
> _nested = Collections.EMPTY_LIST;
> }
> else {
> _nested = new ArrayList(CHILD_COUNT);
> for (int i = 0; i < CHILD_COUNT; i++) {
> _nested.add(new NestedObject(_level + 1));
> }
> }
> }
> ________________________________
>
> So creating a NestedObject, creates a tree of child objects. The Class
> has an associated VM/WM file that looks like this:
> ________________________________
> $this.getIndentation()START LEVEL $this.Level
> #foreach ($obj in $this.Nested)
> $obj
> #end
> $this.getIndentation()END LEVEL $this.Level
> ________________________________
>
> Basically recursing several levels deep.
>
> For the WebMacro part, I use the Macro interface, which means I get an
> outputstream in to write each object to. For Velocity I had to use the
> toString() method, so a seperate String would be created for each level.
>
> Here are the test results as I got them:
> WebMacro 1.1 init memoryConsumption: 1346552
> Run: 0, memoryConsumption: 926968, time: 439ms, size: 67179
> Run: 10, memoryConsumption: 467648, time: 54ms, size: 67179
> Run: 20, memoryConsumption: 468128, time: 54ms, size: 67179
> Run: 30, memoryConsumption: 468320, time: 54ms, size: 67179
> Run: 40, memoryConsumption: 468320, time: 64ms, size: 67179
> Run: 50, memoryConsumption: 468128, time: 56ms, size: 67179
> Run: 60, memoryConsumption: 468320, time: 66ms, size: 67179
> Run: 70, memoryConsumption: 468320, time: 57ms, size: 67179
> Run: 80, memoryConsumption: 468128, time: 55ms, size: 67179
> Run: 90, memoryConsumption: 468320, time: 54ms, size: 67179
> Run: 100, memoryConsumption: 468320, time: 54ms, size: 67179
> Total time for WebMacro: 7416
>
> WebMacro 2.0 init memoryConsumption: 456808
> Run: 0, memoryConsumption: 713456, time: 1668ms, size: 67179
> Run: 10, memoryConsumption: 608160, time: 145ms, size: 67179
> Run: 20, memoryConsumption: 608208, time: 346ms, size: 67179
> Run: 30, memoryConsumption: 608160, time: 157ms, size: 67179
> Run: 40, memoryConsumption: 580208, time: 154ms, size: 67179
> Run: 50, memoryConsumption: 608160, time: 146ms, size: 67179
> Run: 60, memoryConsumption: 608208, time: 1534ms, size: 67179
> Run: 70, memoryConsumption: 608400, time: 144ms, size: 67179
> Run: 80, memoryConsumption: 580208, time: 149ms, size: 67179
> Run: 90, memoryConsumption: 608160, time: 143ms, size: 67179
> Run: 100, memoryConsumption: 608208, time: 146ms, size: 67179
> Total time for WebMacro2: 20406
>
> Velocity 1.4 init memoryConsumption: 2280416
> Run: 0, memoryConsumption: 1872776, time: 2063ms, size: 67179
> Run: 10, memoryConsumption: 1729616, time: 1074ms, size: 67179
> Run: 20, memoryConsumption: 1707664, time: 1037ms, size: 67179
> Run: 30, memoryConsumption: 1707624, time: 1076ms, size: 67179
> Run: 40, memoryConsumption: 1707624, time: 1414ms, size: 67179
> Run: 50, memoryConsumption: 1707624, time: 1018ms, size: 67179
> Run: 60, memoryConsumption: 1707624, time: 1037ms, size: 67179
> Run: 70, memoryConsumption: 1707576, time: 1152ms, size: 67179
> Run: 80, memoryConsumption: 1707624, time: 1090ms, size: 67179
> Run: 90, memoryConsumption: 1707624, time: 1501ms, size: 67179
> Run: 100, memoryConsumption: 1707640, time: 1032ms, size: 67179
> Total time for Velocity: 130396
>
> Please note that WebMacro 2.0 produces errornous output, but thats not
> for this list.
>
> Am I using Velocity totally wrong? Is there interfaces that allows me to
> nest objects and have good performance?
>
> Here is the full code for the NestedObject, which handles the output:
> http://apollo.coretrek.com/~staaleu/NestedObject.java.txt
>
> --
> Ståle Undheim
> Systemutvikler
>
> CoreTrek
> Tlf.: +47 51 97 85 97
> E-post: staaleu@coretrek.no
> WEB: www.coretrek.no
>
> "Simple is beautiful"
>
>
>
>
--
Forio Business Simulations
Will Glass-Husain
wglass@forio.com
www.forio.com
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org
Re: Velocity vs. WebMacro performance
Posted by Robert Koberg <ro...@koberg.com>.
Ståle Undheim wrote:
> Am I using Velocity totally wrong? Is there interfaces that allows me to
> nest objects and have good performance?
>
I would not use velocity for something like this - it sounds like you
need XML and possibly XSL.
You could use a SAX filter to send nested XML to the output stream or
form for use on the server. If you can use the browser and send it the
XML to do an XSL transformation you would get the best performance. If
you need to send pure HTML to the client then you could do the
transformation on the server. Either way you can form your nested
structure and style with CSS.
Another alternative is to just send the XML as is and style simply with CSS.
best,
-Rob
---------------------------------------------------------------------
To unsubscribe, e-mail: velocity-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: velocity-user-help@jakarta.apache.org