You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@velocity.apache.org by Pete <pe...@gmail.com> on 2012/07/12 23:27:58 UTC
partial evaluation / removing directives
Hi all,
I'm working with some velocity files where I want to partially evaluate
them, such that the only things actually evaluated are #parse (includes of
other velocity templates in the same directory) and $references to a
resourceBundle variable passed in the context map. In essence, all this is
doing is combining a number of velocity files into one master velocity
file, and localizing any resource bundle references to a given locale. All
the main velocity directives in the velocity file(s) should be left
unevaluated- #set, #ifelse, #foreach etc. This way, the result could be
redistributed and modified specific to a particular locale, with just one
file instead of n based on the #parse structure.
Up till now, I've been doing this in a somewhat roundabout way:
1. setting strict escape property to true
2. pre-processing the velocity templates by regexp replacing all instances
of #if / #else/ #elseif/ #end / #foreach / #define / #break / #stop #set /
#evaluate with \\#{$1}
This adds a an escape character before each of these directives, and I can
then call mergeTemplate on the result. It works OK, sometimes, but I still
run into some parsing issues now and again (eg not a robust solution). It
also seems kind of heavy handed to have to preprocess these files using a
regular expression and saving them before velocity gets a handle on them.
It would even be a bit better if I could do this by reading them in and
doing the rgexp in-memory rather than on the saved files, then using
evaluate instead of mergeTemplate. But I assume that would cause problems
for the #parse directives.
I know velocity added a removeDirective() method in 1.6.2 which seems like
it should be perfect for me - I could simply rip out all the directives but
#parse. However, I tried the and it seems that it only works for certain
directives, eg #foreach, #include, etc. More basic directives like #if
and #elseif are not in the list of runtimedirectives, or even represented
as Directive objects, so removing them doesn't work.
Any suggestions on the best way to proceed with this? Would it be easy to
change a few velocity files locally, recompile and get this behavior? Or is
it sufficiently complex removing all these more basic directives that I'm
better off continuing to try some kind of preprocessing/ escaping?
thanks,
Pete
Re: partial evaluation / removing directives
Posted by Nathan Bubna <nb...@gmail.com>.
On Fri, Jul 13, 2012 at 9:02 AM, Alex Fedotov <al...@kayak.com> wrote:
> Well, I did not get into the details, but the idea would be to replace the
> ASTIfStatement with a sequence of AST nodes that would output the "#IF"
> text followed by the result of rewriting the children, etc. followed by the
> "#END" text.
> I think the SimpleNode does not have a particularly friendly public
> interface to facilitate this kind of thing. I remember I had to resort to
> using reflection and accessing protected/private members directly.
sounds like something that could be improved. you could always open a
JIRA issue for this and/or provide a patch. :)
> Alex
>
>
> As for the AST rewrite idea, if it was possible to "Burn down" nodes to
>> their corresponding text something like that would work, However, wouldnt'
>> that then possibly miss additional child elements in that AST tree? For
>> instance a clause in an if statement which had a resourceBundle reference?
>> I'm not too familiar with ASTs (been years since I dealt with those at
>> school), maybe I can look at it a bit more today.
>>
>> thanks,
>> Pete
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org
For additional commands, e-mail: user-help@velocity.apache.org
Re: partial evaluation / removing directives
Posted by Alex Fedotov <al...@kayak.com>.
Well, I did not get into the details, but the idea would be to replace the
ASTIfStatement with a sequence of AST nodes that would output the "#IF"
text followed by the result of rewriting the children, etc. followed by the
"#END" text.
I think the SimpleNode does not have a particularly friendly public
interface to facilitate this kind of thing. I remember I had to resort to
using reflection and accessing protected/private members directly.
Alex
As for the AST rewrite idea, if it was possible to "Burn down" nodes to
> their corresponding text something like that would work, However, wouldnt'
> that then possibly miss additional child elements in that AST tree? For
> instance a clause in an if statement which had a resourceBundle reference?
> I'm not too familiar with ASTs (been years since I dealt with those at
> school), maybe I can look at it a bit more today.
>
> thanks,
> Pete
>
>
Re: partial evaluation / removing directives
Posted by Pete <pe...@gmail.com>.
Oops, just managed to just get around this with a tiny class decorating
ClasspathResourceLoader and setting that new class as the loader in the
properties. Not sure about that String loader class but I don't need it for
now.
On Thu, Jul 19, 2012 at 9:59 AM, Pete <pe...@gmail.com> wrote:
> Hey, quick question re: modifying the resource loader .. I think I'm going
> to go with this solution since regexp replace seems to work well, hence if
> I can intercept the vm files and fix them before velocity references them
> I'll be in good shape.
>
> The way I tried to do this so far is as follows:
>
>
> StringResourceLoader.setRepository(StringResourceLoader.REPOSITORY_NAME_DEFAULT,new
> StringResourceRepositoryImpl());
> instance = new Velocity();
> Velocity.setProperty(Velocity.FILE_RESOURCE_LOADER_CACHE, false);
>
> Velocity.setProperty(Velocity.FILE_RESOURCE_LOADER_PATH,
> TEMPLATE_LOCATION);
>
> Velocity.addProperty(Velocity.RESOURCE_LOADER, "classpath");
> Velocity.addProperty(Velocity.RUNTIME_REFERENCES_STRICT +
> ".escape","true");
> Velocity.setProperty("classpath." + Velocity.RESOURCE_LOADER +
> ".class",
> ClasspathResourceLoader.class.getName());
> Velocity.setProperty("classpath." + Velocity.RESOURCE_LOADER +
> ".cache", "false");
> Velocity.setProperty("classpath." + Velocity.RESOURCE_LOADER +
> ".modificationCheckInterval", "2");
> Velocity.setProperty("loader", "string");
> Velocity.setProperty("string.resource.loader.class",
> "org.apache.velocity.runtime.resource.loader.StringResourceLoader");
>
> Velocity.setProperty("string.resource.loader.repository.class","org.apache.velocity.runtime.resource.util.StringResourceRepositoryImpl");
> Velocity.init();
>
>
> Following this, I iterate over all my known velocity templates and try to
> put them in the string resource loader repository, using something like
> this
> repository.putStringResource((template path + name),
> sanitizedAlert);
> it was my understanding that putting these in, when I then call
> mergeTemplate(), rather than using the files on the hard drive it would use
> the copies in this resource loader repository.
> However, it seems to still default to checking the classpath file loader,
> and debugging in I didn't see any sort of conditonal check based on any of
> the properties I set which wold make it use this resource loader repository
> instead.
>
> Am I doing this totally wrong? Is the String Resource Loader Repository
> not what I want here?
>
> thanks,
> Pete
>
>
> On Fri, Jul 13, 2012 at 11:25 AM, Nathan Bubna <nb...@gmail.com> wrote:
>
>> On Fri, Jul 13, 2012 at 8:12 AM, Pete <pe...@gmail.com> wrote:
>> > Thanks for the info guys. To answer that one question
>> >> Why would evaluate cause problems for #parse?? If the engine is
>> >> configured the same, the #parse should work the same whether you enter
>> >> via file or string.
>> > The problem would be even if the #parse directives get picked up OK in
>> my
>> > in-memory, preprocessed string version of the file (since I set template
>> > location path), Velocity would just go and consume those additional
>> > template files on its own, not giving me a chance to preprocess them
>> first.
>> > Maybe I could avoid this if there was way to extend /replacevelocity's
>> > template file loader, I haven't looked at the code enough to know if
>> this
>> > is possible
>>
>> oh yeah, duh. and yes, you can definitely make your own resource
>> loaders. those are pluggable. :)
>>
>> > That's an interesting suggestion re: the other templating Engine... I
>> > considered trying to home-brew my own (just looking for those two
>> > directives), but it seemed like a waste since there's already such
>> powerful
>> > and robust ones out there. Also, there'sa bit of subtley there since it
>> can
>> > be recursive ($resourceBundles can themselves include references to
>> #parse,
>> > which can include references to resource bundles, etc). If there was a
>> > really simple, customizable template engine one that would let me choose
>> > exactly what was parsed, that would probably be best.
>>
>> Hmm. i'm not aware of one, but i'm much less active in this space
>> than i used to be.
>>
>> > My only issue using
>> > another one similar in size to velocity is that there might be some
>> "side
>> > effects", eg statements which are ignored by velocity but which get
>> > interpreted somehow through this other engine due to its own particular
>> > syntax.
>>
>> yeah, that's a concern.
>>
>> > As for the AST rewrite idea, if it was possible to "Burn down" nodes to
>> > their corresponding text something like that would work, However,
>> wouldnt'
>> > that then possibly miss additional child elements in that AST tree? For
>> > instance a clause in an if statement which had a resourceBundle
>> reference?
>> > I'm not too familiar with ASTs (been years since I dealt with those at
>> > school), maybe I can look at it a bit more today.
>>
>> worth a look. it sounded like a reasonable approach to me.
>> definitely easier than trying to remove baked-in directives.
>>
>> >
>> > thanks,
>> > Pete
>> >
>> > On Fri, Jul 13, 2012 at 9:57 AM, Alex Fedotov <al...@kayak.com> wrote:
>> >
>> >> Another option may be rewriting the AST tree after the template is
>> parsed
>> >> and replacing some nodes with text or something else.
>> >> I tried that on the side as an experiment to inject some security
>> checks
>> >> and collect profiling info for macros and templates.
>> >>
>> >> Alex
>> >>
>> >>
>> >>
>> >> On Fri, Jul 13, 2012 at 9:50 AM, Nathan Bubna <nb...@gmail.com>
>> wrote:
>> >>
>> >> > On Thu, Jul 12, 2012 at 2:27 PM, Pete <pe...@gmail.com>
>> wrote:
>> >> > > Hi all,
>> >> > > I'm working with some velocity files where I want to partially
>> evaluate
>> >> > > them, such that the only things actually evaluated are #parse
>> (includes
>> >> > of
>> >> > > other velocity templates in the same directory) and $references
>> to a
>> >> > > resourceBundle variable passed in the context map. In essence, all
>> >> this
>> >> > is
>> >> > > doing is combining a number of velocity files into one master
>> velocity
>> >> > > file, and localizing any resource bundle references to a given
>> locale.
>> >> > All
>> >> > > the main velocity directives in the velocity file(s) should be left
>> >> > > unevaluated- #set, #ifelse, #foreach etc. This way, the result
>> could
>> >> be
>> >> > > redistributed and modified specific to a particular locale, with
>> just
>> >> one
>> >> > > file instead of n based on the #parse structure.
>> >> >
>> >> > interesting challenge! glad i don't have to do it though. :)
>> >> >
>> >> > > Up till now, I've been doing this in a somewhat roundabout way:
>> >> > > 1. setting strict escape property to true
>> >> > > 2. pre-processing the velocity templates by regexp replacing all
>> >> > instances
>> >> > > of #if / #else/ #elseif/ #end / #foreach / #define / #break / #stop
>> >> > #set /
>> >> > > #evaluate with \\#{$1}
>> >> > > This adds a an escape character before each of these directives,
>> and I
>> >> > can
>> >> > > then call mergeTemplate on the result. It works OK, sometimes,
>> but I
>> >> > still
>> >> > > run into some parsing issues now and again (eg not a robust
>> solution).
>> >> It
>> >> > > also seems kind of heavy handed to have to preprocess these files
>> >> using a
>> >> > > regular expression and saving them before velocity gets a handle on
>> >> them.
>> >> > > It would even be a bit better if I could do this by reading them
>> in and
>> >> > > doing the rgexp in-memory rather than on the saved files, then
>> using
>> >> > > evaluate instead of mergeTemplate. But I assume that would cause
>> >> problems
>> >> > > for the #parse directives.
>> >> >
>> >> > Why would evaluate cause problems for #parse?? If the engine is
>> >> > configured the same, the #parse should work the same whether you
>> enter
>> >> > via file or string.
>> >> >
>> >> > > I know velocity added a removeDirective() method in 1.6.2 which
>> seems
>> >> > like
>> >> > > it should be perfect for me - I could simply rip out all the
>> directives
>> >> > but
>> >> > > #parse. However, I tried the and it seems that it only works for
>> >> certain
>> >> > > directives, eg #foreach, #include, etc. More basic directives
>> like
>> >> #if
>> >> > > and #elseif are not in the list of runtimedirectives, or even
>> >> represented
>> >> > > as Directive objects, so removing them doesn't work.
>> >> >
>> >> > Yeah, they are deep in the parser. Their structure doesn't work as a
>> >> > normal directive.
>> >> >
>> >> > > Any suggestions on the best way to proceed with this? Would it be
>> easy
>> >> to
>> >> > > change a few velocity files locally, recompile and get this
>> behavior?
>> >> Or
>> >> > is
>> >> > > it sufficiently complex removing all these more basic directives
>> that
>> >> I'm
>> >> > > better off continuing to try some kind of preprocessing/ escaping?
>> >> >
>> >> > You would have to redesign Velocity's parsing deeply. A major
>> >> > overhaul. You're best off preprocessing. Have you considered using
>> >> > a different template engine to do the first part (the parse and
>> >> > resourceBundle resolution)? Might be overkill, but might be easier,
>> >> > so long as the syntax has no overlap with VTL.
>> >> >
>> >> > > thanks,
>> >> > > Pete
>> >> >
>> >> > ---------------------------------------------------------------------
>> >> > To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org
>> >> > For additional commands, e-mail: user-help@velocity.apache.org
>> >> >
>> >> >
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org
>> For additional commands, e-mail: user-help@velocity.apache.org
>>
>>
>
Re: partial evaluation / removing directives
Posted by Pete <pe...@gmail.com>.
Hey, quick question re: modifying the resource loader .. I think I'm going
to go with this solution since regexp replace seems to work well, hence if
I can intercept the vm files and fix them before velocity references them
I'll be in good shape.
The way I tried to do this so far is as follows:
StringResourceLoader.setRepository(StringResourceLoader.REPOSITORY_NAME_DEFAULT,new
StringResourceRepositoryImpl());
instance = new Velocity();
Velocity.setProperty(Velocity.FILE_RESOURCE_LOADER_CACHE, false);
Velocity.setProperty(Velocity.FILE_RESOURCE_LOADER_PATH,
TEMPLATE_LOCATION);
Velocity.addProperty(Velocity.RESOURCE_LOADER, "classpath");
Velocity.addProperty(Velocity.RUNTIME_REFERENCES_STRICT +
".escape","true");
Velocity.setProperty("classpath." + Velocity.RESOURCE_LOADER +
".class",
ClasspathResourceLoader.class.getName());
Velocity.setProperty("classpath." + Velocity.RESOURCE_LOADER +
".cache", "false");
Velocity.setProperty("classpath." + Velocity.RESOURCE_LOADER +
".modificationCheckInterval", "2");
Velocity.setProperty("loader", "string");
Velocity.setProperty("string.resource.loader.class",
"org.apache.velocity.runtime.resource.loader.StringResourceLoader");
Velocity.setProperty("string.resource.loader.repository.class","org.apache.velocity.runtime.resource.util.StringResourceRepositoryImpl");
Velocity.init();
Following this, I iterate over all my known velocity templates and try to
put them in the string resource loader repository, using something like
this
repository.putStringResource((template path + name),
sanitizedAlert);
it was my understanding that putting these in, when I then call
mergeTemplate(), rather than using the files on the hard drive it would use
the copies in this resource loader repository.
However, it seems to still default to checking the classpath file loader,
and debugging in I didn't see any sort of conditonal check based on any of
the properties I set which wold make it use this resource loader repository
instead.
Am I doing this totally wrong? Is the String Resource Loader Repository not
what I want here?
thanks,
Pete
On Fri, Jul 13, 2012 at 11:25 AM, Nathan Bubna <nb...@gmail.com> wrote:
> On Fri, Jul 13, 2012 at 8:12 AM, Pete <pe...@gmail.com> wrote:
> > Thanks for the info guys. To answer that one question
> >> Why would evaluate cause problems for #parse?? If the engine is
> >> configured the same, the #parse should work the same whether you enter
> >> via file or string.
> > The problem would be even if the #parse directives get picked up OK in my
> > in-memory, preprocessed string version of the file (since I set template
> > location path), Velocity would just go and consume those additional
> > template files on its own, not giving me a chance to preprocess them
> first.
> > Maybe I could avoid this if there was way to extend /replacevelocity's
> > template file loader, I haven't looked at the code enough to know if this
> > is possible
>
> oh yeah, duh. and yes, you can definitely make your own resource
> loaders. those are pluggable. :)
>
> > That's an interesting suggestion re: the other templating Engine... I
> > considered trying to home-brew my own (just looking for those two
> > directives), but it seemed like a waste since there's already such
> powerful
> > and robust ones out there. Also, there'sa bit of subtley there since it
> can
> > be recursive ($resourceBundles can themselves include references to
> #parse,
> > which can include references to resource bundles, etc). If there was a
> > really simple, customizable template engine one that would let me choose
> > exactly what was parsed, that would probably be best.
>
> Hmm. i'm not aware of one, but i'm much less active in this space
> than i used to be.
>
> > My only issue using
> > another one similar in size to velocity is that there might be some "side
> > effects", eg statements which are ignored by velocity but which get
> > interpreted somehow through this other engine due to its own particular
> > syntax.
>
> yeah, that's a concern.
>
> > As for the AST rewrite idea, if it was possible to "Burn down" nodes to
> > their corresponding text something like that would work, However,
> wouldnt'
> > that then possibly miss additional child elements in that AST tree? For
> > instance a clause in an if statement which had a resourceBundle
> reference?
> > I'm not too familiar with ASTs (been years since I dealt with those at
> > school), maybe I can look at it a bit more today.
>
> worth a look. it sounded like a reasonable approach to me.
> definitely easier than trying to remove baked-in directives.
>
> >
> > thanks,
> > Pete
> >
> > On Fri, Jul 13, 2012 at 9:57 AM, Alex Fedotov <al...@kayak.com> wrote:
> >
> >> Another option may be rewriting the AST tree after the template is
> parsed
> >> and replacing some nodes with text or something else.
> >> I tried that on the side as an experiment to inject some security checks
> >> and collect profiling info for macros and templates.
> >>
> >> Alex
> >>
> >>
> >>
> >> On Fri, Jul 13, 2012 at 9:50 AM, Nathan Bubna <nb...@gmail.com> wrote:
> >>
> >> > On Thu, Jul 12, 2012 at 2:27 PM, Pete <pe...@gmail.com> wrote:
> >> > > Hi all,
> >> > > I'm working with some velocity files where I want to partially
> evaluate
> >> > > them, such that the only things actually evaluated are #parse
> (includes
> >> > of
> >> > > other velocity templates in the same directory) and $references to
> a
> >> > > resourceBundle variable passed in the context map. In essence, all
> >> this
> >> > is
> >> > > doing is combining a number of velocity files into one master
> velocity
> >> > > file, and localizing any resource bundle references to a given
> locale.
> >> > All
> >> > > the main velocity directives in the velocity file(s) should be left
> >> > > unevaluated- #set, #ifelse, #foreach etc. This way, the result
> could
> >> be
> >> > > redistributed and modified specific to a particular locale, with
> just
> >> one
> >> > > file instead of n based on the #parse structure.
> >> >
> >> > interesting challenge! glad i don't have to do it though. :)
> >> >
> >> > > Up till now, I've been doing this in a somewhat roundabout way:
> >> > > 1. setting strict escape property to true
> >> > > 2. pre-processing the velocity templates by regexp replacing all
> >> > instances
> >> > > of #if / #else/ #elseif/ #end / #foreach / #define / #break / #stop
> >> > #set /
> >> > > #evaluate with \\#{$1}
> >> > > This adds a an escape character before each of these directives,
> and I
> >> > can
> >> > > then call mergeTemplate on the result. It works OK, sometimes, but
> I
> >> > still
> >> > > run into some parsing issues now and again (eg not a robust
> solution).
> >> It
> >> > > also seems kind of heavy handed to have to preprocess these files
> >> using a
> >> > > regular expression and saving them before velocity gets a handle on
> >> them.
> >> > > It would even be a bit better if I could do this by reading them in
> and
> >> > > doing the rgexp in-memory rather than on the saved files, then using
> >> > > evaluate instead of mergeTemplate. But I assume that would cause
> >> problems
> >> > > for the #parse directives.
> >> >
> >> > Why would evaluate cause problems for #parse?? If the engine is
> >> > configured the same, the #parse should work the same whether you enter
> >> > via file or string.
> >> >
> >> > > I know velocity added a removeDirective() method in 1.6.2 which
> seems
> >> > like
> >> > > it should be perfect for me - I could simply rip out all the
> directives
> >> > but
> >> > > #parse. However, I tried the and it seems that it only works for
> >> certain
> >> > > directives, eg #foreach, #include, etc. More basic directives like
> >> #if
> >> > > and #elseif are not in the list of runtimedirectives, or even
> >> represented
> >> > > as Directive objects, so removing them doesn't work.
> >> >
> >> > Yeah, they are deep in the parser. Their structure doesn't work as a
> >> > normal directive.
> >> >
> >> > > Any suggestions on the best way to proceed with this? Would it be
> easy
> >> to
> >> > > change a few velocity files locally, recompile and get this
> behavior?
> >> Or
> >> > is
> >> > > it sufficiently complex removing all these more basic directives
> that
> >> I'm
> >> > > better off continuing to try some kind of preprocessing/ escaping?
> >> >
> >> > You would have to redesign Velocity's parsing deeply. A major
> >> > overhaul. You're best off preprocessing. Have you considered using
> >> > a different template engine to do the first part (the parse and
> >> > resourceBundle resolution)? Might be overkill, but might be easier,
> >> > so long as the syntax has no overlap with VTL.
> >> >
> >> > > thanks,
> >> > > Pete
> >> >
> >> > ---------------------------------------------------------------------
> >> > To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org
> >> > For additional commands, e-mail: user-help@velocity.apache.org
> >> >
> >> >
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org
> For additional commands, e-mail: user-help@velocity.apache.org
>
>
Re: partial evaluation / removing directives
Posted by Nathan Bubna <nb...@gmail.com>.
On Fri, Jul 13, 2012 at 8:12 AM, Pete <pe...@gmail.com> wrote:
> Thanks for the info guys. To answer that one question
>> Why would evaluate cause problems for #parse?? If the engine is
>> configured the same, the #parse should work the same whether you enter
>> via file or string.
> The problem would be even if the #parse directives get picked up OK in my
> in-memory, preprocessed string version of the file (since I set template
> location path), Velocity would just go and consume those additional
> template files on its own, not giving me a chance to preprocess them first.
> Maybe I could avoid this if there was way to extend /replacevelocity's
> template file loader, I haven't looked at the code enough to know if this
> is possible
oh yeah, duh. and yes, you can definitely make your own resource
loaders. those are pluggable. :)
> That's an interesting suggestion re: the other templating Engine... I
> considered trying to home-brew my own (just looking for those two
> directives), but it seemed like a waste since there's already such powerful
> and robust ones out there. Also, there'sa bit of subtley there since it can
> be recursive ($resourceBundles can themselves include references to #parse,
> which can include references to resource bundles, etc). If there was a
> really simple, customizable template engine one that would let me choose
> exactly what was parsed, that would probably be best.
Hmm. i'm not aware of one, but i'm much less active in this space
than i used to be.
> My only issue using
> another one similar in size to velocity is that there might be some "side
> effects", eg statements which are ignored by velocity but which get
> interpreted somehow through this other engine due to its own particular
> syntax.
yeah, that's a concern.
> As for the AST rewrite idea, if it was possible to "Burn down" nodes to
> their corresponding text something like that would work, However, wouldnt'
> that then possibly miss additional child elements in that AST tree? For
> instance a clause in an if statement which had a resourceBundle reference?
> I'm not too familiar with ASTs (been years since I dealt with those at
> school), maybe I can look at it a bit more today.
worth a look. it sounded like a reasonable approach to me.
definitely easier than trying to remove baked-in directives.
>
> thanks,
> Pete
>
> On Fri, Jul 13, 2012 at 9:57 AM, Alex Fedotov <al...@kayak.com> wrote:
>
>> Another option may be rewriting the AST tree after the template is parsed
>> and replacing some nodes with text or something else.
>> I tried that on the side as an experiment to inject some security checks
>> and collect profiling info for macros and templates.
>>
>> Alex
>>
>>
>>
>> On Fri, Jul 13, 2012 at 9:50 AM, Nathan Bubna <nb...@gmail.com> wrote:
>>
>> > On Thu, Jul 12, 2012 at 2:27 PM, Pete <pe...@gmail.com> wrote:
>> > > Hi all,
>> > > I'm working with some velocity files where I want to partially evaluate
>> > > them, such that the only things actually evaluated are #parse (includes
>> > of
>> > > other velocity templates in the same directory) and $references to a
>> > > resourceBundle variable passed in the context map. In essence, all
>> this
>> > is
>> > > doing is combining a number of velocity files into one master velocity
>> > > file, and localizing any resource bundle references to a given locale.
>> > All
>> > > the main velocity directives in the velocity file(s) should be left
>> > > unevaluated- #set, #ifelse, #foreach etc. This way, the result could
>> be
>> > > redistributed and modified specific to a particular locale, with just
>> one
>> > > file instead of n based on the #parse structure.
>> >
>> > interesting challenge! glad i don't have to do it though. :)
>> >
>> > > Up till now, I've been doing this in a somewhat roundabout way:
>> > > 1. setting strict escape property to true
>> > > 2. pre-processing the velocity templates by regexp replacing all
>> > instances
>> > > of #if / #else/ #elseif/ #end / #foreach / #define / #break / #stop
>> > #set /
>> > > #evaluate with \\#{$1}
>> > > This adds a an escape character before each of these directives, and I
>> > can
>> > > then call mergeTemplate on the result. It works OK, sometimes, but I
>> > still
>> > > run into some parsing issues now and again (eg not a robust solution).
>> It
>> > > also seems kind of heavy handed to have to preprocess these files
>> using a
>> > > regular expression and saving them before velocity gets a handle on
>> them.
>> > > It would even be a bit better if I could do this by reading them in and
>> > > doing the rgexp in-memory rather than on the saved files, then using
>> > > evaluate instead of mergeTemplate. But I assume that would cause
>> problems
>> > > for the #parse directives.
>> >
>> > Why would evaluate cause problems for #parse?? If the engine is
>> > configured the same, the #parse should work the same whether you enter
>> > via file or string.
>> >
>> > > I know velocity added a removeDirective() method in 1.6.2 which seems
>> > like
>> > > it should be perfect for me - I could simply rip out all the directives
>> > but
>> > > #parse. However, I tried the and it seems that it only works for
>> certain
>> > > directives, eg #foreach, #include, etc. More basic directives like
>> #if
>> > > and #elseif are not in the list of runtimedirectives, or even
>> represented
>> > > as Directive objects, so removing them doesn't work.
>> >
>> > Yeah, they are deep in the parser. Their structure doesn't work as a
>> > normal directive.
>> >
>> > > Any suggestions on the best way to proceed with this? Would it be easy
>> to
>> > > change a few velocity files locally, recompile and get this behavior?
>> Or
>> > is
>> > > it sufficiently complex removing all these more basic directives that
>> I'm
>> > > better off continuing to try some kind of preprocessing/ escaping?
>> >
>> > You would have to redesign Velocity's parsing deeply. A major
>> > overhaul. You're best off preprocessing. Have you considered using
>> > a different template engine to do the first part (the parse and
>> > resourceBundle resolution)? Might be overkill, but might be easier,
>> > so long as the syntax has no overlap with VTL.
>> >
>> > > thanks,
>> > > Pete
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org
>> > For additional commands, e-mail: user-help@velocity.apache.org
>> >
>> >
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org
For additional commands, e-mail: user-help@velocity.apache.org
Re: partial evaluation / removing directives
Posted by Pete <pe...@gmail.com>.
Thanks for the info guys. To answer that one question
> Why would evaluate cause problems for #parse?? If the engine is
> configured the same, the #parse should work the same whether you enter
> via file or string.
The problem would be even if the #parse directives get picked up OK in my
in-memory, preprocessed string version of the file (since I set template
location path), Velocity would just go and consume those additional
template files on its own, not giving me a chance to preprocess them first.
Maybe I could avoid this if there was way to extend /replacevelocity's
template file loader, I haven't looked at the code enough to know if this
is possible
That's an interesting suggestion re: the other templating Engine... I
considered trying to home-brew my own (just looking for those two
directives), but it seemed like a waste since there's already such powerful
and robust ones out there. Also, there'sa bit of subtley there since it can
be recursive ($resourceBundles can themselves include references to #parse,
which can include references to resource bundles, etc). If there was a
really simple, customizable template engine one that would let me choose
exactly what was parsed, that would probably be best. My only issue using
another one similar in size to velocity is that there might be some "side
effects", eg statements which are ignored by velocity but which get
interpreted somehow through this other engine due to its own particular
syntax.
As for the AST rewrite idea, if it was possible to "Burn down" nodes to
their corresponding text something like that would work, However, wouldnt'
that then possibly miss additional child elements in that AST tree? For
instance a clause in an if statement which had a resourceBundle reference?
I'm not too familiar with ASTs (been years since I dealt with those at
school), maybe I can look at it a bit more today.
thanks,
Pete
On Fri, Jul 13, 2012 at 9:57 AM, Alex Fedotov <al...@kayak.com> wrote:
> Another option may be rewriting the AST tree after the template is parsed
> and replacing some nodes with text or something else.
> I tried that on the side as an experiment to inject some security checks
> and collect profiling info for macros and templates.
>
> Alex
>
>
>
> On Fri, Jul 13, 2012 at 9:50 AM, Nathan Bubna <nb...@gmail.com> wrote:
>
> > On Thu, Jul 12, 2012 at 2:27 PM, Pete <pe...@gmail.com> wrote:
> > > Hi all,
> > > I'm working with some velocity files where I want to partially evaluate
> > > them, such that the only things actually evaluated are #parse (includes
> > of
> > > other velocity templates in the same directory) and $references to a
> > > resourceBundle variable passed in the context map. In essence, all
> this
> > is
> > > doing is combining a number of velocity files into one master velocity
> > > file, and localizing any resource bundle references to a given locale.
> > All
> > > the main velocity directives in the velocity file(s) should be left
> > > unevaluated- #set, #ifelse, #foreach etc. This way, the result could
> be
> > > redistributed and modified specific to a particular locale, with just
> one
> > > file instead of n based on the #parse structure.
> >
> > interesting challenge! glad i don't have to do it though. :)
> >
> > > Up till now, I've been doing this in a somewhat roundabout way:
> > > 1. setting strict escape property to true
> > > 2. pre-processing the velocity templates by regexp replacing all
> > instances
> > > of #if / #else/ #elseif/ #end / #foreach / #define / #break / #stop
> > #set /
> > > #evaluate with \\#{$1}
> > > This adds a an escape character before each of these directives, and I
> > can
> > > then call mergeTemplate on the result. It works OK, sometimes, but I
> > still
> > > run into some parsing issues now and again (eg not a robust solution).
> It
> > > also seems kind of heavy handed to have to preprocess these files
> using a
> > > regular expression and saving them before velocity gets a handle on
> them.
> > > It would even be a bit better if I could do this by reading them in and
> > > doing the rgexp in-memory rather than on the saved files, then using
> > > evaluate instead of mergeTemplate. But I assume that would cause
> problems
> > > for the #parse directives.
> >
> > Why would evaluate cause problems for #parse?? If the engine is
> > configured the same, the #parse should work the same whether you enter
> > via file or string.
> >
> > > I know velocity added a removeDirective() method in 1.6.2 which seems
> > like
> > > it should be perfect for me - I could simply rip out all the directives
> > but
> > > #parse. However, I tried the and it seems that it only works for
> certain
> > > directives, eg #foreach, #include, etc. More basic directives like
> #if
> > > and #elseif are not in the list of runtimedirectives, or even
> represented
> > > as Directive objects, so removing them doesn't work.
> >
> > Yeah, they are deep in the parser. Their structure doesn't work as a
> > normal directive.
> >
> > > Any suggestions on the best way to proceed with this? Would it be easy
> to
> > > change a few velocity files locally, recompile and get this behavior?
> Or
> > is
> > > it sufficiently complex removing all these more basic directives that
> I'm
> > > better off continuing to try some kind of preprocessing/ escaping?
> >
> > You would have to redesign Velocity's parsing deeply. A major
> > overhaul. You're best off preprocessing. Have you considered using
> > a different template engine to do the first part (the parse and
> > resourceBundle resolution)? Might be overkill, but might be easier,
> > so long as the syntax has no overlap with VTL.
> >
> > > thanks,
> > > Pete
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org
> > For additional commands, e-mail: user-help@velocity.apache.org
> >
> >
>
Re: partial evaluation / removing directives
Posted by Alex Fedotov <al...@kayak.com>.
Another option may be rewriting the AST tree after the template is parsed
and replacing some nodes with text or something else.
I tried that on the side as an experiment to inject some security checks
and collect profiling info for macros and templates.
Alex
On Fri, Jul 13, 2012 at 9:50 AM, Nathan Bubna <nb...@gmail.com> wrote:
> On Thu, Jul 12, 2012 at 2:27 PM, Pete <pe...@gmail.com> wrote:
> > Hi all,
> > I'm working with some velocity files where I want to partially evaluate
> > them, such that the only things actually evaluated are #parse (includes
> of
> > other velocity templates in the same directory) and $references to a
> > resourceBundle variable passed in the context map. In essence, all this
> is
> > doing is combining a number of velocity files into one master velocity
> > file, and localizing any resource bundle references to a given locale.
> All
> > the main velocity directives in the velocity file(s) should be left
> > unevaluated- #set, #ifelse, #foreach etc. This way, the result could be
> > redistributed and modified specific to a particular locale, with just one
> > file instead of n based on the #parse structure.
>
> interesting challenge! glad i don't have to do it though. :)
>
> > Up till now, I've been doing this in a somewhat roundabout way:
> > 1. setting strict escape property to true
> > 2. pre-processing the velocity templates by regexp replacing all
> instances
> > of #if / #else/ #elseif/ #end / #foreach / #define / #break / #stop
> #set /
> > #evaluate with \\#{$1}
> > This adds a an escape character before each of these directives, and I
> can
> > then call mergeTemplate on the result. It works OK, sometimes, but I
> still
> > run into some parsing issues now and again (eg not a robust solution). It
> > also seems kind of heavy handed to have to preprocess these files using a
> > regular expression and saving them before velocity gets a handle on them.
> > It would even be a bit better if I could do this by reading them in and
> > doing the rgexp in-memory rather than on the saved files, then using
> > evaluate instead of mergeTemplate. But I assume that would cause problems
> > for the #parse directives.
>
> Why would evaluate cause problems for #parse?? If the engine is
> configured the same, the #parse should work the same whether you enter
> via file or string.
>
> > I know velocity added a removeDirective() method in 1.6.2 which seems
> like
> > it should be perfect for me - I could simply rip out all the directives
> but
> > #parse. However, I tried the and it seems that it only works for certain
> > directives, eg #foreach, #include, etc. More basic directives like #if
> > and #elseif are not in the list of runtimedirectives, or even represented
> > as Directive objects, so removing them doesn't work.
>
> Yeah, they are deep in the parser. Their structure doesn't work as a
> normal directive.
>
> > Any suggestions on the best way to proceed with this? Would it be easy to
> > change a few velocity files locally, recompile and get this behavior? Or
> is
> > it sufficiently complex removing all these more basic directives that I'm
> > better off continuing to try some kind of preprocessing/ escaping?
>
> You would have to redesign Velocity's parsing deeply. A major
> overhaul. You're best off preprocessing. Have you considered using
> a different template engine to do the first part (the parse and
> resourceBundle resolution)? Might be overkill, but might be easier,
> so long as the syntax has no overlap with VTL.
>
> > thanks,
> > Pete
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org
> For additional commands, e-mail: user-help@velocity.apache.org
>
>
Re: partial evaluation / removing directives
Posted by Nathan Bubna <nb...@gmail.com>.
On Thu, Jul 12, 2012 at 2:27 PM, Pete <pe...@gmail.com> wrote:
> Hi all,
> I'm working with some velocity files where I want to partially evaluate
> them, such that the only things actually evaluated are #parse (includes of
> other velocity templates in the same directory) and $references to a
> resourceBundle variable passed in the context map. In essence, all this is
> doing is combining a number of velocity files into one master velocity
> file, and localizing any resource bundle references to a given locale. All
> the main velocity directives in the velocity file(s) should be left
> unevaluated- #set, #ifelse, #foreach etc. This way, the result could be
> redistributed and modified specific to a particular locale, with just one
> file instead of n based on the #parse structure.
interesting challenge! glad i don't have to do it though. :)
> Up till now, I've been doing this in a somewhat roundabout way:
> 1. setting strict escape property to true
> 2. pre-processing the velocity templates by regexp replacing all instances
> of #if / #else/ #elseif/ #end / #foreach / #define / #break / #stop #set /
> #evaluate with \\#{$1}
> This adds a an escape character before each of these directives, and I can
> then call mergeTemplate on the result. It works OK, sometimes, but I still
> run into some parsing issues now and again (eg not a robust solution). It
> also seems kind of heavy handed to have to preprocess these files using a
> regular expression and saving them before velocity gets a handle on them.
> It would even be a bit better if I could do this by reading them in and
> doing the rgexp in-memory rather than on the saved files, then using
> evaluate instead of mergeTemplate. But I assume that would cause problems
> for the #parse directives.
Why would evaluate cause problems for #parse?? If the engine is
configured the same, the #parse should work the same whether you enter
via file or string.
> I know velocity added a removeDirective() method in 1.6.2 which seems like
> it should be perfect for me - I could simply rip out all the directives but
> #parse. However, I tried the and it seems that it only works for certain
> directives, eg #foreach, #include, etc. More basic directives like #if
> and #elseif are not in the list of runtimedirectives, or even represented
> as Directive objects, so removing them doesn't work.
Yeah, they are deep in the parser. Their structure doesn't work as a
normal directive.
> Any suggestions on the best way to proceed with this? Would it be easy to
> change a few velocity files locally, recompile and get this behavior? Or is
> it sufficiently complex removing all these more basic directives that I'm
> better off continuing to try some kind of preprocessing/ escaping?
You would have to redesign Velocity's parsing deeply. A major
overhaul. You're best off preprocessing. Have you considered using
a different template engine to do the first part (the parse and
resourceBundle resolution)? Might be overkill, but might be easier,
so long as the syntax has no overlap with VTL.
> thanks,
> Pete
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@velocity.apache.org
For additional commands, e-mail: user-help@velocity.apache.org