Including macros using #parse

[Supun Kamburugamuva <su...@gmail.com>]

Hi,

I was looking into the issue of including macros via the #parse
method.  Following is the my understanding of the problem. The main
reason that causes this to be a hard problem to solve without touching
the parser implementation is described below.

At the moment when the parser encounter a call to a macro i.e.
#foo(123) it looks weather a macro definition exists via the runtime
system. If a definition exists everything is going well. This happens
in the AST building face of the parser (not in the rendering face). So
this means with the current implementation, the macros must be
registered with the runtime system before a call occurs and this is
the cause of all the troubles.

Now this is the problem with #parse directive. #parse directive builds
the AST of the files and render them at the same time. This happens
when the main template that include the #parse directive get rendered.
But when this happens, AST of the template that include the #parse
directive is been built already. But the macro calls that are in the
main template get invalidated because at the moment AST is built
macros are not registered with the runtime system.

As I can see the solution for this problem is, parser should not look
into the runtime system to determine weather a directive like #foo()
is a macro call or not. Instead parser should look ahead for the '('
token  when it encounters something like #foo to deside weather it is
a macro call or not.

I would like to know weather this is solvable without breaking the
current system in the way that I have decribed above?

Regards,
Supun.

P.S. If we can do this my first implementation of 'including macros
programmatically' can be done using this method.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org

Re: Including macros using #parse

[Will Glass-Husain <wg...@gmail.com>]

Hi Supun,

Seems like the right approach.

To me this seems analogous to the way we handle references.  If I understand
this right, the literal text for the reference is stored at parse time.  If
the reference turns out to be null then ASTReference renders the invalid
reference text.  Perhaps we need a node to store the literal "potential
macro" text and do the same thing.

WILL

On 8/5/07, Supun Kamburugamuva <su...@gmail.com> wrote:
>
> I was trying to preserve everything including whitespaces incase the
> call to macro does not have a macro definition. But still I couldn't
> find a proper solution. I'm not a Javacc expert and if someone can
> give me a hint I'd really appreciate it.
>
> Regards,
> Supun.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
> For additional commands, e-mail: dev-help@velocity.apache.org
>
>


-- 
Forio Business Simulations

Will Glass-Husain
wglass@forio.com
www.forio.com

Re: Including macros using #parse

[Supun Kamburugamuva <su...@gmail.com>]

I was trying to preserve everything including whitespaces incase the
call to macro does not have a macro definition. But still I couldn't
find a proper solution. I'm not a Javacc expert and if someone can
give me a hint I'd really appreciate it.

Regards,
Supun.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org

Re: Including macros using #parse

[Nathan Bubna <nb...@gmail.com>]

On 7/26/07, Jonathan Revusky <re...@gmail.com> wrote:
> Nathan Bubna wrote:
> > On 7/26/07, Jonathan Revusky <re...@gmail.com> wrote:
> >> Nathan Bubna wrote:
> >> > I believe your analysis is correct.  The parser will have to treat
> >> > anything in the form of #foo( ... ) is a potential macro at parse
> >> > time, then leave the check for a matching macro to render time.  If it
> >> > turns out to be a macro, then we should render the macro, if not (and
> >> > this may turn out to be tricky), we need to render that text exactly
> >> > as it was, whitespace and all.
> >>
> >> Out of idle curiosity, how often do you think that a template writer
> >> actually wants to output something like #foo($bar, $baz) to the output?
> >> In the cases where you do see that there is no #foo macro in the
> >> context, would this not be, in the overwhelming majority of cases, a
> >> result of some kind of coding error?
> >
> > Agreed, it would be good to log a warning (perhaps error?) level
> > message about it as well.
>
>
> Well, FreeMarker simply fails, throws an exception in analogous
> circumstances. But by all means, if you are going to continue processing
> the template, at least log a warning.
>
> But that is all not exactly my point. I am asking what is the point of
> outputting the literal text #foo($bar, $baz) when that was almost
> certainly not the intention of the template writer. If you are going to
> keep processing after what is almost certainly an error, it might make
> more sense just to output nothing at all -- though obviously, I tend
> more to just treating this as an error condition and aborting with a
> very clear error message.

Yeah that sounds ideal for development, but until we're ready to
rethink a lot of Velocity (if we ever are), i would prefer the
outputting of the literal text (as opposed to nothing or aborting)
largely because that's the way it currently works.  You're totally
right #foo($bar, $baz) is hard to imagine to be the desired output,
but #foo() is not at all far-fetched if you consider anchor portions
of URLs or javadoc tags in java code being generated.  I won't pretend
to know all that users do with their templates, so i'd rather keep
backwards compatibility until we're ready to work on 2.0.  Even then,
i'm still likely to lobby for a default stance of WYTIWYG (what you
type is what you get), with the option of turning on a strict "dev
mode" that would treat such things as errors and abort processing.
Outputting nothing but a warning in the logs would be my least
favorite option.

>
> >
> >> Jonathan Revusky
> >> --
> >> lead developer, FreeMarker project, http://freemarker.org/
> >>
> >> >
> >> > On 7/18/07, Supun Kamburugamuva <su...@gmail.com> wrote:
> >> >> Hi,
> >> >>
> >> >> I was looking into the issue of including macros via the #parse
> >> >> method.  Following is the my understanding of the problem. The main
> >> >> reason that causes this to be a hard problem to solve without touching
> >> >> the parser implementation is described below.
> >> >>
> >> >> At the moment when the parser encounter a call to a macro i.e.
> >> >> #foo(123) it looks weather a macro definition exists via the runtime
> >> >> system. If a definition exists everything is going well. This happens
> >> >> in the AST building face of the parser (not in the rendering face). So
> >> >> this means with the current implementation, the macros must be
> >> >> registered with the runtime system before a call occurs and this is
> >> >> the cause of all the troubles.
> >> >>
> >> >> Now this is the problem with #parse directive. #parse directive builds
> >> >> the AST of the files and render them at the same time. This happens
> >> >> when the main template that include the #parse directive get rendered.
> >> >> But when this happens, AST of the template that include the #parse
> >> >> directive is been built already. But the macro calls that are in the
> >> >> main template get invalidated because at the moment AST is built
> >> >> macros are not registered with the runtime system.
> >> >>
> >> >> As I can see the solution for this problem is, parser should not look
> >> >> into the runtime system to determine weather a directive like #foo()
> >> >> is a macro call or not. Instead parser should look ahead for the '('
> >> >> token  when it encounters something like #foo to deside weather it is
> >> >> a macro call or not.
> >> >>
> >> >> I would like to know weather this is solvable without breaking the
> >> >> current system in the way that I have decribed above?
> >> >>
> >> >> Regards,
> >> >> Supun.
> >> >>
> >> >> P.S. If we can do this my first implementation of 'including macros
> >> >> programmatically' can be done using this method.
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
> >> >> For additional commands, e-mail: dev-help@velocity.apache.org
> >> >>
> >> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
> >> For additional commands, e-mail: dev-help@velocity.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
> For additional commands, e-mail: dev-help@velocity.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org

Re: Including macros using #parse

[Jonathan Revusky <re...@gmail.com>]

Nathan Bubna wrote:
> On 7/26/07, Jonathan Revusky <re...@gmail.com> wrote:
>> Nathan Bubna wrote:
>> > I believe your analysis is correct.  The parser will have to treat
>> > anything in the form of #foo( ... ) is a potential macro at parse
>> > time, then leave the check for a matching macro to render time.  If it
>> > turns out to be a macro, then we should render the macro, if not (and
>> > this may turn out to be tricky), we need to render that text exactly
>> > as it was, whitespace and all.
>>
>> Out of idle curiosity, how often do you think that a template writer
>> actually wants to output something like #foo($bar, $baz) to the output?
>> In the cases where you do see that there is no #foo macro in the
>> context, would this not be, in the overwhelming majority of cases, a
>> result of some kind of coding error?
> 
> Agreed, it would be good to log a warning (perhaps error?) level
> message about it as well.


Well, FreeMarker simply fails, throws an exception in analogous 
circumstances. But by all means, if you are going to continue processing 
the template, at least log a warning.

But that is all not exactly my point. I am asking what is the point of 
outputting the literal text #foo($bar, $baz) when that was almost 
certainly not the intention of the template writer. If you are going to 
keep processing after what is almost certainly an error, it might make 
more sense just to output nothing at all -- though obviously, I tend 
more to just treating this as an error condition and aborting with a 
very clear error message.




> 
>> Jonathan Revusky
>> -- 
>> lead developer, FreeMarker project, http://freemarker.org/
>>
>> >
>> > On 7/18/07, Supun Kamburugamuva <su...@gmail.com> wrote:
>> >> Hi,
>> >>
>> >> I was looking into the issue of including macros via the #parse
>> >> method.  Following is the my understanding of the problem. The main
>> >> reason that causes this to be a hard problem to solve without touching
>> >> the parser implementation is described below.
>> >>
>> >> At the moment when the parser encounter a call to a macro i.e.
>> >> #foo(123) it looks weather a macro definition exists via the runtime
>> >> system. If a definition exists everything is going well. This happens
>> >> in the AST building face of the parser (not in the rendering face). So
>> >> this means with the current implementation, the macros must be
>> >> registered with the runtime system before a call occurs and this is
>> >> the cause of all the troubles.
>> >>
>> >> Now this is the problem with #parse directive. #parse directive builds
>> >> the AST of the files and render them at the same time. This happens
>> >> when the main template that include the #parse directive get rendered.
>> >> But when this happens, AST of the template that include the #parse
>> >> directive is been built already. But the macro calls that are in the
>> >> main template get invalidated because at the moment AST is built
>> >> macros are not registered with the runtime system.
>> >>
>> >> As I can see the solution for this problem is, parser should not look
>> >> into the runtime system to determine weather a directive like #foo()
>> >> is a macro call or not. Instead parser should look ahead for the '('
>> >> token  when it encounters something like #foo to deside weather it is
>> >> a macro call or not.
>> >>
>> >> I would like to know weather this is solvable without breaking the
>> >> current system in the way that I have decribed above?
>> >>
>> >> Regards,
>> >> Supun.
>> >>
>> >> P.S. If we can do this my first implementation of 'including macros
>> >> programmatically' can be done using this method.
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
>> >> For additional commands, e-mail: dev-help@velocity.apache.org
>> >>
>> >>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
>> For additional commands, e-mail: dev-help@velocity.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org

Re: Including macros using #parse

[Nathan Bubna <nb...@gmail.com>]

On 7/26/07, Jonathan Revusky <re...@gmail.com> wrote:
> Nathan Bubna wrote:
> > I believe your analysis is correct.  The parser will have to treat
> > anything in the form of #foo( ... ) is a potential macro at parse
> > time, then leave the check for a matching macro to render time.  If it
> > turns out to be a macro, then we should render the macro, if not (and
> > this may turn out to be tricky), we need to render that text exactly
> > as it was, whitespace and all.
>
> Out of idle curiosity, how often do you think that a template writer
> actually wants to output something like #foo($bar, $baz) to the output?
> In the cases where you do see that there is no #foo macro in the
> context, would this not be, in the overwhelming majority of cases, a
> result of some kind of coding error?

Agreed, it would be good to log a warning (perhaps error?) level
message about it as well.

> Jonathan Revusky
> --
> lead developer, FreeMarker project, http://freemarker.org/
>
> >
> > On 7/18/07, Supun Kamburugamuva <su...@gmail.com> wrote:
> >> Hi,
> >>
> >> I was looking into the issue of including macros via the #parse
> >> method.  Following is the my understanding of the problem. The main
> >> reason that causes this to be a hard problem to solve without touching
> >> the parser implementation is described below.
> >>
> >> At the moment when the parser encounter a call to a macro i.e.
> >> #foo(123) it looks weather a macro definition exists via the runtime
> >> system. If a definition exists everything is going well. This happens
> >> in the AST building face of the parser (not in the rendering face). So
> >> this means with the current implementation, the macros must be
> >> registered with the runtime system before a call occurs and this is
> >> the cause of all the troubles.
> >>
> >> Now this is the problem with #parse directive. #parse directive builds
> >> the AST of the files and render them at the same time. This happens
> >> when the main template that include the #parse directive get rendered.
> >> But when this happens, AST of the template that include the #parse
> >> directive is been built already. But the macro calls that are in the
> >> main template get invalidated because at the moment AST is built
> >> macros are not registered with the runtime system.
> >>
> >> As I can see the solution for this problem is, parser should not look
> >> into the runtime system to determine weather a directive like #foo()
> >> is a macro call or not. Instead parser should look ahead for the '('
> >> token  when it encounters something like #foo to deside weather it is
> >> a macro call or not.
> >>
> >> I would like to know weather this is solvable without breaking the
> >> current system in the way that I have decribed above?
> >>
> >> Regards,
> >> Supun.
> >>
> >> P.S. If we can do this my first implementation of 'including macros
> >> programmatically' can be done using this method.
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
> >> For additional commands, e-mail: dev-help@velocity.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
> For additional commands, e-mail: dev-help@velocity.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org

Re: Including macros using #parse

[Jonathan Revusky <re...@gmail.com>]

Nathan Bubna wrote:
> I believe your analysis is correct.  The parser will have to treat
> anything in the form of #foo( ... ) is a potential macro at parse
> time, then leave the check for a matching macro to render time.  If it
> turns out to be a macro, then we should render the macro, if not (and
> this may turn out to be tricky), we need to render that text exactly
> as it was, whitespace and all.

Out of idle curiosity, how often do you think that a template writer 
actually wants to output something like #foo($bar, $baz) to the output? 
In the cases where you do see that there is no #foo macro in the 
context, would this not be, in the overwhelming majority of cases, a 
result of some kind of coding error?

Jonathan Revusky
--
lead developer, FreeMarker project, http://freemarker.org/

> 
> On 7/18/07, Supun Kamburugamuva <su...@gmail.com> wrote:
>> Hi,
>>
>> I was looking into the issue of including macros via the #parse
>> method.  Following is the my understanding of the problem. The main
>> reason that causes this to be a hard problem to solve without touching
>> the parser implementation is described below.
>>
>> At the moment when the parser encounter a call to a macro i.e.
>> #foo(123) it looks weather a macro definition exists via the runtime
>> system. If a definition exists everything is going well. This happens
>> in the AST building face of the parser (not in the rendering face). So
>> this means with the current implementation, the macros must be
>> registered with the runtime system before a call occurs and this is
>> the cause of all the troubles.
>>
>> Now this is the problem with #parse directive. #parse directive builds
>> the AST of the files and render them at the same time. This happens
>> when the main template that include the #parse directive get rendered.
>> But when this happens, AST of the template that include the #parse
>> directive is been built already. But the macro calls that are in the
>> main template get invalidated because at the moment AST is built
>> macros are not registered with the runtime system.
>>
>> As I can see the solution for this problem is, parser should not look
>> into the runtime system to determine weather a directive like #foo()
>> is a macro call or not. Instead parser should look ahead for the '('
>> token  when it encounters something like #foo to deside weather it is
>> a macro call or not.
>>
>> I would like to know weather this is solvable without breaking the
>> current system in the way that I have decribed above?
>>
>> Regards,
>> Supun.
>>
>> P.S. If we can do this my first implementation of 'including macros
>> programmatically' can be done using this method.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
>> For additional commands, e-mail: dev-help@velocity.apache.org
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org

Re: Including macros using #parse

[Nathan Bubna <nb...@gmail.com>]

I believe your analysis is correct.  The parser will have to treat
anything in the form of #foo( ... ) is a potential macro at parse
time, then leave the check for a matching macro to render time.  If it
turns out to be a macro, then we should render the macro, if not (and
this may turn out to be tricky), we need to render that text exactly
as it was, whitespace and all.

On 7/18/07, Supun Kamburugamuva <su...@gmail.com> wrote:
> Hi,
>
> I was looking into the issue of including macros via the #parse
> method.  Following is the my understanding of the problem. The main
> reason that causes this to be a hard problem to solve without touching
> the parser implementation is described below.
>
> At the moment when the parser encounter a call to a macro i.e.
> #foo(123) it looks weather a macro definition exists via the runtime
> system. If a definition exists everything is going well. This happens
> in the AST building face of the parser (not in the rendering face). So
> this means with the current implementation, the macros must be
> registered with the runtime system before a call occurs and this is
> the cause of all the troubles.
>
> Now this is the problem with #parse directive. #parse directive builds
> the AST of the files and render them at the same time. This happens
> when the main template that include the #parse directive get rendered.
> But when this happens, AST of the template that include the #parse
> directive is been built already. But the macro calls that are in the
> main template get invalidated because at the moment AST is built
> macros are not registered with the runtime system.
>
> As I can see the solution for this problem is, parser should not look
> into the runtime system to determine weather a directive like #foo()
> is a macro call or not. Instead parser should look ahead for the '('
> token  when it encounters something like #foo to deside weather it is
> a macro call or not.
>
> I would like to know weather this is solvable without breaking the
> current system in the way that I have decribed above?
>
> Regards,
> Supun.
>
> P.S. If we can do this my first implementation of 'including macros
> programmatically' can be done using this method.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
> For additional commands, e-mail: dev-help@velocity.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org