You are viewing a plain text version of this content. The canonical link for it is here.
Posted to rivet-dev@tcl.apache.org by Massimo Manghi <ma...@unipr.it> on 2013/07/04 15:37:22 UTC
command for parsing strings
Rivet 'parse' command has a very good and clean implementation caching
its results internally and using the full path to a file as a lookup key
in the cache. The file system time stamps tells mod_rivet when a cache
entry need refresh.
Thus the restriction with 'parse' is the templates have to be stored on
the file system, whereas Web applications today may store data (also
templates, why not?) to several classes of data sources, including
Sql/noSql DBMS, sockets, web services etc.. Thus templates could be
accessible as string stored in Tcl variables without an immediate way to
represent and locate them as file system resources (of course you might
want to implement one)
I added in trunk a simple and harmless implementation of a command
parsing a string stored in a Tcl_Obj. It's simple because based on the
code already available for parse, harmless because at the cost of having
a slightly larger module the few lines of code taken from
Rivet_ParseExecFile were replicated in a Rivet_ParseExecString function
so to avoid side effects to parse. If this command will receive your
approval it could evolve and be integrated with its file based counterpart
To test the implementation I made it accessible through the Tcl command
'parsestr'. Admittedly the name is a bad one and I don't stick to it.
Perhaps adding the -string switch to command 'parse' would be a better
choice, something like
parse -string $template
as opposed to the usual
parse <filename>
Command 'parsestr' doesn't cache the template leaving to the caller (the
application) the task to devise a possible way to cache it consistently,
when needed.
--
-- Massimo Manghi
Dipartimento di Neuroscienze
Unità di Biofisica e Fisica Sanitaria
via Volturno 39
43125 Parma
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org
Re: command for parsing strings
Posted by Massimo Manghi <ma...@unipr.it>.
On 07/04/2013 03:53 PM, Harald Oehlmann wrote:
> Sorry, Massimo, I did not get it...
>
> 1) Cache: what is cached:
a Tcl script obtained from either a pure Tcl script (trivial case) or a
template which implies parsing and transformation into pure Tcl
> 1.a) the verbatim .rvt file?
No, as I said it's Tcl code produced by the parser
> 1.b) or the parsed html output of the rvt. file ?
Each chunk of HTML is set as argument of a "puts -nonewline ..." and
interleaved with the embedded Tcl code
>
> I would consider 1.b) as quite dangerous, as already a
> "<? [clock now] ?>" would not work any more...
>
I think I have dissolved your concerns: Rivet has been working that way
ever since. In your case "[clock now]" would go into the cache entry
> 2) About the new command "parse -string xxx" -
> This sounds logic to me. We have something like "<bod><h1><? db get
> header ?></h1></body>" in a data base and then, put it into the tag
> replacement, so essentially do:
> subst $template
> where the command tags are not "[]" but "<??>".
>
> Ok, in which scope, the tags are evaluated ?
> 2.a) current scope
> 2.b) global scope
the scope of the caller, as for the 'parse' command
>
> I think this is useful, no issue Massimo.
>
> So parse may be implemented as:
> proc parse filename {
> set f [open $filename r]
> set res [parse -string [read $f]]
> close $f
> return $res
> }
in this case the caching would be lost unless we build a new C function
::rivet::read_file_cached <filename>
which should return either the cached content or read the file, cache
the content and then return it
--
-- Massimo Manghi
Dipartimento di Neuroscienze
Unità di Biofisica e Fisica Sanitaria
via Volturno 39
43125 Parma
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org
Re: command for parsing strings
Posted by Harald Oehlmann <ha...@elmicron.de>.
Am 04.07.2013 15:37, schrieb Massimo Manghi:
> parse -string $template
>
> as opposed to the usual
>
> parse <filename>
>
> Command 'parsestr' doesn't cache the template leaving to the caller (the
> application) the task to devise a possible way to cache it consistently,
> when needed.
Sorry, Massimo, I did not get it...
1) Cache: what is cached:
1.a) the verbatim .rvt file?
1.b) or the parsed html output of the rvt. file ?
I would consider 1.b) as quite dangerous, as already a
"<? [clock now] ?>" would not work any more...
2) About the new command "parse -string xxx" -
This sounds logic to me. We have something like "<bod><h1><? db get
header ?></h1></body>" in a data base and then, put it into the tag
replacement, so essentially do:
subst $template
where the command tags are not "[]" but "<??>".
Ok, in which scope, the tags are evaluated ?
2.a) current scope
2.b) global scope
I think this is useful, no issue Massimo.
So parse may be implemented as:
proc parse filename {
set f [open $filename r]
set res [parse -string [read $f]]
close $f
return $res
}
Thank you,
Harald
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org
Re: command for parsing strings
Posted by Kirill Shtumf <sh...@gmail.com>.
The output can be stored in a variable. Unfortunately, I don't know how
best to implement it.
06.07.2013 21:03, Massimo Manghi пишет:
> So what exactly you are
> suggesting? Something like having the code parsed then executed but the
> output stored in the channel buffer returned to the caller?
--
With best regards,
Kirill
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org
Re: command for parsing strings
Posted by Massimo Manghi <ma...@unipr.it>.
Hi Kirill
sorry for answering so late
I'm afraid I don't get the picture because in your example the parsed
document is returned and stored into 'data'. So what exactly you are
suggesting? Something like having the code parsed then executed but the
output stored in the channel buffer returned to the caller?
-- Massimo
On 07/05/2013 01:19 PM, Kirill Shtumf wrote:
> For example, I use templates for email-newsletters.
>
> At the moment I use for this the parser implementation on Tcl.
>
> register.html looks like this: <p>Hello <? append output $vars(username)
> ?>!</p> ...
>
> package require tclrivetparser
> set text_for_parse [read_file [file join $::conf(template) {mail}
> {register.html}]]
> set output ""
> ::tclrivetparser::setoutputcmd {append output }
> set data [::tclrivetparser::parserivetdata $text_for_parse]
> eval $data
> functions-> mail $vars(email) $vars(from) $vars(subj) $output
>
> 05.07.2013 14:57, Massimo Manghi wrote:
>> Something that would work like this?
>>
>> set fragment "<?= [::rivet::xml \"a small template\" b] ?>"
>> puts [::rivet::parse_template $fragment]
>> <== puts -nonewline "<b>a small template</b>"
>>
>> while
>>
>> eval [::rivet::parse_template $fragment]
>>
>> would actually evaluate the script and print to the browser the string
>>
>> "<b>a small template</b>"
>>
>> It implies some Tcl level interface to librivetparser. In don't know but
>> it could go into the rivetlib library rather than the core.
>>
>> -- Massimo
>>
>> On 07/04/2013 05:44 PM, Kirill Shtumf wrote:
>>> One problem of the parser implementation is inability to add parsed text
>>> into another variable. Command does not return a result and immediately
>>> sends it to the stdout. Can we somehow change it?
>>>
>
>
--
-- Massimo Manghi
Dipartimento di Neuroscienze
Unità di Biofisica e Fisica Sanitaria
via Volturno 39
43125 Parma
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org
Re: command for parsing strings
Posted by Massimo Manghi <ma...@unipr.it>.
I've just found out that rivet/packages/tclrivet/tclrivet.tcl loaded the
C implementation loading it with command load
load [file join [file dirname [info script]] .. .. \
librivetparser[info sharedlibextension]]
tclrivet.tcl looks like a draft for a pure Tcl environment, but left
unfinished (actually it looks like it never started a real development)
This caused a rather annoying warning because the explicit load let this
file provide 2 packages and also caused some weird problem with
librivetparser.so
I could not figure it out exactly but on my Ubuntu system requiring the
C implementation with
package require rivetparser
caused the Tcl interpreter to execute correctly one request and then
fail on every other subsequent request complaining about errors that
didn't exist actually. I don't understand why that call messed with the
Tcl interpreter internal status.
anyway I committed in trunk this change and I would like to have
feedback on this
-- Massimo
On 07/05/2013 08:00 PM, Damon Courtney wrote:
> We should be doing what tcllib does in these cases. librivetparser
> includes a C implementation for parsing Rivet documents, but we have
> a Tcl version in a package that can be loaded in case the compiled
> version is unavailable. They should be the same command name.
>
> Tcllib does this everywhere. If we can compile one, we will. If we
> can't, we fall back to a slower, Tcl-only implementation of the same
> command.
>
>
> On Jul 5, 2013, at 11:52 AM, Massimo Manghi <ma...@unipr.it>
> wrote:
>
>> I realized there is a Tcl interface to librivetparser implemented
>> in src/parser/parserPkgInit.c.
>>
>> Well this is unforgivable (am I going to be defrocked of my rank
>> of project chair??)....I changed the command creation calls in
>> order to move the commands into the ::rivet namespace but forgot
>> about it altogether.
>>
>> Time for writing some documentation about it, so that the memory of
>> it can fix into my mind.
>>
>> The code implementing the command ::rivet::parserivetdata
>> (Parse_RivetData) is nearly the same code I put in
>> Rivet_ParseExecString except for my version calling also
>> Rivet_ExecuteAndCheck.
>>
>> On 07/05/2013 01:19 PM, Kirill Shtumf wrote:
>>> For example, I use templates for email-newsletters.
>>>
>>> At the moment I use for this the parser implementation on Tcl.
>>>
>>> register.html looks like this: <p>Hello <? append output
>>> $vars(username) ?>!</p> ...
>>>
>>> package require tclrivetparser set text_for_parse [read_file
>>> [file join $::conf(template) {mail} {register.html}]] set output
>>> "" ::tclrivetparser::setoutputcmd {append output } set data
>>> [::tclrivetparser::parserivetdata $text_for_parse] eval $data
>>> functions-> mail $vars(email) $vars(from) $vars(subj) $output
>>>
>>
>> Unless you need a pure Tcl implementation for the parser the way
>> tclrivetparser does, it seems ::rivet::parserivetdata might work
>> out a solution for your problem.
>>
>> -- -- Massimo Manghi
>>
>> Dipartimento di Neuroscienze Unità di Biofisica e Fisica Sanitaria
>> via Volturno 39 43125 Parma
>>
>> ---------------------------------------------------------------------
>>
>>
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
>> For additional commands, e-mail: rivet-dev-help@tcl.apache.org
>>
>
>
> ---------------------------------------------------------------------
>
>
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
> For additional commands, e-mail: rivet-dev-help@tcl.apache.org
>
--
-- Massimo Manghi
Dipartimento di Neuroscienze
Unità di Biofisica e Fisica Sanitaria
via Volturno 39
43125 Parma
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org
Re: command for parsing strings
Posted by Massimo Manghi <ma...@unipr.it>.
the 'parsestr' has been removed from trunk. The ordinary 'parse' command
has been extended to accept the form
parse -string $template_string
which causes the template stored in the variable template_string to be
parsed and executed.
--
-- Massimo Manghi
Dipartimento di Neuroscienze
Unità di Biofisica e Fisica Sanitaria
via Volturno 39
43125 Parma
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org
Re: command for parsing strings
Posted by Damon Courtney <da...@tclhome.com>.
We should be doing what tcllib does in these cases. librivetparser includes a C implementation for parsing Rivet documents, but we have a Tcl version in a package that can be loaded in case the compiled version is unavailable. They should be the same command name.
Tcllib does this everywhere. If we can compile one, we will. If we can't, we fall back to a slower, Tcl-only implementation of the same command.
On Jul 5, 2013, at 11:52 AM, Massimo Manghi <ma...@unipr.it> wrote:
> I realized there is a Tcl interface to librivetparser implemented in
> src/parser/parserPkgInit.c.
>
> Well this is unforgivable (am I going to be defrocked of my rank of
> project chair??)....I changed the command creation calls in order to
> move the commands into the ::rivet namespace but forgot about it
> altogether.
>
> Time for writing some documentation about it, so that the memory of it
> can fix into my mind.
>
> The code implementing the command ::rivet::parserivetdata
> (Parse_RivetData) is nearly the same code I put in Rivet_ParseExecString
> except for my version calling also Rivet_ExecuteAndCheck.
>
> On 07/05/2013 01:19 PM, Kirill Shtumf wrote:
>> For example, I use templates for email-newsletters.
>>
>> At the moment I use for this the parser implementation on Tcl.
>>
>> register.html looks like this: <p>Hello <? append output $vars(username)
>> ?>!</p> ...
>>
>> package require tclrivetparser
>> set text_for_parse [read_file [file join $::conf(template) {mail}
>> {register.html}]]
>> set output ""
>> ::tclrivetparser::setoutputcmd {append output }
>> set data [::tclrivetparser::parserivetdata $text_for_parse]
>> eval $data
>> functions-> mail $vars(email) $vars(from) $vars(subj) $output
>>
>
> Unless you need a pure Tcl implementation for the parser the way
> tclrivetparser does, it seems ::rivet::parserivetdata might work out a
> solution for your problem.
>
> --
> -- Massimo Manghi
>
> Dipartimento di Neuroscienze
> Unità di Biofisica e Fisica Sanitaria
> via Volturno 39
> 43125 Parma
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
> For additional commands, e-mail: rivet-dev-help@tcl.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org
Re: command for parsing strings
Posted by Massimo Manghi <ma...@unipr.it>.
I realized there is a Tcl interface to librivetparser implemented in
src/parser/parserPkgInit.c.
Well this is unforgivable (am I going to be defrocked of my rank of
project chair??)....I changed the command creation calls in order to
move the commands into the ::rivet namespace but forgot about it
altogether.
Time for writing some documentation about it, so that the memory of it
can fix into my mind.
The code implementing the command ::rivet::parserivetdata
(Parse_RivetData) is nearly the same code I put in Rivet_ParseExecString
except for my version calling also Rivet_ExecuteAndCheck.
On 07/05/2013 01:19 PM, Kirill Shtumf wrote:
> For example, I use templates for email-newsletters.
>
> At the moment I use for this the parser implementation on Tcl.
>
> register.html looks like this: <p>Hello <? append output $vars(username)
> ?>!</p> ...
>
> package require tclrivetparser
> set text_for_parse [read_file [file join $::conf(template) {mail}
> {register.html}]]
> set output ""
> ::tclrivetparser::setoutputcmd {append output }
> set data [::tclrivetparser::parserivetdata $text_for_parse]
> eval $data
> functions-> mail $vars(email) $vars(from) $vars(subj) $output
>
Unless you need a pure Tcl implementation for the parser the way
tclrivetparser does, it seems ::rivet::parserivetdata might work out a
solution for your problem.
--
-- Massimo Manghi
Dipartimento di Neuroscienze
Unità di Biofisica e Fisica Sanitaria
via Volturno 39
43125 Parma
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org
Re: command for parsing strings
Posted by Kirill Shtumf <sh...@gmail.com>.
For example, I use templates for email-newsletters.
At the moment I use for this the parser implementation on Tcl.
register.html looks like this: <p>Hello <? append output $vars(username)
?>!</p> ...
package require tclrivetparser
set text_for_parse [read_file [file join $::conf(template) {mail}
{register.html}]]
set output ""
::tclrivetparser::setoutputcmd {append output }
set data [::tclrivetparser::parserivetdata $text_for_parse]
eval $data
functions-> mail $vars(email) $vars(from) $vars(subj) $output
05.07.2013 14:57, Massimo Manghi wrote:
> Something that would work like this?
>
> set fragment "<?= [::rivet::xml \"a small template\" b] ?>"
> puts [::rivet::parse_template $fragment]
> <== puts -nonewline "<b>a small template</b>"
>
> while
>
> eval [::rivet::parse_template $fragment]
>
> would actually evaluate the script and print to the browser the string
>
> "<b>a small template</b>"
>
> It implies some Tcl level interface to librivetparser. In don't know but
> it could go into the rivetlib library rather than the core.
>
> -- Massimo
>
> On 07/04/2013 05:44 PM, Kirill Shtumf wrote:
>> One problem of the parser implementation is inability to add parsed text
>> into another variable. Command does not return a result and immediately
>> sends it to the stdout. Can we somehow change it?
>>
--
With best regards,
Kirill
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org
Re: command for parsing strings
Posted by Massimo Manghi <ma...@unipr.it>.
Something that would work like this?
set fragment "<?= [::rivet::xml \"a small template\" b] ?>"
puts [::rivet::parse_template $fragment]
<== puts -nonewline "<b>a small template</b>"
while
eval [::rivet::parse_template $fragment]
would actually evaluate the script and print to the browser the string
"<b>a small template</b>"
It implies some Tcl level interface to librivetparser. In don't know but
it could go into the rivetlib library rather than the core.
-- Massimo
On 07/04/2013 05:44 PM, Kirill Shtumf wrote:
> One problem of the parser implementation is inability to add parsed text
> into another variable. Command does not return a result and immediately
> sends it to the stdout. Can we somehow change it?
>
--
-- Massimo Manghi
Dipartimento di Neuroscienze
Unità di Biofisica e Fisica Sanitaria
via Volturno 39
43125 Parma
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org
Re: command for parsing strings
Posted by Kirill Shtumf <sh...@gmail.com>.
One problem of the parser implementation is inability to add parsed text
into another variable. Command does not return a result and immediately
sends it to the stdout. Can we somehow change it?
04.07.2013 17:37, Massimo Manghi пишет:
> Rivet 'parse' command has a very good and clean implementation caching
> its results internally and using the full path to a file as a lookup key
> in the cache. The file system time stamps tells mod_rivet when a cache
> entry need refresh.
>
> Thus the restriction with 'parse' is the templates have to be stored on
> the file system, whereas Web applications today may store data (also
> templates, why not?) to several classes of data sources, including
> Sql/noSql DBMS, sockets, web services etc.. Thus templates could be
> accessible as string stored in Tcl variables without an immediate way to
> represent and locate them as file system resources (of course you might
> want to implement one)
>
> I added in trunk a simple and harmless implementation of a command
> parsing a string stored in a Tcl_Obj. It's simple because based on the
> code already available for parse, harmless because at the cost of having
> a slightly larger module the few lines of code taken from
> Rivet_ParseExecFile were replicated in a Rivet_ParseExecString function
> so to avoid side effects to parse. If this command will receive your
> approval it could evolve and be integrated with its file based counterpart
>
> To test the implementation I made it accessible through the Tcl command
> 'parsestr'. Admittedly the name is a bad one and I don't stick to it.
> Perhaps adding the -string switch to command 'parse' would be a better
> choice, something like
>
> parse -string $template
>
> as opposed to the usual
>
> parse <filename>
>
> Command 'parsestr' doesn't cache the template leaving to the caller (the
> application) the task to devise a possible way to cache it consistently,
> when needed.
>
--
With best regards,
Kirill
---------------------------------------------------------------------
To unsubscribe, e-mail: rivet-dev-unsubscribe@tcl.apache.org
For additional commands, e-mail: rivet-dev-help@tcl.apache.org