You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@freemarker.apache.org by Roberto Benitez <rb...@yahoo.com.INVALID> on 2017/01/24 05:27:38 UTC
Template Loader Implementation that loads from a database
(DataSource)
I am working on a TemplateLoader implementation that loads from a database. From a previous e-mail with Daniel Dekany, he mentioned that this is something you still need, and may be interested in integrating.
I have the source code on github: rbenitez22/freemarker-datasource-loader
| |
rbenitez22/freemarker-datasource-loader
| |
This is something I am still tweaking as I learn more about the inner workings of the frame work. Let me know if you have further thoughts on what could be modified to make it integrate better.
--Roberto
Re: Template Loader Implementation that loads from a database
(DataSource)
Posted by Roberto Benitez <rb...@yahoo.com.INVALID>.
I have only had the chance to look it on my phone, so have not read the thread thoroughly.
I will go through it on my desktop later today.
--Roberto
Sent from Yahoo Mail on Android
On Tue, Feb 7, 2017 at 8:14 AM, Daniel Dekany<dd...@freemail.hu> wrote: In case you have missed it in the other thread, note that it seems
that there won't be a new alternative TemplateLoader interface in FM2,
so this will have to be solved with the existing TemplateLoader there.
--
Thanks,
Daniel Dekany
Wednesday, January 25, 2017, 4:17:10 AM, Roberto Benitez wrote:
>>Also, I hope that freemarker-datasource-loader can be simplified over
>>time, so it doesn't employ this many interfaces/classes.
> Currently, the only essential classes are: 1)JdbcMetaData --A
> central place to store DB table/column names. 2) TemplateSourceDao
> --to perform the JDBC/CRUD work 3) DataSourceTemplateLoader -- the
> loader implementation. 4) (Maybe)?) JdbcTemplateSource -- template
> data record including the template content itself. This should be
> helpful to a client such that no additional work should be necessary
> (in most general cases) to maintain the templates
> (name/content)--the DAO and JdbcTemplateSource will do the work.
>
> I am experimenting with the other classes/interfaces. In
> particular, TemplateName and TemplateSource (which extends
> TemplateName). The idea is to describe template metadata that is
> more abstract that is more abstract than File, such that it could
> describe a File or some database record--with the relevant data such
> as name, locale. BUT, as you noted, necessitates pre-fetching data.
>
>
>
>
>>My main problem with FreeMarker's TemplateLoader interface is that it
>>assumes that you execute existence check, last modification query, and
>>oading the actual content as 3 separate steps. While that fits how
>>you deal with plain files (and I guess that's what it was modelled
>>after), it doesn't fit how you do things with a database. In a
>>straightforward implementation we would have 3 SQL round trips per new
>>template loaded, and 2 for each up-to-date checks later. In a smarter
>>implementation, and as you did it too, you can reduce the number of
>>SQL operations be prefetching data into the templateSource object
>>during the existence check (i.e., in findTemplateSource). As
>>closeTemplateSource will be called pretty soon anyway, working from a
>>such "cache" is certainly fine (because closeTemplateSource will
>>invalidate it). But I don't think that you should prefetch the
>>template content there, because then that will be an unnecessarily
>>prefetched for template cache up-to-date checks, which is the dominant
>>TemplateLoader usage in many applications. So it's sill 2 SQL-s per
>>new template loaded, and 1 per template cache up-to-date checks.
>>Certainly not a problem in most apps, but it's still bugs me. In
>>FreeMarker 3 I definitely want to fix this be radically changing the
>>TemplateLoader interface, and it's also possible that I will backport
>>that solution into FreeMarker 2 (if people here will agree), though
>>there it will co-exists with the traditional TemplateLoader, probably
>>under the name TemplateLoader2 or something, so it's kind of messy.
>
> Would the changes to the TemplateLoader be that drastic? Sometimes
> it is feasible to deprecate old/non-conformant methods (perhaps
> provide defaults that rely on the new methods), which can later be removed completely.
>
> --Roberto
>
>
>
>
Re: Template Loader Implementation that loads from a database (DataSource)
Posted by Daniel Dekany <dd...@freemail.hu>.
In case you have missed it in the other thread, note that it seems
that there won't be a new alternative TemplateLoader interface in FM2,
so this will have to be solved with the existing TemplateLoader there.
--
Thanks,
Daniel Dekany
Wednesday, January 25, 2017, 4:17:10 AM, Roberto Benitez wrote:
>>Also, I hope that freemarker-datasource-loader can be simplified over
>>time, so it doesn't employ this many interfaces/classes.
> Currently, the only essential classes are:� 1)JdbcMetaData --A
> central place to store DB table/column names.�2) TemplateSourceDao
> --to perform the JDBC/CRUD work�3) DataSourceTemplateLoader -- the
> loader implementation.�4) (Maybe)?) JdbcTemplateSource -- template
> data record including the template content itself.� This should be
> helpful to a client such that no additional work should be necessary
> (in most general cases) to maintain the templates
> (name/content)--the DAO and JdbcTemplateSource will do the work.
>
> I am experimenting with the other classes/interfaces.� In
> particular, TemplateName and TemplateSource (which extends
> TemplateName).� The idea is to describe template metadata that is
> more abstract that is more abstract than File, such that it could
> describe a File or some database record--with the relevant data such
> as name, locale. BUT, as you noted, necessitates pre-fetching data.
>
>
>
>
>>My main problem with FreeMarker's TemplateLoader interface is that it
>>assumes that you execute existence check, last modification query, and
>>oading the actual content as 3 separate steps. While that fits how
>>you deal with plain files (and I guess that's what it was modelled
>>after), it doesn't fit how you do things with a database. In a
>>straightforward implementation we would have 3 SQL round trips per new
>>template loaded, and 2 for each up-to-date checks later. In a smarter
>>implementation, and as you did it too, you can reduce the number of
>>SQL operations be prefetching data into the templateSource object
>>during the existence check (i.e., in findTemplateSource). As
>>closeTemplateSource will be called pretty soon anyway, working from a
>>such "cache" is certainly fine (because closeTemplateSource will
>>invalidate it). But I don't think that you should prefetch the
>>template content there, because then that will be an unnecessarily
>>prefetched for template cache up-to-date checks, which is the dominant
>>TemplateLoader usage in many applications. So it's sill 2 SQL-s per
>>new template loaded, and 1 per template cache up-to-date checks.
>>Certainly not a problem in most apps, but it's still bugs me. In
>>FreeMarker 3 I definitely want to fix this be radically changing the
>>TemplateLoader interface, and it's also possible that I will backport
>>that solution into FreeMarker 2 (if people here will agree), though
>>there it will co-exists with the traditional TemplateLoader, probably
>>under the name TemplateLoader2 or something, so it's kind of messy.
>
> Would the changes to the TemplateLoader be that drastic?� Sometimes
> it is feasible to deprecate old/non-conformant methods (perhaps
> provide defaults that rely on the new methods), which can later be removed completely.�
>
> --Roberto
>
>
>
>
Re: Template Loader Implementation that loads from a database (DataSource)
Posted by Daniel Dekany <dd...@freemail.hu>.
Wednesday, January 25, 2017, 4:17:10 AM, Roberto Benitez wrote:
>>Also, I hope that freemarker-datasource-loader can be simplified over
>>time, so it doesn't employ this many interfaces/classes.
>
> Currently, the only essential classes are:
> 1) JdbcMetaData --A central place to store DB table/column names.
> 2) TemplateSourceDao --to perform the JDBC/CRUD work
I think we shouldn't have a TemplateSourceDao inside FreeMarker. The
only concern of FreeMarker is reading the templates, so only the R in
CRUD. You might ask, but why not solve other operations as well, it
can be convenient for some users. It's a trap FreeMarker has fell a
few times in the past, and my conclusion is that it's in general not
wise to address things that are outside our scope. It's a maintenance
problem for one (all those little "utilities" we are dragging with
ourselves, being responsible for them). Also, often we aren't even in
the position to give the best solution, as the best solution depends
on the application. Like here, it's likely that the user uses JPA, and
has a hundred other tables that he already manipulates through that,
has his company standards for it, like even some JPA hooks. I think
everyone should just use his own project conventions for this, not a
DAO provided by a template engine. Also we probably don't want to add
yet another thing to our public API (and to the jar...), which the
developers will look at, and contemplate, if they should use that, or
their usual persistence solution. (We don't do anything special in
there really, like we don't invalidate the related template cache
entry when a row is deleted.) Components that stretch their scope too
much are better be in separate jars than inside FreeMarker's core jar.
> 3) DataSourceTemplateLoader -- the loader implementation.
BTW, DatabaseTemplateLoader or JDBCTemplateLoader perhaps would be a
more telling name.
> 4) (Maybe)?) JdbcTemplateSource -- template data record including
> the template content itself. This should be helpful to a client
> such that no additional work should be necessary (in most general
> cases) to maintain the templates (name/content)--the DAO and
> JdbcTemplateSource will do the work.
>
> I am experimenting with the other classes/interfaces. In
> particular, TemplateName and TemplateSource (which extends
> TemplateName). The idea is to describe template metadata that is
> more abstract that is more abstract than File, such that it could
> describe a File or some database record--with the relevant data such
> as name, locale. BUT, as you noted, necessitates pre-fetching data.
>
>
>
>
>>My main problem with FreeMarker's TemplateLoader interface is that it
>>assumes that you execute existence check, last modification query, and
>>oading the actual content as 3 separate steps. While that fits how
>>you deal with plain files (and I guess that's what it was modelled
>>after), it doesn't fit how you do things with a database. In a
>>straightforward implementation we would have 3 SQL round trips per new
>>template loaded, and 2 for each up-to-date checks later. In a smarter
>>implementation, and as you did it too, you can reduce the number of
>>SQL operations be prefetching data into the templateSource object
>>during the existence check (i.e., in findTemplateSource). As
>>closeTemplateSource will be called pretty soon anyway, working from a
>>such "cache" is certainly fine (because closeTemplateSource will
>>invalidate it). But I don't think that you should prefetch the
>>template content there, because then that will be an unnecessarily
>>prefetched for template cache up-to-date checks, which is the dominant
>>TemplateLoader usage in many applications. So it's sill 2 SQL-s per
>>new template loaded, and 1 per template cache up-to-date checks.
>>Certainly not a problem in most apps, but it's still bugs me. In
>>FreeMarker 3 I definitely want to fix this be radically changing the
>>TemplateLoader interface, and it's also possible that I will backport
>>that solution into FreeMarker 2 (if people here will agree), though
>>there it will co-exists with the traditional TemplateLoader, probably
>>under the name TemplateLoader2 or something, so it's kind of messy.
>
>
> Would the changes to the TemplateLoader be that drastic?
It's a completely new interface.
> Sometimes it is feasible to deprecate old/non-conformant methods
> (perhaps provide defaults that rely on the new methods), which can
> later be removed completely.
> --Roberto
--
Thanks,
Daniel Dekany
Re: Template Loader Implementation that loads from a database
(DataSource)
Posted by Roberto Benitez <rb...@yahoo.com.INVALID>.
>Also, I hope that freemarker-datasource-loader can be simplified over
>time, so it doesn't employ this many interfaces/classes.
Currently, the only essential classes are: 1)JdbcMetaData --A central place to store DB table/column names. 2) TemplateSourceDao --to perform the JDBC/CRUD work 3) DataSourceTemplateLoader -- the loader implementation. 4) (Maybe)?) JdbcTemplateSource -- template data record including the template content itself. This should be helpful to a client such that no additional work should be necessary (in most general cases) to maintain the templates (name/content)--the DAO and JdbcTemplateSource will do the work.
I am experimenting with the other classes/interfaces. In particular, TemplateName and TemplateSource (which extends TemplateName). The idea is to describe template metadata that is more abstract that is more abstract than File, such that it could describe a File or some database record--with the relevant data such as name, locale. BUT, as you noted, necessitates pre-fetching data.
>My main problem with FreeMarker's TemplateLoader interface is that it
>assumes that you execute existence check, last modification query, and
>oading the actual content as 3 separate steps. While that fits how
>you deal with plain files (and I guess that's what it was modelled
>after), it doesn't fit how you do things with a database. In a
>straightforward implementation we would have 3 SQL round trips per new
>template loaded, and 2 for each up-to-date checks later. In a smarter
>implementation, and as you did it too, you can reduce the number of
>SQL operations be prefetching data into the templateSource object
>during the existence check (i.e., in findTemplateSource). As
>closeTemplateSource will be called pretty soon anyway, working from a
>such "cache" is certainly fine (because closeTemplateSource will
>invalidate it). But I don't think that you should prefetch the
>template content there, because then that will be an unnecessarily
>prefetched for template cache up-to-date checks, which is the dominant
>TemplateLoader usage in many applications. So it's sill 2 SQL-s per
>new template loaded, and 1 per template cache up-to-date checks.
>Certainly not a problem in most apps, but it's still bugs me. In
>FreeMarker 3 I definitely want to fix this be radically changing the
>TemplateLoader interface, and it's also possible that I will backport
>that solution into FreeMarker 2 (if people here will agree), though
>there it will co-exists with the traditional TemplateLoader, probably
>under the name TemplateLoader2 or something, so it's kind of messy.
Would the changes to the TemplateLoader be that drastic? Sometimes it is feasible to deprecate old/non-conformant methods (perhaps provide defaults that rely on the new methods), which can later be removed completely.
--Roberto
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Posted by Daniel Dekany <dd...@freemail.hu>.
Note that there's quick draft of TemplateResolver committed in the "3" branch:
https://github.com/apache/incubator-freemarker/blob/3/src/main/java/org/apache/freemarker/core/templateresolver/TemplateResolver.java
It's not integrated with the other parts of FM (it's assumed
everywhere that we have a DefaultTemplateResolver, not a generic
TemplateResolver), nor does it deal with the configuration aspects
discussed today. But it shows an idea for the main functionality.
Tuesday, February 7, 2017, 11:59:08 AM, Daniel Dekany wrote:
> Tuesday, February 7, 2017, 7:40:18 AM, David E Jones wrote:
>
>> On Tue, 2017-01-24 at 23:47 +0100, Daniel Dekany wrote:
>>>
>>> Should the TemplateResolver-s (the work name for the new interface)
>>> still get the template name already resolved to absolute and
>>> normalized? So when cfg.getTemplate is called, then it invokes
>>> cfg.templateNameFormat.normalizeAbsoluteName(String) to normalize the
>>> path, and only then it calls the TemplateResolver. Also #import and
>>> such currently calls templateNameFormat.toAbsoluteName(String,
>>> String), and that would happen earlier than TemplateResolver is
>>> involved.
>>>
>>> And how would it look... perhaps like this:
>>>
>>> ����public interface TemplateResolver {
>>>
>>> �������Template getTemplate(String templatePath, Local lookupLocale, Object customLookupCondition)
>>> �������throws IOException;
>>>
>>> ����}
>>>
>>> You may notice that it misses the `encoding` and `parsed` parameter of
>>> cfg.getTemplate. That's because I belive that in FM3 we should not
>>> allow specifying those as getTemplate parameters anymore, but that
>>> will be another thread when we get there.
>>
>> IMO it should pass through the exact text from the include/etc in
>> the template. It might be worth constraining to a valid URI syntax
>> but other than that a TemplateResolver would be much more flexible if no normalization/etc is done.
>
> In FM2 there's a class like this:
>
> public abstract class TemplateNameFormat {
> ...
> abstract String toAbsoluteName(String baseName, String
> targetName) throws MalformedTemplateNameException;
> abstract String normalizeAbsoluteName(String name) throws MalformedTemplateNameException;
> }
>
> My idea was that if we allow users to provide a custom implementation,
> then if the standard rules don't fit your custom TemplateResolver, you
> just create a custom TemplateNameFormat too. But now that I think
> about it more, pushing the TemplateNameFormat functionality on the
> TemplateResolver implementation does indeed look like a better
> approach, because then it can't happen that a configuration
> accidentally contains a TemplateResolver with a TemplateNameFormat
> that doesn't work well with it. But then the above two methods should
> be part of the TemplateResolver interface (because, for example, when
> you #import something, FM has to find out if the same FTL was already
> imported in the current Environment, so it needs to normalize the
> #import argument without loading anything). WDYT?
>
>> Passing in the Locale rather than trying to call it multiple times
>> with different locale extensions is a good idea too as different
>> underlying content or file stores may have their own approach for
>> this.
>
> That's even a problem with the TemplateLoader interface proposed to
> FM3. Like in the recent database template loader thread, the author
> wanted to use a separate column to indicate the locale of the template
> (and the same could happen with custom lookup conditions too), so you
> don't want to encode that information into the template name in the
> case of that particular TemplateLoader. It's something we may try to
> address in a second iteration of the FM3 TemplateLoader interface.
>
>> For example a JCR back end has quite a bit for meta data and
>> content variations.
>>
>> In general more parameters and less built into the path/name is a good thing IMO.
>
> BTW, I still want to see if in FM3 the standard TemplateResolver (the
> one that's basically FM2's TemplateCache, relies on TemplateLoader and
> TemplateLookupStrategy and TemplateNameFormat) could cover more use
> cases without becoming overly intricate. Also, even if you decide to
> use a custom TemplateResolver, perhaps you still want to call some of
> the stock TemplateLoader-s from it (while you replace the caching
> logic, etc., or load from non-TemplateLoader sources too in case of
> more sophisticated storages). I suspect you have experience with these
> things, so it would be good if you can look at the FM3 TemplateLoader
> thread.
>
>> -David
>
--
Thanks,
Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Posted by Daniel Dekany <dd...@freemail.hu>.
Thursday, February 16, 2017, 5:33:01 PM, David E Jones wrote:
> On Thu, 2017-02-16 at 15:56 +0100, Daniel Dekany wrote:
>> Thursday, February 16, 2017, 6:17:00 AM, David E Jones wrote:
>>
>> >
>> > This is cleaner, more obvious what's going on underneath, but since
>> > the DefaultTemplateResolver will be the most commonly used you
>> > could just leave the current setting methods as they are and just
>> > document that if you set a different TemplateResolver they will be
>> > ignored.
>>
>> Or they should just throw exception if the template resolver set is
>> not the default one. It can be quite annoying when someone sets the
>> templateUpdateDelay for example and it has silently no effect.
>
> Good idea, I was thinking more of calls before setting a TemplateResolver.
I would clarify there that it's the build() method that throws the
exception. Because we don't want to constraint the order in which you
set the settings. Like maybe you set the templateLoader on the
builder, and only then set the TemplateLoader to MyTemplateResolver
that doesn't support a templateLoader.
>> Also, it's maybe quite speculative, but perhaps some custom
>> TemplateResolver have some of the standard settings, like for example
>> the templateUpdateDelayMilliseconds. Then it's somewhat confusing that
>> cfg.setTemplateUpdateDelayMilliseconds doesn't work, while
>> myResolver.setTemplateUpdateDelayMilliseconds does. So I guess if we
>> keep those setters in the Configuration, then they should forward to
>> the templateResolver, if it implements the interface that contains the
>> setter.
>
> It isn't too cumbersome to have extra methods to implement, but
> would be nice to have those methods with a default implementation
> that throws the appropriate Exception. In Java 8 you can do this
> with default methods on the interface, but in Java 7- this could
> simply be an abstract class to have the same effect.
Yes, that looks to be the simplest solution. (Even if not the
cleanest... that would be introducing interfaces like
HasTemplateLoaderSetting, and so you can detect this capability
without "probing" the setter methods. Certainly an overkill anyway.)
> -David
>
>
--
Thanks,
Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Posted by Daniel Dekany <dd...@apache.org>.
Friday, June 9, 2017, 12:19:02 AM, Daniel Dekany wrote:
[snip]
> DefaultTemplateResolver happens to support them. If
> Configuration.templateLoader is set to `new MyTemplateResolver()`, and
I meant "If Configuration.templateResolver is set to `new
MyTemplateResolver()`" of course...
--
Thanks,
Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Posted by Daniel Dekany <dd...@apache.org>.
Now this mechanism is working in FM3. At the end I did this:
http://freemarker.org/builds/fm3/api/freemarker-core/org/apache/freemarker/core/templateresolver/TemplateResolver.html
Explanation: If you implement a custom TemplateResolver, then you are
required to declare which of the Configuration settings you support,
out of those that are related to template loading/caching. Then the
user just sets those setting on the Configuration-level, pretty much
like in FM2 (except that in FM3 you set settings in a
Configuration.Builder object, not directly in the Configuration).
DefaultTemplateResolver is not treated specially, it works just like
any custom TemplateResolver; you are allowed to set
Configuration.templateLoader,
Configuration.templateUpdateDellayMilliseconds, etc. because
DefaultTemplateResolver happens to support them. If
Configuration.templateLoader is set to `new MyTemplateResolver()`, and
MyTemplateResolver only cares about templateUpdateDellayMilliseconds,
but can't use the TemplateLoader abstraction of FM, then the user can
set templateUpdateDellayMilliseconds as usual, but can't set
Configuration.templateLoader (they get an exception that tells they
that MyTemplateResolver doesn't support the templateLoader setting).
The author of a custom TemplateResolver admittedly has to face some
extra complexity here, but I believe it's not a big deal in practice.
Also, very few developers will ever implement a custom
TemplateResolver. This thing is optimized for the users who just
configure FreeMarker.
Any insights? Like as a reality check, how well would this work with
Moqui?
Thursday, February 16, 2017, 6:33:01 PM, David E Jones wrote:
> On Thu, 2017-02-16 at 15:56 +0100, Daniel Dekany wrote:
>> Thursday, February 16, 2017, 6:17:00 AM, David E Jones wrote:
>>
>> >
>> > This is cleaner, more obvious what's going on underneath, but since
>> > the DefaultTemplateResolver will be the most commonly used you
>> > could just leave the current setting methods as they are and just
>> > document that if you set a different TemplateResolver they will be
>> > ignored.
>>
>> Or they should just throw exception if the template resolver set is
>> not the default one. It can be quite annoying when someone sets the
>> templateUpdateDelay for example and it has silently no effect.
>
> Good idea, I was thinking more of calls before setting a TemplateResolver.
>
>> Also, it's maybe quite speculative, but perhaps some custom
>> TemplateResolver have some of the standard settings, like for example
>> the templateUpdateDelayMilliseconds. Then it's somewhat confusing that
>> cfg.setTemplateUpdateDelayMilliseconds doesn't work, while
>> myResolver.setTemplateUpdateDelayMilliseconds does. So I guess if we
>> keep those setters in the Configuration, then they should forward to
>> the templateResolver, if it implements the interface that contains the
>> setter.
>
> It isn't too cumbersome to have extra methods to implement, but
> would be nice to have those methods with a default implementation
> that throws the appropriate Exception. In Java 8 you can do this
> with default methods on the interface, but in Java 7- this could
> simply be an abstract class to have the same effect.
>
> -David
--
Thanks,
Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader
Implementation that loads from a database (DataSource))
Posted by David E Jones <de...@dejc.com>.
On Thu, 2017-02-16 at 15:56 +0100, Daniel Dekany wrote:
> Thursday, February 16, 2017, 6:17:00 AM, David E Jones wrote:
>
> >
> > This is cleaner, more obvious what's going on underneath, but since
> > the DefaultTemplateResolver will be the most commonly used you
> > could just leave the current setting methods as they are and just
> > document that if you set a different TemplateResolver they will be
> > ignored.
>
> Or they should just throw exception if the template resolver set is
> not the default one. It can be quite annoying when someone sets the
> templateUpdateDelay for example and it has silently no effect.
Good idea, I was thinking more of calls before setting a TemplateResolver.
> Also, it's maybe quite speculative, but perhaps some custom
> TemplateResolver have some of the standard settings, like for example
> the templateUpdateDelayMilliseconds. Then it's somewhat confusing that
> cfg.setTemplateUpdateDelayMilliseconds doesn't work, while
> myResolver.setTemplateUpdateDelayMilliseconds does. So I guess if we
> keep those setters in the Configuration, then they should forward to
> the templateResolver, if it implements the interface that contains the
> setter.
It isn't too cumbersome to have extra methods to implement, but would be nice to have those methods with a default implementation
that throws the appropriate Exception. In Java 8 you can do this with default methods on the interface, but in Java 7- this could
simply be an abstract class to have the same effect.
-David
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Posted by Daniel Dekany <dd...@freemail.hu>.
Thursday, February 16, 2017, 6:17:00 AM, David E Jones wrote:
>
> This is cleaner, more obvious what's going on underneath, but since
> the DefaultTemplateResolver will be the most commonly used you
> could just leave the current setting methods as they are and just
> document that if you set a different TemplateResolver they will be
> ignored.
Or they should just throw exception if the template resolver set is
not the default one. It can be quite annoying when someone sets the
templateUpdateDelay for example and it has silently no effect.
Also, it's maybe quite speculative, but perhaps some custom
TemplateResolver have some of the standard settings, like for example
the templateUpdateDelayMilliseconds. Then it's somewhat confusing that
cfg.setTemplateUpdateDelayMilliseconds doesn't work, while
myResolver.setTemplateUpdateDelayMilliseconds does. So I guess if we
keep those setters in the Configuration, then they should forward to
the templateResolver, if it implements the interface that contains the
setter.
> -David
>
>
> On Wed, 2017-02-15 at 14:56 +0100, Daniel Dekany wrote:
>> An interesting consequence of introducing the TemplateResolver concept
>> is that it the earlier top-level configuration settings, like
>> templateLoader, templateLookupStrategy, templateNameFormat,
>> cacheStorage, and templateUpdateDelayMillisecond should now be the
>> settings (JavaBean properties) of DefaultTemplateResolver, not of
>> Configuration. In theory at least. It's the same things as with
>> ObjectWrapper, where multiple implementations are possible, and each
>> has different settings.
>>
>> This has some inconvenient consequences for the majority who uses the
>> DefaultTemplateResolver. Earlier, you have written:
>>
>> � cfg.setTemplateLoader(new FooTemplateLoader())
>> � cfg.setTemplateUpdateDelayMilliseconds(10_000);
>>
>> but now you had to write either:
>>
>> � DefaultTemplateResolver dtr = new DefaultTemplateResolver();
>> � dtr.setTemplateLoader(new FooTemplateLoader());
>> � dtr.setTemplateUpdateDelayMilliseconds(10_000);
>> � cfg.setTemplateResolver(dtr);
>>
>> or alternatively:
>>
>> � DefaultTemplateResolver dtr = (DefaultTemplateResolver) dtr.getTemplateResolver();
>> � dtr.setTemplateLoader(new FooTemplateLoader());
>> � dtr.setTemplateUpdateDelayMilliseconds(10_000);
>>
>> Also configuring with j.u.Properties becomes different, because you
>> can't have this anymore:
>>
>> � templateLoader = com.example.FooTemplateLoader
>> � templateUpdateDelay = 10s
>>
>> I guess instead it should be something like:
>>
>> � templateResolver.templateLoader = com.example.FooTemplateLoader
>> � templateResolver.templateUpdateDelay = 10s
>>
>> which is a syntax that we don't yet support (dotted setting names).
>> This syntax we do support though (in FM2):
>>
>> � templateResolver=DefaultTemplateResolver( \
>> ����������templateLoader = com.example.FooTemplateLoader, \
>> ����������templateUpdateDelayMilliseconds = 10_000 \
>> � )
>>
>> but it has the problem that you can't, for example, set the
>> templateUpdateDelay without also specifying the whole templateResolver
>> with all of its settings. So that's where the dotted notation is
>> better. OTOH for some other things the above syntax is better, like
>> for specifying the templateConfigurations (see
>> http://freemarker.org/docs/pgui_config_templateconfigurations.html).
>> So I think we should have both, and since we must ensure that
>> templateResolver=com.example.MyTemplateResolver is "executed" earlier
>> than templateResolver.mySetting=foo anyway, if mySetting is specified
>> on both places (i.e., you have
>> templateResolver=com.example.MyTemplateResolver(mySetting="foo")
>> there), then obviously templateResolver.mySetting wins.
>>
>> And if we support the dotted notation in j.u.Properties, that should
>> work for the objectWrapper setting too, naturally.
>>
>>
>>
>
--
Thanks,
Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader
Implementation that loads from a database (DataSource))
Posted by David E Jones <de...@dejc.com>.
This is cleaner, more obvious what's going on underneath, but since the DefaultTemplateResolver will be the most commonly used you
could just leave the current setting methods as they are and just document that if you set a different TemplateResolver they will be
ignored.
-David
On Wed, 2017-02-15 at 14:56 +0100, Daniel Dekany wrote:
> An interesting consequence of introducing the TemplateResolver concept
> is that it the earlier top-level configuration settings, like
> templateLoader, templateLookupStrategy, templateNameFormat,
> cacheStorage, and templateUpdateDelayMillisecond should now be the
> settings (JavaBean properties) of DefaultTemplateResolver, not of
> Configuration. In theory at least. It's the same things as with
> ObjectWrapper, where multiple implementations are possible, and each
> has different settings.
>
> This has some inconvenient consequences for the majority who uses the
> DefaultTemplateResolver. Earlier, you have written:
>
> � cfg.setTemplateLoader(new FooTemplateLoader())
> � cfg.setTemplateUpdateDelayMilliseconds(10_000);
>
> but now you had to write either:
>
> � DefaultTemplateResolver dtr = new DefaultTemplateResolver();
> � dtr.setTemplateLoader(new FooTemplateLoader());
> � dtr.setTemplateUpdateDelayMilliseconds(10_000);
> � cfg.setTemplateResolver(dtr);
>
> or alternatively:
>
> � DefaultTemplateResolver dtr = (DefaultTemplateResolver) dtr.getTemplateResolver();
> � dtr.setTemplateLoader(new FooTemplateLoader());
> � dtr.setTemplateUpdateDelayMilliseconds(10_000);
>
> Also configuring with j.u.Properties becomes different, because you
> can't have this anymore:
>
> � templateLoader = com.example.FooTemplateLoader
> � templateUpdateDelay = 10s
>
> I guess instead it should be something like:
>
> � templateResolver.templateLoader = com.example.FooTemplateLoader
> � templateResolver.templateUpdateDelay = 10s
>
> which is a syntax that we don't yet support (dotted setting names).
> This syntax we do support though (in FM2):
>
> � templateResolver=DefaultTemplateResolver( \
> ����������templateLoader = com.example.FooTemplateLoader, \
> ����������templateUpdateDelayMilliseconds = 10_000 \
> � )
>
> but it has the problem that you can't, for example, set the
> templateUpdateDelay without also specifying the whole templateResolver
> with all of its settings. So that's where the dotted notation is
> better. OTOH for some other things the above syntax is better, like
> for specifying the templateConfigurations (see
> http://freemarker.org/docs/pgui_config_templateconfigurations.html).
> So I think we should have both, and since we must ensure that
> templateResolver=com.example.MyTemplateResolver is "executed" earlier
> than templateResolver.mySetting=foo anyway, if mySetting is specified
> on both places (i.e., you have
> templateResolver=com.example.MyTemplateResolver(mySetting="foo")
> there), then obviously templateResolver.mySetting wins.
>
> And if we support the dotted notation in j.u.Properties, that should
> work for the objectWrapper setting too, naturally.
>
>
>
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Posted by Daniel Dekany <dd...@freemail.hu>.
An interesting consequence of introducing the TemplateResolver concept
is that it the earlier top-level configuration settings, like
templateLoader, templateLookupStrategy, templateNameFormat,
cacheStorage, and templateUpdateDelayMillisecond should now be the
settings (JavaBean properties) of DefaultTemplateResolver, not of
Configuration. In theory at least. It's the same things as with
ObjectWrapper, where multiple implementations are possible, and each
has different settings.
This has some inconvenient consequences for the majority who uses the
DefaultTemplateResolver. Earlier, you have written:
cfg.setTemplateLoader(new FooTemplateLoader())
cfg.setTemplateUpdateDelayMilliseconds(10_000);
but now you had to write either:
DefaultTemplateResolver dtr = new DefaultTemplateResolver();
dtr.setTemplateLoader(new FooTemplateLoader());
dtr.setTemplateUpdateDelayMilliseconds(10_000);
cfg.setTemplateResolver(dtr);
or alternatively:
DefaultTemplateResolver dtr = (DefaultTemplateResolver) dtr.getTemplateResolver();
dtr.setTemplateLoader(new FooTemplateLoader());
dtr.setTemplateUpdateDelayMilliseconds(10_000);
Also configuring with j.u.Properties becomes different, because you
can't have this anymore:
templateLoader = com.example.FooTemplateLoader
templateUpdateDelay = 10s
I guess instead it should be something like:
templateResolver.templateLoader = com.example.FooTemplateLoader
templateResolver.templateUpdateDelay = 10s
which is a syntax that we don't yet support (dotted setting names).
This syntax we do support though (in FM2):
templateResolver=DefaultTemplateResolver( \
templateLoader = com.example.FooTemplateLoader, \
templateUpdateDelayMilliseconds = 10_000 \
)
but it has the problem that you can't, for example, set the
templateUpdateDelay without also specifying the whole templateResolver
with all of its settings. So that's where the dotted notation is
better. OTOH for some other things the above syntax is better, like
for specifying the templateConfigurations (see
http://freemarker.org/docs/pgui_config_templateconfigurations.html).
So I think we should have both, and since we must ensure that
templateResolver=com.example.MyTemplateResolver is "executed" earlier
than templateResolver.mySetting=foo anyway, if mySetting is specified
on both places (i.e., you have
templateResolver=com.example.MyTemplateResolver(mySetting="foo")
there), then obviously templateResolver.mySetting wins.
And if we support the dotted notation in j.u.Properties, that should
work for the objectWrapper setting too, naturally.
Tuesday, February 7, 2017, 11:59:08 AM, Daniel Dekany wrote:
> Tuesday, February 7, 2017, 7:40:18 AM, David E Jones wrote:
>
>> On Tue, 2017-01-24 at 23:47 +0100, Daniel Dekany wrote:
>>>
>>> Should the TemplateResolver-s (the work name for the new interface)
>>> still get the template name already resolved to absolute and
>>> normalized? So when cfg.getTemplate is called, then it invokes
>>> cfg.templateNameFormat.normalizeAbsoluteName(String) to normalize the
>>> path, and only then it calls the TemplateResolver. Also #import and
>>> such currently calls templateNameFormat.toAbsoluteName(String,
>>> String), and that would happen earlier than TemplateResolver is
>>> involved.
>>>
>>> And how would it look... perhaps like this:
>>>
>>> ����public interface TemplateResolver {
>>>
>>> �������Template getTemplate(String templatePath, Local lookupLocale, Object customLookupCondition)
>>> �������throws IOException;
>>>
>>> ����}
>>>
>>> You may notice that it misses the `encoding` and `parsed` parameter of
>>> cfg.getTemplate. That's because I belive that in FM3 we should not
>>> allow specifying those as getTemplate parameters anymore, but that
>>> will be another thread when we get there.
>>
>> IMO it should pass through the exact text from the include/etc in
>> the template. It might be worth constraining to a valid URI syntax
>> but other than that a TemplateResolver would be much more flexible if no normalization/etc is done.
>
> In FM2 there's a class like this:
>
> public abstract class TemplateNameFormat {
> ...
> abstract String toAbsoluteName(String baseName, String
> targetName) throws MalformedTemplateNameException;
> abstract String normalizeAbsoluteName(String name) throws MalformedTemplateNameException;
> }
>
> My idea was that if we allow users to provide a custom implementation,
> then if the standard rules don't fit your custom TemplateResolver, you
> just create a custom TemplateNameFormat too. But now that I think
> about it more, pushing the TemplateNameFormat functionality on the
> TemplateResolver implementation does indeed look like a better
> approach, because then it can't happen that a configuration
> accidentally contains a TemplateResolver with a TemplateNameFormat
> that doesn't work well with it. But then the above two methods should
> be part of the TemplateResolver interface (because, for example, when
> you #import something, FM has to find out if the same FTL was already
> imported in the current Environment, so it needs to normalize the
> #import argument without loading anything). WDYT?
>
>> Passing in the Locale rather than trying to call it multiple times
>> with different locale extensions is a good idea too as different
>> underlying content or file stores may have their own approach for
>> this.
>
> That's even a problem with the TemplateLoader interface proposed to
> FM3. Like in the recent database template loader thread, the author
> wanted to use a separate column to indicate the locale of the template
> (and the same could happen with custom lookup conditions too), so you
> don't want to encode that information into the template name in the
> case of that particular TemplateLoader. It's something we may try to
> address in a second iteration of the FM3 TemplateLoader interface.
>
>> For example a JCR back end has quite a bit for meta data and
>> content variations.
>>
>> In general more parameters and less built into the path/name is a good thing IMO.
>
> BTW, I still want to see if in FM3 the standard TemplateResolver (the
> one that's basically FM2's TemplateCache, relies on TemplateLoader and
> TemplateLookupStrategy and TemplateNameFormat) could cover more use
> cases without becoming overly intricate. Also, even if you decide to
> use a custom TemplateResolver, perhaps you still want to call some of
> the stock TemplateLoader-s from it (while you replace the caching
> logic, etc., or load from non-TemplateLoader sources too in case of
> more sophisticated storages). I suspect you have experience with these
> things, so it would be good if you can look at the FM3 TemplateLoader
> thread.
>
>> -David
>
--
Thanks,
Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Posted by Daniel Dekany <dd...@freemail.hu>.
Tuesday, February 7, 2017, 7:40:18 AM, David E Jones wrote:
> On Tue, 2017-01-24 at 23:47 +0100, Daniel Dekany wrote:
>>
>> Should the TemplateResolver-s (the work name for the new interface)
>> still get the template name already resolved to absolute and
>> normalized? So when cfg.getTemplate is called, then it invokes
>> cfg.templateNameFormat.normalizeAbsoluteName(String) to normalize the
>> path, and only then it calls the TemplateResolver. Also #import and
>> such currently calls templateNameFormat.toAbsoluteName(String,
>> String), and that would happen earlier than TemplateResolver is
>> involved.
>>
>> And how would it look... perhaps like this:
>>
>> ����public interface TemplateResolver {
>>
>> �������Template getTemplate(String templatePath, Local lookupLocale, Object customLookupCondition)
>> �������throws IOException;
>>
>> ����}
>>
>> You may notice that it misses the `encoding` and `parsed` parameter of
>> cfg.getTemplate. That's because I belive that in FM3 we should not
>> allow specifying those as getTemplate parameters anymore, but that
>> will be another thread when we get there.
>
> IMO it should pass through the exact text from the include/etc in
> the template. It might be worth constraining to a valid URI syntax
> but other than that a TemplateResolver would be much more flexible if no normalization/etc is done.
In FM2 there's a class like this:
public abstract class TemplateNameFormat {
...
abstract String toAbsoluteName(String baseName, String targetName) throws MalformedTemplateNameException;
abstract String normalizeAbsoluteName(String name) throws MalformedTemplateNameException;
}
My idea was that if we allow users to provide a custom implementation,
then if the standard rules don't fit your custom TemplateResolver, you
just create a custom TemplateNameFormat too. But now that I think
about it more, pushing the TemplateNameFormat functionality on the
TemplateResolver implementation does indeed look like a better
approach, because then it can't happen that a configuration
accidentally contains a TemplateResolver with a TemplateNameFormat
that doesn't work well with it. But then the above two methods should
be part of the TemplateResolver interface (because, for example, when
you #import something, FM has to find out if the same FTL was already
imported in the current Environment, so it needs to normalize the
#import argument without loading anything). WDYT?
> Passing in the Locale rather than trying to call it multiple times
> with different locale extensions is a good idea too as different
> underlying content or file stores may have their own approach for
> this.
That's even a problem with the TemplateLoader interface proposed to
FM3. Like in the recent database template loader thread, the author
wanted to use a separate column to indicate the locale of the template
(and the same could happen with custom lookup conditions too), so you
don't want to encode that information into the template name in the
case of that particular TemplateLoader. It's something we may try to
address in a second iteration of the FM3 TemplateLoader interface.
> For example a JCR back end has quite a bit for meta data and
> content variations.
>
> In general more parameters and less built into the path/name is a good thing IMO.
BTW, I still want to see if in FM3 the standard TemplateResolver (the
one that's basically FM2's TemplateCache, relies on TemplateLoader and
TemplateLookupStrategy and TemplateNameFormat) could cover more use
cases without becoming overly intricate. Also, even if you decide to
use a custom TemplateResolver, perhaps you still want to call some of
the stock TemplateLoader-s from it (while you replace the caching
logic, etc., or load from non-TemplateLoader sources too in case of
more sophisticated storages). I suspect you have experience with these
things, so it would be good if you can look at the FM3 TemplateLoader
thread.
> -David
--
Thanks,
Daniel Dekany
Re: Pluggable template retrival/caching logic (Was: Template Loader
Implementation that loads from a database (DataSource))
Posted by David E Jones <de...@dejc.com>.
On Tue, 2017-01-24 at 23:47 +0100, Daniel Dekany wrote:
>
> Should the TemplateResolver-s (the work name for the new interface)
> still get the template name already resolved to absolute and
> normalized? So when cfg.getTemplate is called, then it invokes
> cfg.templateNameFormat.normalizeAbsoluteName(String) to normalize the
> path, and only then it calls the TemplateResolver. Also #import and
> such currently calls templateNameFormat.toAbsoluteName(String,
> String), and that would happen earlier than TemplateResolver is
> involved.
>
> And how would it look... perhaps like this:
>
> ����public interface TemplateResolver {
>
> �������Template getTemplate(String templatePath, Local lookupLocale, Object customLookupCondition)
> �������throws IOException;
>
> ����}
>
> You may notice that it misses the `encoding` and `parsed` parameter of
> cfg.getTemplate. That's because I belive that in FM3 we should not
> allow specifying those as getTemplate parameters anymore, but that
> will be another thread when we get there.
IMO it should pass through the exact text from the include/etc in the template. It might be worth constraining to a valid URI syntax
but other than that a TemplateResolver would be much more flexible if no normalization/etc is done.
Passing in the Locale rather than trying to call it multiple times with different locale extensions is a good idea too as different
underlying content or file stores may have their own approach for this. For example a JCR back end has quite a bit for meta data and
content variations.
In general more parameters and less built into the path/name is a good thing IMO.
-David
Re: Pluggable template retrival/caching logic (Was: Template Loader Implementation that loads from a database (DataSource))
Posted by Daniel Dekany <dd...@freemail.hu>.
Tuesday, January 24, 2017, 9:27:37 PM, David E Jones wrote:
> On Jan 24 2017, at 10:25 am, Daniel Dekany <dd...@freemail.hu> wrote:
>
>> Tuesday, January 24, 2017, 6:48:16 PM, David E Jones wrote:
>
>>
>
>> > but all other Configuration.getTemplate() methods call this method
>
>>
>
>> > so it is an easy way to override all Template loading (assuming you
>> are doing your own caching and so on, which is part of what I wanted
>> in this case, ie use a managed cache external to FreeMarker that
>> other things also use instead of the cache internal to FreeMarker).
>
>>
>
>> That's another thing I definitely want to solve in FM3 (I'm not sure
> if I want to go into that in FM2 though), but by providing a setting
> like cfg.templateResolver (TemplateResolver interface), which has some
> minimalistic "give me a template like this somehow" method, and
> default implementation that delegates to TemplateCache.
>
>
>
> Being able to implement an interface and register it with the Configuration
> would be great, much better than hacking around in the implementation of
> FreeMarker while still being able to override the Template loading behavior
> completely (including caching, location resolution, etc). So yes, this sounds
> great.
Should the TemplateResolver-s (the work name for the new interface)
still get the template name already resolved to absolute and
normalized? So when cfg.getTemplate is called, then it invokes
cfg.templateNameFormat.normalizeAbsoluteName(String) to normalize the
path, and only then it calls the TemplateResolver. Also #import and
such currently calls templateNameFormat.toAbsoluteName(String,
String), and that would happen earlier than TemplateResolver is
involved.
And how would it look... perhaps like this:
public interface TemplateResolver {
Template getTemplate(String templatePath, Local lookupLocale, Object customLookupCondition)
throws IOException;
}
You may notice that it misses the `encoding` and `parsed` parameter of
cfg.getTemplate. That's because I belive that in FM3 we should not
allow specifying those as getTemplate parameters anymore, but that
will be another thread when we get there.
> -David
>
>
>
>
>
--
Thanks,
Daniel Dekany
Re: Template Loader Implementation that loads from a database
(DataSource)
Posted by David E Jones <de...@dejc.com>.
On Jan 24 2017, at 10:25 am, Daniel Dekany <dd...@freemail.hu> wrote:
> Tuesday, January 24, 2017, 6:48:16 PM, David E Jones wrote:
>
> > but all other Configuration.getTemplate() methods call this method
>
> > so it is an easy way to override all Template loading (assuming you
> are doing your own caching and so on, which is part of what I wanted
> in this case, ie use a managed cache external to FreeMarker that
> other things also use instead of the cache internal to FreeMarker).
>
> That's another thing I definitely want to solve in FM3 (I'm not sure
if I want to go into that in FM2 though), but by providing a setting
like cfg.templateResolver (TemplateResolver interface), which has some
minimalistic "give me a template like this somehow" method, and
default implementation that delegates to TemplateCache.
Being able to implement an interface and register it with the Configuration
would be great, much better than hacking around in the implementation of
FreeMarker while still being able to override the Template loading behavior
completely (including caching, location resolution, etc). So yes, this sounds
great.
-David
Re: Template Loader Implementation that loads from a database (DataSource)
Posted by Daniel Dekany <dd...@freemail.hu>.
Tuesday, January 24, 2017, 6:48:16 PM, David E Jones wrote:
> A simpler TemplateLoader would be helpful.
>
>
>
> One approach I've used to simplify it is to override all loading by creating
> an object that extends the freemarker.template.Configuration and override this
> method:
>
>
>
> Template getTemplate(final String name, Locale locale, Object
> customLookupCondition, String encoding,
> boolean parseAsFTL, boolean ignoreMissing)
>
>
>
> The method to override has changed a little over time and I don't think it is
> meant to be used this way,
It's not designed for that for sure. (When it comes to widely used
libraries like FreeMarker, I'm in camp who wants every class to only
have methods that are either final or abstract, and maybe protected.
That is, something is either designed to be overridden, or can't be
overridden.)
> but all other Configuration.getTemplate() methods call this method
> so it is an easy way to override all Template loading (assuming you
> are doing your own caching and so on, which is part of what I wanted
> in this case, ie use a managed cache external to FreeMarker that
> other things also use instead of the cache internal to FreeMarker).
That's another thing I definitely want to solve in FM3 (I'm not sure
if I want to go into that in FM2 though), but by providing a setting
like cfg.templateResolver (TemplateResolver interface), which has some
minimalistic "give me a template like this somehow" method, and
default implementation that delegates to TemplateCache.
> Hopefully this is helpful feedback, ie the need for and one approach I've
> found to a simple override for all Template loading.
>
>
>
> -David
>
>
>
>
> On Jan 24 2017, at 4:28 am, Daniel Dekany <dd...@freemail.hu> wrote:
>
>> Tuesday, January 24, 2017, 6:27:38 AM, Roberto Benitez wrote:
>
>>
>
>> > I am working on a TemplateLoader implementation that loads from a
>> database.� From a previous e-mail with Daniel Dekany, he mentioned
>> that this is something you still need, and may be interested in integrating.
>> I have the source code on github:
>> rbenitez22/freemarker-datasource-loader
>>
>>
>> | |
>> rbenitez22/freemarker-datasource-loader
>> | |
>
>>
>
>> Your link above didn't go through in one piece somehow, so again:
> https://github.com/rbenitez22/freemarker-datasource-loader
> (To clone:
> https://github.com/rbenitez22/freemarker-datasource-loader.git)
>
>>
>
>> > This is something I am still tweaking as I learn more about the
>> inner workings of the frame work.� Let me know if you have further
>> thoughts on what could be modified to make it integrate better.
>
>>
>
>> The TemplateLoader interface have some tricky aspects, so reading the
> JavaDoc carefully is important. (It took a while for me to figure all
> details out too, but then I have JavaDoc-ed them.) Especially look
> into how a templateSource works (the return value of findTemplate). In
> your case I think it should just be the things that you are using in
> the SQL `where`: name and locale. And note that it must implement
> `equals` (and `hash`) properly.
>
>>
>
>> Also, I hope that freemarker-datasource-loader can be simplified over
> time, so it doesn't employ this many interfaces/classes.
>
>>
>
>> My main problem with FreeMarker's TemplateLoader interface is that it
> assumes that you execute existence check, last modification query, and
> loading the actual content as 3 separate steps. While that fits how
> you deal with plain files (and I guess that's what it was modelled
> after), it doesn't fit how you do things with a database. In a
> straightforward implementation we would have 3 SQL round trips per new
> template loaded, and 2 for each up-to-date checks later. In a smarter
> implementation, and as you did it too, you can reduce the number of
> SQL operations be prefetching data into the templateSource object
> during the existence check (i.e., in findTemplateSource). As
> closeTemplateSource will be called pretty soon anyway, working from a
> such "cache" is certainly fine (because closeTemplateSource will
> invalidate it). But I don't think that you should prefetch the
> template content there, because then that will be an unnecessarily
> prefetched for template cache up-to-date checks, which is the dominant
> TemplateLoader usage in many applications. So it's sill 2 SQL-s per
> new template loaded, and 1 per template cache up-to-date checks.
> Certainly not a problem in most apps, but it's still bugs me. In
> FreeMarker 3 I definitely want to fix this be radically changing the
> TemplateLoader interface, and it's also possible that I will backport
> that solution into FreeMarker 2 (if people here will agree), though
> there it will co-exists with the traditional TemplateLoader, probably
> under the name TemplateLoader2 or something, so it's kind of messy.
>
>>
>
>> > \--Roberto
>
>>
>
>> \--
> Thanks,
> �Daniel Dekany
>
--
Thanks,
Daniel Dekany
Re: Template Loader Implementation that loads from a database
(DataSource)
Posted by David E Jones <de...@dejc.com>.
A simpler TemplateLoader would be helpful.
One approach I've used to simplify it is to override all loading by creating
an object that extends the freemarker.template.Configuration and override this
method:
Template getTemplate(final String name, Locale locale, Object
customLookupCondition, String encoding,
boolean parseAsFTL, boolean ignoreMissing)
The method to override has changed a little over time and I don't think it is
meant to be used this way, but all other Configuration.getTemplate() methods
call this method so it is an easy way to override all Template loading
(assuming you are doing your own caching and so on, which is part of what I
wanted in this case, ie use a managed cache external to FreeMarker that other
things also use instead of the cache internal to FreeMarker).
Hopefully this is helpful feedback, ie the need for and one approach I've
found to a simple override for all Template loading.
-David
On Jan 24 2017, at 4:28 am, Daniel Dekany <dd...@freemail.hu> wrote:
> Tuesday, January 24, 2017, 6:27:38 AM, Roberto Benitez wrote:
>
> > I am working on a TemplateLoader implementation that loads from a
> database. From a previous e-mail with Daniel Dekany, he mentioned
> that this is something you still need, and may be interested in integrating.
> I have the source code on github:
> rbenitez22/freemarker-datasource-loader
>
>
> | |
> rbenitez22/freemarker-datasource-loader
> | |
>
> Your link above didn't go through in one piece somehow, so again:
https://github.com/rbenitez22/freemarker-datasource-loader
(To clone: https://github.com/rbenitez22/freemarker-datasource-loader.git)
>
> > This is something I am still tweaking as I learn more about the
> inner workings of the frame work. Let me know if you have further
> thoughts on what could be modified to make it integrate better.
>
> The TemplateLoader interface have some tricky aspects, so reading the
JavaDoc carefully is important. (It took a while for me to figure all
details out too, but then I have JavaDoc-ed them.) Especially look
into how a templateSource works (the return value of findTemplate). In
your case I think it should just be the things that you are using in
the SQL `where`: name and locale. And note that it must implement
`equals` (and `hash`) properly.
>
> Also, I hope that freemarker-datasource-loader can be simplified over
time, so it doesn't employ this many interfaces/classes.
>
> My main problem with FreeMarker's TemplateLoader interface is that it
assumes that you execute existence check, last modification query, and
loading the actual content as 3 separate steps. While that fits how
you deal with plain files (and I guess that's what it was modelled
after), it doesn't fit how you do things with a database. In a
straightforward implementation we would have 3 SQL round trips per new
template loaded, and 2 for each up-to-date checks later. In a smarter
implementation, and as you did it too, you can reduce the number of
SQL operations be prefetching data into the templateSource object
during the existence check (i.e., in findTemplateSource). As
closeTemplateSource will be called pretty soon anyway, working from a
such "cache" is certainly fine (because closeTemplateSource will
invalidate it). But I don't think that you should prefetch the
template content there, because then that will be an unnecessarily
prefetched for template cache up-to-date checks, which is the dominant
TemplateLoader usage in many applications. So it's sill 2 SQL-s per
new template loaded, and 1 per template cache up-to-date checks.
Certainly not a problem in most apps, but it's still bugs me. In
FreeMarker 3 I definitely want to fix this be radically changing the
TemplateLoader interface, and it's also possible that I will backport
that solution into FreeMarker 2 (if people here will agree), though
there it will co-exists with the traditional TemplateLoader, probably
under the name TemplateLoader2 or something, so it's kind of messy.
>
> > \--Roberto
>
> \--
Thanks,
Daniel Dekany
Re: Template Loader Implementation that loads from a database (DataSource)
Posted by Daniel Dekany <dd...@freemail.hu>.
Tuesday, January 24, 2017, 6:27:38 AM, Roberto Benitez wrote:
> I am working on a TemplateLoader implementation that loads from a
> database.� From a previous e-mail with Daniel Dekany, he mentioned
> that this is something you still need, and may be interested in integrating.
> I have the source code on github:
> rbenitez22/freemarker-datasource-loader
>
>
> | |
> rbenitez22/freemarker-datasource-loader
> | |
Your link above didn't go through in one piece somehow, so again:
https://github.com/rbenitez22/freemarker-datasource-loader
(To clone: https://github.com/rbenitez22/freemarker-datasource-loader.git)
> This is something I am still tweaking as I learn more about the
> inner workings of the frame work.� Let me know if you have further
> thoughts on what could be modified to make it integrate better.
The TemplateLoader interface have some tricky aspects, so reading the
JavaDoc carefully is important. (It took a while for me to figure all
details out too, but then I have JavaDoc-ed them.) Especially look
into how a templateSource works (the return value of findTemplate). In
your case I think it should just be the things that you are using in
the SQL `where`: name and locale. And note that it must implement
`equals` (and `hash`) properly.
Also, I hope that freemarker-datasource-loader can be simplified over
time, so it doesn't employ this many interfaces/classes.
My main problem with FreeMarker's TemplateLoader interface is that it
assumes that you execute existence check, last modification query, and
loading the actual content as 3 separate steps. While that fits how
you deal with plain files (and I guess that's what it was modelled
after), it doesn't fit how you do things with a database. In a
straightforward implementation we would have 3 SQL round trips per new
template loaded, and 2 for each up-to-date checks later. In a smarter
implementation, and as you did it too, you can reduce the number of
SQL operations be prefetching data into the templateSource object
during the existence check (i.e., in findTemplateSource). As
closeTemplateSource will be called pretty soon anyway, working from a
such "cache" is certainly fine (because closeTemplateSource will
invalidate it). But I don't think that you should prefetch the
template content there, because then that will be an unnecessarily
prefetched for template cache up-to-date checks, which is the dominant
TemplateLoader usage in many applications. So it's sill 2 SQL-s per
new template loaded, and 1 per template cache up-to-date checks.
Certainly not a problem in most apps, but it's still bugs me. In
FreeMarker 3 I definitely want to fix this be radically changing the
TemplateLoader interface, and it's also possible that I will backport
that solution into FreeMarker 2 (if people here will agree), though
there it will co-exists with the traditional TemplateLoader, probably
under the name TemplateLoader2 or something, so it's kind of messy.
> --Roberto
--
Thanks,
Daniel Dekany