Re: The problem of "too human" text transformations

[Daniel Dekany <dd...@apache.org>]

Back to topic... let's not suck here. In FreeMarker 2.x we already
have a lot of built-ins, so I think it doesn't make sense to miss a
few more that's actually asked for, only because for those we should
also support overriding the default implementations, as it's harder to
find a default that satisfies the 99% of users. Maybe FM3 should do
this differently, but the FM2 way of doing this is already what it is
(i.e., users need to do "x"? We add `?x`... problem solved.)


Thursday, May 31, 2018, 11:03:09 AM, Daniel Dekany wrote:

> Thursday, May 31, 2018, 1:16:26 AM, David E Jones wrote:
>
>> My general opinion on this sort of thing is there are lots of libraries and
>> tools around in the Java world for doing string manipulations and such,
>> they don't need to be built into a template language. Taking that one step
>> further, I'd actually prefer that the expression syntax in the template
>> language makes it easy to call an existing API vs trying to make everything
>> built into the template language.
>>
>> For example in addition to using FTL I also use Groovy heavily and Groovy
>> already has a great API including standard Java API augmentation for all
>> sorts of things. Yes, FTL could use the Groovy String and other expanded
>> objects as a reference for built in design and have a good set of commonly
>> needed things, but why not just defer to Groovy for those that use it or
>> whatever other API/library anyone might prefer.
>>
>> I know this is a VERY different direction from FM2. What I find most useful
>> in FTL is features like macros and the extensive directives and other
>> things that make a template language flexible and powerful. The built-ins
>> are handy sometimes, but often don't do what I need anyway so other than
>> things like ?has_content I don't use them a lot.
>
> An important (and as far as I'm concerned, the main) reason for many
> built-ins is/was that frameworks and other products drop in FreeMarker
> for templates and then often don't give much attention to give tools
> to the template authors. Then the poor people who actually has to
> create templates come and ask, "I must do X... how can I do that in
> FreeMaker?". The less often we have to say "Please pester the
> developers (who are often from another company, or are on another
> sub-project by now => you have near 0 chance) to expose such
> functionality to the templates", the better. Instead we can say "Like
> foo?x", and furthermore, ?x is documented, and people can actually
> find its documentation, even years later. Surely if you have some
> small company with the sharp people, this is not a problem for you.
> Actually, then you very often don't want the one-size-fits-all
> solutions anyway.
>
>> I've mentioned this before but what would be a killer feature for FTL is to
>> support pluggable expression evaluators. The FTL expression syntax isn't
>> awful
>
> (Yes it is... but that's besides the point until I have fixed it in
> FM3.)
>
>> but is not nearly as convenient as Groovy, especially when you are
>> using it extensively elsewhere.
>
> The theoretical problem is of course that both languages (the "outer"
> template language and the language from where you rip off the
> expression language from) have its own ideas, and so there will be
> mismatches. The expression language in a language and other parts of
> the language aren't separate in general anyway. Which leads to the
> question that if you are using Groovy expressions, why not also use
> for loops and whatnot from Groovy? (This is fairly common way of
> creating template language, just taking an existing language, and add
> some convenience syntax to embed it into the static text.) So then we
> don't need our own #list, #if, etc. implementation either. But then,
> the original idea was that a template language is a domain specific
> language, and as such can have more specialized, say, loop constructs
> for that domain. (I'm not saying that FM2 did great there, but there
> was a lack of determination there as well.) If that's not the case
> though...
>
> Anyway, when and if FM3 will be more mature, we can see if how doable
> this is in reality. It's also a question if we are talking about a
> syntax, that in 99% looks like Groovy, or we are talking about actual
> Groovy. In the last case TemplateModel-s and the particular way null
> handling will work will be an obstacle for sure. Not to mention
> TemplateMarkupOutputModel-s, that AFAIK don't even have commonly used
> equivalent (like a class in the standard Java API).
>
>> If the default FTL expressions were not used you couldn't use
>> built-ins and instead would just have expressions that are expected
>> to evaluate to a boolean, string, or arbitrary object and can be
>> whatever.
>
> At least in FM3 built-ins will be just functions you have defined in
> the "core" FTL namespace (whose content is specified in template
> language Dialect you use) that's "statically linked" to the templates
> (allowing some parse-time checks and somewhat faster calls). So
> `foo?bar(1)` is basically just a shorthand syntax for `<#import core =
> 'freemarker://dialect-or-something'>` and `core.bar(foo, 1)`. There's
> a standard/default Dialect of course, which has the big advantage that
> it's documented in the official Manual, is used in SO answers, etc.
>
>> How important is this? Important enough that I've
>> considered using a different template language. If FTL wasn't so
>> darn convenience for XML transformation using macros I'd probably be
>> using Groovy GString templates instead, and just for the far better
>> syntax in Groovy.
>>
>> This also goes back to the object wrapping discussion which is another can
>> of worms...
>
> Yeah, it's one of the things that I really wanted to get rid of, but
> it's a very disrupting transition to do. I don't mean for most of the
> users, but for the source code. On the brighter side, what really
> happens when you do that transition is that instead of something like:
>
>   if (value is TemplateMapLikeModel) {
>       ... ((TemplateMapLikeModel) value).get(k) ...
>   }
>
> you, in first approach, rewrite it to something like:
>
>   if (mop.isMapLike(value)) {
>       ... mop.get(value, k) ...
>   }
>
> Of course that would be a silly MOP API design, so this below is more
> likely, but now it's visible that one comes from the other trivially:
>
>
>   MapLike m = mop.tryGetAsMapLike(value);
>   if (m != null) {
>       ... m.get(k) ...
>   }
>
> Hence when we have our TemplateModel-s cleaned up, it's easier to
> migrate to the MOP approach. The "type system" (just a mental model
> really) depicted be the TemplateModel still exists, you just use a MOP
> to implement it.
>
>> I don't mean to derail this discussion, but especially these days in a
>> world of a huge number of great Java libraries out there it is often better
>> to not try to do everything and instead focus on core strengths that you
>> can't get anywhere else, and make it easy for people to use other open
>> source projects for their core strengths. This could even go as far as
>> FreeMarker builds that have dependencies on certain commonly used open
>> source projects, there are many even in the ASF that do various things that
>> FTL built-ins do but with far more options and well known behavior for the
>> many who use them (like various Commons projects).
>
> It's really the duty of the Dialects feature of FM3. You can put
> together your "toolbox" from whatever you want, like call Apache
> Commons. I mean it's unrelated to question of expression languages, as
> far as I see. (Like, you can call plain Java methods from FM2 as
> well.)
>
> If we want to use Apache Commons etc. in implementing the functions of
> the standard dialect... I *guess* we won't benefit from it much at
> this point, but I need to see it case-by-case to tell, like, the FM
> implementation of function x sucks, while Apache Commons StringUtils.x
> is great, etc. In any case, users don't care, as far as they can call
> ?x and it does its job. My point in this thread is that we should be
> able to tell users to "just write ?x" (see earlier), instead of add
> your own solution somehow. (Also, if ?x works, they prefer if we have
> less dependencies. Especially if we want to support Andorid.)
>
>> Would supporting different expression languages cause huge problems for IDE
>> support? Possibly, but IDEs these days have lots of support for language
>> injection and layers that are meant for just this sort of thing.
>
> At least looking from big enough distance it's easy. :) Note that so
> far we fail to deliver an editor for Eclipse that integrates the
> existing HTML editor functionality with FTL... :-> Please everyone go
> and help out Angelo Zerr so that this will change:
>
> https://github.com/angelozerr/lsp4e-freemarker
> https://github.com/angelozerr/freemarker-languageserver
>
> Anyway, IDE support won't be the obstacle. At worse only the default
> expression syntax will have reasonable IDE support.
>
>> In theory FreeMarker itself could be a heck of a lot smaller and
>> simpler and at the same time be far more flexible and powerful. IMO
>> that could be a game changer in the world of Java-based template
>> languages. Some of this would be more work than not supporting this
>> sort of flexibility (ie support Groovy syntax as an option, ie
>> without requiring the much larger Groovy jar files as dependencies
>> by having a default built-in expression syntax), but by simplifying
>> other things it would save a heck of a lot of work (including design
>> work).
>
> So, I'm not sure I follow what exactly are you proposing. To use
> vanilla Groovy (the actual implementation) for expressions? How does
> that blend with the directives (which mostly also have similar pairs
> in Groovy), and other (planned) FM features? (And... why's the result
> called FreeMarker at all? ;) )
>
>> -David
>>
>>
>>
>> On Wed, May 30, 2018 at 5:50 AM, Taher Alkhateeb <slidingfilaments@gmail.com
>>> wrote:
>>
>>> I'm not a subject matter expert here, but I always thought sensible
>>> defaults with the ability to override using configuration is a wise
>>> move because you appeal to many people out of the box and then allow
>>> minority to also not feel locked out when they need to override. Also,
>>> as a general rule, I found from the projects we worked on that
>>> internationalization is always a challenge, but you can also resolve
>>> that with the same solution (sensible defaults with configuration
>>> overrides); then you simply let the community adopt and improve the
>>> code for non-English parts.
>>>
>>> Of course then the challenge becomes, what's a sensible default? I
>>> wish I had a dollar every time I asked that! I'd favor simplicity
>>> where possible. In your example above maybe "No space f..." is the
>>> simplest, and then people can get fancy if they want to?
>>>
>>> My 2 cents, sorry for the noise :)
>>>
>>> On Wed, May 30, 2018 at 11:59 AM, Daniel Dekany <dd...@apache.org>
>>> wrote:
>>> > There's this old frequently requested built-in,
>>> > maybeTooLongText?truncate(n). Like "No space for such a long
>>> > title"?truncate(13) would give "No space...". The problem with these
>>> > is that there's no single algorithm for this that fits the need of
>>> > everyone. Like maybe the result should be "No space f...", or "No
>>> > space [...]", etc. This can even change depending on the current value
>>> > of the `locale` FreeMarker setting.
>>> >
>>> > Now there's another similar wish, for "?title_case", which has the
>>> > same problem, plus it's much more locale dependent. See:
>>> >
>>> > https://issues.apache.org/jira/browse/FREEMARKER-97
>>> >
>>> > We do provide locale dependent functions, like number and date
>>> > formatting. This is pretty much an FM tradition, to treat non-English
>>> > users as equals (well, except that the keywords aren't localizable).
>>> > However, we push those tasks to the Java platform. But it doesn't
>>> > support title case (as far as I know). Besides there can be different
>>> > preferences even inside the same country, just as with ?truncate.
>>> >
>>> > So, it's acceptable to chicken out ;) saying that none of the above
>>> > has a widely accepted default, so the correct solution is letting the
>>> > user solve this. But I think in practice what happens is that people
>>> > need the functionality (especially ?truncate), and then has to come up
>>> > with some ad-hoc solution on the spot, which will be almost always
>>> > worse than the default we could provide. So we might as well just
>>> > provide some decent default, sparing time for lot of users. Then, then
>>> > if they are particular about the algorithm (which is certainly rare
>>> > nowadays), let them plug their own via the Configurable API. So there
>>> > would be something like
>>> > cfg.setTruncateImplementation(localToImplementationMap), where the
>>> > parameter maps locale-s to Java functions and defines a fallback
>>> > function. (I haven't worked out the API details, as you can see.)
>>> >
>>> > What do you think?
>>> >
>>> > --
>>> > Thanks,
>>> >  Daniel Dekany
>>> >
>>>
>

-- 
Thanks,
 Daniel Dekany