Lambda Expressions - filter list without <#list> directive

[Christoph Rüger <c....@synesty.com>]

Hello,
I was just wondering if it is planned (or maybe there is a way already) to
filter a List/Sequence for specific elements and get a subset of matching
elements: preferably with a one-liner without a <#list> directive

e.g. in Java 8 pseudo code I can write something like this:

*xmlNode.getChildren().filter(c -> c.attr("xml:lang") ==
"x-default").first();*

E.g. to access the first XML by attribute lang="x-default" sub-node this
this:
[image: unnamed.png]

I am wondering if this is (theoretically) possible with freemarker.

e.g.
*xmlNode?children?filterList(node -> attr(node, "lang") ==
"x-default")?first*

Note: attr() would be a custom TemplateMethodModel function from us.

Thanks in advance for thoughts. Maybe for FM3 something like this is
planned, although we would appreciate such syntactical sugar also in FM2 :)

Thanks
Christoph
-- 
Christoph Rüger, Geschäftsführer
Synesty <https://synesty.com/> - Anbinden und Automatisieren ohne
Programmieren - Automatisierung, Schnittstellen, Datenfeeds

Xing: https://www.xing.com/profile/Christoph_Rueger2
LinkedIn: http://www.linkedin.com/pub/christoph-rueger/a/685/198

-- 
Synesty GmbH
Moritz-von-Rohr-Str. 1a
07745 Jena
Tel.: +49 3641 5596493Fax.: 
+49 3641 5596499
Internet: https://synesty.com <https://synesty.com>


Geschäftsführer: Christoph Rüger
Unternehmenssitz: Jena
Handelsregister B 
beim Amtsgericht: Jena
Handelsregister-Nummer: HRB 508766
Ust-IdNr.: 
DE287564982

Re: Lambda Expressions - filter list without <#list> directive

[Christoph Rüger <c....@synesty.com>]

Am Fr., 9. Nov. 2018 um 22:55 Uhr schrieb Daniel Dekany <ddekany@apache.org
>:

> It's certainly tricky, but as far as I see possible (but then, who

knows what will one find when actually working on it). It's also a
> feature missing a lot. It's especially missing for #list (I know that
> you need it for something else), because if you filter the items
> inside #list with #if-s, then #sep, ?hasNext, etc. will not be usable.
>
> As of your use case, XML has XPath for such filtering/queries. Our XML
> wrapper supports that. XPath is maybe too difficult/different for many
> template authors though.
>
I used XML only as one example. We also traverse normal Java-objects /
lists (which got passed to the template context).
It would also help for non-XML use cases.


>
> As of FM2 vs FM3, it should go to FM2 and then forward ported to FM3.
> Unless, there's something in FM2 that seriously complicate lambdas.
>
Great.


>
>
> Some quick mussing on the technical details follows...
>
> We need closures for this, at least if we want to make this universal,
> rather than a special case feature for a ?filter/?map hack. Generally,
> the closure have to store the list of all the visible local/loop
> variable scopes (contexts), plus a reference to the template
> namespace, and be created every time the lambda is reached during
> execution. Like for each method call where you have a lambda as
> parameter, even if the lambda is never evaluated, this capturing has
> to be done. In simple (majority of?) cases though, it can be proven
> that you need not capture anything. Like in in `x -> x.y?abs == 1`,
> where you only access the lambda parameter and built-ins and literals.
> The problem with capturing the variable scopes is not only the CPU
> overhead (storing the references to the scopes, also, when evaluating
> the lambda expression, you have to swap the context stack of the
> runtime environment to that, and then at the end back), but that
> capturing a whole variable scope makes memory leaks. Although template
> execution is short lived, so it's maybe OK in practice, unlike in a
> general purpose language where you might have long running processes.
>
> I think the main use case for lambdas in templates is ?filter(lambda)
> and ?map(lambda). But unlike in Java Stream API, I strongly believe
> that here making them lazy would be a very bad idea, especially
> combined with the average template author. That's because in Java, the
> captured variables must be effectively final, but here, they can
> change later. So somebody writes <#assign filtereXs = xs?filter(x ->
> x.grp == onlyThisGrp)>. Then at some later point onlyThisGrp changes,
> and then later (or even worse, meanwhile) filteredXs is traversed.
> Then you will get wrong output.

Agreed. Since there are no "final variables" in Freemarker, it should not
be lazy.

Also if they are eager, then we can
> always optimize out closure creation, since we know that ?filter and
> ?map doesn't leak the function, nor they create a template-visible
> variable scope. Actually, this means that in the first iteration of
> the lambdas feature we can restrict using lambdas to ?filter/?map, and

Sounds ok to me. Our use case is mainly ?filter I guess.




thus we can get away without closures. Thus these can get earlier into
> a release. Yeah, maybe that should be attempted first.
>
Would be great. I can't think of all the details of closures vs.
no-closures, but if there is a way to start small without the complexities
of scope-capturing, it would be great.


>
> Any thoughts?
>

Just want to add an example to the discussion which shows more obvious what
could be achieved in terms of syntax:


*Before:*
<#assign mylist = ["a","b","c"] />
<#assign filteredResult = "" />
<#list mylist as l>
 <#if l == "b">
   <#assign filteredResult = l />
 </#if>
</#list>

The filtered result is ${filteredResult}.
## output is 'b'

*After:*
The filtered result is ${mylist?filter(x -> x == "b")?first}.
## output is 'b'




>
> --
> Thanks,
>  Daniel Dekany
>
>
> Friday, November 9, 2018, 2:32:14 PM, Christoph Rüger wrote:
>
> > Hello,
> > I was just wondering if it is planned (or maybe there is a way
> > already) to filter a List/Sequence for specific elements and get a
> > subset of matching elements: preferably with a one-liner without a
> <#list> directive
> >
> > e.g. in Java 8 pseudo code I can write something like this:
> >
> > xmlNode.getChildren().filter(c -> c.attr("xml:lang") ==
> "x-default").first();
> >
> > E.g. to access the first XML by attribute lang="x-default" sub-node this
> this:
> >
> >
> > I am wondering if this is (theoretically) possible with freemarker.
> >
> > e.g.
> > xmlNode?children?filterList(node -> attr(node, "lang") ==
> "x-default")?first
> >
> > Note: attr() would be a custom TemplateMethodModel function from us.
> >
> > Thanks in advance for thoughts. Maybe for FM3 something like this
> > is planned, although we would appreciate such syntactical sugar also in
> FM2
> >
> > Thanks
> > Christoph
>
>

-- 
Synesty GmbH
Moritz-von-Rohr-Str. 1a
07745 Jena
Tel.: +49 3641 5596493Fax.: 
+49 3641 5596499
Internet: https://synesty.com <https://synesty.com>


Geschäftsführer: Christoph Rüger
Unternehmenssitz: Jena
Handelsregister B 
beim Amtsgericht: Jena
Handelsregister-Nummer: HRB 508766
Ust-IdNr.: 
DE287564982

Re: Lambda Expressions - filter list without <#list> directive

[Daniel Dekany <dd...@apache.org>]

It's certainly tricky, but as far as I see possible (but then, who
knows what will one find when actually working on it). It's also a
feature missing a lot. It's especially missing for #list (I know that
you need it for something else), because if you filter the items
inside #list with #if-s, then #sep, ?hasNext, etc. will not be usable.

As of your use case, XML has XPath for such filtering/queries. Our XML
wrapper supports that. XPath is maybe too difficult/different for many
template authors though.

As of FM2 vs FM3, it should go to FM2 and then forward ported to FM3.
Unless, there's something in FM2 that seriously complicate lambdas.


Some quick mussing on the technical details follows...

We need closures for this, at least if we want to make this universal,
rather than a special case feature for a ?filter/?map hack. Generally,
the closure have to store the list of all the visible local/loop
variable scopes (contexts), plus a reference to the template
namespace, and be created every time the lambda is reached during
execution. Like for each method call where you have a lambda as
parameter, even if the lambda is never evaluated, this capturing has
to be done. In simple (majority of?) cases though, it can be proven
that you need not capture anything. Like in in `x -> x.y?abs == 1`,
where you only access the lambda parameter and built-ins and literals.
The problem with capturing the variable scopes is not only the CPU
overhead (storing the references to the scopes, also, when evaluating
the lambda expression, you have to swap the context stack of the
runtime environment to that, and then at the end back), but that
capturing a whole variable scope makes memory leaks. Although template
execution is short lived, so it's maybe OK in practice, unlike in a
general purpose language where you might have long running processes.

I think the main use case for lambdas in templates is ?filter(lambda)
and ?map(lambda). But unlike in Java Stream API, I strongly believe
that here making them lazy would be a very bad idea, especially
combined with the average template author. That's because in Java, the
captured variables must be effectively final, but here, they can
change later. So somebody writes <#assign filtereXs = xs?filter(x ->
x.grp == onlyThisGrp)>. Then at some later point onlyThisGrp changes,
and then later (or even worse, meanwhile) filteredXs is traversed.
Then you will get wrong output. Also if they are eager, then we can
always optimize out closure creation, since we know that ?filter and
?map doesn't leak the function, nor they create a template-visible
variable scope. Actually, this means that in the first iteration of
the lambdas feature we can restrict using lambdas to ?filter/?map, and
thus we can get away without closures. Thus these can get earlier into
a release. Yeah, maybe that should be attempted first.

Any thoughts?

-- 
Thanks,
 Daniel Dekany


Friday, November 9, 2018, 2:32:14 PM, Christoph Rüger wrote:

> Hello,
> I was just wondering if it is planned (or maybe there is a way
> already) to filter a List/Sequence for specific elements and get a
> subset of matching elements: preferably with a one-liner without a <#list> directive
>
> e.g. in Java 8 pseudo code I can write something like this: 
>
> xmlNode.getChildren().filter(c -> c.attr("xml:lang") == "x-default").first();
>
> E.g. to access the first XML by attribute lang="x-default" sub-node this this:
>  
>
> I am wondering if this is (theoretically) possible with freemarker. 
>
> e.g. 
> xmlNode?children?filterList(node -> attr(node, "lang") == "x-default")?first
>
> Note: attr() would be a custom TemplateMethodModel function from us. 
>
> Thanks in advance for thoughts. Maybe for FM3 something like this
> is planned, although we would appreciate such syntactical sugar also in FM2
>
> Thanks
> Christoph