You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@freemarker.apache.org by dd...@apache.org on 2019/02/23 16:31:18 UTC
[freemarker] branch 2.3-gae updated (8783907 -> 0635b53)
This is an automated email from the ASF dual-hosted git repository.
ddekany pushed a change to branch 2.3-gae
in repository https://gitbox.apache.org/repos/asf/freemarker.git.
from 8783907 Some code cleanup, mostly around function and macro calls. Also added generic parameters at some places.
new 7eeeea1 (SequenceIterator to be internally usable on its own)
new 7e4ab15 Added "local lambdas" to parser, which look like Java lambda-s, but are only allowed as parameters in certain built-ins, as they can't work outside the variable scope where they were created. To added new built-ins, ?filter(...) and ?map(...) to try the concept. These can get a "local lambda", or an #ftl function, or a method as parameter. These built-ins support the lazy processing of elements when they are the child of an AST node that explicitly enables that (for now [...]
new 0635b53 Fixed issue with local lambdas and #list: The visibility scope of loop variables were wider than necessary, which now become visible, because the lambda applied on the listed value is possibly evaluated lazily after the nested content of #list was already executed for the first iteration, and then the lambda saw the loop variables from the previous iteration. As the lambda expression is outside the nested content of #list, it should never see the loop variables of the sa [...]
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
src/main/java/freemarker/core/BuiltIn.java | 14 +-
.../freemarker/core/BuiltInForLoopVariable.java | 2 +-
.../core/BuiltInWithParseTimeParameters.java | 21 +-
.../java/freemarker/core/BuiltInsForSequences.java | 345 ++++++++++++++++++++-
.../freemarker/core/CollectionAndSequence.java | 19 --
src/main/java/freemarker/core/Environment.java | 71 +++++
...ression.java => ExpressionWithFixedResult.java} | 77 ++---
src/main/java/freemarker/core/Items.java | 2 +-
src/main/java/freemarker/core/IteratorBlock.java | 231 ++++++++------
.../freemarker/core/LocalLambdaExpression.java | 152 +++++++++
src/main/java/freemarker/core/MethodCall.java | 2 +-
.../java/freemarker/core/NonMethodException.java | 7 +-
src/main/java/freemarker/core/Sep.java | 2 +-
...iltInForSequence.java => SequenceIterator.java} | 31 +-
.../core/SingleIterationCollectionModel.java | 51 +++
src/main/javacc/FTL.jj | 127 +++++++-
src/test/java/freemarker/core/FilterBiTest.java | 142 +++++++++
.../core/ListWithStreamLikeBuiltinsTest.java | 104 +++++++
src/test/java/freemarker/core/MapBiTest.java | 206 ++++++++++++
19 files changed, 1404 insertions(+), 202 deletions(-)
copy src/main/java/freemarker/core/{ExistsExpression.java => ExpressionWithFixedResult.java} (50%)
mode change 100755 => 100644
create mode 100644 src/main/java/freemarker/core/LocalLambdaExpression.java
copy src/main/java/freemarker/core/{BuiltInForSequence.java => SequenceIterator.java} (64%)
create mode 100644 src/main/java/freemarker/core/SingleIterationCollectionModel.java
create mode 100644 src/test/java/freemarker/core/FilterBiTest.java
create mode 100644 src/test/java/freemarker/core/ListWithStreamLikeBuiltinsTest.java
create mode 100644 src/test/java/freemarker/core/MapBiTest.java
[freemarker] 03/03: Fixed issue with local lambdas and #list: The
visibility scope of loop variables were wider than necessary,
which now become visible,
because the lambda applied on the listed value is possibly evaluated lazily
after the nested content of #list was already executed for the first
iteration,
and then the lambda saw the loop variables from the previous iteration. As
the lambda expression is outside the nested content of #list,
it should never see the loop variables of the same #list. Also did [...]
Posted by dd...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ddekany pushed a commit to branch 2.3-gae
in repository https://gitbox.apache.org/repos/asf/freemarker.git
commit 0635b53be2d95e511e3c3488ee06cbe094e0a663
Author: ddekany <dd...@apache.org>
AuthorDate: Sat Feb 23 17:28:44 2019 +0100
Fixed issue with local lambdas and #list: The visibility scope of loop variables were wider than necessary, which now become visible, because the lambda applied on the listed value is possibly evaluated lazily after the nested content of #list was already executed for the first iteration, and then the lambda saw the loop variables from the previous iteration. As the lambda expression is outside the nested content of #list, it should never see the loop variables of the same #list. Also [...]
---
.../freemarker/core/BuiltInForLoopVariable.java | 2 +-
src/main/java/freemarker/core/Environment.java | 33 ++++
src/main/java/freemarker/core/Items.java | 2 +-
src/main/java/freemarker/core/IteratorBlock.java | 192 ++++++++++++---------
src/main/java/freemarker/core/Sep.java | 2 +-
.../core/ListWithStreamLikeBuiltinsTest.java | 104 +++++++++++
src/test/java/freemarker/core/MapBiTest.java | 52 ------
7 files changed, 248 insertions(+), 139 deletions(-)
diff --git a/src/main/java/freemarker/core/BuiltInForLoopVariable.java b/src/main/java/freemarker/core/BuiltInForLoopVariable.java
index 8ba7f45..20cb6d3 100644
--- a/src/main/java/freemarker/core/BuiltInForLoopVariable.java
+++ b/src/main/java/freemarker/core/BuiltInForLoopVariable.java
@@ -33,7 +33,7 @@ abstract class BuiltInForLoopVariable extends SpecialBuiltIn {
@Override
TemplateModel _eval(Environment env) throws TemplateException {
- IterationContext iterCtx = IteratorBlock.findEnclosingIterationContext(env, loopVarName);
+ IterationContext iterCtx = env.findEnclosingIterationContextWithVisibleVariable(loopVarName);
if (iterCtx == null) {
// The parser should prevent this situation
throw new _MiscTemplateException(
diff --git a/src/main/java/freemarker/core/Environment.java b/src/main/java/freemarker/core/Environment.java
index 3498a03..eff4c91 100644
--- a/src/main/java/freemarker/core/Environment.java
+++ b/src/main/java/freemarker/core/Environment.java
@@ -650,6 +650,39 @@ public final class Environment extends Configurable {
}
/**
+ * @param loopVarName
+ * Then name of the loop variable that's also visible in FTL at the moment, whose context we are looking
+ * for.
+ * @return The matching context or {@code null} if no such context exists.
+ */
+ IteratorBlock.IterationContext findEnclosingIterationContextWithVisibleVariable(String loopVarName) {
+ return findEnclosingIterationContext(loopVarName);
+ }
+
+ /**
+ * @return The matching context or {@code null} if no such context exists.
+ */
+ IteratorBlock.IterationContext findClosestEnclosingIterationContext() {
+ return findEnclosingIterationContext(null);
+ }
+
+ private IteratorBlock.IterationContext findEnclosingIterationContext(String visibleLoopVarName) {
+ LocalContextStack ctxStack = getLocalContextStack();
+ if (ctxStack != null) {
+ for (int i = ctxStack.size() - 1; i >= 0; i--) {
+ Object ctx = ctxStack.get(i);
+ if (ctx instanceof IteratorBlock.IterationContext
+ && (visibleLoopVarName == null
+ || ((IteratorBlock.IterationContext) ctx)
+ .hasVisibleLoopVar(visibleLoopVarName))) {
+ return (IteratorBlock.IterationContext) ctx;
+ }
+ }
+ }
+ return null;
+ }
+
+ /**
* Evaluate expression with shadowing a single variable with a new local variable.
*
* @since 2.3.29
diff --git a/src/main/java/freemarker/core/Items.java b/src/main/java/freemarker/core/Items.java
index 47366f7..122742c 100644
--- a/src/main/java/freemarker/core/Items.java
+++ b/src/main/java/freemarker/core/Items.java
@@ -44,7 +44,7 @@ class Items extends TemplateElement {
@Override
TemplateElement[] accept(Environment env) throws TemplateException, IOException {
- final IterationContext iterCtx = IteratorBlock.findEnclosingIterationContext(env, null);
+ final IterationContext iterCtx = env.findClosestEnclosingIterationContext();
if (iterCtx == null) {
// The parser should prevent this situation
throw new _MiscTemplateException(env,
diff --git a/src/main/java/freemarker/core/IteratorBlock.java b/src/main/java/freemarker/core/IteratorBlock.java
index e8ed614..59e3c35 100644
--- a/src/main/java/freemarker/core/IteratorBlock.java
+++ b/src/main/java/freemarker/core/IteratorBlock.java
@@ -33,7 +33,6 @@ import freemarker.template.TemplateHashModelEx2;
import freemarker.template.TemplateHashModelEx2.KeyValuePair;
import freemarker.template.TemplateHashModelEx2.KeyValuePairIterator;
import freemarker.template.TemplateModel;
-import freemarker.template.TemplateModelException;
import freemarker.template.TemplateModelIterator;
import freemarker.template.TemplateScalarModel;
import freemarker.template.TemplateSequenceModel;
@@ -45,7 +44,7 @@ import freemarker.template.utility.Constants;
final class IteratorBlock extends TemplateElement {
private final Expression listedExp;
- private final String loopVarName;
+ private final String loopVar1Name;
private final String loopVar2Name;
private final boolean hashListing;
private final boolean forEach;
@@ -54,7 +53,7 @@ final class IteratorBlock extends TemplateElement {
/**
* @param listedExp
* a variable referring to a sequence or collection or extended hash to list
- * @param loopVarName
+ * @param loopVar1Name
* The name of the variable that will hold the value of the current item when looping through listed value,
* or {@code null} if we have a nested {@code #items}. If this is a hash listing then this variable will holds the value
* of the hash key.
@@ -72,13 +71,13 @@ final class IteratorBlock extends TemplateElement {
* Whether this is {@code #foreach} or a {@code #list}.
*/
IteratorBlock(Expression listedExp,
- String loopVarName,
+ String loopVar1Name,
String loopVar2Name,
TemplateElements childrenBeforeElse,
boolean hashListing,
boolean forEach) {
this.listedExp = listedExp;
- this.loopVarName = loopVarName;
+ this.loopVar1Name = loopVar1Name;
this.loopVar2Name = loopVar2Name;
setChildren(childrenBeforeElse);
this.hashListing = hashListing;
@@ -112,33 +111,9 @@ final class IteratorBlock extends TemplateElement {
}
}
- return env.visitIteratorBlock(new IterationContext(listedValue, loopVarName, loopVar2Name));
+ return env.visitIteratorBlock(new IterationContext(listedValue, loopVar1Name, loopVar2Name));
}
- /**
- * @param loopVariableName
- * Then name of the loop variable whose context we are looking for, or {@code null} if we simply look for
- * the innermost context.
- * @return The matching context or {@code null} if no such context exists.
- */
- static IterationContext findEnclosingIterationContext(Environment env, String loopVariableName)
- throws _MiscTemplateException {
- LocalContextStack ctxStack = env.getLocalContextStack();
- if (ctxStack != null) {
- for (int i = ctxStack.size() - 1; i >= 0; i--) {
- Object ctx = ctxStack.get(i);
- if (ctx instanceof IterationContext
- && (loopVariableName == null
- || loopVariableName.equals(((IterationContext) ctx).getLoopVariableName())
- || loopVariableName.equals(((IterationContext) ctx).getLoopVariable2Name())
- )) {
- return (IterationContext) ctx;
- }
- }
- }
- return null;
- }
-
@Override
protected String dump(boolean canonical) {
StringBuilder buf = new StringBuilder();
@@ -146,14 +121,14 @@ final class IteratorBlock extends TemplateElement {
buf.append(getNodeTypeSymbol());
buf.append(' ');
if (forEach) {
- buf.append(_CoreStringUtils.toFTLTopLevelIdentifierReference(loopVarName));
+ buf.append(_CoreStringUtils.toFTLTopLevelIdentifierReference(loopVar1Name));
buf.append(" in ");
buf.append(listedExp.getCanonicalForm());
} else {
buf.append(listedExp.getCanonicalForm());
- if (loopVarName != null) {
+ if (loopVar1Name != null) {
buf.append(" as ");
- buf.append(_CoreStringUtils.toFTLTopLevelIdentifierReference(loopVarName));
+ buf.append(_CoreStringUtils.toFTLTopLevelIdentifierReference(loopVar1Name));
if (loopVar2Name != null) {
buf.append(", ");
buf.append(_CoreStringUtils.toFTLTopLevelIdentifierReference(loopVar2Name));
@@ -174,7 +149,7 @@ final class IteratorBlock extends TemplateElement {
@Override
int getParameterCount() {
- return 1 + (loopVarName != null ? 1 : 0) + (loopVar2Name != null ? 1 : 0);
+ return 1 + (loopVar1Name != null ? 1 : 0) + (loopVar2Name != null ? 1 : 0);
}
@Override
@@ -183,8 +158,8 @@ final class IteratorBlock extends TemplateElement {
case 0:
return listedExp;
case 1:
- if (loopVarName == null) throw new IndexOutOfBoundsException();
- return loopVarName;
+ if (loopVar1Name == null) throw new IndexOutOfBoundsException();
+ return loopVar1Name;
case 2:
if (loopVar2Name == null) throw new IndexOutOfBoundsException();
return loopVar2Name;
@@ -198,7 +173,7 @@ final class IteratorBlock extends TemplateElement {
case 0:
return ParameterRole.LIST_SOURCE;
case 1:
- if (loopVarName == null) throw new IndexOutOfBoundsException();
+ if (loopVar1Name == null) throw new IndexOutOfBoundsException();
return ParameterRole.TARGET_LOOP_VARIABLE;
case 2:
if (loopVar2Name == null) throw new IndexOutOfBoundsException();
@@ -214,7 +189,7 @@ final class IteratorBlock extends TemplateElement {
@Override
boolean isNestedBlockRepeater() {
- return loopVarName != null;
+ return loopVar1Name != null;
}
/**
@@ -227,22 +202,47 @@ final class IteratorBlock extends TemplateElement {
private Object openedIterator;
private boolean hasNext;
- private TemplateModel loopVar;
- private TemplateModel loopVar2;
+ private TemplateModel loopVar1Value;
+ private TemplateModel loopVar2Value;
private int index;
private boolean alreadyEntered;
- private Collection localVarNames = null;
+ private Collection<String> localVarNames = null;
- /** If the {@code #list} has nested {@code #items}, it's {@code null} outside the {@code #items}. */
- private String loopVarName;
- /** Used if we list key-value pairs */
+ /**
+ * The name of the 1st loop variable.
+ * If the {@code #list} has nested {@code #items}, it's {@code null} outside the {@code #items}.
+ * Do not use this to resolve {@link LocalContext#getLocalVariable(String)} and such, as the loop variable might
+ * be still out of scope in FTL when this is already filled; use {@link #visibleLoopVar1Name} for that
+ * instead.
+ */
+ private String loopVar1Name;
+ /**
+ * The name of the 1st loop variable in the {@link LocalContext}. Either {@code null} or {@link #loopVar1Name}.
+ * When {@code null}, none of the loop variables are in scope in FTL.
+ * It would be more intuitive if the {@link LocalContext} is not in the local stack when they aren't visible,
+ * but the {@link LocalContext} is also used for {@code #items} to find its parent, for which we need the tricky
+ * scoping of the local context stack {@link Environment#getLocalContextStack()}.
+ *
+ * (It would be cleaner to have
+ * {@code boolean loopVarsVisible} instead, but it's a trick to decrease runtime overhead added because of
+ * lambdas. Certainly an unmeasurable difference... yet it just doesn't feel right when new features slows
+ * down every existing template a tiny bit, so we try to mitigate that effect.)
+ *
+ * @since 2.3.29
+ */
+ private String visibleLoopVar1Name;
+ /*
+ * The name of the 2nd loop variable, only used if we list key-value pairs.
+ * Do not use this to resolve {@link LocalContext#getLocalVariable} and such, when {@link
+ * #localContextLoopVar1Name} is {@code null}, as then this is not yet in scope as FTL variable.
+ */
private String loopVar2Name;
-
+
private final TemplateModel listedValue;
- public IterationContext(TemplateModel listedValue, String loopVarName, String loopVar2Name) {
+ public IterationContext(TemplateModel listedValue, String loopVar1Name, String loopVar2Name) {
this.listedValue = listedValue;
- this.loopVarName = loopVarName;
+ this.loopVar1Name = loopVar1Name;
this.loopVar2Name = loopVar2Name;
}
@@ -258,17 +258,17 @@ final class IteratorBlock extends TemplateElement {
"The #items directive was already entered earlier for this listing.");
}
alreadyEntered = true;
- this.loopVarName = loopVarName;
+ this.loopVar1Name = loopVarName;
this.loopVar2Name = loopVar2Name;
executeNestedContent(env, childBuffer);
} finally {
- this.loopVarName = null;
+ this.loopVar1Name = null;
this.loopVar2Name = null;
}
}
/**
- * Executes the given block for the {@link #listedValue}: if {@link #loopVarName} is non-{@code null}, then for
+ * Executes the given block for the {@link #listedValue}: if {@link #loopVar1Name} is non-{@code null}, then for
* each list item once, otherwise once if {@link #listedValue} isn't empty.
*/
private boolean executeNestedContent(Environment env, TemplateElement[] childBuffer)
@@ -288,16 +288,19 @@ final class IteratorBlock extends TemplateElement {
: ((TemplateModelIterator) openedIterator);
listNotEmpty = iterModel.hasNext();
if (listNotEmpty) {
- if (loopVarName != null) {
+ if (loopVar1Name != null) {
listLoop: do {
- loopVar = iterModel.next();
+ loopVar1Value = iterModel.next();
hasNext = iterModel.hasNext();
try {
+ visibleLoopVar1Name = loopVar1Name; // Makes all loop variables visible in FTL
env.visit(childBuffer);
} catch (BreakOrContinueException br) {
if (br == BreakOrContinueException.BREAK_INSTANCE) {
break listLoop;
}
+ } finally {
+ visibleLoopVar1Name = null; // Hides all loop variables in FTL
}
index++;
} while (hasNext);
@@ -306,6 +309,7 @@ final class IteratorBlock extends TemplateElement {
// We must reuse this later, because TemplateCollectionModel-s that wrap an Iterator only
// allow one iterator() call. (Also those returned by ?filter, etc. with lazy processing on.)
openedIterator = iterModel;
+ // Note: Loop variables will only become visible inside #items
env.visit(childBuffer);
}
}
@@ -314,32 +318,39 @@ final class IteratorBlock extends TemplateElement {
final int size = seqModel.size();
listNotEmpty = size != 0;
if (listNotEmpty) {
- if (loopVarName != null) {
+ if (loopVar1Name != null) {
listLoop: for (index = 0; index < size; index++) {
- loopVar = seqModel.get(index);
+ loopVar1Value = seqModel.get(index);
hasNext = (size > index + 1);
try {
+ visibleLoopVar1Name = loopVar1Name; // Makes all loop variables visible in FTL
env.visit(childBuffer);
} catch (BreakOrContinueException br) {
if (br == BreakOrContinueException.BREAK_INSTANCE) {
break listLoop;
}
+ } finally {
+ visibleLoopVar1Name = null; // Hides all loop variables in FTL
}
}
} else {
+ // Note: Loop variables will only become visible inside #items
env.visit(childBuffer);
}
}
} else if (env.isClassicCompatible()) {
listNotEmpty = true;
- if (loopVarName != null) {
- loopVar = listedValue;
+ if (loopVar1Name != null) {
+ loopVar1Value = listedValue;
hasNext = false;
}
try {
+ visibleLoopVar1Name = loopVar1Name; // Makes all loop variables visible in FTL
env.visit(childBuffer);
} catch (BreakOrContinueException br) {
// Silently exit "loop"
+ } finally {
+ visibleLoopVar1Name = null; // Hides all loop variables in FTL
}
} else if (listedValue instanceof TemplateHashModelEx
&& !NonSequenceOrCollectionException.isWrappedIterable(listedValue)) {
@@ -357,7 +368,7 @@ final class IteratorBlock extends TemplateElement {
}
private boolean executedNestedContentForHashListing(Environment env, TemplateElement[] childBuffer)
- throws TemplateModelException, IOException, TemplateException {
+ throws IOException, TemplateException {
final boolean hashNotEmpty;
if (listedValue instanceof TemplateHashModelEx) {
TemplateHashModelEx listedHash = (TemplateHashModelEx) listedValue;
@@ -367,25 +378,29 @@ final class IteratorBlock extends TemplateElement {
: (KeyValuePairIterator) openedIterator;
hashNotEmpty = kvpIter.hasNext();
if (hashNotEmpty) {
- if (loopVarName != null) {
+ if (loopVar1Name != null) {
listLoop: do {
KeyValuePair kvp = kvpIter.next();
- loopVar = kvp.getKey();
- loopVar2 = kvp.getValue();
+ loopVar1Value = kvp.getKey();
+ loopVar2Value = kvp.getValue();
hasNext = kvpIter.hasNext();
try {
+ visibleLoopVar1Name = loopVar1Name; // Makes all loop variables visible in FTL
env.visit(childBuffer);
} catch (BreakOrContinueException br) {
if (br == BreakOrContinueException.BREAK_INSTANCE) {
break listLoop;
}
+ } finally {
+ visibleLoopVar1Name = null; // Hides all loop variables in FTL
}
index++;
} while (hasNext);
openedIterator = null;
} else {
- // We will reuse this at the #iterms
+ // We will reuse this at #items
openedIterator = kvpIter;
+ // Note: Loop variables will only become visible inside #items
env.visit(childBuffer);
}
}
@@ -393,25 +408,29 @@ final class IteratorBlock extends TemplateElement {
TemplateModelIterator keysIter = listedHash.keys().iterator();
hashNotEmpty = keysIter.hasNext();
if (hashNotEmpty) {
- if (loopVarName != null) {
+ if (loopVar1Name != null) {
listLoop: do {
- loopVar = keysIter.next();
- if (!(loopVar instanceof TemplateScalarModel)) {
+ loopVar1Value = keysIter.next();
+ if (!(loopVar1Value instanceof TemplateScalarModel)) {
throw _MessageUtil.newKeyValuePairListingNonStringKeyExceptionMessage(
- loopVar, (TemplateHashModelEx) listedValue);
+ loopVar1Value, (TemplateHashModelEx) listedValue);
}
- loopVar2 = listedHash.get(((TemplateScalarModel) loopVar).getAsString());
+ loopVar2Value = listedHash.get(((TemplateScalarModel) loopVar1Value).getAsString());
hasNext = keysIter.hasNext();
try {
+ visibleLoopVar1Name = loopVar1Name; // Makes all loop variables visible in FTL
env.visit(childBuffer);
} catch (BreakOrContinueException br) {
if (br == BreakOrContinueException.BREAK_INSTANCE) {
break listLoop;
}
+ } finally {
+ visibleLoopVar1Name = null; // Hides all loop variables in FTL
}
index++;
} while (hasNext);
} else {
+ // Note: Loop variables will only become visible inside #items
env.visit(childBuffer);
}
}
@@ -431,20 +450,25 @@ final class IteratorBlock extends TemplateElement {
return hashNotEmpty;
}
- String getLoopVariableName() {
- return this.loopVarName;
+ boolean hasVisibleLoopVar(String visibleLoopVarName) {
+ String visibleLoopVar1Name = this.visibleLoopVar1Name;
+ if (visibleLoopVar1Name == null) {
+ return false; // Loop vars aren't in scope in FTL
+ }
+ return visibleLoopVarName.equals(visibleLoopVar1Name) || visibleLoopVarName.equals(loopVar2Name);
}
- String getLoopVariable2Name() {
- return this.loopVar2Name;
- }
-
public TemplateModel getLocalVariable(String name) {
- String loopVariableName = this.loopVarName;
- if (loopVariableName != null && name.startsWith(loopVariableName)) {
- switch(name.length() - loopVariableName.length()) {
+ String visibleLoopVar1Name = this.visibleLoopVar1Name; // Not this.loopVar1Name!
+ if (visibleLoopVar1Name == null) {
+ // Loop variables aren't yet in scope in FTL
+ return null;
+ }
+
+ if (name.startsWith(visibleLoopVar1Name)) {
+ switch(name.length() - visibleLoopVar1Name.length()) {
case 0:
- return loopVar;
+ return loopVar1Value;
case 6:
if (name.endsWith(LOOP_STATE_INDEX)) {
return new SimpleNumber(index);
@@ -459,24 +483,24 @@ final class IteratorBlock extends TemplateElement {
}
if (name.equals(loopVar2Name)) {
- return loopVar2;
+ return loopVar2Value;
}
return null;
}
- public Collection getLocalVariableNames() {
- String loopVariableName = this.loopVarName;
- if (loopVariableName != null) {
+ public Collection<String> getLocalVariableNames() {
+ String visibleLoopVar1Name = this.visibleLoopVar1Name; // Not this.loopVar1Name!
+ if (visibleLoopVar1Name != null) {
if (localVarNames == null) {
localVarNames = new ArrayList(3);
- localVarNames.add(loopVariableName);
- localVarNames.add(loopVariableName + LOOP_STATE_INDEX);
- localVarNames.add(loopVariableName + LOOP_STATE_HAS_NEXT);
+ localVarNames.add(visibleLoopVar1Name);
+ localVarNames.add(visibleLoopVar1Name + LOOP_STATE_INDEX);
+ localVarNames.add(visibleLoopVar1Name + LOOP_STATE_HAS_NEXT);
}
return localVarNames;
} else {
- return Collections.EMPTY_LIST;
+ return Collections.emptyList();
}
}
diff --git a/src/main/java/freemarker/core/Sep.java b/src/main/java/freemarker/core/Sep.java
index d25981f..b26567b 100644
--- a/src/main/java/freemarker/core/Sep.java
+++ b/src/main/java/freemarker/core/Sep.java
@@ -34,7 +34,7 @@ class Sep extends TemplateElement {
@Override
TemplateElement[] accept(Environment env) throws TemplateException, IOException {
- final IterationContext iterCtx = IteratorBlock.findEnclosingIterationContext(env, null);
+ final IterationContext iterCtx = env.findClosestEnclosingIterationContext();
if (iterCtx == null) {
// The parser should prevent this situation
throw new _MiscTemplateException(env,
diff --git a/src/test/java/freemarker/core/ListWithStreamLikeBuiltinsTest.java b/src/test/java/freemarker/core/ListWithStreamLikeBuiltinsTest.java
new file mode 100644
index 0000000..1438304
--- /dev/null
+++ b/src/test/java/freemarker/core/ListWithStreamLikeBuiltinsTest.java
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package freemarker.core;
+
+import org.junit.Test;
+
+import freemarker.template.Configuration;
+import freemarker.test.TemplateTest;
+
+public class ListWithStreamLikeBuiltinsTest extends TemplateTest {
+
+ @Override
+ protected Configuration createConfiguration() throws Exception {
+ Configuration cfg = super.createConfiguration();
+ cfg.setNumberFormat("0.####");
+ cfg.setBooleanFormat("c");
+ return cfg;
+ }
+
+ @Test
+ public void testLambdaScope() throws Exception {
+ // The loop variables aren't visible during the lazy processing done for the elements
+ assertOutput("<#list (1..3)?map(p -> p * 10 + it!'-') as it>${it}<#sep>, </#list>",
+ "10-, 20-, 30-");
+ assertOutput("<#list (1..3)?map(p -> p * 10 + it_has_next!'-') as it>${it}<#sep>, </#list>",
+ "10-, 20-, 30-");
+ assertOutput("<#list (1..3)?map(p -> p * 10 + it!'-')><#items as it>${it}<#sep>, </#items></#list>",
+ "10-, 20-, 30-");
+
+ // #else scope wasn't messed up
+ assertOutput("<#list []?map(p -> p) as it>${it}<#else>${it_has_next!'-'}</#list>",
+ "-");
+ }
+
+ @Test
+ public void testListEnablesLaziness() throws Exception {
+ // #list enables lazy evaluation:
+ assertOutput(
+ "" +
+ "<#assign s = ''>" +
+ "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
+ "<#list (1..3)?map(tenTimes) as x>" +
+ "<#assign s += x>" +
+ "<#sep><#assign s += ', '>" +
+ "</#list>" +
+ "${s}",
+ "1->10, 2->20, 3->30");
+ // Most other context causes eager behavior:
+ assertOutput(
+ "" +
+ "<#assign s = ''>" +
+ "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
+ "<#assign xs = (1..3)?map(tenTimes)>" +
+ "<#list xs as x>" +
+ "<#assign s += x>" +
+ "<#sep><#assign s += ', '>" +
+ "</#list>" +
+ "${s}",
+ "1->2->3->10, 20, 30");
+
+ // ?map-s can be chained and all is "streaming":
+ assertOutput(
+ "" +
+ "<#assign s = ''>" +
+ "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
+ "<#list (1..3)?map(tenTimes)?map(tenTimes)?map(tenTimes) as x>" +
+ "<#assign s += x>" +
+ "<#sep><#assign s += ', '>" +
+ "</#list>" +
+ "${s}",
+ "1->10->100->1000, 2->20->200->2000, 3->30->300->3000");
+
+ // Rest of the elements not consumed after #break:
+ assertOutput(
+ "" +
+ "<#assign s = ''>" +
+ "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
+ "<#list (1..3)?map(tenTimes) as x>" +
+ "<#assign s += x>" +
+ "<#sep><#assign s += ', '>" +
+ "<#if x == 20><#break></#if>" +
+ "</#list>" +
+ "${s}",
+ "1->10, 2->20, ");
+ }
+
+}
diff --git a/src/test/java/freemarker/core/MapBiTest.java b/src/test/java/freemarker/core/MapBiTest.java
index 56bca58..49934be 100644
--- a/src/test/java/freemarker/core/MapBiTest.java
+++ b/src/test/java/freemarker/core/MapBiTest.java
@@ -164,58 +164,6 @@ public class MapBiTest extends TemplateTest {
}
@Test
- public void testLaziness() throws Exception {
- // #list enables lazy evaluation:
- assertOutput(
- "" +
- "<#assign s = ''>" +
- "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
- "<#list (1..3)?map(tenTimes) as x>" +
- "<#assign s += x>" +
- "<#sep><#assign s += ', '>" +
- "</#list>" +
- "${s}",
- "1->10, 2->20, 3->30");
- // Most other context causes eager behavior:
- assertOutput(
- "" +
- "<#assign s = ''>" +
- "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
- "<#assign xs = (1..3)?map(tenTimes)>" +
- "<#list xs as x>" +
- "<#assign s += x>" +
- "<#sep><#assign s += ', '>" +
- "</#list>" +
- "${s}",
- "1->2->3->10, 20, 30");
-
- // ?map-s can be chained and all is "streaming":
- assertOutput(
- "" +
- "<#assign s = ''>" +
- "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
- "<#list (1..3)?map(tenTimes)?map(tenTimes)?map(tenTimes) as x>" +
- "<#assign s += x>" +
- "<#sep><#assign s += ', '>" +
- "</#list>" +
- "${s}",
- "1->10->100->1000, 2->20->200->2000, 3->30->300->3000");
-
- // Rest of the elements not consumed after #break:
- assertOutput(
- "" +
- "<#assign s = ''>" +
- "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
- "<#list (1..3)?map(tenTimes) as x>" +
- "<#assign s += x>" +
- "<#sep><#assign s += ', '>" +
- "<#if x == 20><#break></#if>" +
- "</#list>" +
- "${s}",
- "1->10, 2->20, ");
- }
-
- @Test
public void testErrorMessages() {
assertErrorContains("${1?map(it -> it)}", TemplateException.class,
"sequence or collection", "number");
[freemarker] 01/03: (SequenceIterator to be internally usable on
its own)
Posted by dd...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ddekany pushed a commit to branch 2.3-gae
in repository https://gitbox.apache.org/repos/asf/freemarker.git
commit 7eeeea1e95b467b74d93e5b4651b19ecc33ea2ab
Author: ddekany <dd...@apache.org>
AuthorDate: Sat Feb 16 21:06:27 2019 +0100
(SequenceIterator to be internally usable on its own)
---
.../freemarker/core/CollectionAndSequence.java | 19 ----------
.../java/freemarker/core/SequenceIterator.java | 44 ++++++++++++++++++++++
2 files changed, 44 insertions(+), 19 deletions(-)
diff --git a/src/main/java/freemarker/core/CollectionAndSequence.java b/src/main/java/freemarker/core/CollectionAndSequence.java
index 33556ff..affb7fe 100644
--- a/src/main/java/freemarker/core/CollectionAndSequence.java
+++ b/src/main/java/freemarker/core/CollectionAndSequence.java
@@ -85,23 +85,4 @@ implements TemplateCollectionModel, TemplateSequenceModel, Serializable {
}
}
- private static class SequenceIterator
- implements TemplateModelIterator {
- private final TemplateSequenceModel sequence;
- private final int size;
- private int index = 0;
-
- SequenceIterator(TemplateSequenceModel sequence) throws TemplateModelException {
- this.sequence = sequence;
- this.size = sequence.size();
-
- }
- public TemplateModel next() throws TemplateModelException {
- return sequence.get(index++);
- }
-
- public boolean hasNext() {
- return index < size;
- }
- }
}
diff --git a/src/main/java/freemarker/core/SequenceIterator.java b/src/main/java/freemarker/core/SequenceIterator.java
new file mode 100644
index 0000000..3c8e84a
--- /dev/null
+++ b/src/main/java/freemarker/core/SequenceIterator.java
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package freemarker.core;
+
+import freemarker.template.TemplateModel;
+import freemarker.template.TemplateModelException;
+import freemarker.template.TemplateModelIterator;
+import freemarker.template.TemplateSequenceModel;
+
+class SequenceIterator implements TemplateModelIterator {
+ private final TemplateSequenceModel sequence;
+ private final int size;
+ private int index = 0;
+
+ SequenceIterator(TemplateSequenceModel sequence) throws TemplateModelException {
+ this.sequence = sequence;
+ this.size = sequence.size();
+
+ }
+ public TemplateModel next() throws TemplateModelException {
+ return sequence.get(index++);
+ }
+
+ public boolean hasNext() {
+ return index < size;
+ }
+}
[freemarker] 02/03: Added "local lambdas" to parser,
which look like Java lambda-s,
but are only allowed as parameters in certain built-ins,
as they can't work outside the variable scope where they were created. To
added new built-ins,
?filter(...) and ?map(...) to try the concept. These can get a "local
lambda", or an #ftl function,
or a method as parameter. These built-ins support the lazy processing of
elements when they are the child of an AST node that explicitly enables
that (for now only #list and a fe [...]
Posted by dd...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
ddekany pushed a commit to branch 2.3-gae
in repository https://gitbox.apache.org/repos/asf/freemarker.git
commit 7e4ab15adb09ea6347ee5052b6b027c69cbdeafe
Author: ddekany <dd...@apache.org>
AuthorDate: Sun Feb 17 15:39:38 2019 +0100
Added "local lambdas" to parser, which look like Java lambda-s, but are only allowed as parameters in certain built-ins, as they can't work outside the variable scope where they were created. To added new built-ins, ?filter(...) and ?map(...) to try the concept. These can get a "local lambda", or an #ftl function, or a method as parameter. These built-ins support the lazy processing of elements when they are the child of an AST node that explicitly enables that (for now only #list and [...]
---
src/main/java/freemarker/core/BuiltIn.java | 14 +-
.../core/BuiltInWithParseTimeParameters.java | 21 +-
.../java/freemarker/core/BuiltInsForSequences.java | 345 ++++++++++++++++++++-
src/main/java/freemarker/core/Environment.java | 38 +++
.../freemarker/core/ExpressionWithFixedResult.java | 73 +++++
src/main/java/freemarker/core/IteratorBlock.java | 41 +--
.../freemarker/core/LocalLambdaExpression.java | 152 +++++++++
src/main/java/freemarker/core/MethodCall.java | 2 +-
.../java/freemarker/core/NonMethodException.java | 7 +-
.../core/SingleIterationCollectionModel.java | 51 +++
src/main/javacc/FTL.jj | 127 +++++++-
src/test/java/freemarker/core/FilterBiTest.java | 142 +++++++++
src/test/java/freemarker/core/MapBiTest.java | 258 +++++++++++++++
13 files changed, 1234 insertions(+), 37 deletions(-)
diff --git a/src/main/java/freemarker/core/BuiltIn.java b/src/main/java/freemarker/core/BuiltIn.java
index 7d063ab..450dd6d 100644
--- a/src/main/java/freemarker/core/BuiltIn.java
+++ b/src/main/java/freemarker/core/BuiltIn.java
@@ -84,7 +84,7 @@ abstract class BuiltIn extends Expression implements Cloneable {
static final Set<String> CAMEL_CASE_NAMES = new TreeSet<String>();
static final Set<String> SNAKE_CASE_NAMES = new TreeSet<String>();
- static final int NUMBER_OF_BIS = 279;
+ static final int NUMBER_OF_BIS = 281;
static final HashMap<String, BuiltIn> BUILT_INS_BY_NAME = new HashMap(NUMBER_OF_BIS * 3 / 2 + 1, 1f);
static {
@@ -115,6 +115,7 @@ abstract class BuiltIn extends Expression implements Cloneable {
putBI("esc", new escBI());
putBI("eval", new evalBI());
putBI("exists", new BuiltInsForExistenceHandling.existsBI());
+ putBI("filter", new BuiltInsForSequences.filterBI());
putBI("first", new firstBI());
putBI("float", new floatBI());
putBI("floor", new floorBI());
@@ -238,6 +239,7 @@ abstract class BuiltIn extends Expression implements Cloneable {
putBI("long", new longBI());
putBI("lower_abc", "lowerAbc", new BuiltInsForNumbers.lower_abcBI());
putBI("lower_case", "lowerCase", new BuiltInsForStringsBasic.lower_caseBI());
+ putBI("map", new BuiltInsForSequences.mapBI());
putBI("namespace", new BuiltInsForMultipleTypes.namespaceBI());
putBI("new", new NewBI());
putBI("markup_string", "markupString", new markup_stringBI());
@@ -385,9 +387,19 @@ abstract class BuiltIn extends Expression implements Cloneable {
}
bi.key = key;
bi.target = target;
+ if (bi.isSingleIterationCollectionTargetSupported()) {
+ if (target instanceof BuiltInsForSequences.IntermediateStreamOperationLikeBuiltIn) {
+ ((BuiltInsForSequences.IntermediateStreamOperationLikeBuiltIn) target)
+ .setLazyProcessingAllowed(true);
+ }
+ }
return bi;
}
+ protected boolean isSingleIterationCollectionTargetSupported() {
+ return false;
+ }
+
@Override
public String getCanonicalForm() {
return target.getCanonicalForm() + "?" + key;
diff --git a/src/main/java/freemarker/core/BuiltInWithParseTimeParameters.java b/src/main/java/freemarker/core/BuiltInWithParseTimeParameters.java
index f17c204..383506f 100644
--- a/src/main/java/freemarker/core/BuiltInWithParseTimeParameters.java
+++ b/src/main/java/freemarker/core/BuiltInWithParseTimeParameters.java
@@ -85,13 +85,25 @@ abstract class BuiltInWithParseTimeParameters extends SpecialBuiltIn {
}
}
- protected ParseException newArgumentCountException(String ordinalityDesc, Token openParen, Token closeParen) {
+ protected final ParseException newArgumentCountException(String ordinalityDesc, Token openParen, Token closeParen) {
return new ParseException(
"?" + key + "(...) " + ordinalityDesc + " parameters", this.getTemplate(),
openParen.beginLine, openParen.beginColumn,
closeParen.endLine, closeParen.endColumn);
}
+ protected final void checkLocalLambdaParamCount(LocalLambdaExpression localLambdaExp, int expectedParamCount)
+ throws ParseException {
+ int actualParamCount = localLambdaExp.getLambdaParameterList().getParameters().size();
+ if (actualParamCount != expectedParamCount) {
+ throw new ParseException(
+ "?" + key + "(...) parameter lambda expression must declare exactly " + expectedParamCount + " "
+ + "parameter" + (expectedParamCount > 1 ? "s" : "") + ", but it declared "
+ + actualParamCount + ".",
+ localLambdaExp);
+ }
+ }
+
@Override
protected Expression deepCloneWithIdentifierReplaced_inner(
String replacedIdentifier, Expression replacement, ReplacemenetState replacementState) {
@@ -108,5 +120,12 @@ abstract class BuiltInWithParseTimeParameters extends SpecialBuiltIn {
protected abstract void cloneArguments(Expression clone, String replacedIdentifier,
Expression replacement, ReplacemenetState replacementState);
+
+ /**
+ * If parameter expressions can be {@link LocalLambdaExpression}-s.
+ */
+ protected boolean isLocalLambdaParameterSupported() {
+ return false;
+ }
}
diff --git a/src/main/java/freemarker/core/BuiltInsForSequences.java b/src/main/java/freemarker/core/BuiltInsForSequences.java
index 9bc8d95..0bb2bd9 100644
--- a/src/main/java/freemarker/core/BuiltInsForSequences.java
+++ b/src/main/java/freemarker/core/BuiltInsForSequences.java
@@ -37,6 +37,7 @@ import freemarker.template.TemplateCollectionModelEx;
import freemarker.template.TemplateDateModel;
import freemarker.template.TemplateException;
import freemarker.template.TemplateHashModel;
+import freemarker.template.TemplateMethodModel;
import freemarker.template.TemplateMethodModelEx;
import freemarker.template.TemplateModel;
import freemarker.template.TemplateModelException;
@@ -141,7 +142,12 @@ class BuiltInsForSequences {
}
static class firstBI extends BuiltIn {
-
+
+ @Override
+ protected boolean isSingleIterationCollectionTargetSupported() {
+ return true;
+ }
+
@Override
TemplateModel _eval(Environment env)
throws TemplateException {
@@ -177,7 +183,12 @@ class BuiltInsForSequences {
}
static class joinBI extends BuiltIn {
-
+
+ @Override
+ protected boolean isSingleIterationCollectionTargetSupported() {
+ return true;
+ }
+
private class BIMethodForCollection implements TemplateMethodModelEx {
private final Environment env;
@@ -246,7 +257,7 @@ class BuiltInsForSequences {
throw new NonSequenceOrCollectionException(target, model, env);
}
}
-
+
}
static class lastBI extends BuiltInForSequence {
@@ -289,6 +300,12 @@ class BuiltInsForSequences {
}
static class seq_containsBI extends BuiltIn {
+
+ @Override
+ protected boolean isSingleIterationCollectionTargetSupported() {
+ return true;
+ }
+
private class BIMethodForCollection implements TemplateMethodModelEx {
private TemplateCollectionModel m_coll;
private Environment m_env;
@@ -355,7 +372,12 @@ class BuiltInsForSequences {
}
static class seq_index_ofBI extends BuiltIn {
-
+
+ @Override
+ protected boolean isSingleIterationCollectionTargetSupported() {
+ return true;
+ }
+
private class BIMethod implements TemplateMethodModelEx {
protected final TemplateSequenceModel m_seq;
@@ -893,6 +915,11 @@ class BuiltInsForSequences {
}
@Override
+ protected boolean isSingleIterationCollectionTargetSupported() {
+ return true;
+ }
+
+ @Override
TemplateModel _eval(Environment env)
throws TemplateException {
TemplateModel model = target.eval(env);
@@ -951,8 +978,314 @@ class BuiltInsForSequences {
}
}
-
+
+ /**
+ * Built-in that's similar to an Java 8 Stream intermediate operation. To be on the safe side, by default these
+ * are eager, and just produce a {@link TemplateSequenceModel}. But when circumstances allow, they become
+ * lazy, similarly to Java 8 Stream-s. Another characteristic of the built-ins that they usually accept
+ * lambda expressions as parameters.
+ */
+ static abstract class IntermediateStreamOperationLikeBuiltIn extends BuiltInWithParseTimeParameters {
+
+ private Expression elementTransformerExp;
+ private ElementTransformer precreatedElementTransformer;
+ private boolean lazyProcessingAllowed;
+
+ @Override
+ void bindToParameters(List<Expression> parameters, Token openParen, Token closeParen) throws ParseException {
+ if (parameters.size() != 1) {
+ throw newArgumentCountException("requires exactly 1", openParen, closeParen);
+ }
+ this.elementTransformerExp = parameters.get(0);
+ if (elementTransformerExp instanceof LocalLambdaExpression) {
+ LocalLambdaExpression localLambdaExp = (LocalLambdaExpression) elementTransformerExp;
+ checkLocalLambdaParamCount(localLambdaExp, 1);
+ // We can't do this with other kind of expressions, as they need to be evaluated on runtime:
+ precreatedElementTransformer = new LocalLambdaElementTransformer(localLambdaExp);
+ }
+
+ if (target instanceof IntermediateStreamOperationLikeBuiltIn) {
+ ((IntermediateStreamOperationLikeBuiltIn) target).setLazyProcessingAllowed(true);
+ }
+ }
+
+ @Override
+ protected boolean isLocalLambdaParameterSupported() {
+ return true;
+ }
+
+ boolean isLazyProcessingAllowed() {
+ return lazyProcessingAllowed;
+ }
+
+ /**
+ * Used to allow processing of the collection or sequence elements on an as-needed basis, similarly as
+ * Java 8 Stream intermediate operations do it. This is initially {@code false}. The containing expression or
+ * directive sets it to {@code true} if it can ensure that:
+ * <ul>
+ * <li>The returned {@link TemplateCollectionModel} is traversed only once, more specifically,
+ * {@link TemplateCollectionModel#iterator()} is called only once.
+ * <li>When the methods of the collection or iterator are called, the context provided by
+ * the {@link Environment} (such as the local context stack) is similar to the context from where the
+ * built-in was called. This is required as lambda expression are {@link LocalLambdaExpression}-s.
+ * </ul>
+ */
+ void setLazyProcessingAllowed(boolean lazyProcessingAllowed) {
+ this.lazyProcessingAllowed = lazyProcessingAllowed;
+ }
+
+ protected List<Expression> getArgumentsAsList() {
+ return Collections.singletonList(elementTransformerExp);
+ }
+
+ protected int getArgumentsCount() {
+ return 1;
+ }
+
+ protected Expression getArgumentParameterValue(int argIdx) {
+ if (argIdx != 0) {
+ throw new IndexOutOfBoundsException();
+ }
+ return elementTransformerExp;
+ }
+
+ protected Expression getElementTransformerExp() {
+ return elementTransformerExp;
+ }
+
+ protected void cloneArguments(
+ Expression clone, String replacedIdentifier, Expression replacement, ReplacemenetState replacementState) {
+ ((IntermediateStreamOperationLikeBuiltIn) clone).elementTransformerExp
+ = elementTransformerExp.deepCloneWithIdentifierReplaced(
+ replacedIdentifier, replacement, replacementState);
+ }
+
+ TemplateModel _eval(Environment env) throws TemplateException {
+ TemplateModel lho = target.eval(env);
+ TemplateModelIterator lhoIterator = getTemplateModelIterator(env, lho);
+ return calculateResult(lhoIterator, lho, evalElementTransformerExp(env), env);
+ }
+
+ private ElementTransformer evalElementTransformerExp(Environment env) throws TemplateException {
+ if (precreatedElementTransformer != null) {
+ return precreatedElementTransformer;
+ }
+
+ TemplateModel elementTransformerModel = elementTransformerExp.eval(env);
+ if (elementTransformerModel instanceof TemplateMethodModel) {
+ return new MethodElementTransformer((TemplateMethodModel) elementTransformerModel);
+ } else if (elementTransformerModel instanceof Macro) {
+ return new FunctionElementTransformer((Macro) elementTransformerModel, elementTransformerExp);
+ } else {
+ throw new NonMethodException(elementTransformerExp, elementTransformerModel, true, true, null, env);
+ }
+ }
+
+ private TemplateModelIterator getTemplateModelIterator(Environment env, TemplateModel model) throws TemplateModelException,
+ NonSequenceOrCollectionException, InvalidReferenceException {
+ if (model instanceof TemplateCollectionModel) {
+ return ((TemplateCollectionModel) model).iterator();
+ } else if (model instanceof TemplateSequenceModel) {
+ return new SequenceIterator((TemplateSequenceModel) model);
+ } else if (model instanceof TemplateModelIterator) { // For a stream mode LHO
+ return (TemplateModelIterator) model;
+ } else {
+ throw new NonSequenceOrCollectionException(target, model, env);
+ }
+ }
+
+ /**
+ * @param lhoIterator Use this to iterate through the items
+ * @param lho Maybe needed for operations specific to the built-in, like getting the size
+ *
+ * @return {@link TemplateSequenceModel} or {@link TemplateCollectionModel} or {@link TemplateModelIterator}.
+ */
+ protected abstract TemplateModel calculateResult(
+ TemplateModelIterator lhoIterator, TemplateModel lho, ElementTransformer elementTransformer,
+ Environment env) throws TemplateException;
+
+ interface ElementTransformer {
+ TemplateModel transformElement(TemplateModel element, Environment env) throws TemplateException;
+ }
+
+ private static class LocalLambdaElementTransformer implements ElementTransformer {
+ private final LocalLambdaExpression elementTransformerExp;
+
+ public LocalLambdaElementTransformer(LocalLambdaExpression elementTransformerExp) {
+ this.elementTransformerExp = elementTransformerExp;
+ }
+
+ public TemplateModel transformElement(TemplateModel element, Environment env) throws TemplateException {
+ return elementTransformerExp.invokeLambdaDefinedFunction(element, env);
+ }
+ }
+
+ private static class MethodElementTransformer implements ElementTransformer {
+ private final TemplateMethodModel elementTransformer;
+
+ public MethodElementTransformer(TemplateMethodModel elementTransformer) {
+ this.elementTransformer = elementTransformer;
+ }
+
+ public TemplateModel transformElement(TemplateModel element, Environment env)
+ throws TemplateModelException {
+ Object result = elementTransformer.exec(Collections.singletonList(element));
+ return result instanceof TemplateModel ? (TemplateModel) result : env.getObjectWrapper().wrap(result);
+ }
+ }
+
+ private static class FunctionElementTransformer implements ElementTransformer {
+ private final Macro templateTransformer;
+ private final Expression elementTransformerExp;
+
+ public FunctionElementTransformer(Macro templateTransformer, Expression elementTransformerExp) {
+ this.templateTransformer = templateTransformer;
+ this.elementTransformerExp = elementTransformerExp;
+ }
+
+ public TemplateModel transformElement(TemplateModel element, Environment env) throws
+ TemplateException {
+ ExpressionWithFixedResult functionArgExp = new ExpressionWithFixedResult(
+ element, elementTransformerExp);
+ return env.invokeFunction(env, templateTransformer,
+ Collections.singletonList(functionArgExp),
+ elementTransformerExp);
+ }
+ }
+
+ }
+
+ static class filterBI extends IntermediateStreamOperationLikeBuiltIn {
+
+ protected TemplateModel calculateResult(
+ final TemplateModelIterator lhoIterator, final TemplateModel lho,
+ final ElementTransformer elementTransformer,
+ final Environment env) throws TemplateException {
+ if (!isLazyProcessingAllowed()) {
+ List<TemplateModel> resultList = new ArrayList<TemplateModel>();
+ while (lhoIterator.hasNext()) {
+ TemplateModel element = lhoIterator.next();
+ if (elementMatches(element, elementTransformer, env)) {
+ resultList.add(element);
+ }
+ }
+ return new TemplateModelListSequence(resultList);
+ } else {
+ return new SingleIterationCollectionModel(
+ new TemplateModelIterator() {
+ boolean prefetchDone;
+ TemplateModel prefetchedElement;
+ boolean prefetchedEndOfIterator;
+
+ public TemplateModel next() throws TemplateModelException {
+ ensurePrefetchDone();
+ if (prefetchedEndOfIterator) {
+ throw new IllegalStateException("next() was called when hasNext() is false");
+ }
+ prefetchDone = false;
+ return prefetchedElement;
+ }
+
+ public boolean hasNext() throws TemplateModelException {
+ ensurePrefetchDone();
+ return !prefetchedEndOfIterator;
+ }
+
+ private void ensurePrefetchDone() throws TemplateModelException {
+ if (prefetchDone) {
+ return;
+ }
+
+ boolean conclusionReached = false;
+ do {
+ if (lhoIterator.hasNext()) {
+ TemplateModel element = lhoIterator.next();
+ boolean elementMatched;
+ try {
+ elementMatched = elementMatches(element, elementTransformer, env);
+ } catch (TemplateException e) {
+ throw new _TemplateModelException(e, env, "Failed to transform element");
+ }
+ if (elementMatched) {
+ prefetchedElement = element;
+ conclusionReached = true;
+ }
+ } else {
+ prefetchedEndOfIterator = true;
+ prefetchedElement = null;
+ conclusionReached = true;
+ }
+ } while (!conclusionReached);
+ prefetchDone = true;
+ }
+ }
+ );
+ }
+ }
+
+ private boolean elementMatches(TemplateModel element, ElementTransformer elementTransformer, Environment env) throws
+ TemplateException {
+ TemplateModel transformedElement = elementTransformer.transformElement(element, env);
+ if (!(transformedElement instanceof TemplateBooleanModel)) {
+ if (transformedElement == null) {
+ throw new _TemplateModelException(getElementTransformerExp(), env,
+ "The element transformer function has returned no return value (has returned null) " +
+ "instead of a boolean.");
+ }
+ throw new _TemplateModelException(getElementTransformerExp(), env,
+ "The element transformer function had to return a boolean value, but it has returned ",
+ new _DelayedAOrAn(new _DelayedFTLTypeDescription(transformedElement)),
+ " instead.");
+ }
+ return ((TemplateBooleanModel) transformedElement).getAsBoolean();
+ }
+
+ }
+
+ static class mapBI extends IntermediateStreamOperationLikeBuiltIn {
+
+ protected TemplateModel calculateResult(
+ final TemplateModelIterator lhoIterator, TemplateModel lho, final ElementTransformer elementTransformer,
+ final Environment env) throws TemplateException {
+ if (!isLazyProcessingAllowed()) {
+ List<TemplateModel> resultList = new ArrayList<TemplateModel>();
+ while (lhoIterator.hasNext()) {
+ resultList.add(fetchAndTransformNextElement(lhoIterator, elementTransformer, env));
+ }
+ return new TemplateModelListSequence(resultList);
+ } else {
+ return new SingleIterationCollectionModel(
+ new TemplateModelIterator() {
+
+ public TemplateModel next() throws TemplateModelException {
+ try {
+ return fetchAndTransformNextElement(lhoIterator, elementTransformer, env);
+ } catch (TemplateException e) {
+ throw new _TemplateModelException(e, env, "Failed to transform element");
+ }
+ }
+
+ public boolean hasNext() throws TemplateModelException {
+ return lhoIterator.hasNext();
+ }
+ });
+ }
+ }
+
+ private TemplateModel fetchAndTransformNextElement(
+ TemplateModelIterator lhoIterator, ElementTransformer elementTransformer, Environment env)
+ throws TemplateException {
+ TemplateModel transformedElement = elementTransformer.transformElement(lhoIterator.next(), env);
+ if (transformedElement == null) {
+ throw new _TemplateModelException(getElementTransformerExp(), env,
+ "The element transformer function has returned no return value (has returned null).");
+ }
+ return transformedElement;
+ }
+
+ }
+
// Can't be instantiated
private BuiltInsForSequences() { }
-
+
}
\ No newline at end of file
diff --git a/src/main/java/freemarker/core/Environment.java b/src/main/java/freemarker/core/Environment.java
index 376b838..3498a03 100644
--- a/src/main/java/freemarker/core/Environment.java
+++ b/src/main/java/freemarker/core/Environment.java
@@ -31,6 +31,7 @@ import java.text.DecimalFormatSymbols;
import java.text.NumberFormat;
import java.util.ArrayList;
import java.util.Collection;
+import java.util.Collections;
import java.util.Date;
import java.util.HashMap;
import java.util.IdentityHashMap;
@@ -649,6 +650,43 @@ public final class Environment extends Configurable {
}
/**
+ * Evaluate expression with shadowing a single variable with a new local variable.
+ *
+ * @since 2.3.29
+ */
+ TemplateModel evaluateWithNewLocal(Expression exp, String lambdaArgName, TemplateModel lamdaArgValue)
+ throws TemplateException {
+ pushLocalContext(new LocalContextWithNewLocal(lambdaArgName, lamdaArgValue));
+ try {
+ return exp.eval(this);
+ } finally {
+ localContextStack.pop();
+ }
+ }
+
+ /**
+ * Specialization for 1 local variables.
+ */
+ private static class LocalContextWithNewLocal implements LocalContext {
+ private final String lambdaArgName;
+ private final TemplateModel lambdaArgValue;
+
+ public LocalContextWithNewLocal(String lambdaArgName, TemplateModel lambdaArgValue) {
+ this.lambdaArgName = lambdaArgName;
+ this.lambdaArgValue = lambdaArgValue;
+ }
+
+ public TemplateModel getLocalVariable(String name) throws TemplateModelException {
+ // TODO [lambda] Do not allow fallback (i.e., introduce untransparent null-s)
+ return name.equals(lambdaArgName) ? lambdaArgValue : null;
+ }
+
+ public Collection getLocalVariableNames() throws TemplateModelException {
+ return Collections.singleton(lambdaArgName);
+ }
+ }
+
+ /**
* Used for {@code #visit} and {@code #recurse}.
*/
void invokeNodeHandlerFor(TemplateNodeModel node, TemplateSequenceModel namespaces)
diff --git a/src/main/java/freemarker/core/ExpressionWithFixedResult.java b/src/main/java/freemarker/core/ExpressionWithFixedResult.java
new file mode 100644
index 0000000..37aee8f
--- /dev/null
+++ b/src/main/java/freemarker/core/ExpressionWithFixedResult.java
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package freemarker.core;
+
+import freemarker.template.TemplateException;
+import freemarker.template.TemplateModel;
+
+/**
+ * Mimics an expression (the "source expression"), but returns the predefined "fixed result" whenever it's evaluated.
+ */
+class ExpressionWithFixedResult extends Expression {
+ private final TemplateModel fixedResult;
+ private final Expression sourceExpression;
+
+ ExpressionWithFixedResult(TemplateModel fixedResult, Expression sourceExpression) {
+ this.fixedResult = fixedResult;
+ this.sourceExpression = sourceExpression;
+ }
+
+ TemplateModel _eval(Environment env) throws TemplateException {
+ return fixedResult;
+ }
+
+ boolean isLiteral() {
+ return sourceExpression.isLiteral();
+ }
+
+ protected Expression deepCloneWithIdentifierReplaced_inner(String replacedIdentifier, Expression replacement,
+ ReplacemenetState replacementState) {
+ return new ExpressionWithFixedResult(
+ fixedResult,
+ sourceExpression.deepCloneWithIdentifierReplaced(replacedIdentifier, replacement, replacementState));
+ }
+
+ public String getCanonicalForm() {
+ return sourceExpression.getCanonicalForm();
+ }
+
+ String getNodeTypeSymbol() {
+ return sourceExpression.getNodeTypeSymbol();
+ }
+
+ int getParameterCount() {
+ return sourceExpression.getParameterCount();
+ }
+
+ Object getParameterValue(int idx) {
+ return sourceExpression.getParameterValue(idx);
+ }
+
+ ParameterRole getParameterRole(int idx) {
+ return sourceExpression.getParameterRole(idx);
+ }
+
+
+}
diff --git a/src/main/java/freemarker/core/IteratorBlock.java b/src/main/java/freemarker/core/IteratorBlock.java
index bbb6773..e8ed614 100644
--- a/src/main/java/freemarker/core/IteratorBlock.java
+++ b/src/main/java/freemarker/core/IteratorBlock.java
@@ -49,6 +49,7 @@ final class IteratorBlock extends TemplateElement {
private final String loopVar2Name;
private final boolean hashListing;
private final boolean forEach;
+ private final boolean fetchElementsOutsideLoopVarContext;
/**
* @param listedExp
@@ -82,6 +83,13 @@ final class IteratorBlock extends TemplateElement {
setChildren(childrenBeforeElse);
this.hashListing = hashListing;
this.forEach = forEach;
+
+ if (listedExp instanceof BuiltInsForSequences.IntermediateStreamOperationLikeBuiltIn) {
+ ((BuiltInsForSequences.IntermediateStreamOperationLikeBuiltIn) listedExp).setLazyProcessingAllowed(true);
+ fetchElementsOutsideLoopVarContext = true;
+ } else {
+ fetchElementsOutsideLoopVarContext = false;
+ }
}
boolean isHashListing() {
@@ -243,8 +251,7 @@ final class IteratorBlock extends TemplateElement {
}
void loopForItemsElement(Environment env, TemplateElement[] childBuffer, String loopVarName, String loopVar2Name)
- throws NonSequenceOrCollectionException, TemplateModelException, InvalidReferenceException,
- TemplateException, IOException {
+ throws TemplateException, IOException {
try {
if (alreadyEntered) {
throw new _MiscTemplateException(env,
@@ -265,16 +272,14 @@ final class IteratorBlock extends TemplateElement {
* each list item once, otherwise once if {@link #listedValue} isn't empty.
*/
private boolean executeNestedContent(Environment env, TemplateElement[] childBuffer)
- throws TemplateModelException, TemplateException, IOException, NonSequenceOrCollectionException,
- InvalidReferenceException {
+ throws TemplateException, IOException {
return !hashListing
? executedNestedContentForCollOrSeqListing(env, childBuffer)
: executedNestedContentForHashListing(env, childBuffer);
}
private boolean executedNestedContentForCollOrSeqListing(Environment env, TemplateElement[] childBuffer)
- throws TemplateModelException, IOException, TemplateException,
- NonSequenceOrCollectionException, InvalidReferenceException {
+ throws IOException, TemplateException {
final boolean listNotEmpty;
if (listedValue instanceof TemplateCollectionModel) {
final TemplateCollectionModel collModel = (TemplateCollectionModel) listedValue;
@@ -284,22 +289,22 @@ final class IteratorBlock extends TemplateElement {
listNotEmpty = iterModel.hasNext();
if (listNotEmpty) {
if (loopVarName != null) {
- listLoop: do {
- loopVar = iterModel.next();
- hasNext = iterModel.hasNext();
- try {
- env.visit(childBuffer);
- } catch (BreakOrContinueException br) {
- if (br == BreakOrContinueException.BREAK_INSTANCE) {
- break listLoop;
- }
+ listLoop: do {
+ loopVar = iterModel.next();
+ hasNext = iterModel.hasNext();
+ try {
+ env.visit(childBuffer);
+ } catch (BreakOrContinueException br) {
+ if (br == BreakOrContinueException.BREAK_INSTANCE) {
+ break listLoop;
}
- index++;
- } while (hasNext);
+ }
+ index++;
+ } while (hasNext);
openedIterator = null;
} else {
// We must reuse this later, because TemplateCollectionModel-s that wrap an Iterator only
- // allow one iterator() call.
+ // allow one iterator() call. (Also those returned by ?filter, etc. with lazy processing on.)
openedIterator = iterModel;
env.visit(childBuffer);
}
diff --git a/src/main/java/freemarker/core/LocalLambdaExpression.java b/src/main/java/freemarker/core/LocalLambdaExpression.java
new file mode 100644
index 0000000..537efb1
--- /dev/null
+++ b/src/main/java/freemarker/core/LocalLambdaExpression.java
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package freemarker.core;
+
+import java.util.List;
+
+import freemarker.template.TemplateException;
+import freemarker.template.TemplateModel;
+
+/**
+ * A local lambada expression is a lambda expression that creates a function that can only be called from the same
+ * context where it was created, and thus it doesn't need closure support. As of this writing (2019-02), this is the
+ * only kind of lambda expression supported, as supporting closures would add overhead to many basic operations, while
+ * local lambdas are "good enough" for the main use cases in templates (for filtering/transforming lists). Also,
+ * closures can be quite confusing when the lambda expression refers to variables that are not effectively final,
+ * such as a loop variable. So that's yet another issue to address if we go for less restricted lambdas.
+ */
+final class LocalLambdaExpression extends Expression {
+
+ private final LambdaParameterList lho;
+ private final Expression rho;
+
+ LocalLambdaExpression(LambdaParameterList lho, Expression rho) {
+ this.lho = lho;
+ this.rho = rho;
+ }
+
+ @Override
+ public String getCanonicalForm() {
+ return lho.getCanonicalForm() + " -> " + rho.getCanonicalForm();
+ }
+
+ @Override
+ String getNodeTypeSymbol() {
+ return "->";
+ }
+
+ TemplateModel _eval(Environment env) throws TemplateException {
+ throw new TemplateException("Can't get lambda expression as a value: Lambdas currently can only be used on a " +
+ "few special places.",
+ env);
+ }
+
+ /**
+ * Call the function defined by the lambda expression; overload specialized for 1 argument, the most common case.
+ */
+ TemplateModel invokeLambdaDefinedFunction(TemplateModel argValue, Environment env) throws TemplateException {
+ return env.evaluateWithNewLocal(rho, lho.getParameters().get(0).getName(), argValue);
+ }
+
+ @Override
+ boolean isLiteral() {
+ // As we don't support true lambdas, they can't be evaluted in parse time.
+ return false;
+ }
+
+ @Override
+ protected Expression deepCloneWithIdentifierReplaced_inner(
+ String replacedIdentifier, Expression replacement, ReplacemenetState replacementState) {
+ // TODO [lambda] replacement in lho is illegal; detect it
+ return new LocalLambdaExpression(
+ lho,
+ rho.deepCloneWithIdentifierReplaced(replacedIdentifier, replacement, replacementState));
+ }
+
+ @Override
+ int getParameterCount() {
+ return 2;
+ }
+
+ @Override
+ Object getParameterValue(int idx) {
+ // TODO [lambda] should be similar to #function
+ switch (idx) {
+ case 0: return lho;
+ case 1: return rho;
+ default: throw new IndexOutOfBoundsException();
+ }
+ }
+
+ @Override
+ ParameterRole getParameterRole(int idx) {
+ return ParameterRole.forBinaryOperatorOperand(idx);
+ }
+
+ LambdaParameterList getLambdaParameterList() {
+ return lho;
+ }
+
+ /** The left side of the `->`. */
+ static class LambdaParameterList {
+ private final Token openingParenthesis;
+ private final Token closingParenthesis;
+ private final List<Identifier> parameters;
+
+ public LambdaParameterList(Token openingParenthesis, List<Identifier> parameters, Token closingParenthesis) {
+ this.openingParenthesis = openingParenthesis;
+ this.closingParenthesis = closingParenthesis;
+ this.parameters = parameters;
+ }
+
+ /** Maybe {@code null} */
+ public Token getOpeningParenthesis() {
+ return openingParenthesis;
+ }
+
+ /** Maybe {@code null} */
+ public Token getClosingParenthesis() {
+ return closingParenthesis;
+ }
+
+ public List<Identifier> getParameters() {
+ return parameters;
+ }
+
+ public String getCanonicalForm() {
+ if (parameters.size() == 1) {
+ return parameters.get(0).getCanonicalForm();
+ } else {
+ StringBuilder sb = new StringBuilder();
+ sb.append('(');
+ for (int i = 0; i < parameters.size(); i++) {
+ if (i != 0) {
+ sb.append(", ");
+ }
+ Identifier parameter = parameters.get(i);
+ sb.append(parameter.getCanonicalForm());
+ }
+ sb.append(')');
+ return sb.toString();
+ }
+ }
+ }
+
+}
diff --git a/src/main/java/freemarker/core/MethodCall.java b/src/main/java/freemarker/core/MethodCall.java
index afe3c83..74e20a4 100644
--- a/src/main/java/freemarker/core/MethodCall.java
+++ b/src/main/java/freemarker/core/MethodCall.java
@@ -64,7 +64,7 @@ final class MethodCall extends Expression {
} else if (targetModel instanceof Macro) {
return env.invokeFunction(env, (Macro) targetModel, arguments.items, this);
} else {
- throw new NonMethodException(target, targetModel, true, null, env);
+ throw new NonMethodException(target, targetModel, true, false, null, env);
}
}
diff --git a/src/main/java/freemarker/core/NonMethodException.java b/src/main/java/freemarker/core/NonMethodException.java
index a6eff89..4ba3009 100644
--- a/src/main/java/freemarker/core/NonMethodException.java
+++ b/src/main/java/freemarker/core/NonMethodException.java
@@ -59,7 +59,7 @@ public class NonMethodException extends UnexpectedTypeException {
NonMethodException(
Expression blamed, TemplateModel model, String[] tips, Environment env) throws InvalidReferenceException {
- this(blamed, model, false, tips, env);
+ this(blamed, model, false, false, tips, env);
}
/**
@@ -68,10 +68,11 @@ public class NonMethodException extends UnexpectedTypeException {
* @since 2.3.29
*/
NonMethodException(
- Expression blamed, TemplateModel model, boolean allowFTLFunction, String[] tips, Environment env)
+ Expression blamed, TemplateModel model, boolean allowFTLFunction, boolean allowLambdaExp,
+ String[] tips, Environment env)
throws InvalidReferenceException {
super(blamed, model,
- allowFTLFunction ? "method or function" : "method",
+ "method" + (allowFTLFunction ? " or function" : "") + (allowLambdaExp ? " or lambda expression" : ""),
allowFTLFunction ? EXPECTED_TYPES_WITH_FUNCTION : EXPECTED_TYPES,
tips, env);
}
diff --git a/src/main/java/freemarker/core/SingleIterationCollectionModel.java b/src/main/java/freemarker/core/SingleIterationCollectionModel.java
new file mode 100644
index 0000000..292bcd3
--- /dev/null
+++ b/src/main/java/freemarker/core/SingleIterationCollectionModel.java
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package freemarker.core;
+
+import freemarker.template.TemplateCollectionModel;
+import freemarker.template.TemplateModel;
+import freemarker.template.TemplateModelException;
+import freemarker.template.TemplateModelIterator;
+import freemarker.template.utility.NullArgumentException;
+
+/**
+ * Used where we really want to return/pass a {@link TemplateModelIterator}, but the API requires us to return
+ * a {@link TemplateModel}.
+ *
+ * @since 2.3.29
+ */
+class SingleIterationCollectionModel implements TemplateCollectionModel {
+ private TemplateModelIterator iterator;
+
+ public SingleIterationCollectionModel(TemplateModelIterator iterator) {
+ NullArgumentException.check(iterator);
+ this.iterator = iterator;
+ }
+
+ public TemplateModelIterator iterator() throws TemplateModelException {
+ if (iterator == null) {
+ throw new IllegalStateException(
+ "Can't return the iterator again, as this TemplateCollectionModel can only be iterated once.");
+ }
+ TemplateModelIterator result = iterator;
+ iterator = null;
+ return result;
+ }
+}
diff --git a/src/main/javacc/FTL.jj b/src/main/javacc/FTL.jj
index a2abb4d..a157871 100644
--- a/src/main/javacc/FTL.jj
+++ b/src/main/javacc/FTL.jj
@@ -29,6 +29,7 @@ PARSER_BEGIN(FMParser)
package freemarker.core;
+import freemarker.core.LocalLambdaExpression.LambdaParameterList;
import freemarker.template.*;
import freemarker.template.utility.*;
import java.io.*;
@@ -1278,6 +1279,8 @@ TOKEN:
|
<ESCAPED_GTE : "gte" | "\\gte" | ">=">
|
+ <LAMBDA_ARROW : "->" | "->">
+ |
<PLUS : "+">
|
<MINUS : "-">
@@ -1993,9 +1996,6 @@ Expression RangeExpression() :
}
}
-
-
-
Expression AndExpression() :
{
Expression lhs, rhs, result;
@@ -2122,8 +2122,10 @@ BuiltinVariable BuiltinVariable() :
parseTimeValue = new SimpleScalar(outputFormat.getName());
} else if (nameStr.equals(BuiltinVariable.AUTO_ESC) || nameStr.equals(BuiltinVariable.AUTO_ESC_CC)) {
parseTimeValue = autoEscaping ? TemplateBooleanModel.TRUE : TemplateBooleanModel.FALSE;
- } else {
parseTimeValue = null;
- }
+ } else {
+ parseTimeValue = null;
+ }
+
result = new BuiltinVariable(name, token_source, parseTimeValue);
result.setLocation(template, dot, name);
@@ -2235,13 +2237,31 @@ Expression BuiltIn(Expression lhoExp) :
}
}
[
- LOOKAHEAD({ result instanceof BuiltInWithParseTimeParameters })
+ LOOKAHEAD({
+ result instanceof BuiltInWithParseTimeParameters
+ && !((BuiltInWithParseTimeParameters) result).isLocalLambdaParameterSupported() })
openParen = <OPEN_PAREN>
args = PositionalArgs()
closeParen = <CLOSE_PAREN> {
result.setLocation(template, lhoExp, closeParen);
((BuiltInWithParseTimeParameters) result).bindToParameters(args, openParen, closeParen);
-
+
+ return result;
+ }
+ ]
+ // In principle we should embed the BuiltInWithParseTimeParameters LOOKAHEAD into the
+ // isLocalLambdaParameterSupported LOOKAHEAD, but then the `result` variable was out of scope in the code
+ // generated for the nested LOOKAHEAD. So we had to flatten this by checking isLocalLambdaParameterSupported twice.
+ [
+ LOOKAHEAD({
+ result instanceof BuiltInWithParseTimeParameters
+ && ((BuiltInWithParseTimeParameters) result).isLocalLambdaParameterSupported() })
+ openParen = <OPEN_PAREN>
+ args = PositionalMaybeLambdaArgs()
+ closeParen = <CLOSE_PAREN> {
+ result.setLocation(template, lhoExp, closeParen);
+ ((BuiltInWithParseTimeParameters) result).bindToParameters(args, openParen, closeParen);
+
return result;
}
]
@@ -2251,6 +2271,78 @@ Expression BuiltIn(Expression lhoExp) :
}
}
+// Only supported as the argument of certain built-ins, so it's not called inside Expression.
+Expression LocalLambdaExpression() :
+{
+ LambdaParameterList lhs;
+ Expression rhs, result;
+}
+{
+ (
+ LOOKAHEAD(LambdaExpressionParameterList() <LAMBDA_ARROW>)
+ (
+ lhs = LambdaExpressionParameterList()
+ <LAMBDA_ARROW>
+ rhs = OrExpression()
+ {
+ result = new LocalLambdaExpression(lhs, rhs);
+ if (lhs.getOpeningParenthesis() != null) {
+ // (args) -> exp
+ result.setLocation(template, lhs.getOpeningParenthesis(), rhs);
+ } else {
+ // singleArg -> exp
+ result.setLocation(template, lhs.getParameters().get(0), rhs);
+ }
+}
+ )
+ |
+ result = OrExpression()
+ )
+ {
+ return result;
+ }
+}
+
+LambdaParameterList LambdaExpressionParameterList() :
+{
+ Token openParen = null;
+ Token closeParen = null;
+ List<Identifier> params = null;
+ Identifier param;
+}
+{
+ (
+ (
+ openParen = <OPEN_PAREN>
+ [
+ param = Identifier()
+ {
+ params = new ArrayList<Identifier>(4);
+ params.add(param);
+ }
+ (
+ <COMMA>
+ param = Identifier()
+ {
+ params.add(param);
+ }
+ )*
+ ]
+ closeParen = <CLOSE_PAREN>
+ )
+ |
+ param = Identifier()
+ {
+ params = Collections.<Identifier>singletonList(param);
+ }
+ )
+ {
+ return new LambdaParameterList(
+ openParen,
+ params != null ? params : Collections.<Identifier>emptyList(),
+ closeParen);
+ }
+}
/**
* production for when a key is specified by <DOT> + keyname
@@ -3636,6 +3728,27 @@ ArrayList PositionalArgs() :
}
}
+/**
+ * Like PositionalArgs, but allows lambdas. This is separate as it's slower, while lambdas are only allowed on a few
+ * places.
+ */
+ArrayList PositionalMaybeLambdaArgs() :
+{
+ ArrayList result = new ArrayList();
+ Expression arg;
+}
+{
+ [
+ arg = LocalLambdaExpression() { result.add(arg); }
+ (
+ [<COMMA>]
+ arg = LocalLambdaExpression() { result.add(arg); }
+ )*
+ ]
+ {
+ return result;
+ }
+}
Comment Comment() :
{
diff --git a/src/test/java/freemarker/core/FilterBiTest.java b/src/test/java/freemarker/core/FilterBiTest.java
new file mode 100644
index 0000000..14e37ac
--- /dev/null
+++ b/src/test/java/freemarker/core/FilterBiTest.java
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package freemarker.core;
+
+import java.util.List;
+
+import org.junit.Test;
+
+import com.google.common.collect.ImmutableList;
+
+import freemarker.template.TemplateException;
+import freemarker.test.TemplateTest;
+
+public class FilterBiTest extends TemplateTest {
+
+ private static class TestParam {
+ private final List<?> list;
+ private final String result;
+
+ public TestParam(List<?> list, String result) {
+ this.list = list;
+ this.result = result;
+ }
+ }
+
+ private static final List<TestParam> TEST_PARAMS = ImmutableList.of(
+ new TestParam(ImmutableList.of("a", "aX", "bX", "b", "cX", "c"), "a, b, c"),
+ new TestParam(ImmutableList.of("a", "b", "c"), "a, b, c"),
+ new TestParam(ImmutableList.of("aX", "bX", "a", "b", "c", "cX", "cX"), "a, b, c"),
+ new TestParam(ImmutableList.of("aX", "bX", "cX"), ""),
+ new TestParam(ImmutableList.of(), "")
+ );
+
+ @Test
+ public void testFilterWithLambda() throws Exception {
+ for (TestParam testParam : TEST_PARAMS) {
+ addToDataModel("xs", testParam.list);
+ assertOutput(
+ "<#list xs?filter(it -> !it?contains('X')) as x>${x}<#sep>, </#list>",
+ testParam.result);
+ assertOutput(
+ "<#assign fxs = xs?filter(it -> !it?contains('X'))>" +
+ "${fxs?join(', ')}",
+ testParam.result);
+ }
+ }
+
+ @Test
+ public void testFilterWithFunction() throws Exception {
+ for (TestParam testParam : TEST_PARAMS) {
+ addToDataModel("xs", testParam.list);
+ String functionDef = "<#function noX s><#return !s?contains('X')></#function>";
+ assertOutput(
+ functionDef +
+ "<#list xs?filter(noX) as x>${x}<#sep>, </#list>",
+ testParam.result);
+ assertOutput(
+ functionDef +
+ "<#assign fxs = xs?filter(noX)>" +
+ "${fxs?join(', ')}",
+ testParam.result);
+ }
+ }
+
+ @Test
+ public void testFilterWithMethod() throws Exception {
+ for (TestParam testParam : TEST_PARAMS) {
+ addToDataModel("xs", testParam.list);
+ addToDataModel("obj", new FilterObject());
+ assertOutput(
+ "<#list xs?filter(obj.noX) as x>${x}<#sep>, </#list>",
+ testParam.result);
+ assertOutput(
+ "<#assign fxs = xs?filter(obj.noX)>" +
+ "${fxs?join(', ')}",
+ testParam.result);
+ }
+ }
+
+ @Test
+ public void testWithNumberElements() throws Exception {
+ addToDataModel("xs", ImmutableList.of(1, 1.5, 2, 2.3, 3));
+ addToDataModel("obj", new FilterObject());
+ assertOutput(
+ "<#list xs?filter(n -> n == n?int) as x>${x}<#sep>, </#list>",
+ "1, 2, 3");
+ assertOutput(
+ "<#function isInteger n><#return n == n?int></#function>" +
+ "<#list xs?filter(isInteger) as x>${x}<#sep>, </#list>",
+ "1, 2, 3");
+ assertOutput(
+ "<#list xs?filter(obj.isInteger) as x>${x}<#sep>, </#list>",
+ "1, 2, 3");
+ }
+
+ @Test
+ public void testErrorMessages() {
+ assertErrorContains("${1?filter(it -> true)}", TemplateException.class,
+ "sequence or collection", "number");
+ assertErrorContains("${[]?filter(1)}", TemplateException.class,
+ "method or function or lambda", "number");
+ assertErrorContains("${['x']?filter(it -> 1)}", TemplateException.class,
+ "boolean", "number");
+ assertErrorContains("<#function f></#function>${['x']?filter(f)}", TemplateException.class,
+ "Function", "0 parameters", "1");
+ assertErrorContains("<#function f x y z></#function>${['x']?filter(f)}", TemplateException.class,
+ "function", "parameter \"y\"");
+ assertErrorContains("<#function f x></#function>${['x']?filter(f)}", TemplateException.class,
+ "boolean", "null");
+ assertErrorContains("${[]?filter(() -> true)}", ParseException.class,
+ "lambda", "1 parameter", "declared 0");
+ assertErrorContains("${[]?filter((i, j) -> true)}", ParseException.class,
+ "lambda", "1 parameter", "declared 2");
+ }
+
+ public static class FilterObject {
+ public boolean noX(String s) {
+ return !s.contains("X");
+ }
+ public boolean isInteger(double n) {
+ return n == (int) n;
+ }
+ }
+
+}
diff --git a/src/test/java/freemarker/core/MapBiTest.java b/src/test/java/freemarker/core/MapBiTest.java
new file mode 100644
index 0000000..56bca58
--- /dev/null
+++ b/src/test/java/freemarker/core/MapBiTest.java
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package freemarker.core;
+
+import java.math.BigDecimal;
+import java.util.List;
+
+import org.junit.Test;
+
+import com.google.common.collect.ImmutableList;
+
+import freemarker.template.Configuration;
+import freemarker.template.TemplateException;
+import freemarker.test.TemplateTest;
+
+public class MapBiTest extends TemplateTest {
+
+ private static class TestParam {
+ private final List<?> list;
+ private final String result;
+
+ public TestParam(List<?> list, String result) {
+ this.list = list;
+ this.result = result;
+ }
+ }
+
+ private static final List<TestParam> TEST_PARAMS = ImmutableList.of(
+ new TestParam(ImmutableList.of("a", "b", "c"), "A, B, C"),
+ new TestParam(ImmutableList.of("a"), "A"),
+ new TestParam(ImmutableList.of(), "")
+ );
+
+ @Override
+ protected Configuration createConfiguration() throws Exception {
+ Configuration cfg = super.createConfiguration();
+ cfg.setNumberFormat("0.####");
+ cfg.setBooleanFormat("c");
+ return cfg;
+ }
+
+ @Test
+ public void testFilterWithLambda() throws Exception {
+ for (TestParam testParam : TEST_PARAMS) {
+ addToDataModel("xs", testParam.list);
+ // Lazy:
+ assertOutput(
+ "<#list xs?map(it -> it?upperCase) as x>${x}<#sep>, </#list>",
+ testParam.result);
+ // Eager:
+ assertOutput(
+ "<#assign fxs = xs?map(it -> it?upperCase)>" +
+ "${fxs?join(', ')}",
+ testParam.result);
+ }
+ }
+
+ @Test
+ public void testFilterWithFunction() throws Exception {
+ for (TestParam testParam : TEST_PARAMS) {
+ addToDataModel("xs", testParam.list);
+ String functionDef = "<#function toUpper s><#return s?upperCase></#function>";
+ // Lazy:
+ assertOutput(
+ functionDef +
+ "<#list xs?map(toUpper) as x>${x}<#sep>, </#list>",
+ testParam.result);
+ // Eager:
+ assertOutput(
+ functionDef +
+ "<#assign fxs = xs?map(toUpper)>" +
+ "${fxs?join(', ')}",
+ testParam.result);
+ }
+ }
+
+ @Test
+ public void testFilterWithMethod() throws Exception {
+ for (TestParam testParam : TEST_PARAMS) {
+ addToDataModel("xs", testParam.list);
+ addToDataModel("obj", new MapperObject());
+ // Lazy:
+ assertOutput(
+ "<#list xs?map(obj.toUpper) as x>${x}<#sep>, </#list>",
+ testParam.result);
+ // Eager:
+ assertOutput(
+ "<#assign fxs = xs?map(obj.toUpper)>" +
+ "${fxs?join(', ')}",
+ testParam.result);
+ }
+ }
+
+ @Test
+ public void testWithNumberElements() throws Exception {
+ addToDataModel("xs", ImmutableList.of(1, 1.55, 3));
+ addToDataModel("obj", new MapperObject());
+ assertOutput(
+ "<#list xs?map(n -> n * 10) as x>${x}<#sep>, </#list>",
+ "10, 15.5, 30");
+ assertOutput(
+ "<#function tenTimes n><#return n * 10></#function>" +
+ "<#list xs?map(tenTimes) as x>${x}<#sep>, </#list>",
+ "10, 15.5, 30");
+ assertOutput(
+ "<#list xs?map(obj.tenTimes) as x>${x}<#sep>, </#list>",
+ "10, 15.5, 30");
+ }
+
+ @Test
+ public void testWithBeanElements() throws Exception {
+ addToDataModel("xs", ImmutableList.of(new User("a"), new User("b"), new User("c")));
+ addToDataModel("obj", new MapperObject());
+ assertOutput(
+ "<#list xs?map(user -> user.name) as x>${x}<#sep>, </#list>",
+ "a, b, c");
+ assertOutput(
+ "<#function extractName user><#return user.name></#function>" +
+ "<#list xs?map(extractName) as x>${x}<#sep>, </#list>",
+ "a, b, c");
+ assertOutput(
+ "<#list xs?map(obj.extractName) as x>${x}<#sep>, </#list>",
+ "a, b, c");
+ }
+
+ @Test
+ public void testBuiltInsThatAllowLazyEval() throws Exception {
+ assertOutput("" +
+ "<#assign s = ''>" +
+ "<#function tenTimes(x)><#assign s += '${x};'><#return x * 10></#function>" +
+ "${(1..3)?map(tenTimes)?first} ${s}", "10 1;");
+
+ assertOutput("" +
+ "<#assign s = ''>" +
+ "<#function tenTimes(x)><#assign s += '${x};'><#return x * 10></#function>" +
+ "${(1..3)?map(tenTimes)?seqContains(20)} ${s}", "true 1;2;");
+
+ assertOutput("" +
+ "<#assign s = ''>" +
+ "<#function tenTimes(x)><#assign s += '${x};'><#return x * 10></#function>" +
+ "${(1..3)?map(tenTimes)?seqIndexOf(20)} ${s}", "1 1;2;");
+
+ // For these this test can't check that there was no sequence built, but at least we know they are working:
+ assertOutput("${(1..3)?map(it -> it * 10)?min}", "10");
+ assertOutput("${(1..3)?map(it -> it * 10)?max}", "30");
+ assertOutput("${(1..3)?map(it -> it * 10)?join(', ')}", "10, 20, 30");
+ }
+
+ @Test
+ public void testLaziness() throws Exception {
+ // #list enables lazy evaluation:
+ assertOutput(
+ "" +
+ "<#assign s = ''>" +
+ "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
+ "<#list (1..3)?map(tenTimes) as x>" +
+ "<#assign s += x>" +
+ "<#sep><#assign s += ', '>" +
+ "</#list>" +
+ "${s}",
+ "1->10, 2->20, 3->30");
+ // Most other context causes eager behavior:
+ assertOutput(
+ "" +
+ "<#assign s = ''>" +
+ "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
+ "<#assign xs = (1..3)?map(tenTimes)>" +
+ "<#list xs as x>" +
+ "<#assign s += x>" +
+ "<#sep><#assign s += ', '>" +
+ "</#list>" +
+ "${s}",
+ "1->2->3->10, 20, 30");
+
+ // ?map-s can be chained and all is "streaming":
+ assertOutput(
+ "" +
+ "<#assign s = ''>" +
+ "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
+ "<#list (1..3)?map(tenTimes)?map(tenTimes)?map(tenTimes) as x>" +
+ "<#assign s += x>" +
+ "<#sep><#assign s += ', '>" +
+ "</#list>" +
+ "${s}",
+ "1->10->100->1000, 2->20->200->2000, 3->30->300->3000");
+
+ // Rest of the elements not consumed after #break:
+ assertOutput(
+ "" +
+ "<#assign s = ''>" +
+ "<#function tenTimes(x)><#assign s += '${x}->'><#return x * 10></#function>" +
+ "<#list (1..3)?map(tenTimes) as x>" +
+ "<#assign s += x>" +
+ "<#sep><#assign s += ', '>" +
+ "<#if x == 20><#break></#if>" +
+ "</#list>" +
+ "${s}",
+ "1->10, 2->20, ");
+ }
+
+ @Test
+ public void testErrorMessages() {
+ assertErrorContains("${1?map(it -> it)}", TemplateException.class,
+ "sequence or collection", "number");
+ assertErrorContains("${[]?map(1)}", TemplateException.class,
+ "method or function or lambda", "number");
+ assertErrorContains("<#function f></#function>${['x']?map(f)}", TemplateException.class,
+ "Function", "0 parameters", "1");
+ assertErrorContains("<#function f x y z></#function>${['x']?map(f)}", TemplateException.class,
+ "function", "parameter \"y\"");
+ assertErrorContains("<#function f x></#function>${['x']?map(f)}", TemplateException.class,
+ "null");
+ assertErrorContains("${[]?map(() -> 1)}", ParseException.class,
+ "lambda", "1 parameter", "declared 0");
+ assertErrorContains("${[]?map((i, j) -> 1)}", ParseException.class,
+ "lambda", "1 parameter", "declared 2");
+ }
+
+ public static class MapperObject {
+ public String toUpper(String s) {
+ return s.toUpperCase();
+ }
+ public BigDecimal tenTimes(BigDecimal n) {
+ return n.movePointRight(1);
+ }
+ public String extractName(User user) { return user.getName(); }
+ }
+
+ public static class User {
+ private final String name;
+
+ public User(String name) {
+ this.name = name;
+ }
+
+ public String getName() {
+ return name;
+ }
+ }
+
+}