[jira] [Created] (FREEMARKER-213) Handling AutoCloseable Iterators in ?filter / ?first when the iterator is not fully consumed

["Christoph Rueger (Jira)" <ji...@apache.org>]

Christoph Rueger created FREEMARKER-213:
-------------------------------------------

             Summary: Handling AutoCloseable Iterators in ?filter / ?first when the iterator is not fully consumed 
                 Key: FREEMARKER-213
                 URL: https://issues.apache.org/jira/browse/FREEMARKER-213
             Project: Apache Freemarker
          Issue Type: Improvement
          Components: engine
    Affects Versions: 2.3.31
            Reporter: Christoph Rueger


HI, 
We have expressions like:

{code:java}
${myCatalog.getCSVFileRowsIterator()?filter(row -> row.get("categoryId") == "123")?first}
{code}


Unter the hood this becomes an IteratorModel().

The problem is that:
- the underlying iterator is working on a resource (file, stream) 
- which needs to be *closed when finished*. 
- but ?filter and ?first can abandon and leave the iterator somewhere in the middle once it has identified a match
- this caused Resource Leaks in some cases.

I debugged down to the following code in _freemarker.core.new TemplateModelIterator() {...}.ensurePrefetchDone())_

{code:java}
private void ensurePrefetchDone() throws TemplateModelException {
                                if (prefetchDone) {
                                    return;
                                }

                                boolean conclusionReached = false;
                                do {
                                    if (lhoIterator.hasNext()) {
                                        TemplateModel element = lhoIterator.next();
                                        boolean elementMatched;
                                        try {
                                            elementMatched = elementMatches(element, elementTransformer, env);
                                        } catch (TemplateException e) {
                                            throw new _TemplateModelException(e, env, "Failed to transform element");
                                        }
                                        if (elementMatched) {
                                            prefetchedElement = element;
                                            conclusionReached = true;
                                        }
                                    } else {
                                        prefetchedEndOfIterator = true;
                                        prefetchedElement = null;
                                        conclusionReached = true;
                                    }
                                } while (!conclusionReached);
                                prefetchDone = true;
                            }
{code}

It looks like at this place we have the "iterator" and also know when we stop iterating, when finding a match.

*Question*

Would it be possible to check if the underlying Iterator implements _java.lang.AutoCloseable_ and if yes, call _.close()_ on it, when a match is found?

I am not sure if that is the right thing to do here.

We have built workarounds so that AutoCloseable objects in the datamodel are closed after rendering is finished.
But it would be better to close it as soon as possible, when the iterator is stopped to be consumed. This would help in scenarios where the template does that multiple times, e.g.:

{code:java}
${myCustomObject.getCSVRowsIterator()?filter(row -> row.get("categoryId") == "123")?first}

${myCustomObject.getCSVRowsIterator()?filter(row -> row.get("categoryId") == "234")?first}
{code}

Ideas would be welcome, where this kind of "closing" should happen.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)