You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Jungtaek Lim (Jira)" <ji...@apache.org> on 2020/06/10 08:01:00 UTC

[jira] [Created] (SPARK-31951) Explicitly close iterator in KVStoreView

Jungtaek Lim created SPARK-31951:
------------------------------------

             Summary: Explicitly close iterator in KVStoreView
                 Key: SPARK-31951
                 URL: https://issues.apache.org/jira/browse/SPARK-31951
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 3.1.0
            Reporter: Jungtaek Lim


While reviewing SPARK-31929 we realized there're places which retrieve KVStoreView and leverage it but don't close the underlying iterator not knowing that it's actually closeable.

The culprit is that KVStoreView implements Iterable and hides the fact that it should be closed. The only way to close the underlying iterator properly is calling closeableIterator to get underlying iterator.

I think this is the way to go; force caller to call closeableIterator and let caller deal with underlying iterator with responsibility to close the iterator.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org