You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Tengfei Huang (Jira)" <ji...@apache.org> on 2023/03/09 07:34:00 UTC

[jira] [Commented] (SPARK-42626) Add Destructive Iterator for SparkResult

    [ https://issues.apache.org/jira/browse/SPARK-42626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698220#comment-17698220 ] 

Tengfei Huang commented on SPARK-42626:
---------------------------------------

I will take a look! Thanks

> Add Destructive Iterator for SparkResult
> ----------------------------------------
>
>                 Key: SPARK-42626
>                 URL: https://issues.apache.org/jira/browse/SPARK-42626
>             Project: Spark
>          Issue Type: New Feature
>          Components: Connect
>    Affects Versions: 3.4.0
>            Reporter: Herman van Hövell
>            Priority: Major
>
> Add a destructive iterator to SparkResult. Instead of keeping everything in memory for the life time of SparkResult object, clean it up as soon as we know we are done with it. We can use this for Dataset.toLocalIterator.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org