Re: [PR] [SPARK-53901][CONNECT] Handle potential memory leaks in`SparkConnectAddArtifactsHandler` upon flush/error [spark]

["hvanhovell (via GitHub)" <gi...@apache.org>]

hvanhovell commented on code in PR #52610:
URL: https://github.com/apache/spark/pull/52610#discussion_r2429732240


##########
sql/connect/server/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectAddArtifactsHandler.scala:
##########
@@ -259,11 +279,25 @@ class SparkConnectAddArtifactsHandler(val responseObserver: StreamObserver[AddAr
 
     def close(): Unit = {
       if (artifactSummary == null) {
-        checksumOut.close()
+        // Close streams defensively: even if outer stream close fails, try inner streams.
+        // Normally checksumOut.close() cascades to inner streams, but if it throws an
+        // exception, we still attempt to close the inner streams to prevent resource leaks.
+        var closeException: Throwable = null
+        try {
+          checksumOut.close()
+        } catch {
+          case NonFatal(e) =>
+            closeException = e
+            // Try to close inner streams directly as fallback
+            try countingOut.close() catch { case NonFatal(_) => }

Review Comment:
   Use `SparkErrorUtils.closeQuietly(...)`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org