You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/11/23 10:47:06 UTC

[GitHub] [beam] mosche opened a new issue, #24330: [Bug]: Spark dataset runner might leak SparkSession if translation fails

mosche opened a new issue, #24330:
URL: https://github.com/apache/beam/issues/24330

   ### What happened?
   
   If pipeline translation fails with an exception, it bubbles all the way up to the caller of run without closing the SparkSession.
   
   Considering that we're potentially already evaluating side-input datasets, translation should be done in the same thread as the evaluation. This way exceptions will result in a failed `PipelineResult` and close the session properly.
   
   
   ### Issue Priority
   
   Priority: 2
   
   ### Issue Component
   
   Component: runner-spark


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] aromanenko-dev closed issue #24330: [Bug]: Spark dataset runner might leak SparkSession if translation fails

Posted by GitBox <gi...@apache.org>.

aromanenko-dev closed issue #24330: [Bug]: Spark dataset runner might leak SparkSession if translation fails
URL: https://github.com/apache/beam/issues/24330


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org