You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/11/07 13:56:49 UTC

[GitHub] [beam] mosche opened a new issue, #24008: Cache datasets in Spark Dataset runner if used multiple times.

mosche opened a new issue, #24008:
URL: https://github.com/apache/beam/issues/24008

   ### What would you like to happen?
   
   This is an outstanding feature compared to the Spark RDD runner.
   If datasets are used multiple times, they should be cached to avoid repeated evaluation.
   
   Caching can be disabled setting the Pipeline option storageLevel to `NONE`.
   
   ### Issue Priority
   
   Priority: 2
   
   ### Issue Component
   
   Component: runner-spark


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] echauchot closed issue #24008: Cache datasets in Spark Dataset runner if used multiple times.

Posted by GitBox <gi...@apache.org>.

echauchot closed issue #24008: Cache datasets in Spark Dataset runner if used multiple times.
URL: https://github.com/apache/beam/issues/24008


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org