You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2019/07/04 18:36:42 UTC
[GitHub] [spark] vgankidi commented on issue #24991: [SPARK-28188]
Materialize Dataframe API
vgankidi commented on issue #24991: [SPARK-28188] Materialize Dataframe API
URL: https://github.com/apache/spark/pull/24991#issuecomment-508554739
@rxin Yes, materializing on a query plan without a shuffle just wastes resources. We usually recommend repartitioning the dataframe before invoking materialize to add a shuffle. I think this can be used for your use case of measuring execution time of a query plan. What name would you suggest for this api?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org