You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "pmenon (via GitHub)" <gi...@apache.org> on 2024/10/09 18:38:07 UTC
[PR] [CORE] Use read-only access to conf in `SparkContext` where appropriate [spark]
pmenon opened a new pull request, #48402:
URL: https://github.com/apache/spark/pull/48402
### What changes were proposed in this pull request?
This PR switches all read-only calls to `SparkContext.getConf` to `SparkContext.conf`. The former method creates a clone of the entire conf, which is unnecessary when the caller only reads the conf. `SparkContext.conf` provides read-only access to the conf.
### Why are the changes needed?
Cloning the entire conf adds unnecessary CPU overhead due to copying, and GC overhead due to cleanup, and both affect tail latencies on certain workloads.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing tests.
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49918][CORE] Use read-only access to conf in `SparkContext` where appropriate [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #48402:
URL: https://github.com/apache/spark/pull/48402#issuecomment-2403716189
The GA jobs actually all passed. Thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-49918][CORE] Use read-only access to conf in `SparkContext` where appropriate [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan closed pull request #48402: [SPARK-49918][CORE] Use read-only access to conf in `SparkContext` where appropriate
URL: https://github.com/apache/spark/pull/48402
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org