You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "WangJianfei (JIRA)" <ji...@apache.org> on 2016/09/15 02:11:20 UTC
[jira] [Comment Edited] (SPARK-17535) Performance Improvement of
Signleton pattern in SparkContext
[ https://issues.apache.org/jira/browse/SPARK-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15492046#comment-15492046 ]
WangJianfei edited comment on SPARK-17535 at 9/15/16 2:10 AM:
--------------------------------------------------------------
First: this logic and code is simple too.
Second:After JDK1.5, The volatile can avoid the situation you say.We can't get into some strange situations where the reference is available but the synchronized init hasn't completed,because the volatile will tell the JVM to not do instruction reordering about the init of sparkContext.
was (Author: codlife):
After JDK1.5, The volatile can avoid the situation you say.We can't get into some strange situations where the reference is available but the synchronized init hasn't completed,because the volatile will tell the JVM to not do instruction reordering about the init of sparkContext.
> Performance Improvement of Signleton pattern in SparkContext
> ------------------------------------------------------------
>
> Key: SPARK-17535
> URL: https://issues.apache.org/jira/browse/SPARK-17535
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.0.0
> Reporter: WangJianfei
> Labels: easyfix, performance
>
> I think the singleton pattern of SparkContext is inefficient if there are many request to get the SparkContext,
> So we can write the singleton pattern as below,The second way if more efficient when there are many request to get the SparkContext.
> {code}
> // the current version
> def getOrCreate(): SparkContext = {
> SPARK_CONTEXT_CONSTRUCTOR_LOCK.synchronized {
> if (activeContext.get() == null) {
> setActiveContext(new SparkContext(), allowMultipleContexts = false)
> }
> activeContext.get()
> }
> }
> // by myself
> def getOrCreate(): SparkContext = {
> if (activeContext.get() == null) {
> SPARK_CONTEXT_CONSTRUCTOR_LOCK.synchronized {
> if(activeContext.get == null) {
> @volatile val sparkContext = new SparkContext()
> setActiveContext(sparkContext, allowMultipleContexts = false)
> }
> }
> activeContext.get()
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org