You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2015/06/06 01:34:00 UTC
[jira] [Assigned] (SPARK-8135) In SerializableWritable, don't load
defaults when instantiating Configuration
[ https://issues.apache.org/jira/browse/SPARK-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-8135:
-----------------------------------
Assignee: Sandy Ryza (was: Apache Spark)
> In SerializableWritable, don't load defaults when instantiating Configuration
> -----------------------------------------------------------------------------
>
> Key: SPARK-8135
> URL: https://issues.apache.org/jira/browse/SPARK-8135
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
>
> Calling "new Configuration()" is an expensive operation because it loads any Hadoop configuration XMLs from disk.
> In SerializableWritable, we call new Configuration needlessly when instantiating an ObjectWritable. The ObjectWritable only needs the Configuration for its class cache, not for any Hadoop properties that might be in XML files, so it should be ok to call new Configuration with loadDefaults = false.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org