You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sandy Ryza (JIRA)" <ji...@apache.org> on 2015/06/06 01:25:02 UTC
[jira] [Created] (SPARK-8135) In SerializableWritable, don't load
defaults when instantiating Configuration
Sandy Ryza created SPARK-8135:
---------------------------------
Summary: In SerializableWritable, don't load defaults when instantiating Configuration
Key: SPARK-8135
URL: https://issues.apache.org/jira/browse/SPARK-8135
Project: Spark
Issue Type: Improvement
Components: Spark Core
Reporter: Sandy Ryza
Assignee: Sandy Ryza
Calling "new Configuration()" is an expensive operation because it loads any Hadoop configuration XMLs from disk.
In SerializableWritable, we call new Configuration needlessly when instantiating an ObjectWritable. The ObjectWritable only needs the Configuration for its class cache, not for any Hadoop properties that might be in XML files, so it should be ok to call new Configuration with loadDefaults = false.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org