You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2018/09/18 15:38:59 UTC
spark git commit: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to
use JDK8
Repository: spark
Updated Branches:
refs/heads/master 1c0423b28 -> 182da81e9
[SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JDK8
## What changes were proposed in this pull request?
Update `tuning.md` and `rdd-programming-guide.md` to use JDK8.
## How was this patch tested?
manual tests
Closes #22446 from wangyum/java8.
Authored-by: Yuming Wang <yu...@ebay.com>
Signed-off-by: Sean Owen <se...@databricks.com>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/182da81e
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/182da81e
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/182da81e
Branch: refs/heads/master
Commit: 182da81e9e75ac1658a39014beb90e60495bf544
Parents: 1c0423b
Author: Yuming Wang <yu...@ebay.com>
Authored: Tue Sep 18 10:38:55 2018 -0500
Committer: Sean Owen <se...@databricks.com>
Committed: Tue Sep 18 10:38:55 2018 -0500
----------------------------------------------------------------------
docs/rdd-programming-guide.md | 4 ++--
docs/tuning.md | 8 ++++----
2 files changed, 6 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/182da81e/docs/rdd-programming-guide.md
----------------------------------------------------------------------
diff --git a/docs/rdd-programming-guide.md b/docs/rdd-programming-guide.md
index d95b757..0054257 100644
--- a/docs/rdd-programming-guide.md
+++ b/docs/rdd-programming-guide.md
@@ -859,7 +859,7 @@ We could also use `counts.sortByKey()`, for example, to sort the pairs alphabeti
**Note:** when using custom objects as the key in key-value pair operations, you must be sure that a
custom `equals()` method is accompanied with a matching `hashCode()` method. For full details, see
the contract outlined in the [Object.hashCode()
-documentation](http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#hashCode()).
+documentation](https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#hashCode--).
</div>
@@ -896,7 +896,7 @@ We could also use `counts.sortByKey()`, for example, to sort the pairs alphabeti
**Note:** when using custom objects as the key in key-value pair operations, you must be sure that a
custom `equals()` method is accompanied with a matching `hashCode()` method. For full details, see
the contract outlined in the [Object.hashCode()
-documentation](http://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#hashCode()).
+documentation](https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#hashCode--).
</div>
http://git-wip-us.apache.org/repos/asf/spark/blob/182da81e/docs/tuning.md
----------------------------------------------------------------------
diff --git a/docs/tuning.md b/docs/tuning.md
index f60971a..cd0f9cd 100644
--- a/docs/tuning.md
+++ b/docs/tuning.md
@@ -26,12 +26,12 @@ Often, this will be the first thing you should tune to optimize a Spark applicat
Spark aims to strike a balance between convenience (allowing you to work with any Java type
in your operations) and performance. It provides two serialization libraries:
-* [Java serialization](http://docs.oracle.com/javase/6/docs/api/java/io/Serializable.html):
+* [Java serialization](https://docs.oracle.com/javase/8/docs/api/java/io/Serializable.html):
By default, Spark serializes objects using Java's `ObjectOutputStream` framework, and can work
with any class you create that implements
- [`java.io.Serializable`](http://docs.oracle.com/javase/6/docs/api/java/io/Serializable.html).
+ [`java.io.Serializable`](https://docs.oracle.com/javase/8/docs/api/java/io/Serializable.html).
You can also control the performance of your serialization more closely by extending
- [`java.io.Externalizable`](http://docs.oracle.com/javase/6/docs/api/java/io/Externalizable.html).
+ [`java.io.Externalizable`](https://docs.oracle.com/javase/8/docs/api/java/io/Externalizable.html).
Java serialization is flexible but often quite slow, and leads to large
serialized formats for many classes.
* [Kryo serialization](https://github.com/EsotericSoftware/kryo): Spark can also use
@@ -230,7 +230,7 @@ temporary objects created during task execution. Some steps which may be useful
* Monitor how the frequency and time taken by garbage collection changes with the new settings.
Our experience suggests that the effect of GC tuning depends on your application and the amount of memory available.
-There are [many more tuning options](http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html) described online,
+There are [many more tuning options](https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/index.html) described online,
but at a high level, managing how frequently full GC takes place can help in reducing the overhead.
GC tuning flags for executors can be specified by setting `spark.executor.extraJavaOptions` in
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org