You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Xianyang Liu (JIRA)" <ji...@apache.org> on 2019/08/16 02:21:00 UTC
[jira] [Updated] (SPARK-28751) Imporve java serializer
deserialization performance
[ https://issues.apache.org/jira/browse/SPARK-28751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xianyang Liu updated SPARK-28751:
---------------------------------
Description:
Improve the performance of java serializer deserialization by caching the resolved class. Java serializer is used in many places: closure, RPC and others. This change could improve the performance of deserialization 1.3X ~ 1.5X. Especially for java primitive instances.
And also, add new UT tests and benchmarks case.
was:
### What changes were proposed in this pull request?
Improve the performance of java serializer deserialization by caching the resolved class. Java serializer is used in many places: closure, RPC and others. This change could improve the performance of deserialization 1.3X ~ 1.5X. Especially for java primitive instances.
And also, add new UT tests and benchmarks case.
### Why are the changes needed?
This change could improve the performance of deserialization 1.3X ~ 1.5X. Especially for java primitive instances.
### Does this PR introduce any user-facing change?
Add new config to enable disable this feature, default is true. This shouldn't influence the existed code.
### How was this patch tested?
New UT tests.
> Imporve java serializer deserialization performance
> ---------------------------------------------------
>
> Key: SPARK-28751
> URL: https://issues.apache.org/jira/browse/SPARK-28751
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.4.3
> Reporter: Xianyang Liu
> Priority: Major
>
> Improve the performance of java serializer deserialization by caching the resolved class. Java serializer is used in many places: closure, RPC and others. This change could improve the performance of deserialization 1.3X ~ 1.5X. Especially for java primitive instances.
> And also, add new UT tests and benchmarks case.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org