You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean R. Owen (Jira)" <ji...@apache.org> on 2023/05/24 23:26:00 UTC
[jira] [Resolved] (SPARK-43334) Error while serializing ExecutorPeakMetricsDistributions into API JSON response
[ https://issues.apache.org/jira/browse/SPARK-43334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean R. Owen resolved SPARK-43334.
----------------------------------
Fix Version/s: 3.5.0
Resolution: Fixed
Issue resolved by pull request 41017
[https://github.com/apache/spark/pull/41017]
> Error while serializing ExecutorPeakMetricsDistributions into API JSON response
> -------------------------------------------------------------------------------
>
> Key: SPARK-43334
> URL: https://issues.apache.org/jira/browse/SPARK-43334
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 3.3.3
> Reporter: Thejdeep
> Assignee: Thejdeep
> Priority: Major
> Fix For: 3.5.0
>
>
> When we try to get the ExecutorPeakMetricsDistributions [through the API|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/status/api/v1/api.scala#L463] (/stages), there is a possibility of encountering an issue while serializing the StagesData into a JSON if the executor metrics are empty.
>
> The following error is thrown :
> {code:java}
> Caused by: com.fasterxml.jackson.databind.JsonMappingException: -1 (through reference chain: scala.c
> ollection.immutable.$colon$colon[0]->org.apache.spark.status.api.v1.StageData["executorMetricsDistri
> butions"]->org.apache.spark.status.api.v1.ExecutorMetricsDistributions["peakMemoryMetrics"])
> at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.jav
> a:390)
> at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.jav
> a:349) {code}
> This happens because the indices for the quartiles are populated incorrectly as -1 since the metrics itself are empty and this leads to this exception being thrown.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org