You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2021/02/25 07:32:00 UTC
[jira] [Assigned] (SPARK-34534) New protocol FetchShuffleBlocks in
OneForOneBlockFetcher lead to data loss or correctness
[ https://issues.apache.org/jira/browse/SPARK-34534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-34534:
------------------------------------
Assignee: Apache Spark
> New protocol FetchShuffleBlocks in OneForOneBlockFetcher lead to data loss or correctness
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-34534
> URL: https://issues.apache.org/jira/browse/SPARK-34534
> Project: Spark
> Issue Type: Bug
> Components: Shuffle
> Affects Versions: 3.0.0, 3.0.1, 3.0.2
> Reporter: haiyangyu
> Assignee: Apache Spark
> Priority: Major
> Labels: Correctness, data-loss
> Attachments: image-2021-02-25-11-17-12-714.png, image-2021-02-25-11-27-34-429.png, image-2021-02-25-11-28-31-255.png, image-2021-02-25-11-30-03-834.png, image-2021-02-25-11-31-59-110.png
>
>
> We will build a new rpc message `FetchShuffleBlocks` when `OneForOneBlockFetcher` init in replace of `OpenBlocks` to use adaptive feature, this introduce additional problems as follows.
> `OneForOneBlockFetcher` will init a `blockIds` String array to catch chunk fetch success, it will use index in `blockIds` to fetch blocks and match blockId in `blockIds` when chunk data return. So the `blockIds` 's order must be consistent with fetchChunk index, but the new `FetchShuffleBlocks` return chunk order is not same as `blockIds`.
> This will lead to the return data not match the blockId, and this can lead to data corretness when retry to fetch after fetch block chunk failed.
> Fetch chunk orker code and match blockId when rerun data code as follows:
> !image-2021-02-25-11-27-34-429.png|width=446,height=251!!image-2021-02-25-11-28-31-255.png|width=445,height=159!
> Howerver, the fetch order in shuffle service,
> !image-2021-02-25-11-30-03-834.png|width=510,height=361!
> So, it will fetch some wrong block data when chunk fetch failed beause the blocks's wrong order.
> !image-2021-02-25-11-31-59-110.png|width=601,height=204!
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org