You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2020/10/21 04:47:00 UTC

[jira] [Commented] (SPARK-33200) Validate that all shuffle files are migrated during E2E testing

    [ https://issues.apache.org/jira/browse/SPARK-33200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218106#comment-17218106 ] 

Apache Spark commented on SPARK-33200:
--------------------------------------

User 'holdenk' has created a pull request for this issue:
https://github.com/apache/spark/pull/30115

> Validate that all shuffle files are migrated during E2E testing
> ---------------------------------------------------------------
>
>                 Key: SPARK-33200
>                 URL: https://issues.apache.org/jira/browse/SPARK-33200
>             Project: Spark
>          Issue Type: Improvement
>          Components: Kubernetes, Tests
>    Affects Versions: 3.1.0
>            Reporter: Holden Karau
>            Priority: Major
>
> Currently the end to end testing only migrates a small number of shuffle files. While we can't migrate large amounts of data in the time of the integration suite we can extend the decommissioning test to migrate multiple shuffle files of different sizes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org