You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Sanford Ryza (Jira)" <ji...@apache.org> on 2025/10/01 02:38:00 UTC

[jira] [Created] (SPARK-53772) Refactor PythonPipelineSuite on top of APITest

Sanford Ryza created SPARK-53772:
------------------------------------

             Summary: Refactor PythonPipelineSuite on top of APITest
                 Key: SPARK-53772
                 URL: https://issues.apache.org/jira/browse/SPARK-53772
             Project: Spark
          Issue Type: Improvement
          Components: Declarative Pipelines
    Affects Versions: 4.1.0
            Reporter: Sanford Ryza


The tests in PythonPipelineSuite are essentially API tests, meaning that they could be written to extend the APITest class. That would allow them to rely on the APITest infrastructure and be tested in a higher fidelity way.

 

Doing this will Spark Connect protocol additions to allow the client to access metadata about the dataflow graph.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org