You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sa...@apache.org on 2025/05/30 14:54:35 UTC
(spark) branch master updated (6a89f654bb54 -> e3321aa44ea2)
This is an automated email from the ASF dual-hosted git repository.
sandy pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
from 6a89f654bb54 [SPARK-52130][FOLLOW-UP][ML][CONNECT] Refine error message when model.summary is evicted
add e3321aa44ea2 [SPARK-52238][PYTHON] Python client for Declarative Pipelines
No new revisions were added by this update.
Summary of changes:
.github/workflows/build_and_test.yml | 2 +-
bin/{beeline => spark-pipelines} | 17 +-
dev/requirements.txt | 1 +
dev/sparktestsupport/modules.py | 12 +
dev/sparktestsupport/utils.py | 12 +-
python/mypy.ini | 6 +
python/packaging/client/setup.py | 2 +
python/pyspark/errors/error-conditions.json | 60 +++
.../pipelines}/__init__.py | 19 +-
python/pyspark/pipelines/api.py | 462 +++++++++++++++++++++
python/pyspark/pipelines/block_connect_access.py | 58 +++
python/pyspark/pipelines/cli.py | 276 ++++++++++++
python/pyspark/pipelines/dataset.py | 76 ++++
python/pyspark/pipelines/flow.py | 45 ++
python/pyspark/pipelines/graph_element_registry.py | 83 ++++
python/pyspark/pipelines/init_cli.py | 70 ++++
.../source_code_location.py} | 36 +-
.../spark_connect_graph_element_registry.py | 112 +++++
python/pyspark/pipelines/spark_connect_pipeline.py | 75 ++++
.../pyspark/pipelines/tests}/__init__.py | 0
.../tests/local_graph_element_registry.py | 58 +++
.../pipelines/tests/test_block_connect_access.py | 63 +++
python/pyspark/pipelines/tests/test_cli.py | 324 +++++++++++++++
python/pyspark/pipelines/tests/test_decorators.py | 72 ++++
.../pipelines/tests/test_graph_element_registry.py | 126 ++++++
python/pyspark/pipelines/tests/test_init_cli.py | 62 +++
python/pyspark/pipelines/type_error_utils.py | 51 +++
python/pyspark/sql/connect/proto/__init__.py | 1 +
28 files changed, 2149 insertions(+), 32 deletions(-)
copy bin/{beeline => spark-pipelines} (70%)
copy python/{packaging/connect/pyspark_connect => pyspark/pipelines}/__init__.py (76%)
create mode 100644 python/pyspark/pipelines/api.py
create mode 100644 python/pyspark/pipelines/block_connect_access.py
create mode 100644 python/pyspark/pipelines/cli.py
create mode 100644 python/pyspark/pipelines/dataset.py
create mode 100644 python/pyspark/pipelines/flow.py
create mode 100644 python/pyspark/pipelines/graph_element_registry.py
create mode 100644 python/pyspark/pipelines/init_cli.py
copy python/pyspark/{ml/tests/typing/test_param.yml => pipelines/source_code_location.py} (50%)
create mode 100644 python/pyspark/pipelines/spark_connect_graph_element_registry.py
create mode 100644 python/pyspark/pipelines/spark_connect_pipeline.py
copy {examples/src/main/python => python/pyspark/pipelines/tests}/__init__.py (100%)
create mode 100644 python/pyspark/pipelines/tests/local_graph_element_registry.py
create mode 100644 python/pyspark/pipelines/tests/test_block_connect_access.py
create mode 100644 python/pyspark/pipelines/tests/test_cli.py
create mode 100644 python/pyspark/pipelines/tests/test_decorators.py
create mode 100644 python/pyspark/pipelines/tests/test_graph_element_registry.py
create mode 100644 python/pyspark/pipelines/tests/test_init_cli.py
create mode 100644 python/pyspark/pipelines/type_error_utils.py
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org