You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@spark.apache.org by gu...@apache.org on 2023/03/22 01:18:27 UTC

[spark] branch master updated (da19e0de05b -> b4f02248972)

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


    from da19e0de05b [SPARK-42816][CONNECT] Support Max Message size up to 128MB
     add b4f02248972 [SPARK-42889][CONNECT][PYTHON] Implement cache, persist, unpersist, and storageLevel

No new revisions were added by this update.

Summary of changes:
 .../src/main/protobuf/spark/connect/base.proto     |  50 ++++
 .../common/StorageLevelProtoConverter.scala        |  31 ++-
 .../service/SparkConnectAnalyzeHandler.scala       |  33 ++-
 python/pyspark/sql/connect/client.py               |  36 +++
 python/pyspark/sql/connect/dataframe.py            |  56 +++-
 python/pyspark/sql/connect/proto/base_pb2.py       | 298 ++++++++++++++-------
 python/pyspark/sql/connect/proto/base_pb2.pyi      | 228 ++++++++++++++++
 python/pyspark/sql/dataframe.py                    |  20 ++
 .../sql/tests/connect/test_connect_basic.py        |   3 -
 python/pyspark/sql/tests/test_dataframe.py         |  26 +-
 python/pyspark/storagelevel.py                     |  14 +-
 11 files changed, 671 insertions(+), 124 deletions(-)
 copy sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ColumnUtils.scala => connector/connect/common/src/main/scala/org/apache/spark/sql/connect/common/StorageLevelProtoConverter.scala (53%)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org