You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by ru...@apache.org on 2021/12/04 17:09:11 UTC

[iceberg] branch master updated (c6e68d8 -> b6554fc)

This is an automated email from the ASF dual-hosted git repository.

russellspitzer pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/iceberg.git.


    from c6e68d8  Core: Throw CommitStateUnknownException in old RewriteDatafilesAction (#2932)
     add b6554fc  Spark: Support CALL procedure for rewrite_data_files (#3375)

No new revisions were added by this update.

Summary of changes:
 site/docs/spark-procedures.md                      |  53 ++++
 .../extensions/TestRewriteDataFilesProcedure.java  | 275 +++++++++++++++++++++
 .../procedures/RewriteDataFilesProcedure.java      | 186 ++++++++++++++
 .../iceberg/spark/procedures/SparkProcedures.java  |   1 +
 .../logical/SetWriteDistributionAndOrdering.scala  |   0
 .../plans/logical/SortOrderParserUtil.scala}       |  28 +--
 .../datasources/SparkExpressionConverter.scala}    |  17 +-
 7 files changed, 535 insertions(+), 25 deletions(-)
 create mode 100644 spark/v3.2/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRewriteDataFilesProcedure.java
 create mode 100644 spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java
 rename spark/v3.2/{spark-extensions/src/main/scala => spark/src/main/java}/org/apache/spark/sql/catalyst/plans/logical/SetWriteDistributionAndOrdering.scala (100%)
 copy spark/{v3.0/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/SetWriteDistributionAndOrdering.scala => v3.2/spark/src/main/java/org/apache/spark/sql/catalyst/plans/logical/SortOrderParserUtil.scala} (58%)
 copy spark/{v3.0/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Call.scala => v3.2/spark/src/main/java/org/apache/spark/sql/execution/datasources/SparkExpressionConverter.scala} (58%)