You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by gi...@apache.org on 2023/05/08 10:38:36 UTC
[iceberg] branch dependabot/gradle/net.ltgt.gradle-gradle-errorprone-plugin-3.1.0 updated (04ea778534 -> 28a1ff9777)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/gradle/net.ltgt.gradle-gradle-errorprone-plugin-3.1.0
in repository https://gitbox.apache.org/repos/asf/iceberg.git
omit 04ea778534 Build: Bump net.ltgt.gradle:gradle-errorprone-plugin from 3.0.1 to 3.1.0
add c9bdae12ce Core: Remove compile-time dependency to ResolvingFileIO (#7488)
add 5117b6bfa9 Docs: Update new catalog features (#7433)
add c19e7e9d5d Spark 3.3: Add read and write support for UUIDs (#7496)
add b58bf17ae3 Spark 3.2: Add read and write support for UUIDs (#7497)
add 4f9584cea6 Spark 3.1: Add read and write support for UUIDs (#7508)
add 3db4e89658 API: Remove deprecated AssertHelpers usage (#7468)
add f0149a5860 API: Update Javadoc for listTables and listViews (#7336)
add c751cb24b6 Core: Simplify partition coercion code in PartitionsTable (#7503)
add 8d8db30aeb Python: Support Iceberg UUIDType from Parquet (#7523)
add 4aa9e9fe3b Core, Spark: Add case-sensitivity configuration to CachingCatalog (#7469)
add db3800d3c4 AWS: Add finalizer to S3FileIO (#7513)
add c0253429d2 AWS: Move all S3FileIO related properties into a separate class S3FileIOProperties (#7505)
add 7fa5fca9b1 Spark 3.4: Handle skew in writes (#7520)
add d8142d1035 Spec: Update View spec to reflect that schema is defined at the version level and is required (#7485)
add 667fd86033 Spark 3.4: Action for rewriting position deletes (#7389)
add 7e71c3fd98 Spark 3.4: Tests for coalescing small writing tasks (#7532)
add 1cb1320e20 Core: Support delete file stats in partitions metadata table (#6661)
add 7406098ee7 Core, Spark: Revert SparkCatalog case sensitive CachingCatalog default (#7535)
add 445ee94e2c Core: Remove duplicate check for ManifestEntry.dataSequenceNumber() (#7538)
add 3ab00171b4 Remove usages of S3 fields from AwsProperties within s3 package (#7534)
add f5de7fb80c Build: Bump mkdocstrings from 0.20.0 to 0.21.2 in /python (#7547)
add ed105e1b70 Build: Bump griffe from 0.26.0 to 0.27.3 in /python (#7546)
add 9980ef7b6e Build: Bump rich from 13.3.4 to 13.3.5 in /python (#7545)
add 475fc65b66 Build: Bump requests from 2.28.2 to 2.30.0 in /python (#7544)
add 492d60e3ce Build: Bump mkdocs-material from 9.1.8 to 9.1.9 in /python (#7542)
add 6a0732f7c4 Python: Add REST support for SigV4 (#7519)
add 5da1bfaab7 Flink: fix typo in namespace (#7527)
add 98f4bc427f Arrow: Fix errorprone warnings (#7498)
add fbdbaec5a5 Spark: Use UUIDUtil.convertToByteBuffer to avoid rewinding buffer (#7525)
add f3c90f9d53 Build: Bump me.champeau.jmh:jmh-gradle-plugin from 0.7.0 to 0.7.1 (#7408)
add 28a1ff9777 Build: Bump net.ltgt.gradle:gradle-errorprone-plugin from 3.0.1 to 3.1.0
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (04ea778534)
\
N -- N -- N refs/heads/dependabot/gradle/net.ltgt.gradle-gradle-errorprone-plugin-3.1.0 (28a1ff9777)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.palantir/revapi.yml | 4 +
.../apache/iceberg/actions/ActionsProvider.java | 6 +
.../actions/RewritePositionDeleteFiles.java | 125 +++-
.../java/org/apache/iceberg/catalog/Catalog.java | 4 +-
.../org/apache/iceberg/catalog/ViewCatalog.java | 4 +-
.../iceberg/TestPartitionSpecValidation.java | 335 +++++-----
.../java/org/apache/iceberg/TestSnapshotRef.java | 58 +-
.../org/apache/iceberg/catalog/TestNamespace.java | 25 +-
.../apache/iceberg/expressions/TestEvaluator.java | 148 ++--
.../iceberg/expressions/TestExpressionHelpers.java | 26 +-
.../TestInclusiveManifestEvaluator.java | 20 +-
.../expressions/TestInclusiveMetricsEvaluator.java | 20 +-
.../iceberg/expressions/TestPredicateBinding.java | 17 +-
.../expressions/TestStrictMetricsEvaluator.java | 11 +-
.../apache/iceberg/io/TestCloseableIterable.java | 8 +-
.../apache/iceberg/transforms/TestBucketing.java | 10 +-
.../apache/iceberg/transforms/TestProjection.java | 21 +-
.../apache/iceberg/transforms/TestTruncate.java | 10 +-
.../org/apache/iceberg/types/TestTypeUtil.java | 43 +-
.../iceberg/arrow/vectorized/ColumnVector.java | 12 +-
.../iceberg/aws/s3/TestS3FileIOIntegration.java | 52 +-
.../iceberg/aws/s3/TestS3MultipartUpload.java | 23 +-
.../java/org/apache/iceberg/aws/AwsProperties.java | 431 ++++++++++--
.../java/org/apache/iceberg/aws/s3/BaseS3File.java | 14 +-
.../java/org/apache/iceberg/aws/s3/S3FileIO.java | 75 ++-
.../apache/iceberg/aws/s3/S3FileIOProperties.java | 653 ++++++++++++++++++
.../org/apache/iceberg/aws/s3/S3InputFile.java | 22 +-
.../org/apache/iceberg/aws/s3/S3InputStream.java | 14 +-
.../org/apache/iceberg/aws/s3/S3OutputFile.java | 19 +-
.../org/apache/iceberg/aws/s3/S3OutputStream.java | 30 +-
.../org/apache/iceberg/aws/s3/S3RequestUtil.java | 54 +-
.../apache/iceberg/aws/TestAwsClientFactories.java | 7 +-
.../org/apache/iceberg/aws/s3/TestS3FileIO.java | 8 +
.../iceberg/aws/s3/TestS3FileIOProperties.java | 397 +++++++++++
.../apache/iceberg/aws/s3/TestS3OutputStream.java | 27 +-
.../apache/iceberg/aws/s3/TestS3RequestUtil.java | 33 +-
baseline.gradle | 1 +
build.gradle | 5 +-
.../java/org/apache/iceberg/CatalogProperties.java | 5 +
.../main/java/org/apache/iceberg/CatalogUtil.java | 2 +-
.../java/org/apache/iceberg/PartitionsTable.java | 244 ++++---
.../actions/RewritePositionDeleteStrategy.java | 7 +-
.../RewritePositionDeletesCommitManager.java | 129 ++++
.../actions/RewritePositionDeletesGroup.java | 125 ++++
.../iceberg/actions/SizeBasedFileRewriter.java | 12 +
.../actions/SizeBasedPositionDeletesRewriter.java | 58 ++
.../java/org/apache/iceberg/avro/ValueWriters.java | 9 +-
.../apache/iceberg/rest/RESTSessionCatalog.java | 5 +-
.../org/apache/iceberg/util/PartitionUtil.java | 2 +-
.../apache/iceberg/MetadataTableScanTestBase.java | 12 +-
.../java/org/apache/iceberg/TableTestBase.java | 8 -
.../org/apache/iceberg/TestMetadataTableScans.java | 208 ++++--
...stMetadataTableScansWithPartitionEvolution.java | 18 +-
docs/spark-configuration.md | 18 +-
.../iceberg/flink/TestFlinkCatalogDatabase.java | 4 +-
.../iceberg/flink/TestFlinkCatalogTable.java | 4 +-
.../iceberg/flink/TestFlinkCatalogDatabase.java | 4 +-
.../iceberg/flink/TestFlinkCatalogTable.java | 4 +-
.../iceberg/flink/TestFlinkCatalogDatabase.java | 4 +-
.../iceberg/flink/TestFlinkCatalogTable.java | 4 +-
format/view-spec.md | 22 +-
.../org/apache/hadoop/hive/ql/io/orc/OrcSplit.java | 12 +-
python/mkdocs/docs/configuration.md | 9 +
python/mkdocs/requirements.txt | 6 +-
python/poetry.lock | 16 +-
python/pyiceberg/catalog/rest.py | 123 +++-
python/pyiceberg/schema.py | 9 +
python/tests/catalog/test_rest.py | 18 +-
.../org/apache/iceberg/spark/SparkCatalog.java | 12 +-
.../java/org/apache/iceberg/spark/SparkUtil.java | 4 +
.../apache/iceberg/spark/data/SparkAvroWriter.java | 2 +-
.../apache/iceberg/spark/data/SparkOrcReader.java | 3 +
.../iceberg/spark/data/SparkOrcValueReaders.java | 20 +
.../iceberg/spark/data/SparkOrcValueWriters.java | 20 +
.../apache/iceberg/spark/data/SparkOrcWriter.java | 12 +-
.../iceberg/spark/data/SparkParquetReaders.java | 17 +
.../iceberg/spark/data/SparkParquetWriters.java | 33 +
.../iceberg/spark/data/SparkValueWriters.java | 8 +-
.../vectorized/ArrowVectorAccessorFactory.java | 7 +
.../data/vectorized/VectorizedSparkOrcReaders.java | 5 +-
.../apache/iceberg/spark/data/AvroDataTest.java | 2 +-
.../org/apache/iceberg/spark/data/RandomData.java | 2 +
.../iceberg/spark/data/TestSparkParquetWriter.java | 2 +-
.../spark/source/TestIcebergSourceTablesBase.java | 40 +-
.../org/apache/iceberg/spark/SparkCatalog.java | 12 +-
.../apache/iceberg/spark/data/SparkAvroWriter.java | 2 +-
.../apache/iceberg/spark/data/SparkOrcReader.java | 3 +
.../iceberg/spark/data/SparkOrcValueReaders.java | 20 +
.../iceberg/spark/data/SparkOrcValueWriters.java | 20 +
.../apache/iceberg/spark/data/SparkOrcWriter.java | 12 +-
.../iceberg/spark/data/SparkParquetReaders.java | 17 +
.../iceberg/spark/data/SparkParquetWriters.java | 33 +
.../iceberg/spark/data/SparkValueWriters.java | 8 +-
.../vectorized/ArrowVectorAccessorFactory.java | 7 +
.../data/vectorized/VectorizedSparkOrcReaders.java | 5 +-
.../apache/iceberg/spark/data/AvroDataTest.java | 2 +-
.../org/apache/iceberg/spark/data/RandomData.java | 2 +
.../iceberg/spark/data/TestSparkParquetWriter.java | 2 +-
.../spark/source/TestIcebergSourceTablesBase.java | 198 +++++-
.../org/apache/iceberg/spark/SparkCatalog.java | 25 +-
.../spark/actions/SparkBinPackDataRewriter.java | 12 -
.../apache/iceberg/spark/data/SparkAvroWriter.java | 2 +-
.../apache/iceberg/spark/data/SparkOrcReader.java | 3 +
.../iceberg/spark/data/SparkOrcValueReaders.java | 20 +
.../iceberg/spark/data/SparkOrcValueWriters.java | 20 +
.../apache/iceberg/spark/data/SparkOrcWriter.java | 12 +-
.../iceberg/spark/data/SparkParquetReaders.java | 17 +
.../iceberg/spark/data/SparkParquetWriters.java | 33 +
.../iceberg/spark/data/SparkValueWriters.java | 8 +-
.../vectorized/ArrowVectorAccessorFactory.java | 7 +
.../data/vectorized/VectorizedSparkOrcReaders.java | 5 +-
.../apache/iceberg/spark/data/AvroDataTest.java | 2 +-
.../org/apache/iceberg/spark/data/RandomData.java | 2 +
.../iceberg/spark/data/TestSparkParquetWriter.java | 2 +-
.../spark/source/TestIcebergSourceTablesBase.java | 198 +++++-
.../iceberg/spark/extensions/TestDelete.java | 128 ++++
.../apache/iceberg/spark/extensions/TestMerge.java | 147 ++++
.../iceberg/spark/extensions/TestUpdate.java | 132 ++++
.../org/apache/iceberg/spark/SparkCatalog.java | 25 +-
.../actions/RewritePositionDeleteSparkAction.java} | 315 ++++-----
.../apache/iceberg/spark/actions/SparkActions.java | 5 +
.../spark/actions/SparkBinPackDataRewriter.java | 12 -
.../SparkBinPackPositionDeletesRewriter.java | 141 ++++
.../iceberg/spark/data/SparkOrcValueWriters.java | 3 +
.../iceberg/spark/data/SparkValueWriters.java | 8 +-
.../spark/source/SparkPositionDeletesRewrite.java | 2 +-
.../spark/source/SparkPositionDeltaWrite.java | 5 +
.../apache/iceberg/spark/source/SparkWrite.java | 5 +
.../TestRewritePositionDeleteFilesAction.java | 742 +++++++++++++++++++++
.../iceberg/spark/source/FourColumnRecord.java} | 37 +-
.../spark/source/TestIcebergSourceTablesBase.java | 198 +++++-
131 files changed, 5400 insertions(+), 1278 deletions(-)
create mode 100644 aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java
create mode 100644 aws/src/test/java/org/apache/iceberg/aws/s3/TestS3FileIOProperties.java
create mode 100644 core/src/main/java/org/apache/iceberg/actions/RewritePositionDeletesCommitManager.java
create mode 100644 core/src/main/java/org/apache/iceberg/actions/RewritePositionDeletesGroup.java
create mode 100644 core/src/main/java/org/apache/iceberg/actions/SizeBasedPositionDeletesRewriter.java
copy spark/{v3.3/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteDataFilesSparkAction.java => v3.4/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteSparkAction.java} (56%)
create mode 100644 spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/actions/SparkBinPackPositionDeletesRewriter.java
create mode 100644 spark/v3.4/spark/src/test/java/org/apache/iceberg/spark/actions/TestRewritePositionDeleteFilesAction.java
copy spark/{v3.3/spark/src/test/java/org/apache/iceberg/spark/source/ThreeColumnRecord.java => v3.4/spark/src/test/java/org/apache/iceberg/spark/source/FourColumnRecord.java} (71%)