You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by gi...@apache.org on 2022/09/01 22:38:02 UTC
[iceberg] branch dependabot/gradle/org.apache.avro-avro-1.11.1 updated (2d5b2562f2 -> d44972a23f)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/gradle/org.apache.avro-avro-1.11.1
in repository https://gitbox.apache.org/repos/asf/iceberg.git
discard 2d5b2562f2 Bump avro from 1.9.2 to 1.11.1
add 4f90472d61 Python: Fix CLI after properties refactor (#5594)
add 9d26979a0c Python: Add logic for loading custom FileIO (#5588)
add 73a6c905c4 Build: Enforce LambdaMethodReference check at compile-time (#5529)
add f06c4f7dfc API: Extend FileIO in optional interfaces (#5576)
add c48f6fc721 Flink: Fix Javadoc in ContinuousSplitPlannerImpl (#5551)
add 7f2bd24049 Core: Add expression JSON parser (#5602)
add 1ce3b76a23 Build: Bump AWS SDK version (#5612)
add 254bc49ea7 Build: Bump tezVersion from 0.10.1 to 0.10.2 (#5520)
add 96ca8ab803 Docs: Flink `Streaming upsert write` (#5380)
add e5ec43a19b Fix message pattern in checkArgument invocation (#5621)
add a87d1b6837 Core: Add snapshot references metadata table (#4807)
add 12cd59b97d Python: Add additional information to the describe command (#5609)
add 007ef6eaf4 Core: Add TableMetadata support for statistics information (#5450)
add 65b8ac99f8 Docs: Added missing doc for REPLACE PARTITION FIELD (#5624)
add af1d405204 Python: Add types to bin packing (#5617)
add 1526c1f663 Core: Transform parquet bloom filter props when updating schema (#5426)
add a40dbd2e8b AWS: Deprecate AwsClientFactories.s3Configuration() (#5592)
add e227c521a0 Spark 3.3: Add SparkV2Filters (#5302)
add 80ec14ff36 API: Deprecate old incremental append scans (#5577)
add 023f3f5f07 Remove deprecations for Rollback and Overwrite Files (#5639)
add 487c6d2d4e Core: Use Bulk Delete when dropping table data and metadata (#5459)
add 509b817ebc Python: Add annotations typedef (#5628)
add d2b5e94a62 MR: Remove deprecated properties for 1.0 release (#5657)
add 324897e781 Aliyun: Remove deprecated classes for 1.0 release (#5654)
add ea217e9469 Build: Bump spotless-plugin-gradle from 6.9.1 to 6.10.0 (#5650)
add 7891536bac Docs: Switch post- and pre- around (#5633)
add 64e31684c9 Python: Add year, month, day, and hour transforms (#5462)
add 5d9c1d37af Deprections for 1.0 release: remove dynamo lock manager and OBE properties (#5655)
add 57195c4d27 AWS: Support S3 Dual Stack (#5644)
add 97c85aa03e Python: Fix date/time transforms. (#5667)
add cc7e6d9fa2 API: Add API changes for table statistics tracking (#5021)
add 6a1f351414 Python: Include PyYaml as a dependency (#5674)
add 08bb3e2589 Python: Reassign schema/partition-spec/sort-order IDs (#5627)
add 5764c93a59 AWS: fix wrong config key for useArnRegionEnabled in AssumeRoleAwsClientFactory (#5680)
add ec0195185d Spark 3.3: Add row-based changelog reader (#5578)
add 84f40cff9b Spark 3.2: Add row-based changelog reader (#5682)
add b2c4ec2620 [Python] FsspecFileIO that wraps any fsspec filesystem (#5332)
add 9f946a116e Core: Fix exception handling in BaseTaskWriter (#5683)
add 26cdc7a018 Core: Support deleting tables without metadata files (#5510)
add ec9643e9f7 Core, AWS: Fix Kryo serialization failure for FileIO (#5437)
add d44972a23f Bump avro from 1.9.2 to 1.11.1
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (2d5b2562f2)
\
N -- N -- N refs/heads/dependabot/gradle/org.apache.avro-avro-1.11.1 (d44972a23f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.palantir/revapi.yml | 61 +-
.../apache/iceberg/aliyun/oss/OSSInputFile.java | 4 +-
.../apache/iceberg/aliyun/oss/OSSInputStream.java | 8 +-
.../iceberg/{DeleteFile.java => BlobMetadata.java} | 28 +-
.../java/org/apache/iceberg/OverwriteFiles.java | 26 -
api/src/main/java/org/apache/iceberg/Rollback.java | 55 --
api/src/main/java/org/apache/iceberg/RowDelta.java | 28 -
.../{FileScanTask.java => StatisticsFile.java} | 42 +-
api/src/main/java/org/apache/iceberg/Table.java | 18 +-
.../main/java/org/apache/iceberg/TableScan.java | 6 +
.../main/java/org/apache/iceberg/Transaction.java | 11 +
.../{UpdateLocation.java => UpdateStatistics.java} | 17 +-
.../apache/iceberg/expressions/BoundReference.java | 9 +-
.../iceberg/expressions/ExpressionVisitors.java | 184 ++++++
.../apache/iceberg/expressions/Expressions.java | 4 +
.../apache/iceberg/expressions/NamedReference.java | 3 +-
.../org/apache/iceberg/expressions/Reference.java | 4 +-
.../apache/iceberg/io/SupportsBulkOperations.java | 2 +-
.../iceberg/io/SupportsPrefixOperations.java | 2 +-
.../java/org/apache/iceberg/types/TypeUtil.java | 4 +-
.../test/java/org/apache/iceberg/TestHelpers.java | 34 ++
.../org/apache/iceberg/aws/glue/GlueTestBase.java | 1 +
.../iceberg/aws/glue/TestGlueCatalogTable.java | 25 +
.../iceberg/aws/AssumeRoleAwsClientFactory.java | 2 +-
.../org/apache/iceberg/aws/AwsClientFactories.java | 14 +
.../java/org/apache/iceberg/aws/AwsProperties.java | 15 +-
.../iceberg/aws/dynamodb/DynamoDbCatalog.java | 13 +-
.../apache/iceberg/aws/glue/DynamoLockManager.java | 52 --
.../org/apache/iceberg/aws/glue/GlueCatalog.java | 13 +-
.../java/org/apache/iceberg/aws/s3/S3FileIO.java | 11 +-
.../org/apache/iceberg/aws/s3/S3OutputStream.java | 13 -
.../org/apache/iceberg/aws/s3/TestS3FileIO.java | 23 +
.../apache/iceberg/aws/s3/TestS3OutputStream.java | 6 +-
baseline.gradle | 2 +
build.gradle | 6 +-
.../java/org/apache/iceberg/BaseMetadataTable.java | 8 +-
.../main/java/org/apache/iceberg/BaseTable.java | 5 -
.../java/org/apache/iceberg/BaseTransaction.java | 8 +-
.../main/java/org/apache/iceberg/CatalogUtil.java | 96 +--
.../apache/iceberg/CommitCallbackTransaction.java | 5 +
.../org/apache/iceberg/GenericBlobMetadata.java | 108 ++++
.../org/apache/iceberg/GenericStatisticsFile.java | 105 ++++
.../java/org/apache/iceberg/MetadataTableType.java | 1 +
.../org/apache/iceberg/MetadataTableUtils.java | 2 +
.../java/org/apache/iceberg/MetadataUpdate.java | 40 ++
.../org/apache/iceberg/MetadataUpdateParser.java | 43 +-
.../java/org/apache/iceberg/MetricsConfig.java | 33 -
.../main/java/org/apache/iceberg/RefsTable.java | 108 ++++
.../org/apache/iceberg/RollbackToSnapshot.java | 38 --
.../main/java/org/apache/iceberg/SchemaUpdate.java | 15 +-
.../java/org/apache/iceberg/SerializableTable.java | 8 +-
...aultValueParser.java => SingleValueParser.java} | 4 +-
.../org/apache/iceberg/StatisticsFileParser.java | 136 +++++
.../java/org/apache/iceberg/TableMetadata.java | 33 +
.../org/apache/iceberg/TableMetadataParser.java | 23 +
.../iceberg/expressions/ExpressionParser.java | 424 +++++++++++++
.../java/org/apache/iceberg/io/BaseTaskWriter.java | 97 ++-
.../org/apache/iceberg/io/ResolvingFileIO.java | 7 +-
.../apache/iceberg/io/SortedPosDeleteWriter.java | 12 +-
.../java/org/apache/iceberg/jdbc/JdbcCatalog.java | 16 +-
.../java/org/apache/iceberg/util/PropertyUtil.java | 40 ++
.../iceberg/TestIncrementalDataTableScan.java | 4 +-
.../apache/iceberg/TestMetadataUpdateParser.java | 115 ++++
.../test/java/org/apache/iceberg/TestRowDelta.java | 15 +-
.../apache/iceberg/TestSchemaAndMappingUpdate.java | 23 +
...ValueParser.java => TestSingleValueParser.java} | 6 +-
.../java/org/apache/iceberg/TestTableMetadata.java | 114 ++++
.../iceberg/expressions/TestExpressionParser.java | 553 +++++++++++++++++
.../apache/iceberg/hadoop/HadoopFileIOTest.java | 26 +
.../org/apache/iceberg/io/TestResolvingIO.java | 51 ++
.../org/apache/iceberg/jdbc/TestJdbcCatalog.java | 14 +
.../resources/TableMetadataStatisticsFiles.json | 70 +++
docs/aws.md | 18 +
docs/flink-getting-started.md | 48 ++
docs/hive.md | 1 -
docs/spark-ddl.md | 10 +
.../enumerator/ContinuousSplitPlannerImpl.java | 2 +-
.../enumerator/ContinuousSplitPlannerImpl.java | 2 +-
.../java/org/apache/iceberg/gcp/gcs/GCSFileIO.java | 11 +-
.../org/apache/iceberg/gcp/gcs/GCSFileIOTest.java | 26 +
.../java/org/apache/iceberg/hive/HiveCatalog.java | 16 +-
.../org/apache/iceberg/hive/TestHiveCatalog.java | 18 +
hive3/build.gradle | 2 +-
.../main/java/org/apache/iceberg/mr/Catalogs.java | 51 +-
.../org/apache/iceberg/mr/InputFormatConfig.java | 22 -
.../java/org/apache/iceberg/mr/TestCatalogs.java | 90 +--
.../apache/iceberg/mr/TestIcebergInputFormats.java | 2 +-
.../iceberg/mr/TestInputFormatReaderDeletes.java | 3 +-
python/Makefile | 4 +-
python/poetry.lock | 668 +++++++++++----------
python/pyiceberg/catalog/__init__.py | 17 -
python/pyiceberg/catalog/hive.py | 26 +-
python/pyiceberg/catalog/rest.py | 22 +-
python/pyiceberg/cli/console.py | 7 +-
python/pyiceberg/cli/output.py | 23 +-
python/pyiceberg/io/__init__.py | 90 ++-
python/pyiceberg/io/fsspec.py | 196 ++++++
python/pyiceberg/manifest.py | 12 +
python/pyiceberg/schema.py | 154 ++++-
python/pyiceberg/serializers.py | 9 +-
python/pyiceberg/table/__init__.py | 11 +-
python/pyiceberg/table/metadata.py | 56 +-
python/pyiceberg/table/partitioning.py | 22 +
python/pyiceberg/table/snapshots.py | 7 +
python/pyiceberg/table/sorting.py | 33 +
python/pyiceberg/transforms.py | 227 ++++++-
python/pyiceberg/typedef.py | 4 +-
python/pyiceberg/types.py | 1 +
python/pyiceberg/utils/bin_packing.py | 69 ++-
python/pyiceberg/utils/datetime.py | 40 +-
python/pyiceberg/utils/iceberg_base_model.py | 19 +-
python/pyproject.toml | 18 +
python/tests/catalog/test_hive.py | 22 +-
python/tests/catalog/test_rest.py | 9 +-
python/tests/cli/test_console.py | 13 +-
python/tests/conftest.py | 21 +
python/tests/io/test_fsspec.py | 200 ++++++
python/tests/io/test_io.py | 68 +++
python/tests/table/test_init.py | 3 +-
python/tests/table/test_metadata.py | 208 ++++++-
python/tests/table/test_sorting.py | 4 +-
python/tests/test_transforms.py | 185 ++++++
python/tests/test_typedef.py | 4 +-
python/tests/utils/test_bin_packing.py | 13 +-
.../examples/SnapshotFunctionalityTest.java | 2 +-
.../spark/source/TestIcebergSourceTablesBase.java | 6 +-
.../spark/source/TestIcebergSourceTablesBase.java | 6 +-
.../spark/source/TestIcebergSourceTablesBase.java | 6 +-
.../spark/extensions/TestMetadataTables.java | 123 ++++
.../iceberg/spark/source/ChangelogRowReader.java | 154 +++++
.../iceberg/spark/source/TestChangelogReader.java | 255 ++++++++
.../spark/source/TestIcebergSourceTablesBase.java | 6 +-
.../org/apache/iceberg/spark/SparkV2Filters.java | 338 +++++++++++
.../iceberg/spark/source/ChangelogRowReader.java | 154 +++++
.../apache/iceberg/spark/TestSparkV2Filters.java | 267 ++++++++
.../iceberg/spark/source/TestChangelogReader.java | 255 ++++++++
.../spark/source/TestIcebergSourceTablesBase.java | 6 +-
versions.props | 3 +-
138 files changed, 6412 insertions(+), 1128 deletions(-)
copy api/src/main/java/org/apache/iceberg/{DeleteFile.java => BlobMetadata.java} (58%)
delete mode 100644 api/src/main/java/org/apache/iceberg/Rollback.java
copy api/src/main/java/org/apache/iceberg/{FileScanTask.java => StatisticsFile.java} (53%)
copy api/src/main/java/org/apache/iceberg/{UpdateLocation.java => UpdateStatistics.java} (63%)
delete mode 100644 aws/src/main/java/org/apache/iceberg/aws/glue/DynamoLockManager.java
create mode 100644 core/src/main/java/org/apache/iceberg/GenericBlobMetadata.java
create mode 100644 core/src/main/java/org/apache/iceberg/GenericStatisticsFile.java
create mode 100644 core/src/main/java/org/apache/iceberg/RefsTable.java
delete mode 100644 core/src/main/java/org/apache/iceberg/RollbackToSnapshot.java
rename core/src/main/java/org/apache/iceberg/{DefaultValueParser.java => SingleValueParser.java} (99%)
create mode 100644 core/src/main/java/org/apache/iceberg/StatisticsFileParser.java
create mode 100644 core/src/main/java/org/apache/iceberg/expressions/ExpressionParser.java
rename core/src/test/java/org/apache/iceberg/{TestDefaultValueParser.java => TestSingleValueParser.java} (97%)
create mode 100644 core/src/test/java/org/apache/iceberg/expressions/TestExpressionParser.java
create mode 100644 core/src/test/java/org/apache/iceberg/io/TestResolvingIO.java
create mode 100644 core/src/test/resources/TableMetadataStatisticsFiles.json
create mode 100644 python/pyiceberg/io/fsspec.py
create mode 100644 python/tests/io/test_fsspec.py
create mode 100644 spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/source/ChangelogRowReader.java
create mode 100644 spark/v3.2/spark/src/test/java/org/apache/iceberg/spark/source/TestChangelogReader.java
create mode 100644 spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/SparkV2Filters.java
create mode 100644 spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/ChangelogRowReader.java
create mode 100644 spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/TestSparkV2Filters.java
create mode 100644 spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/source/TestChangelogReader.java