You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/03/14 23:36:30 UTC
[GitHub] [iceberg] szehon-ho commented on a change in pull request #4005: Core: Add delete file details for PartitionsTable
szehon-ho commented on a change in pull request #4005:
URL: https://github.com/apache/iceberg/pull/4005#discussion_r826462290
##########
File path: core/src/test/java/org/apache/iceberg/TestMetadataTableScans.java
##########
@@ -516,6 +522,101 @@ public void testPartitionColumnNamedPartition() throws Exception {
validateIncludesPartitionScan(tasksAndEq, 0);
}
+ @Test
+ public void testPartitionsTableScanWithDeleteFilesInFilter() throws IOException {
+ Assume.assumeTrue(formatVersion == 2);
+ Configuration conf = new Configuration();
+ HadoopTables tables = new HadoopTables(conf);
Review comment:
Seems all the rest of the test in this class just use the constructor of PartitionsTable (or whatever metadata table), is there any reason not to do it here?
##########
File path: core/src/main/java/org/apache/iceberg/PartitionsTable.java
##########
@@ -48,7 +48,10 @@
this.schema = new Schema(
Types.NestedField.required(1, "partition", table.spec().partitionType()),
Types.NestedField.required(2, "record_count", Types.LongType.get()),
- Types.NestedField.required(3, "file_count", Types.IntegerType.get())
+ Types.NestedField.required(3, "file_count", Types.IntegerType.get()),
+ Types.NestedField.required(4, "delete_file_count", Types.IntegerType.get()),
+ Types.NestedField.required(5, "equality_delete_count", Types.LongType.get()),
Review comment:
The names still do not seem consistent to me ("delete_file" and "file" but "equality_delete" and "position_delete"). But yes good point this is following SnapshotSummary, though I think this schema is already quite different than that. Will leave to @rdblue or others to see what they think.
##########
File path: core/src/test/java/org/apache/iceberg/TestMetadataTableScans.java
##########
@@ -516,6 +522,101 @@ public void testPartitionColumnNamedPartition() throws Exception {
validateIncludesPartitionScan(tasksAndEq, 0);
}
+ @Test
+ public void testPartitionsTableScanWithDeleteFilesInFilter() throws IOException {
+ Assume.assumeTrue(formatVersion == 2);
+ Configuration conf = new Configuration();
+ HadoopTables tables = new HadoopTables(conf);
+ File tableLocation = new File(temp.newFolder(), "table");
+ Table table = tables.create(
Review comment:
On same line, I think we do not need to create a table again, as 'table' is already setup in this test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org