You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/01/06 23:52:32 UTC
[GitHub] [iceberg] huaxingao commented on a change in pull request #3845: Partition Metadata table breaks with a partition column named 'partition'
huaxingao commented on a change in pull request #3845:
URL: https://github.com/apache/iceberg/pull/3845#discussion_r779941363
##########
File path: core/src/test/java/org/apache/iceberg/TestMetadataTableScans.java
##########
@@ -459,6 +460,65 @@ public void testDataFilesTableSelection() throws IOException {
Assert.assertEquals(expected, scan.schema().asStruct());
}
+ @Test
+ public void testPartitionColumnNamedPartition() throws Exception {
+ TestTables.clearTables();
+ this.tableDir = temp.newFolder();
+ tableDir.delete();
+
+ Schema schema = new Schema(
+ required(1, "id", Types.IntegerType.get()),
+ // create a column named "partition", to recreate the problem in
+ // https://github.com/apache/iceberg/issues/3709
+ required(2, "partition", Types.IntegerType.get())
+ );
+ this.metadataDir = new File(tableDir, "metadata");
+ PartitionSpec spec = PartitionSpec.builderFor(schema)
+ .identity("partition")
+ .build();
+
+ DataFile par0 = DataFiles.builder(spec)
+ .withPath("/path/to/data-0.parquet")
+ .withFileSizeInBytes(10)
+ .withPartition(TestHelpers.Row.of(0))
+ .withRecordCount(1)
+ .build();
+ DataFile par1 = DataFiles.builder(spec)
+ .withPath("/path/to/data-0.parquet")
+ .withFileSizeInBytes(10)
+ .withPartition(TestHelpers.Row.of(1))
+ .withRecordCount(1)
+ .build();
+ DataFile par2 = DataFiles.builder(spec)
+ .withPath("/path/to/data-0.parquet")
+ .withFileSizeInBytes(10)
+ .withPartition(TestHelpers.Row.of(2))
+ .withRecordCount(1)
+ .build();
+
+ this.table = create(schema, spec);
+ table.newFastAppend()
+ .appendFile(par0)
+ .commit();
+ table.newFastAppend()
+ .appendFile(par1)
+ .commit();
+ table.newFastAppend()
+ .appendFile(par2)
+ .commit();
+
+ Table partitionsTable = new PartitionsTable(table.ops(), table);
+
+ Expression andEquals = Expressions.and(
+ Expressions.equal("partition.partition", 0),
+ Expressions.greaterThan("record_count", 0));
+ TableScan scanAndEq = partitionsTable.newScan().filter(andEquals);
+ CloseableIterable<FileScanTask> tasksAndEq = PartitionsTable.planFiles((StaticTableScan) scanAndEq);
+ Assert.assertEquals(1, Iterators.size(tasksAndEq.iterator()));
+ validateIncludesPartitionScan(tasksAndEq, 0);
+ TestTables.clearTables();
+ }
Review comment:
Yes.
This is actually not needed. I only need to manually set up table because my `SCHEMA` and `SPEC` are different. No need to manually call `clearTables`. I will remove line 519.
##########
File path: core/src/test/java/org/apache/iceberg/TestMetadataTableScans.java
##########
@@ -459,6 +460,65 @@ public void testDataFilesTableSelection() throws IOException {
Assert.assertEquals(expected, scan.schema().asStruct());
}
+ @Test
+ public void testPartitionColumnNamedPartition() throws Exception {
+ TestTables.clearTables();
+ this.tableDir = temp.newFolder();
+ tableDir.delete();
+
+ Schema schema = new Schema(
+ required(1, "id", Types.IntegerType.get()),
+ // create a column named "partition", to recreate the problem in
+ // https://github.com/apache/iceberg/issues/3709
Review comment:
I remove the comments. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org