You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "laskoviymishka (via GitHub)" <gi...@apache.org> on 2026/09/10 11:34:57 UTC
[PR] Core: add labels metadata table [iceberg]
laskoviymishka opened a new pull request, #18048:
URL: https://github.com/apache/iceberg/pull/18048
Part 4 of the catalog-labels stack (spec: #15750). **Based on #18046.**
Adds a `LABELS` metadata table that exposes catalog-provided labels as rows (object- and field-level), wired through `MetadataTableType` and `MetadataTableUtils`. This is the queryable read path — `SELECT * FROM tbl.labels`.
Stacked on #18046.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: add labels metadata table [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #18048:
URL: https://github.com/apache/iceberg/pull/18048#discussion_r4045536165
##########
core/src/main/java/org/apache/iceberg/LabelsTable.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iceberg;
+
+import java.util.List;
+import java.util.Map;
+import org.apache.iceberg.io.CloseableIterable;
+import org.apache.iceberg.relocated.com.google.common.collect.Lists;
+import org.apache.iceberg.types.Types;
+
+/**
+ * A {@link Table} implementation that exposes the catalog-provided labels of its base table as
+ * rows.
+ *
+ * <p>Each label key-value pair is a row. Object-level labels have {@code scope = "object"} and a
+ * null {@code field_id}; field-level labels have {@code scope = "field"} and the field id they are
+ * attached to. Labels are catalog-provided enrichment obtained at load time; a re-scan re-reads the
+ * base table's labels and may observe different values.
+ */
+public class LabelsTable extends BaseMetadataTable {
+ private static final String OBJECT_SCOPE = "object";
+ private static final String FIELD_SCOPE = "field";
+
+ private static final Schema LABELS_SCHEMA =
+ new Schema(
+ Types.NestedField.required(1, "scope", Types.StringType.get()),
+ Types.NestedField.optional(2, "field_id", Types.IntegerType.get()),
+ Types.NestedField.optional(3, "field_name", Types.StringType.get()),
+ Types.NestedField.required(4, "key", Types.StringType.get()),
+ Types.NestedField.required(5, "value", Types.StringType.get()));
+
+ LabelsTable(Table table) {
+ this(table, table.name() + ".labels");
+ }
+
+ LabelsTable(Table table, String name) {
+ super(table, name);
+ }
+
+ @Override
+ MetadataTableType metadataTableType() {
+ return MetadataTableType.LABELS;
+ }
+
+ @Override
+ public TableScan newScan() {
+ return new LabelsScan(table());
+ }
+
+ @Override
+ public Schema schema() {
+ return LABELS_SCHEMA;
+ }
+
+ private Labels labels() {
Review Comment:
this seems to be used at a single place only, so maybe just inline it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: add labels metadata table [iceberg]
Posted by "laskoviymishka (via GitHub)" <gi...@apache.org>.
laskoviymishka commented on PR #18048:
URL: https://github.com/apache/iceberg/pull/18048#issuecomment-5927705174
@pvary - thanks, that make sense, here is a rationale and discussion - https://www.mail-archive.com/dev@iceberg.apache.org/msg15281.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: add labels metadata table [iceberg]
Posted by "laskoviymishka (via GitHub)" <gi...@apache.org>.
laskoviymishka closed pull request #18048: Core: add labels metadata table
URL: https://github.com/apache/iceberg/pull/18048
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: add labels metadata table [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #18048:
URL: https://github.com/apache/iceberg/pull/18048#discussion_r4045525555
##########
core/src/main/java/org/apache/iceberg/LabelsTable.java:
##########
@@ -0,0 +1,120 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iceberg;
+
+import java.util.List;
+import java.util.Map;
+import org.apache.iceberg.io.CloseableIterable;
+import org.apache.iceberg.relocated.com.google.common.collect.Lists;
+import org.apache.iceberg.types.Types;
+
+/**
+ * A {@link Table} implementation that exposes the catalog-provided labels of its base table as
+ * rows.
+ *
+ * <p>Each label key-value pair is a row. Object-level labels have {@code scope = "object"} and a
Review Comment:
I guess we can remove this entire paragraph because it's not adding value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: add labels metadata table [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on PR #18048:
URL: https://github.com/apache/iceberg/pull/18048#issuecomment-5923921566
@laskoviymishka: Could you please link the dev list discussion, so future readers can understand why this PR was closed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: add labels metadata table [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #18048:
URL: https://github.com/apache/iceberg/pull/18048#discussion_r4045559534
##########
core/src/test/java/org/apache/iceberg/TestLabelsTable.java:
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iceberg;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.io.File;
+import java.util.List;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.iceberg.hadoop.HadoopTables;
+import org.apache.iceberg.io.CloseableIterable;
+import org.apache.iceberg.relocated.com.google.common.collect.ImmutableList;
+import org.apache.iceberg.relocated.com.google.common.collect.Maps;
+import org.apache.iceberg.types.Types;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+public class TestLabelsTable {
+
+ private static final HadoopTables TABLES = new HadoopTables(new Configuration());
+ private static final Schema SCHEMA =
+ new Schema(
+ Types.NestedField.required(1, "id", Types.LongType.get()),
+ Types.NestedField.optional(2, "data", Types.StringType.get()));
+
+ @TempDir private File tableDir;
+ private Table table;
+
+ @BeforeEach
+ public void createTable() {
+ this.table =
+ TABLES.create(
+ SCHEMA, PartitionSpec.unpartitioned(), Maps.newHashMap(), tableDir.toURI().toString());
+ }
+
+ @Test
+ public void labelsTableResolvesThroughMetadataTableUtils() {
+ Table labelsTable =
+ MetadataTableUtils.createMetadataTableInstance(table, MetadataTableType.LABELS);
+
+ assertThat(labelsTable).isInstanceOf(LabelsTable.class);
Review Comment:
can we create a table that actually has labels and ensure that the actual values are properly shown through the metadata table?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: add labels metadata table [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on PR #18048:
URL: https://github.com/apache/iceberg/pull/18048#issuecomment-5816341135
These would be the first metadata tables which contain data which doesn't come directly from the Iceberg metadata files, and might return "transient" data. I think we should discuss this on the dev list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org