[GitHub] [incubator-iceberg] rdblue commented on a change in pull request #627: Update indexing to handle nested lists

[GitBox <gi...@apache.org>]

rdblue commented on a change in pull request #627: Update indexing to handle nested lists
URL: https://github.com/apache/incubator-iceberg/pull/627#discussion_r345338437
 
 

 ##########
 File path: api/src/main/java/org/apache/iceberg/types/IndexByName.java
 ##########
 @@ -20,53 +20,99 @@
 package org.apache.iceberg.types;
 
 import com.google.common.base.Joiner;
+import com.google.common.collect.Lists;
 import com.google.common.collect.Maps;
-import java.util.List;
+import java.util.Deque;
 import java.util.Map;
+import java.util.concurrent.Callable;
+import java.util.function.Supplier;
 import org.apache.iceberg.Schema;
+import org.apache.iceberg.exceptions.ValidationException;
 
-public class IndexByName extends TypeUtil.SchemaVisitor<Map<String, Integer>> {
+public class IndexByName extends TypeUtil.CustomOrderSchemaVisitor<Map<String, Integer>> {
   private static final Joiner DOT = Joiner.on(".");
 
+  private final Deque<String> fieldNames = Lists.newLinkedList();
   private final Map<String, Integer> nameToId = Maps.newHashMap();
 
   @Override
-  public Map<String, Integer> schema(Schema schema, Map<String, Integer> structResult) {
-    return nameToId;
+  public Map<String, Integer> schema(Schema schema, Supplier<Map<String, Integer>> structResult) {
+    return structResult.get();
   }
 
   @Override
-  public Map<String, Integer> struct(Types.StructType struct, List<Map<String, Integer>> fieldResults) {
+  public Map<String, Integer> struct(Types.StructType struct, Iterable<Map<String, Integer>> fieldResults) {
+    // iterate through the fields to update the index for each one, use size to avoid errorprone failure
+    Lists.newArrayList(fieldResults).size();
     return nameToId;
   }
 
   @Override
-  public Map<String, Integer> field(Types.NestedField field, Map<String, Integer> fieldResult) {
+  public Map<String, Integer> field(Types.NestedField field, Supplier<Map<String, Integer>> fieldResult) {
+    withName(field.name(), fieldResult::get);
     addField(field.name(), field.fieldId());
     return null;
   }
 
   @Override
-  public Map<String, Integer> list(Types.ListType list, Map<String, Integer> elementResult) {
+  public Map<String, Integer> list(Types.ListType list, Supplier<Map<String, Integer>> elementResult) {
+    // add element
     for (Types.NestedField field : list.fields()) {
       addField(field.name(), field.fieldId());
     }
+
+    if (list.elementType().isStructType()) {
+      // return to avoid errorprone failure
+      return elementResult.get();
+    }
+
+    withName("element", elementResult::get);
+
     return null;
   }
 
   @Override
-  public Map<String, Integer> map(Types.MapType map, Map<String, Integer> keyResult, Map<String, Integer> valueResult) {
+  public Map<String, Integer> map(Types.MapType map,
+                                  Supplier<Map<String, Integer>> keyResult,
+                                  Supplier<Map<String, Integer>> valueResult) {
+    withName("key", keyResult::get);
+
+    // add key and value
     for (Types.NestedField field : map.fields()) {
       addField(field.name(), field.fieldId());
     }
+
+    if (map.valueType().isStructType()) {
 
 Review comment:
   Yeah, I tried adding both variants, but it fails when we create a BiMap from the index due to duplicate values (list.field and list.element.field with the same ID). We can fix this later by adding these as aliases, but we'd have to store it separately so I thought it would be better to start with the simple solution here.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org