[GitHub] [iceberg] hililiwei commented on a change in pull request #3991: Flink: Support nested projection

[GitBox <gi...@apache.org>]

hililiwei commented on a change in pull request #3991:
URL: https://github.com/apache/iceberg/pull/3991#discussion_r793274140



##########
File path: flink/v1.14/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSource.java
##########
@@ -90,12 +94,8 @@ private IcebergTableSource(TableLoader loader, TableSchema schema, Map<String, S
 
   @Override
   public void applyProjection(int[][] projectFields) {
-    this.projectedFields = new int[projectFields.length];
-    for (int i = 0; i < projectFields.length; i++) {
-      Preconditions.checkArgument(projectFields[i].length == 1,
-          "Don't support nested projection in iceberg source now.");
-      this.projectedFields[i] = projectFields[i][0];
-    }
+    this.projectedFields = projectFields;
+    this.topProjectedFields = Arrays.stream(projectFields).mapToInt(array -> array[0]).distinct().toArray();

Review comment:
       Use topProjectedFields to extract data from the original file. In this case, the nested fields in the data remain the same.
   Use projectedFields to flatten the original data. In this case, nested fields are extracted.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org