[GitHub] [iceberg] gustavoatt commented on a diff in pull request #7352: Fix IcebergGenerics::read to read metadata tables

["gustavoatt (via GitHub)" <gi...@apache.org>]

gustavoatt commented on code in PR #7352:
URL: https://github.com/apache/iceberg/pull/7352#discussion_r1256437201


##########
core/src/main/java/org/apache/iceberg/StaticDataTask.java:
##########
@@ -71,8 +71,10 @@ public List<DeleteFile> deletes() {
 
   @Override
   public CloseableIterable<StructLike> rows() {
-    StructProjection projection = StructProjection.create(tableSchema, projectedSchema);
-    Iterable<StructLike> projectedRows = Iterables.transform(Arrays.asList(rows), projection::wrap);
+    Iterable<StructLike> projectedRows =

Review Comment:
   @szehon-ho I believe it work in Spark, because the Spark API [iterates one row at a time](https://github.com/apache/iceberg/blob/master/spark/v3.1/spark/src/main/java/org/apache/iceberg/spark/source/BaseDataReader.java#L103), using [this iterator](https://github.com/apache/iceberg/blob/master/spark/v3.1/spark/src/main/java/org/apache/iceberg/spark/source/RowDataReader.java#L87).
   
   I tested changing `reuseContainer` in `ManifestReader`, but it does not work. I believe the problem is with always using the same `StructProjection`. One alternative I see is modifying the `DataTask` interface and add a `reuseContainer` method there that that Generic can then set to false but it will be set to true everywhere else. What do you think?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org