[GitHub] [iceberg] guilload commented on pull request #1267: Single jar for input formats

[GitBox <gi...@apache.org>]

guilload commented on pull request #1267:
URL: https://github.com/apache/iceberg/pull/1267#issuecomment-665845026


   I run MR in local mode whereas this seems to be running in distributed mode with YARN. I'll have to dig deeper but my guess is that `table` is null because in that case the calls to `getSplits` and then `getRecordReader` happens in two different processes.
   
   You mind trying this [branch](https://github.com/guilload/iceberg/tree/guilload--carry-table-schema-and-projection-within-iceberg-split)? I've added a `schema` and a `projection` attribute to `IcebergSplit`, this way we can remove the "forwarding" logic in the input format. @rdsr, what do you think of this approach? One downside is the increase in size of the serialized splits.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org