Re: [PR] Bug Fix: Position Deletes + row_filter yields less data when the DataFile is large [iceberg-python]

["kevinjqliu (via GitHub)" <gi...@apache.org>]

kevinjqliu commented on PR #1141:
URL: https://github.com/apache/iceberg-python/pull/1141#issuecomment-2350674380

   > In cases where a single RecordBatch -> a Table -> multiple RecordBatches because of how Arrow automatically chunks a Table into multiple RecordBatches, we would lose the remaining RecordBatches in the returned output. Essentially, there's no guarantee that a single RecordBatch will remain a single one in a round trip conversion to Arrow Table, and back.
   
   This is specifically about this piece of code? 
   ```
        batch = arrow_table.to_batches()[0] 
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org