[I] [Python] Cast one chunk at a time in ChunkedArray ArrowStream export with requested type [arrow]

["jorisvandenbossche (via GitHub)" <gi...@apache.org>]

jorisvandenbossche opened a new issue, #41064:
URL: https://github.com/apache/arrow/issues/41064

   Follow-up on https://github.com/apache/arrow/pull/40818, see https://github.com/apache/arrow/pull/40818#discussion_r1551271374 for context.
   
   `ChunkedArray.__arrow_c_stream__` currently casts the full ChunkedArray up front when providing a `requested_type`. Instead, we can also catst one chunk at a time when the stream is being read, to reduce memory consumption. 
   
   This is similar to what was implemented for RecordBatchReader in https://github.com/apache/arrow/pull/40070 (`CastingRecordBatchReader`)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Python] Cast one chunk at a time in ChunkedArray ArrowStream export with requested type [arrow]

["paleolimbot (via GitHub)" <gi...@apache.org>]

paleolimbot commented on issue #41064:
URL: https://github.com/apache/arrow/issues/41064#issuecomment-2042572102

   Thanks for opening this...the `CastingRecordBatchReader` was slightly easier because we already had a `RecordBatchReader`. I will try to squeeze this one in (unless somebody else gets here first 🙂 ).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org