[GitHub] [arrow] elithrar commented on issue #34363: [Python] Writing to cloudflare r2 fails for mutlipart upload

["elithrar (via GitHub)" <gi...@apache.org>]

elithrar commented on issue #34363:
URL: https://github.com/apache/arrow/issues/34363#issuecomment-1544274719

   (For context: I work at Cloudflare)
   
   Here's the latest:
   
   - R2 _currently_ requires that: all parts are of equal size, a maximum of 10K parts, and per-part limit of 5GB, and a total file size of 5TB.
   - DuckDB updated their custom http-fs implementation to work with this 
   - We don't yet have any concrete plans to relax this in the medium term: s3fs would need to support the ability (or a client using it) to work within R2's requirements
   
   This doesn't mean we're not open to relaxing this, but it's a non-trivial change.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org