[GitHub] [arrow] nealrichardson commented on issue #8732: arrow::write_feather error: Capacity error: array cannot contain more than 2147483646 bytes

[GitBox <gi...@apache.org>]

nealrichardson commented on issue #8732:
URL: https://github.com/apache/arrow/issues/8732#issuecomment-732297194


   Thanks for the report, and that's odd about your ASF jira account--maybe password reset would fix it? 
   
   The issue is that the R package isn't chunking the data.frame when converting to Arrow--that is, it's not a limitation of Arrow/Feather format but just of the R package as it stands now. We're working on it (https://issues.apache.org/jira/browse/ARROW-9293 among others) and hope to have some improvements in the next release.
   
   If you're interested, you may be able to work around this now by doing the chunking yourself, something like
   
   ```r
   write_feather(Table$create(d[1:2e8, , drop = FALSE], d[2e8 + 1:2e8, , drop = FALSE]), "data.feather")
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org