You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Anthony Abate (Jira)" <ji...@apache.org> on 2019/11/13 16:10:00 UTC
[jira] [Comment Edited] (ARROW-7156) [R] [C++] get_batch - fails
for large batches
[ https://issues.apache.org/jira/browse/ARROW-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973478#comment-16973478 ]
Anthony Abate edited comment on ARROW-7156 at 11/13/19 4:09 PM:
----------------------------------------------------------------
this is still a problem in 15.1.1 but the failure is different
{code:java}
rbn <- data_rbfr$get_batch(x) works!
df <- as.data.frame(rbn) - Crashes R Studio!
{code}
was (Author: abbot):
this is still a problem in 15.1.1 but the failure is slightly different
rbn <- data_rbfr$get_batch(x) works!
df <- as.data.frame(rbn) - Crashes R Studio!
> [R] [C++] get_batch - fails for large batches
> ---------------------------------------------
>
> Key: ARROW-7156
> URL: https://issues.apache.org/jira/browse/ARROW-7156
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, R
> Affects Versions: 0.14.1, 0.15.1
> Reporter: Anthony Abate
> Priority: Major
>
> I have a 30 gig arrow file with 100 batches. the largest batch in the file causes get batch to fail - All other batches load fine.
> I dont know if this is fixed in 15.x because 15.x fails to load the file (another bug)
>
> {color:#0000FF}> {color}{color:#0000FF} rbn <- data_rbfr$get_batch(4){color}{color:#c5060b}Error in ipc___RecordBatchFileReader__ReadRecordBatch(self, i) :
> Invalid: negative malloc size{color}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)