[jira] [Updated] (ARROW-2579) [Python] Appending to streamable table file format doesn't seem to work

["Wes McKinney (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-2579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wes McKinney updated ARROW-2579:
--------------------------------
    Fix Version/s: 0.14.0

> [Python] Appending to streamable table file format doesn't seem to work
> -----------------------------------------------------------------------
>
>                 Key: ARROW-2579
>                 URL: https://issues.apache.org/jira/browse/ARROW-2579
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++
>    Affects Versions: 0.9.0
>            Reporter: Rob Ambalu
>            Priority: Major
>             Fix For: 0.14.0
>
>
> As far as I can tell it looks like appending to a streaming file format isn’t currently supported, is that right?
> RecordBatchStreamWriter always writes the schema up front, and it doesn’t look like a schema is expected mid file ( assuming im doing this append test correctly, this is the error I hit when I try to read back this file into python:
>  Traceback (most recent call last):
>   File "/home/ra7293/rba_arrow_mmap.py", line 9, in <module>
>     table = reader.read_all()
>   File "ipc.pxi", line 302, in pyarrow.lib._RecordBatchReader.read_all
>   File "error.pxi", line 79, in pyarrow.lib.check_status
> pyarrow.lib.ArrowIOError: Message not expected type: record batch, was: 1
>  
> This reader script works fine if I write once / don’t append.
> Seeing as IO interfaces support Append, streaming should support it as well ( if for whatever reason this cant be supported, RecordBatchStreamWriter should throw if configured with an OutputStreamer that is attempting to append )



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)