You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "Wes McKinney (JIRA)" <ji...@apache.org> on 2017/07/16 03:30:00 UTC

[jira] [Commented] (ARROW-989) [Python] Write pyarrow.Table to FileWriter or StreamWriter

    [ https://issues.apache.org/jira/browse/ARROW-989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088792#comment-16088792 ] 

Wes McKinney commented on ARROW-989:
------------------------------------

I think what I'm going to do is implement a stream reader for table that handles different chunking across columns, producing a sequence of clean record batches. 

> [Python] Write pyarrow.Table to FileWriter or StreamWriter
> ----------------------------------------------------------
>
>                 Key: ARROW-989
>                 URL: https://issues.apache.org/jira/browse/ARROW-989
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: Python
>            Reporter: Wes McKinney
>             Fix For: 0.5.0
>
>
> As part of this, we need to be able to get an iterator of record batches from a table. We may want to write this iteration logic in C++ as it will be generally useful. The chunking between columns may be different, so there is some amount of complexity there



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)