You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/07/23 17:21:00 UTC

[jira] [Updated] (ARROW-13441) [CSV] Streaming reader conversion should skip empty blocks

     [ https://issues.apache.org/jira/browse/ARROW-13441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated ARROW-13441:
-----------------------------------
    Labels: pull-request-available  (was: )

> [CSV] Streaming reader conversion should skip empty blocks
> ----------------------------------------------------------
>
>                 Key: ARROW-13441
>                 URL: https://issues.apache.org/jira/browse/ARROW-13441
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++
>    Affects Versions: 5.0.0
>            Reporter: Nate Clark
>            Assignee: Nate Clark
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> The csv streaming reader hardens the schema after the first block is processed. However if the first block does not have any rows then the schema will be hardened with all columns as NAType. This is made worse with the skip_rows_after_names options which will create empty batches until the specified number of rows are skipped.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)