You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "nevi_me (JIRA)" <ji...@apache.org> on 2018/11/21 21:51:00 UTC

[jira] [Commented] (ARROW-3839) [Rust] Add ability to infer schema in CSV reader

    [ https://issues.apache.org/jira/browse/ARROW-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16695294#comment-16695294 ] 

nevi_me commented on ARROW-3839:
--------------------------------

Hi [~andygrove], I've created a prototype of schema inference in [https://github.com/nevi-me/arrow/commit/e9875c482056a697177222a5fdd70371d7549ed5|[https://github.com/nevi-me/arrow/commit/e9875c482056a697177222a5fdd70371d7549ed5],] where I use regex on a sample of records to infer the schema of the csv.

The code isn't clean, and I was more creating it as a prototype, but may you(or anyone else) please have a brief look if you can. I couldn't figure out how to read the file once to infer the schema, then use the same reader to create an iterator of records. I'll try figure it out when I get some time.

Thanks

> [Rust] Add ability to infer schema in CSV reader
> ------------------------------------------------
>
>                 Key: ARROW-3839
>                 URL: https://issues.apache.org/jira/browse/ARROW-3839
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Rust
>    Affects Versions: 0.11.1
>            Reporter: Andy Grove
>            Priority: Minor
>
> A CSV reader is being added in ARROW-3726 and it currently requires an explicit schema to be provided.
> It would be nice to have an option where the schema can be inferred automatically.
> The user should be able to specify some defaults, such as date/time formats.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)