You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Ian Cook (Jira)" <ji...@apache.org> on 2021/10/31 16:47:00 UTC

[jira] [Created] (ARROW-14528) [R] Add option to attempt 32-bit integer type inference in CSV reader

Ian Cook created ARROW-14528:
--------------------------------

             Summary: [R] Add option to attempt 32-bit integer type inference in CSV reader
                 Key: ARROW-14528
                 URL: https://issues.apache.org/jira/browse/ARROW-14528
             Project: Apache Arrow
          Issue Type: Improvement
          Components: R
            Reporter: Ian Cook
             Fix For: 7.0.0


Expose ARROW-13028 to the R bindings so that users can choose to have {{read_csv_arrow(as_data_frame = FALSE)}} and {{open_dataset(format = "csv)}} infer 32-bit integer fields instead of always inferring 64-bit integer fields for all integers (the current behavior).

Note that there is an existing option in the R bindings that controls something similar: {{arrow.int64_downcast}}. See ARROW-10093 for details. I think we can _not_ reuse this option to control the CSV reader, behavior because (a) users might want to control these behaviors separately, and (b) the default value of {{arrow.int64_downcast}} is {{TRUE}} which does not align with the existing behavior of the CSV reader (always inferring 64-bit integer fields, i.e. _not_ downcasting) and we probably want to retain that as the default behavior. So we will want to add a new argument or a new option to control this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)