You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/10/25 16:23:53 UTC
[GitHub] [arrow-cookbook] thisisnic opened a new issue #92: [R] Add content on Tables vs. Datasets
thisisnic opened a new issue #92:
URL: https://github.com/apache/arrow-cookbook/issues/92
There are fundamental differences between working with Tables/InMemoryDatasets and file-based datasets. There should be content about working with datasets which has an intro covering those differences, and recipes for working with them.
Possible topics:
* Adding new data to a dataset
* Link to reading/writing sections
* Mention of reading only reading in relevant data
This content potentially could be part of other chapters, but is definitely needed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-cookbook] thisisnic edited a comment on issue #92: [R] Add content on Tables vs. Datasets
Posted by GitBox <gi...@apache.org>.
thisisnic edited a comment on issue #92:
URL: https://github.com/apache/arrow-cookbook/issues/92#issuecomment-1055869736
Topics:
- different formats (csv/feather/parquet)
- partitioning (via group_by or just supplying column names)
- customising filenames via basename_template param
- hive_style vs bare values
- overwrite existing data via existing_data_behavior param
- fine control over file structure via max_partitions/max_open_files/max_rows_per_file/min_rows_per_group/max_rows_per_group [note: this functionality is not in version 7.0.0 of the R package]
- CSV datasets - how to read ones in with or without headers
- CSV datasets - similarities and differences compared to read_csv_arrow
- CSV datasets - working with schemas
- converting datasets from one format to another without loading it all into memory
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-cookbook] stephhazlitt commented on issue #92: [R] Add content on Tables vs. Datasets
Posted by GitBox <gi...@apache.org>.
stephhazlitt commented on issue #92:
URL: https://github.com/apache/arrow-cookbook/issues/92#issuecomment-1055944724
– working with a multi-source dataset
– it might be useful to have an example join workflow using `Dataset`
– a link to existing content or repeated content explaining the different behaviour with `Dataset` if/when a user supplies an unmapped function (an error instead of the warning+pull into R with `Table`)
– a link to and/or some content on optimal number of partitions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-cookbook] thisisnic commented on issue #92: [R] Add content on Tables vs. Datasets
Posted by GitBox <gi...@apache.org>.
thisisnic commented on issue #92:
URL: https://github.com/apache/arrow-cookbook/issues/92#issuecomment-1055869736
Topics:
- different formats (csv/feather/parquet)
- partitioning (via group_by or just supplying column names)
- customising filenames via basename_template param
- hive_style vs bare values
- overwrite existing data via existing_data_behavior param
- fine control over file structure via max_partitions/max_open_files/max_rows_per_file/min_rows_per_group/max_rows_per_group
- CSV datasets - how to read ones in with or without headers
- CSV datasets - similarities and differences compared to read_csv_arrow
- CSV datasets - working with schemas
- converting datasets from one format to another without loading it all into memory
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org