You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "csgillespie (via GitHub)" <gi...@apache.org> on 2023/10/25 12:37:15 UTC
[I] Inconsistent naming [arrow]
csgillespie opened a new issue, #38456:
URL: https://github.com/apache/arrow/issues/38456
### Describe the bug, including details regarding any error messages, version, and platform.
You've probably thought and discussed this already, but the naming system for the read_ APIs is a little inconsistent.
```
read_parquet # No arrow
read_csv_arrow
open_csv_dataset # Why not open_csv_arrow_dataset
```
I guess you added `_arrow` to differentiate between {readr}?
### Component(s)
R
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Inconsistent naming [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #38456:
URL: https://github.com/apache/arrow/issues/38456#issuecomment-1782360840
The thing is though, they're almost but not directly interchangeable. There are some params in readr::read_csv which arrow doesn't have, and reasons why folks might want both in 1 script. And while you may think it's best practice to namespace read/write function calls (I also do this for clarity in my scripts) it's not something which is necessarily widely accepted/done, and we can't assume everyone else is doing that.
An example of where you might want both is needing to read in a CSV as an Arrow Table, but the CSV not being valid (a missing comma on one row perhaps); readr::read_csv allows you to read in just 1 row of data from any point in the file (arrow doesn't work quite like this), so you can use the error output from arrow to work out which row to get readr to read in so you can inspect it.
I mean, the benefit of arrow is being able to work with Arrow objects when that's needed (larger than memory data, tight control of data types, we have some file reading options that readr does not); while our API could be cleaner, I'm not sure that it follows logically that that fact negates those benefits :)
Maybe the distinction here is more subtle; whereas you could view the file reading functions in e.g. data.table and readr as different frameworks where one is less likely to have a mix of both in a script, I'd consider readr and arrow as much more complementary.
It all depends on what angle you're coming at it from - if you recall, in the posit::conf workshop, the focus was on `open_dataset()` and mych less on `arrow::read_*`, as ultimately, the former is where you get the most benefit of working with arrow.
There are a whole load of API changes which would make sense, but I don't personally have time and they're lower priority to me than bugfixes and docs. However, if you were to open "enhancement request" issues for individual changes, with the reasoning there too (as opposed to asking a question, which prompts a different kind of conversation/interaction), then we can have a wider conversation there and if there's no reason *not* to make that change, you're welcome to submit a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Inconsistent naming [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #38456:
URL: https://github.com/apache/arrow/issues/38456#issuecomment-1782826624
Thinking more about this (a hypothetical API overhaul), I quite like how sparklyr names its file reading/writing functions: https://therinspark.com/data.html#data-file-formats
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Inconsistent naming [arrow]
Posted by "csgillespie (via GitHub)" <gi...@apache.org>.
csgillespie commented on issue #38456:
URL: https://github.com/apache/arrow/issues/38456#issuecomment-1781487093
Naming things is hard!
So, I've tried to think about the "right thing to do" from the start (purely personal POV).
* It doesn't bother me that there's a namespace issue with {readr}. Having the same name would benefit me, as I could switch back ends more easily.It also indicates that `read_csv()` are (sort of) interchangeable
* The same applies to `write` functions. The fact there are `_file`, `_csv_arrow`, `_feather`, etc. is a little odd
* I also think that people should namespace functions for reading/writing data. But that's just me :)
* Going back to the first point, if the function name was the same, I could more easily see the benefit of {arrow}.
So, it would make sense to step back and consider renaming some of the functions. The easiest way could be to create aliases, print a message about the new name, and then deprecate at some point in the distant future.
---
Ideally, the argument should be `as_tibble`, but changing it now is probably annoying.
---
> I've been thinking it might be worth us spelling out the different functions more directly in an article - what do you think?
The docs are excellent. I'm just being annoying. I don't think you need an article.
> I'd be keen to hear more about what's made you think about this - your own usage, comments from clients of yours who are using arrow, etc.
So, I'm writing some blog posts on this stuff. So I understand what's going on. So things like
> as_data_frame: Should the function return a tibble (default) or an Arrow Table?
Automatically made me think, "Is {arrow} being sneaky and loading {tibble}"? The answer is no, but it's the way I learn.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Inconsistent naming [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #38456:
URL: https://github.com/apache/arrow/issues/38456#issuecomment-3547150815
Not working on this, but could see the benefit of `arrow_...` functions if anyone did want to pick this up in future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Inconsistent naming [arrow]
Posted by "github-actions[bot] (via GitHub) via github" <gi...@arrow.apache.org>.
github-actions[bot] commented on issue #38456:
URL: https://github.com/apache/arrow/issues/38456#issuecomment-7007521080
This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this usage question has evolved into a feature request or docs update, please remove the 'Type: usage' label and add the 'Type: enhancement' label instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Inconsistent naming [arrow]
Posted by "thisisnic (via GitHub)" <gi...@apache.org>.
thisisnic commented on issue #38456:
URL: https://github.com/apache/arrow/issues/38456#issuecomment-1781072827
I've swapped this over to a "usage question" label as it's not a bug, but no need to apologise - it's useful having this kind of user feedback!
I agree that the API is a little inconsistent (in more places than this) and yeah, I believe that `read_csv_arrow()` is named like that so we don't have a clash when folks have `readr` loaded at the same time.
When I implemented `open_csv_dataset()`, I prioritised the parameters matching `read_csv_arrow()` rather than the function name, but I see your point.
> as_data_frame should probably be as_tibble
I think this used to return a `data.frame` object, which is the origin of the name, and I'd be hesitant to change it given that a tibble is a kind of data frame *and* not wanting to break folks' existing code.
FWIW I agree with a lot of your points. There are lots of other little bits and pieces like this that could be prettier, but there have always been higher priorities in terms of my own dev time, and I just don't know whether the community would prefer a stable API versus cleaner UX, though I'll count this issue as a vote for the latter!
I've been thinking it might be worth us spelling out the different functions more directly in an article - what do you think?
I'd be keen to hear more about what's made you think about this - your own usage, comments from clients of yours who are using arrow, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [R] Inconsistent naming [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #38456:
URL: https://github.com/apache/arrow/issues/38456#issuecomment-3547062723
This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this usage question has evolved into a feature request or docs update, please remove the 'Type: usage' label and add the 'Type: enhancement' label instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org