[DISCUSS][C++] C++ API as a user-facing API

[Will Jones <wi...@gmail.com>]

In a discussion about new additions to C++ docs, someone had a question:
Should we even be documenting this?

Long-time contributors to Arrow C++ noted that many parts were written
without the intention that those APIs would not be used directly. Instead,
they were intended as an implementation detail of the high-level bindings
(such as Python, R, Ruby, and so on). This made sense early in the project,
as I understand it, because the high level bindings had the most potential
users and more libraries to collaborate with (for example, PyArrow +
Pandas).

However, we now have several projects that use the C++ libraries, so this
older outlook on C++ as mostly being an internal library no longer makes
sense.

Which parts of the C++ libraries are considered public? And are we
effectively delineating that for our users?

Re: [DISCUSS][C++] C++ API as a user-facing API

[Antoine Pitrou <an...@python.org>]

On Thu, 29 Sep 2022 11:19:44 -0700
Will Jones <wi...@gmail.com> wrote:
> In a discussion about new additions to C++ docs, someone had a question:
> Should we even be documenting this?
> 
> Long-time contributors to Arrow C++ noted that many parts were written
> without the intention that those APIs would not be used directly.

Well, the parts that were not written for outside use are typically not
documented :-) I suppose that answers the question...

> Which parts of the C++ libraries are considered public? And are we
> effectively delineating that for our users?

The publicly documented parts, effectively. Now *some* non-documented
parts may also be considered public, if we simply forgot documenting
them (which is bad, but happens unfortunately).

It's also true that our header files expose some APIs that are not
means to be public (and normally not documented). That's difficult to
avoid with C++. Some of them are explicitly marked "internal" (for
example by virtue of being in the "arrow::internal" namespace), but we
haven't been very consistent in this (*).

(*) Making things more consistent would be welcome. It requires some
work and careful thinking.

Regards

Antoine.

Re: [DISCUSS][C++] C++ API as a user-facing API

[Micah Kornfield <em...@gmail.com>]

I think the convention we have been using is that headers included directly
in the "api.h" headers were considered public, those that aren't were
considered a gray area.

On Thu, Sep 29, 2022 at 11:51 AM Aldrin <ak...@ucsc.edu.invalid> wrote:

> > many parts were written without the intention that those APIs would not
> be used directly
>
> From the proceeding sentence, I assume you mean that many parts were not
> intended for direct use?
>
>
> I think my personal take is that I consider all parts of the C++ libraries
> to be public. Otherwise, I have
> trouble imagining how a C/C++ project benefits from Arrow or how to use it
> before "the perfect API"
> is available.
>
> In general, I agree that higher-level APIs should be used where possible,
> but when that can't be done it
> seems clear that one would have to use the lower-level APIs, especially
> while Arrow is still growing. In
> addition, if I want to contribute to Arrow, I would also need to interact
> with the lower-level API at some
> point and I wouldn't necessarily want to start with trying to contribute
> code before using it in my own
> project(s).
>
>
> Aldrin Montana
> Computer Science PhD Student
> UC Santa Cruz
>
>
> On Thu, Sep 29, 2022 at 11:21 AM Will Jones <wi...@gmail.com>
> wrote:
>
> > In a discussion about new additions to C++ docs, someone had a question:
> > Should we even be documenting this?
> >
> > Long-time contributors to Arrow C++ noted that many parts were written
> > without the intention that those APIs would not be used directly.
> Instead,
> > they were intended as an implementation detail of the high-level bindings
> > (such as Python, R, Ruby, and so on). This made sense early in the
> project,
> > as I understand it, because the high level bindings had the most
> potential
> > users and more libraries to collaborate with (for example, PyArrow +
> > Pandas).
> >
> > However, we now have several projects that use the C++ libraries, so this
> > older outlook on C++ as mostly being an internal library no longer makes
> > sense.
> >
> > Which parts of the C++ libraries are considered public? And are we
> > effectively delineating that for our users?
> >
>

Re: [DISCUSS][C++] C++ API as a user-facing API

[Aldrin <ak...@ucsc.edu.INVALID>]

> many parts were written without the intention that those APIs would not
be used directly

From the proceeding sentence, I assume you mean that many parts were not
intended for direct use?


I think my personal take is that I consider all parts of the C++ libraries
to be public. Otherwise, I have
trouble imagining how a C/C++ project benefits from Arrow or how to use it
before "the perfect API"
is available.

In general, I agree that higher-level APIs should be used where possible,
but when that can't be done it
seems clear that one would have to use the lower-level APIs, especially
while Arrow is still growing. In
addition, if I want to contribute to Arrow, I would also need to interact
with the lower-level API at some
point and I wouldn't necessarily want to start with trying to contribute
code before using it in my own
project(s).


Aldrin Montana
Computer Science PhD Student
UC Santa Cruz


On Thu, Sep 29, 2022 at 11:21 AM Will Jones <wi...@gmail.com> wrote:

> In a discussion about new additions to C++ docs, someone had a question:
> Should we even be documenting this?
>
> Long-time contributors to Arrow C++ noted that many parts were written
> without the intention that those APIs would not be used directly. Instead,
> they were intended as an implementation detail of the high-level bindings
> (such as Python, R, Ruby, and so on). This made sense early in the project,
> as I understand it, because the high level bindings had the most potential
> users and more libraries to collaborate with (for example, PyArrow +
> Pandas).
>
> However, we now have several projects that use the C++ libraries, so this
> older outlook on C++ as mostly being an internal library no longer makes
> sense.
>
> Which parts of the C++ libraries are considered public? And are we
> effectively delineating that for our users?
>