You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by Aldrin <ak...@ucsc.edu> on 2021/06/05 00:26:03 UTC
Understanding R and C++ using cpp11
Hello!
I am trying to wrap some C++ code I have that uses arrow with R to enable
users to continue working in R. Unfortunately, I get a compilation error of
`arrow does not name a type` when it gets to the return type of my function
which is
`arrow::Result<std::shared_ptr<arrow::ipc::RecordBatchStreamReader>>`. I
posted in RStudio in case this is purely a cpp11 library issue happening (
https://community.rstudio.com/t/cpp11-does-not-name-a-type/106740).
However, I am trying to get this working with arrow data types. So, I just
wanted to check with this list the following things:
- If I import the arrow library in R before trying to source C++ code
that returns an arrow data type, am I able to use the object as if I
constructed it in R?
- If I want to have an R package that I can import like the arrow
package (e.g. `library(arrow)`), what is the best way to either:
- import the arrow package from my package
- or, specify the arrow package as a dependency?
I apologize since I know that to some extent these are primarily R
questions and not arrow questions, but I am just hoping that since the
arrow R package seems complex, it makes sense to ask these questions here
as well.
Thank you very much for any help!
Aldrin Montana
Computer Science PhD Student
UC Santa Cruz
Re: Understanding R and C++ using cpp11
Posted by Aldrin <ak...@ucsc.edu>.
An update:
I have organized my code like an R package (after reading more about
NAMESPACE file), and I used `devtools::document()` to generate source files
(per cpp11 docs <https://cpp11.r-lib.org/articles/cpp11.html#package>).
Unfortunately, the generated source does not propagate my include
directives, which seems to make clear why using `cpp_source` does not work
for my code. Here is a code excerpt of what is generated:
// skytethr.cpp
arrow::Result<BatchStreamRdrPtr> R_GetReaderForKey(cpp11::r_string
table_key);
extern "C" SEXP _skytethr_R_GetReaderForKey(SEXP table_key) {
BEGIN_CPP11
return
cpp11::as_sexp(R_GetReaderForKey(cpp11::as_cpp<cpp11::decay_t<cpp11::r_string>>(table_key)));
END_CPP11
}
I manually added the include of arrow to the generated source to see if
that is my only problem, but I get many errors related to the return value
being wrapped in `cpp11::as_sexp`.
To continue my proof-of-concept, I can change my C++ code to return a
simpler type (just an arrow table, maybe), but I think I ultimately will
need to figure out how to return a type that is defined by the `arrow`
package. Other than doing a short-term copy-paste of headers, is there a
recommended path forward on how to return a type that R knows about from
the `arrow` package? My first thought is that maybe I can import some
variables from the arrow namespace in my namespace file
<https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Specifying-imports-and-exports>,
maybe in the essence of this example
<https://cran.r-project.org/doc/manuals/r-release/R-exts.html#An-example>?
Thanks for any help or feedback!
Aldrin Montana
Computer Science PhD Student
UC Santa Cruz
On Tue, Jun 8, 2021 at 8:54 AM Aldrin <ak...@ucsc.edu> wrote:
> Thank you Neal! I'll check those out and I appreciate the tips!
>
> Hi Antoine, I do include those. That's a reason I am especially confused,
> since I don't get any issue on including them. Although, interestingly, the
> error " 'arrow' does not name a type " is there whether or not I include
> the arrow headers. If they are not included though, compilation stops at
> that step.
>
> Here's a gist to make it easier to see the difference. I've also included
> more code just for completeness. If there's anything else I can include
> that would be helpful, please let me know!
>
> https://gist.github.com/drin/d9f4400670eae69f5c367b37ca8c1b24
>
>
>
> Aldrin Montana
> Computer Science PhD Student
> UC Santa Cruz
>
>
> On Tue, Jun 8, 2021 at 12:14 AM Antoine Pitrou <an...@python.org> wrote:
>
>> On Fri, 4 Jun 2021 17:26:03 -0700
>> Aldrin <ak...@ucsc.edu> wrote:
>> > Hello!
>> >
>> > I am trying to wrap some C++ code I have that uses arrow with R to
>> enable
>> > users to continue working in R. Unfortunately, I get a compilation
>> error of
>> > `arrow does not name a type` when it gets to the return type of my
>> function
>> > which is
>> > `arrow::Result<std::shared_ptr<arrow::ipc::RecordBatchStreamReader>>`. I
>> > posted in RStudio in case this is purely a cpp11 library issue
>> happening (
>> > https://community.rstudio.com/t/cpp11-does-not-name-a-type/106740).
>>
>> Did you include the Arrow headers? For example `#include "arrow/api.h"`?
>>
>>
>>
>>
Re: Understanding R and C++ using cpp11
Posted by Aldrin <ak...@ucsc.edu>.
Thank you Neal! I'll check those out and I appreciate the tips!
Hi Antoine, I do include those. That's a reason I am especially confused,
since I don't get any issue on including them. Although, interestingly, the
error " 'arrow' does not name a type " is there whether or not I include
the arrow headers. If they are not included though, compilation stops at
that step.
Here's a gist to make it easier to see the difference. I've also included
more code just for completeness. If there's anything else I can include
that would be helpful, please let me know!
https://gist.github.com/drin/d9f4400670eae69f5c367b37ca8c1b24
Aldrin Montana
Computer Science PhD Student
UC Santa Cruz
On Tue, Jun 8, 2021 at 12:14 AM Antoine Pitrou <an...@python.org> wrote:
> On Fri, 4 Jun 2021 17:26:03 -0700
> Aldrin <ak...@ucsc.edu> wrote:
> > Hello!
> >
> > I am trying to wrap some C++ code I have that uses arrow with R to enable
> > users to continue working in R. Unfortunately, I get a compilation error
> of
> > `arrow does not name a type` when it gets to the return type of my
> function
> > which is
> > `arrow::Result<std::shared_ptr<arrow::ipc::RecordBatchStreamReader>>`. I
> > posted in RStudio in case this is purely a cpp11 library issue happening
> (
> > https://community.rstudio.com/t/cpp11-does-not-name-a-type/106740).
>
> Did you include the Arrow headers? For example `#include "arrow/api.h"`?
>
>
>
>
Re: Understanding R and C++ using cpp11
Posted by Antoine Pitrou <an...@python.org>.
On Fri, 4 Jun 2021 17:26:03 -0700
Aldrin <ak...@ucsc.edu> wrote:
> Hello!
>
> I am trying to wrap some C++ code I have that uses arrow with R to enable
> users to continue working in R. Unfortunately, I get a compilation error of
> `arrow does not name a type` when it gets to the return type of my function
> which is
> `arrow::Result<std::shared_ptr<arrow::ipc::RecordBatchStreamReader>>`. I
> posted in RStudio in case this is purely a cpp11 library issue happening (
> https://community.rstudio.com/t/cpp11-does-not-name-a-type/106740).
Did you include the Arrow headers? For example `#include "arrow/api.h"`?
Re: Understanding R and C++ using cpp11
Posted by Neal Richardson <ne...@gmail.com>.
This is not my area of expertise, but to answer your second question, if
you're trying to depend on {arrow} in C++ in an R package, you'll need to
list arrow in LinkingTo in your DESCRIPTION, and you may need to add
useDynLib in your NAMESPACE. See
https://adv-r.hadley.nz/rcpp.html#rcpp-package, and also search those two
terms in https://cran.r-project.org/doc/manuals/r-release/R-exts.html for
more details. It's possible that getting this right will solve your
compilation problem, but I'm not sure--I'm not aware of any other R
packages that depend on {arrow} in the C++ layer.
Neal
On Fri, Jun 4, 2021 at 5:26 PM Aldrin <ak...@ucsc.edu> wrote:
> Hello!
>
> I am trying to wrap some C++ code I have that uses arrow with R to enable
> users to continue working in R. Unfortunately, I get a compilation error of
> `arrow does not name a type` when it gets to the return type of my function
> which is
> `arrow::Result<std::shared_ptr<arrow::ipc::RecordBatchStreamReader>>`. I
> posted in RStudio in case this is purely a cpp11 library issue happening (
> https://community.rstudio.com/t/cpp11-does-not-name-a-type/106740).
>
> However, I am trying to get this working with arrow data types. So, I just
> wanted to check with this list the following things:
>
> - If I import the arrow library in R before trying to source C++ code
> that returns an arrow data type, am I able to use the object as if I
> constructed it in R?
> - If I want to have an R package that I can import like the arrow
> package (e.g. `library(arrow)`), what is the best way to either:
> - import the arrow package from my package
> - or, specify the arrow package as a dependency?
>
> I apologize since I know that to some extent these are primarily R
> questions and not arrow questions, but I am just hoping that since the
> arrow R package seems complex, it makes sense to ask these questions here
> as well.
>
> Thank you very much for any help!
>
> Aldrin Montana
> Computer Science PhD Student
> UC Santa Cruz
>