You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/10/08 10:36:38 UTC

[GitHub] [arrow] romainfrancois opened a new pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

romainfrancois opened a new pull request #11366:
URL: https://github.com/apache/arrow/pull/11366


   ``` r
   library(arrow, warn.conflicts = FALSE)
   #> See arrow_info() for available features
   
   a <- Array$create(c(1, 2, 3))
   b <- Array$create(a$as_vector())
   
   # different shared_ptr
   a$pointer()
   #> <pointer: 0x7f9617c99040>
   b$pointer()
   #> <pointer: 0x7f9617cf9b50>
   
   # but they both point to the same underlying Array*
   arrow:::test_same_Array(a$pointer(), b$pointer())
   #> [1] TRUE
   ```
   
   <sup>Created on 2021-10-08 by the [reprex package](https://reprex.tidyverse.org) (v2.0.1.9000)</sup>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] github-actions[bot] commented on pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11366:
URL: https://github.com/apache/arrow/pull/11366#issuecomment-938536441






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot edited a comment on pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11366:
URL: https://github.com/apache/arrow/pull/11366#issuecomment-940054239


   Benchmark runs are scheduled for baseline = 39875f1c9581226c38f24ad3515f831b729fe4d6 and contender = edcea6c6b183b991e1906415e73a3f56d328e553. edcea6c6b183b991e1906415e73a3f56d328e553 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/61fcc86e084549f79ecb9e9341eac477...b92270f8194d44fb91c50bf48ca7bab7/)
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/29f6e42790c84a80be2e27afa7de24f2...13837040f60d480db24f51fd37c739df/)
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/3a7380e03efd4b5d88f1761f331663e2...9235a01d99da4ca98c880c83a8f7e56b/)
   Supported benchmarks:
   ursa-i9-9960x: langs = Python, R, JavaScript
   ursa-thinkcentre-m75q: langs = C++, Java
   ec2-t3-xlarge-us-east-2: cloud = True
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot commented on pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #11366:
URL: https://github.com/apache/arrow/pull/11366#issuecomment-940054239


   Benchmark runs are scheduled for baseline = 39875f1c9581226c38f24ad3515f831b729fe4d6 and contender = edcea6c6b183b991e1906415e73a3f56d328e553. edcea6c6b183b991e1906415e73a3f56d328e553 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Scheduled] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/61fcc86e084549f79ecb9e9341eac477...b92270f8194d44fb91c50bf48ca7bab7/)
   [Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/29f6e42790c84a80be2e27afa7de24f2...13837040f60d480db24f51fd37c739df/)
   [Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/3a7380e03efd4b5d88f1761f331663e2...9235a01d99da4ca98c880c83a8f7e56b/)
   Supported benchmarks:
   ursa-i9-9960x: langs = Python, R, JavaScript
   ursa-thinkcentre-m75q: langs = C++, Java
   ec2-t3-xlarge-us-east-2: cloud = True
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] nealrichardson commented on a change in pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

Posted by GitBox <gi...@apache.org>.
nealrichardson commented on a change in pull request #11366:
URL: https://github.com/apache/arrow/pull/11366#discussion_r725000828



##########
File path: r/src/altrep.cpp
##########
@@ -629,6 +629,24 @@ SEXP MakeAltrepVector(const std::shared_ptr<ChunkedArray>& chunked_array) {
   return R_NilValue;
 }
 
+// borrowed from
+// https://github.com/r-devel/r-svn/blob/6418faeb6f5d87d3d9b92b8978773bc3856b4b6f/src/main/altrep.c#L37
+#define ALTREP_CLASS_SERIALIZED_CLASS(x) ATTRIB(x)
+#define ALTREP_SERIALIZED_CLASS_PKGSYM(x) CADR(x)
+
+std::shared_ptr<Array> vec_to_arrow_altrep_bypass(SEXP x) {
+  if (ALTREP(x)) {
+    SEXP info = ALTREP_CLASS_SERIALIZED_CLASS(ALTREP_CLASS(x));
+    SEXP pkg = ALTREP_SERIALIZED_CLASS_PKGSYM(info);

Review comment:
       We have to watch out not to "borrow" GPL code; does this work?
   
   ```suggestion
   // For context, see
   // https://github.com/r-devel/r-svn/blob/6418faeb6f5d87d3d9b92b8978773bc3856b4b6f/src/main/altrep.c#L37
   #define ALTREP_CLASS_SERIALIZED_CLASS(x) ATTRIB(x)
   #define ALTREP_SERIALIZED_CLASS_PKGSYM(x) CADR(x)
   
   std::shared_ptr<Array> vec_to_arrow_altrep_bypass(SEXP x) {
     if (ALTREP(x)) {
       SEXP info = ATTRIB(ALTREP_CLASS(x));
       SEXP pkg = CADR(info);
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot edited a comment on pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11366:
URL: https://github.com/apache/arrow/pull/11366#issuecomment-940054239


   Benchmark runs are scheduled for baseline = 39875f1c9581226c38f24ad3515f831b729fe4d6 and contender = edcea6c6b183b991e1906415e73a3f56d328e553. edcea6c6b183b991e1906415e73a3f56d328e553 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/61fcc86e084549f79ecb9e9341eac477...b92270f8194d44fb91c50bf48ca7bab7/)
   [Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/29f6e42790c84a80be2e27afa7de24f2...13837040f60d480db24f51fd37c739df/)
   [Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/3a7380e03efd4b5d88f1761f331663e2...9235a01d99da4ca98c880c83a8f7e56b/)
   Supported benchmarks:
   ursa-i9-9960x: langs = Python, R, JavaScript
   ursa-thinkcentre-m75q: langs = C++, Java
   ec2-t3-xlarge-us-east-2: cloud = True
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] nealrichardson commented on pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

Posted by GitBox <gi...@apache.org>.
nealrichardson commented on pull request #11366:
URL: https://github.com/apache/arrow/pull/11366#issuecomment-940007958


   @romainfrancois which of the two PRs do you want to merge first, this one or the ChunkedArray one? Either way there will be extra work to do on the second PR extending this change to support ChunkedArrays.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] romainfrancois commented on a change in pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

Posted by GitBox <gi...@apache.org>.
romainfrancois commented on a change in pull request #11366:
URL: https://github.com/apache/arrow/pull/11366#discussion_r725456816



##########
File path: r/src/altrep.cpp
##########
@@ -629,6 +629,24 @@ SEXP MakeAltrepVector(const std::shared_ptr<ChunkedArray>& chunked_array) {
   return R_NilValue;
 }
 
+// borrowed from
+// https://github.com/r-devel/r-svn/blob/6418faeb6f5d87d3d9b92b8978773bc3856b4b6f/src/main/altrep.c#L37
+#define ALTREP_CLASS_SERIALIZED_CLASS(x) ATTRIB(x)
+#define ALTREP_SERIALIZED_CLASS_PKGSYM(x) CADR(x)
+
+std::shared_ptr<Array> vec_to_arrow_altrep_bypass(SEXP x) {
+  if (ALTREP(x)) {
+    SEXP info = ALTREP_CLASS_SERIALIZED_CLASS(ALTREP_CLASS(x));
+    SEXP pkg = ALTREP_SERIALIZED_CLASS_PKGSYM(info);

Review comment:
       Sure. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot edited a comment on pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11366:
URL: https://github.com/apache/arrow/pull/11366#issuecomment-940054239


   Benchmark runs are scheduled for baseline = 39875f1c9581226c38f24ad3515f831b729fe4d6 and contender = edcea6c6b183b991e1906415e73a3f56d328e553. edcea6c6b183b991e1906415e73a3f56d328e553 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/61fcc86e084549f79ecb9e9341eac477...b92270f8194d44fb91c50bf48ca7bab7/)
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/29f6e42790c84a80be2e27afa7de24f2...13837040f60d480db24f51fd37c739df/)
   [Scheduled] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/3a7380e03efd4b5d88f1761f331663e2...9235a01d99da4ca98c880c83a8f7e56b/)
   Supported benchmarks:
   ursa-i9-9960x: langs = Python, R, JavaScript
   ursa-thinkcentre-m75q: langs = C++, Java
   ec2-t3-xlarge-us-east-2: cloud = True
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] romainfrancois commented on pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

Posted by GitBox <gi...@apache.org>.
romainfrancois commented on pull request #11366:
URL: https://github.com/apache/arrow/pull/11366#issuecomment-940034489


   @nealrichardson this one please :-) I'll rebase right after


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] romainfrancois commented on a change in pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

Posted by GitBox <gi...@apache.org>.
romainfrancois commented on a change in pull request #11366:
URL: https://github.com/apache/arrow/pull/11366#discussion_r725942531



##########
File path: r/src/altrep.cpp
##########
@@ -629,6 +629,24 @@ SEXP MakeAltrepVector(const std::shared_ptr<ChunkedArray>& chunked_array) {
   return R_NilValue;
 }
 
+// For context, see
+// https://github.com/r-devel/r-svn/blob/6418faeb6f5d87d3d9b92b8978773bc3856b4b6f/src/main/altrep.c#L37
+#define ALTREP_CLASS_SERIALIZED_CLASS(x) ATTRIB(x)
+#define ALTREP_SERIALIZED_CLASS_PKGSYM(x) CADR(x)
+
+std::shared_ptr<Array> vec_to_arrow_altrep_bypass(SEXP x) {
+  if (ALTREP(x)) {
+    SEXP info = ATTRIB(ALTREP_CLASS(x));
+    SEXP pkg = CADR(info);

Review comment:
       and this should be `ALTREP_SERIALIZED_CLASS_PKGSYM`

##########
File path: r/src/altrep.cpp
##########
@@ -629,6 +629,24 @@ SEXP MakeAltrepVector(const std::shared_ptr<ChunkedArray>& chunked_array) {
   return R_NilValue;
 }
 
+// For context, see
+// https://github.com/r-devel/r-svn/blob/6418faeb6f5d87d3d9b92b8978773bc3856b4b6f/src/main/altrep.c#L37
+#define ALTREP_CLASS_SERIALIZED_CLASS(x) ATTRIB(x)
+#define ALTREP_SERIALIZED_CLASS_PKGSYM(x) CADR(x)
+
+std::shared_ptr<Array> vec_to_arrow_altrep_bypass(SEXP x) {
+  if (ALTREP(x)) {
+    SEXP info = ATTRIB(ALTREP_CLASS(x));

Review comment:
       If we do define the macros, then we should use them, this should be `ALTREP_CLASS_SERIALIZED_CLASS` instead pf `ATTRIB`

##########
File path: r/src/altrep.cpp
##########
@@ -629,6 +629,24 @@ SEXP MakeAltrepVector(const std::shared_ptr<ChunkedArray>& chunked_array) {
   return R_NilValue;
 }
 
+// For context, see
+// https://github.com/r-devel/r-svn/blob/6418faeb6f5d87d3d9b92b8978773bc3856b4b6f/src/main/altrep.c#L37
+#define ALTREP_CLASS_SERIALIZED_CLASS(x) ATTRIB(x)
+#define ALTREP_SERIALIZED_CLASS_PKGSYM(x) CADR(x)

Review comment:
       The macros should perhaps go to arrow_cpp11.h, e.g. this pull request would use them: https://github.com/apache/arrow/pull/11369 too. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] nealrichardson closed pull request #11366: ARROW-14259: [R] converting from R vector to Array when the R vector is altrep

Posted by GitBox <gi...@apache.org>.
nealrichardson closed pull request #11366:
URL: https://github.com/apache/arrow/pull/11366


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org