You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "felipecrv (via GitHub)" <gi...@apache.org> on 2023/06/26 20:51:25 UTC

[GitHub] [arrow] felipecrv opened a new pull request, #36310: GH-36309: [C++] Add ability to cast between scalars of list-like types

felipecrv opened a new pull request, #36310:
URL: https://github.com/apache/arrow/pull/36310

   ### Rationale for this change
   
   Makes it easier to work generically with different list types.
   
   ### What changes are included in this PR?
   
   More cast implementations between list-like types of scalars.
   
   ### Are these changes tested?
   
   Yes. With new unit tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] felipecrv commented on a diff in pull request #36310: GH-36309: [C++] Add ability to cast between scalars of list-like types

Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #36310:
URL: https://github.com/apache/arrow/pull/36310#discussion_r1244104763


##########
cpp/src/arrow/scalar.cc:
##########
@@ -1127,6 +1127,28 @@ Status CastImpl(const StructScalar& from, StringScalar* to) {
   return Status::OK();
 }
 
+// casts between variable-length and fixed-length list types
+template <typename ToScalar>
+enable_if_t<is_list_like_type<typename ToScalar::TypeClass>::value &&
+                ToScalar::TypeClass::type_id != Type::MAP,
+            Status>
+CastImpl(const BaseListScalar& from, ToScalar* to) {
+  DCHECK(from.type->id() == Type::LIST || from.type->id() == Type::LARGE_LIST ||
+         from.type->id() == Type::FIXED_SIZE_LIST);

Review Comment:
   Used a better predicate for the enable_if here and extended it to gracefully handle Map as well.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou merged pull request #36310: GH-36309: [C++] Add ability to cast between scalars of list-like types

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou merged PR #36310:
URL: https://github.com/apache/arrow/pull/36310


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] felipecrv commented on a diff in pull request #36310: GH-36309: [C++] Add ability to cast between scalars of list-like types

Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #36310:
URL: https://github.com/apache/arrow/pull/36310#discussion_r1243884320


##########
cpp/src/arrow/scalar.cc:
##########
@@ -1127,6 +1127,28 @@ Status CastImpl(const StructScalar& from, StringScalar* to) {
   return Status::OK();
 }
 
+// casts between variable-length and fixed-length list types
+template <typename ToScalar>
+enable_if_t<is_list_like_type<typename ToScalar::TypeClass>::value &&
+                ToScalar::TypeClass::type_id != Type::MAP,
+            Status>
+CastImpl(const BaseListScalar& from, ToScalar* to) {
+  DCHECK(from.type->id() == Type::LIST || from.type->id() == Type::LARGE_LIST ||
+         from.type->id() == Type::FIXED_SIZE_LIST);

Review Comment:
   I will use `is_var_length_list` in the enable-if and expand the test cases.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] pitrou commented on a diff in pull request #36310: GH-36309: [C++] Add ability to cast between scalars of list-like types

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #36310:
URL: https://github.com/apache/arrow/pull/36310#discussion_r1243605216


##########
cpp/src/arrow/scalar.cc:
##########
@@ -1127,6 +1127,28 @@ Status CastImpl(const StructScalar& from, StringScalar* to) {
   return Status::OK();
 }
 
+// casts between variable-length and fixed-length list types
+template <typename ToScalar>
+enable_if_t<is_list_like_type<typename ToScalar::TypeClass>::value &&
+                ToScalar::TypeClass::type_id != Type::MAP,
+            Status>
+CastImpl(const BaseListScalar& from, ToScalar* to) {
+  DCHECK(from.type->id() == Type::LIST || from.type->id() == Type::LARGE_LIST ||
+         from.type->id() == Type::FIXED_SIZE_LIST);

Review Comment:
   Careful: `MapScalar` inherits from `BaseListScalar`. Are you sure it can't use this overload?



##########
cpp/src/arrow/scalar_test.cc:
##########
@@ -391,6 +390,8 @@ class TestRealScalar : public ::testing::Test {
     ASSERT_TRUE(list_nan.ApproxEquals(list_other_nan, options));
   }
 
+  void TestListOf() { TestListOf<ListScalar>(list(type_)); }

Review Comment:
   And can we add a cast test between map and non-map-list-like somewhere?



##########
cpp/src/arrow/scalar_test.cc:
##########
@@ -1135,6 +1136,25 @@ class TestListScalar : public ::testing::Test {
     ASSERT_NE(a0->hash(), b0->hash());
   }
 
+  void TestCast(ScalarType& scalar, const std::shared_ptr<DataType>& to_type) {

Review Comment:
   Can we avoid non-const refs?
   ```suggestion
     void TestCast(const ScalarType& scalar, const std::shared_ptr<DataType>& to_type) {
   ```



##########
cpp/src/arrow/scalar_test.cc:
##########
@@ -391,6 +390,8 @@ class TestRealScalar : public ::testing::Test {
     ASSERT_TRUE(list_nan.ApproxEquals(list_other_nan, options));
   }
 
+  void TestListOf() { TestListOf<ListScalar>(list(type_)); }

Review Comment:
   Is it me, or is `TestListOf` never called with types other than plain list?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] felipecrv commented on a diff in pull request #36310: GH-36309: [C++] Add ability to cast between scalars of list-like types

Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #36310:
URL: https://github.com/apache/arrow/pull/36310#discussion_r1244106016


##########
cpp/src/arrow/scalar_test.cc:
##########
@@ -391,6 +390,8 @@ class TestRealScalar : public ::testing::Test {
     ASSERT_TRUE(list_nan.ApproxEquals(list_other_nan, options));
   }
 
+  void TestListOf() { TestListOf<ListScalar>(list(type_)); }

Review Comment:
   I added `large_list` here, but note that the cast tests are the ones relevant here.



##########
cpp/src/arrow/scalar_test.cc:
##########
@@ -391,6 +390,8 @@ class TestRealScalar : public ::testing::Test {
     ASSERT_TRUE(list_nan.ApproxEquals(list_other_nan, options));
   }
 
+  void TestListOf() { TestListOf<ListScalar>(list(type_)); }

Review Comment:
   I added `large_list` here, but note that the cast tests are the ones relevant in this PR.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] conbench-apache-arrow[bot] commented on pull request #36310: GH-36309: [C++] Add ability to cast between scalars of list-like types

Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #36310:
URL: https://github.com/apache/arrow/pull/36310#issuecomment-1616249618

   Conbench analyzed the 6 benchmark runs on commit `3e359116`.
   
   There were 3 benchmark results indicating a performance regression:
   
   - Commit Run on `ursa-thinkcentre-m75q` at [2023-06-30 07:07:53Z](http://conbench.ursa.dev/compare/runs/a4248842c8c140dba3d39e368fe31b15...f11277222e7541efb1d727390bd82293/)
     - [params=524288/1, source=cpp-micro, suite=arrow-int-util-benchmark](http://conbench.ursa.dev/compare/benchmarks/0649e566dc157e6580003d3c3074955e...0649e7f6d580774f80001b3e298ca897)
     - [params=524288/1, source=cpp-micro, suite=arrow-int-util-benchmark](http://conbench.ursa.dev/compare/benchmarks/0649e566d01d7053800024b34a782e56...0649e7f6ca1975078000e7d0b504244f)
   - and 1 more (see the report linked below)
   
   The [full Conbench report](https://github.com/apache/arrow/runs/14713102592) has more details.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow] felipecrv commented on a diff in pull request #36310: GH-36309: [C++] Add ability to cast between scalars of list-like types

Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on code in PR #36310:
URL: https://github.com/apache/arrow/pull/36310#discussion_r1243884320


##########
cpp/src/arrow/scalar.cc:
##########
@@ -1127,6 +1127,28 @@ Status CastImpl(const StructScalar& from, StringScalar* to) {
   return Status::OK();
 }
 
+// casts between variable-length and fixed-length list types
+template <typename ToScalar>
+enable_if_t<is_list_like_type<typename ToScalar::TypeClass>::value &&
+                ToScalar::TypeClass::type_id != Type::MAP,
+            Status>
+CastImpl(const BaseListScalar& from, ToScalar* to) {
+  DCHECK(from.type->id() == Type::LIST || from.type->id() == Type::LARGE_LIST ||
+         from.type->id() == Type::FIXED_SIZE_LIST);

Review Comment:
   ~I will use `is_var_length_list` in the enable-if and expand the test cases.~



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org