You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/10/26 16:36:35 UTC
[GitHub] [arrow] pprudhvi opened a new pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
pprudhvi opened a new pull request #8530:
URL: https://github.com/apache/arrow/pull/8530
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8530:
URL: https://github.com/apache/arrow/pull/8530#issuecomment-717129780
Revision: eec6dfdec0f4e0d0bb2a19bdde03fb1472e876d5
Submitted crossbow builds: [ursa-labs/crossbow @ actions-682](https://github.com/ursa-labs/crossbow/branches/all?query=actions-682)
|Task|Status|
|----|------|
|gandiva-jar-osx|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|gandiva-jar-xenial|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pprudhvi commented on pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
pprudhvi commented on pull request #8530:
URL: https://github.com/apache/arrow/pull/8530#issuecomment-717335283
_MinGW link problem is caused by msys2/MINGW-packages#7170 .
It'll be fixed in upstream._
https://github.com/apache/arrow/pull/8521#issuecomment-716210482
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pprudhvi commented on pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
pprudhvi commented on pull request #8530:
URL: https://github.com/apache/arrow/pull/8530#issuecomment-716918217
still couldn't avoid string copy - the copy is now happening inside HashTable
`#0 0x00007f8b9e690159 in __lll_unlock_wake_private () from /lib64/libc.so.6
#1 0x00007f8b9e60df78 in _L_unlock_14902 () from /lib64/libc.so.6
#2 0x00007f8b9e60afe5 in malloc () from /lib64/libc.so.6
#3 0x00007f8b9d3d20cd in operator new(unsigned long) () from /lib64/libstdc++.so.6
#4 0x00007f8b9d430c79 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#5 0x00007f8b9d432531 in char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) () from /lib64/libstdc++.so.6
#6 0x00007f8b9d4325ed in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#7 0x00007f8b595c7c4a in std::_Hashtable<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> >, nonstd::sv_lite::basic_string_view<char, std::char_traits<char> >, std::allocator<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::__detail::_Identity, std::equal_to<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::hash<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::count(nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > const&) const () from /data/yarn/data/usercache/dremio/appcache/application_1603758047742_0001/container_1603758047742_0001_01_000003/tmp/libgandiva_jni.so87f06318-3d22-4bda-be14-ef2458081d87
#8 0x00007f8b595c7258 in gdv_fn_in_expr_lookup_utf8 () from /data/yarn/data/usercache/dremio/appcache/application_1603758047742_0001/container_1603758047742_0001_01_000003/tmp/libgandiva_jni.so87f06318-3d22-4bda-be14-ef2458081d87`
@bkietz , any reason why the copy is happening even when we are looking for a string_view in hashtable<string_view>?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pprudhvi commented on pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
pprudhvi commented on pull request #8530:
URL: https://github.com/apache/arrow/pull/8530#issuecomment-716989866
Turns out string_view hash function internally creates a string
https://github.com/martinmoene/string-view-lite/blob/master/include/nonstd/string_view.hpp#L1583
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8530:
URL: https://github.com/apache/arrow/pull/8530#issuecomment-716720548
Revision: 84515e88c39a6ae20ffc9ae86c4eb02273a6aa70
Submitted crossbow builds: [ursa-labs/crossbow @ actions-680](https://github.com/ursa-labs/crossbow/branches/all?query=actions-680)
|Task|Status|
|----|------|
|gandiva-jar-osx|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
|gandiva-jar-xenial|[](https://travis-ci.org/ursa-labs/crossbow/branches)|
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pprudhvi edited a comment on pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
pprudhvi edited a comment on pull request #8530:
URL: https://github.com/apache/arrow/pull/8530#issuecomment-716918217
still couldn't avoid string copy - the copy is now happening inside HashTable
`#0 0x00007f8b9e690159 in __lll_unlock_wake_private () from /lib64/libc.so.6
#1 0x00007f8b9e60df78 in _L_unlock_14902 () from /lib64/libc.so.6
#2 0x00007f8b9e60afe5 in malloc () from /lib64/libc.so.6
#3 0x00007f8b9d3d20cd in operator new(unsigned long) () from /lib64/libstdc++.so.6
#4 0x00007f8b9d430c79 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#5 0x00007f8b9d432531 in char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) () from /lib64/libstdc++.so.6
#6 0x00007f8b9d4325ed in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#7 0x00007f8b595c7c4a in std::_Hashtable<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> >, nonstd::sv_lite::basic_string_view<char, std::char_traits<char> >, std::allocator<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::__detail::_Identity, std::equal_to<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::hash<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::count(nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > const&) const () from /data/yarn/data/usercache/dremio/appcache/application_1603758047742_0001/container_1603758047742_0001_01_000003/tmp/libgandiva_jni.so87f06318-3d22-4bda-be14-ef2458081d87
#8 0x00007f8b595c7258 in gdv_fn_in_expr_lookup_utf8 () from /data/yarn/data/usercache/dremio/appcache/application_1603758047742_0001/container_1603758047742_0001_01_000003/tmp/libgandiva_jni.so87f06318-3d22-4bda-be14-ef2458081d87`
@bkietz , any reason why the copy is happening even when we are looking for a string_view in hashtable<string_view>?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pprudhvi commented on pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
pprudhvi commented on pull request #8530:
URL: https://github.com/apache/arrow/pull/8530#issuecomment-716706701
@github-actions crossbow submit -g gandiva
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pprudhvi edited a comment on pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
pprudhvi edited a comment on pull request #8530:
URL: https://github.com/apache/arrow/pull/8530#issuecomment-716918217
still couldn't avoid string copy - the copy is now happening inside HashTable
```
#0 0x00007f8b9e690159 in __lll_unlock_wake_private () from /lib64/libc.so.6
#1 0x00007f8b9e60df78 in _L_unlock_14902 () from /lib64/libc.so.6
#2 0x00007f8b9e60afe5 in malloc () from /lib64/libc.so.6
#3 0x00007f8b9d3d20cd in operator new(unsigned long) () from /lib64/libstdc++.so.6
#4 0x00007f8b9d430c79 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#5 0x00007f8b9d432531 in char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) () from /lib64/libstdc++.so.6
#6 0x00007f8b9d4325ed in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#7 0x00007f8b595c7c4a in std::_Hashtable<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> >, nonstd::sv_lite::basic_string_view<char, std::char_traits<char> >, std::allocator<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::__detail::_Identity, std::equal_to<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::hash<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::count(nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > const&) const () from /data/yarn/data/usercache/dremio/appcache/application_1603758047742_0001/container_1603758047742_0001_01_000003/tmp/libgandiva_jni.so87f06318-3d22-4bda-be14-ef2458081d87
#8 0x00007f8b595c7258 in gdv_fn_in_expr_lookup_utf8 () from /data/yarn/data/usercache/dremio/appcache/application_1603758047742_0001/container_1603758047742_0001_01_000003/tmp/libgandiva_jni.so87f06318-3d22-4bda-be14-ef2458081d87
```
@bkietz , any reason why the copy is happening even when we are looking for a string_view in hashtable<string_view>?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pravindra closed pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
pravindra closed pull request #8530:
URL: https://github.com/apache/arrow/pull/8530
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8530:
URL: https://github.com/apache/arrow/pull/8530#issuecomment-716717270
https://issues.apache.org/jira/browse/ARROW-10392
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pprudhvi commented on pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
pprudhvi commented on pull request #8530:
URL: https://github.com/apache/arrow/pull/8530#issuecomment-717117927
@github-actions crossbow submit -g gandiva
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pprudhvi edited a comment on pull request #8530: ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression
Posted by GitBox <gi...@apache.org>.
pprudhvi edited a comment on pull request #8530:
URL: https://github.com/apache/arrow/pull/8530#issuecomment-716918217
still couldn't avoid string copy - the copy is now happening inside HashTable
#0 0x00007f8b9e690159 in __lll_unlock_wake_private () from /lib64/libc.so.6
#1 0x00007f8b9e60df78 in _L_unlock_14902 () from /lib64/libc.so.6
#2 0x00007f8b9e60afe5 in malloc () from /lib64/libc.so.6
#3 0x00007f8b9d3d20cd in operator new(unsigned long) () from /lib64/libstdc++.so.6
#4 0x00007f8b9d430c79 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#5 0x00007f8b9d432531 in char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) () from /lib64/libstdc++.so.6
#6 0x00007f8b9d4325ed in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#7 0x00007f8b595c7c4a in std::_Hashtable<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> >, nonstd::sv_lite::basic_string_view<char, std::char_traits<char> >, std::allocator<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::__detail::_Identity, std::equal_to<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::hash<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::count(nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > const&) const () from /data/yarn/data/usercache/dremio/appcache/application_1603758047742_0001/container_1603758047742_0001_01_000003/tmp/libgandiva_jni.so87f06318-3d22-4bda-be14-ef2458081d87
#8 0x00007f8b595c7258 in gdv_fn_in_expr_lookup_utf8 () from /data/yarn/data/usercache/dremio/appcache/application_1603758047742_0001/container_1603758047742_0001_01_000003/tmp/libgandiva_jni.so87f06318-3d22-4bda-be14-ef2458081d87
@bkietz , any reason why the copy is happening even when we are looking for a string_view in hashtable<string_view>?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org