You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "jorisvandenbossche (via GitHub)" <gi...@apache.org> on 2024/01/16 13:17:09 UTC

[I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

jorisvandenbossche opened a new issue, #39634:
URL: https://github.com/apache/arrow/issues/39634

   PR https://github.com/apache/arrow/pull/37792 added a basic implementation of the new BinaryView and Utf8View (StringView) format additions. This included mostly just the basic objects (Array, DataType, Builder) and IPC support. 
   For being able to handle input that uses those new formats, there are some additional features we will have to support (eg receiving a dataset, doing some filtering/projection, and then converting to standard string type).
   
   - [x] Array equality (included in initial implementation)
     - [ ] also expose this as the "equal" kernel
   - [ ] Conversion (cast) to/from the standard binary and string types
   - [ ] Take / filter
   - [x] Unique / dictionary encoding (included in initial implementation)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

Posted by "kevinli1993 (via GitHub)" <gi...@apache.org>.
kevinli1993 commented on issue #39634:
URL: https://github.com/apache/arrow/issues/39634#issuecomment-2441582603

   Hi! I found this issue after creating #44540 above - thanks Joris for organizing/triaging. 
   
   I'm wondering if the cast functionality will be made available in `pyarrow` in a later release?  Or we focusing on polishing up the remaining C++ tasks first?
   
   On another note, another missing functionality I might like to see added is a `count_unique` kernel for Utf8View. (Hopefully not a big lift as the dictionary implementation already exists.)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

Posted by "Periecle (via GitHub)" <gi...@apache.org>.
Periecle commented on issue #39634:
URL: https://github.com/apache/arrow/issues/39634#issuecomment-4433615483

   Implementation of basic comparison kernels for binary/utf8 views - #49964 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

Posted by "WillAyd (via GitHub)" <gi...@apache.org>.
WillAyd commented on issue #39634:
URL: https://github.com/apache/arrow/issues/39634#issuecomment-2444409981

   I opened https://github.com/apache/arrow/issues/44336 to track the development of compute kernels - should we just roll that into this PR or keep separate?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

Posted by "Periecle (via GitHub)" <gi...@apache.org>.
Periecle commented on issue #39634:
URL: https://github.com/apache/arrow/issues/39634#issuecomment-4430610785

   take


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #39634:
URL: https://github.com/apache/arrow/issues/39634#issuecomment-2441741715

   > I'm wondering if the cast functionality will be made available in pyarrow in a later release? Or we focusing on polishing up the remaining C++ tasks first?
   
   Which version are you using here? I'd try 18.0 for this later


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

Posted by "kevinli1993 (via GitHub)" <gi...@apache.org>.
kevinli1993 commented on issue #39634:
URL: https://github.com/apache/arrow/issues/39634#issuecomment-2451775440

   > > I'm wondering if the cast functionality will be made available in pyarrow in a later release? Or we focusing on polishing up the remaining C++ tasks first?
   > 
   > Which version are you using here? I'd try 18.0 for this later, and just support cast if it cannot.
   
   I could confirm that casting to pa.string() works very well now, on the 18.0 version of pyarrow!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #39634:
URL: https://github.com/apache/arrow/issues/39634#issuecomment-2447612280

   Either way is fine for me. In addition to the "basic" kernels (cast, take, filter, etc), there are also all the binary or string-specific kernels (`binary_..`/`ascii_..`/`utf8_..`)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

Posted by "Periecle (via GitHub)" <gi...@apache.org>.
Periecle commented on issue #39634:
URL: https://github.com/apache/arrow/issues/39634#issuecomment-4691513371

   Implementation of take/filter for binary/utf8 views: #50164 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #39634:
URL: https://github.com/apache/arrow/issues/39634#issuecomment-2440890517

   https://github.com/apache/arrow/pull/43302/files
   
   This Cast is implemented in C++, I don't know what should be added then 🤔


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #39634:
URL: https://github.com/apache/arrow/issues/39634#issuecomment-2441273345

   Updated the top post to indicate `cast` is implemented now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++] Implement basic functionality for new Binary/Utf8View types [arrow]

Posted by "adamreeve (via GitHub)" <gi...@apache.org>.
adamreeve commented on issue #39634:
URL: https://github.com/apache/arrow/issues/39634#issuecomment-6277340021

   I've opened #54668 which is related to this, for implementing comparison of Utf8View with Utf8 and LargeUtf8.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org