You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "timsaucer (via GitHub)" <gi...@apache.org> on 2024/11/09 18:42:18 UTC
[I] Add support for string view to a few functions [datafusion]
timsaucer opened a new issue, #13330:
URL: https://github.com/apache/datafusion/issues/13330
### Is your feature request related to a problem or challenge?
These functions are failing when passing in a string view instead of a string. Now that string view is the default type, it is causing some regressions on existing code that have to be modified.
### Describe the solution you'd like
Add options of passing in string view to these:
- [ ] ArrayToString
- [ ] NamedStructFunc
- [ ] DateBinFunc (I couldn't quite track down where the Utf->Interval or Timestamp conversion occurs
- [ ] BitLengthFunc
- [ ] ConcatFunc (gives error on optimizer not matching schema)
- [ ] EncodeFunc
### Describe alternatives you've considered
_No response_
### Additional context
These were discovered during the `datafusion-python` migration to DF43.0.0. This commit is where I made adjustments in that repo to account for these: https://github.com/apache/datafusion-python/pull/905/commits/fcb5f968e3f11cbcdf77d4c86e5d437de7427f11
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #13330:
URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2468324874
regexp_match filed as https://github.com/apache/datafusion/issues/13357
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #13330:
URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2468807910
[NullIf](https://github.com/apache/datafusion/blob/main/datafusion/functions/src/core/nullif.rs):
```Rust
static SUPPORTED_NULLIF_TYPES: &[DataType] = &[
DataType::Boolean,
DataType::UInt8,
DataType::UInt16,
DataType::UInt32,
DataType::UInt64,
DataType::Int8,
DataType::Int16,
DataType::Int32,
DataType::Int64,
DataType::Float32,
DataType::Float64,
DataType::Utf8,
DataType::LargeUtf8,
];
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #13330:
URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2469191937
[nvl](https://github.com/apache/datafusion/blob/main/datafusion/functions/src/core/nvl.rs):
```Rust
static SUPPORTED_NVL_TYPES: &[DataType] = &[
DataType::Boolean,
DataType::UInt8,
DataType::UInt16,
DataType::UInt32,
DataType::UInt64,
DataType::Int8,
DataType::Int16,
DataType::Int32,
DataType::Int64,
DataType::Float32,
DataType::Float64,
DataType::Utf8,
DataType::LargeUtf8,
];
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n commented on issue #13330:
URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2466412377
This could be made into an EPIC. Should we apply this to the regexp functions as well? Off the top of the head, I remember `regexp_match` still needing string view support.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #13330:
URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2468804783
None of the crypto functions support UTF8View directly as a input type. For example [digest](https://github.com/apache/datafusion/blob/main/datafusion/functions/src/crypto/digest.rs):
```Rust
signature: Signature::one_of(
vec![
Exact(vec![Utf8, Utf8]),
Exact(vec![LargeUtf8, Utf8]),
Exact(vec![Binary, Utf8]),
Exact(vec![LargeBinary, Utf8]),
],
Volatility::Immutable,
),
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #13330:
URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2468481874
Related:
- https://github.com/apache/arrow-rs/issues/6713
- https://github.com/apache/arrow-rs/issues/6714
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #13330:
URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2468786850
[array_to_string](https://github.com/apache/datafusion/blob/main/datafusion/functions-nested/src/string.rs) aslso does not appear to support Utf8View
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #13330:
URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2467120727
regexp_match looks to not support stringview either.
```
// TODO: Native support Utf8View for regexp_match.
TypeSignature::Exact(vec![Utf8, Utf8]),
TypeSignature::Exact(vec![LargeUtf8, LargeUtf8]),
TypeSignature::Exact(vec![Utf8, Utf8, Utf8]),
TypeSignature::Exact(vec![LargeUtf8, LargeUtf8, LargeUtf8]),
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on issue #13330:
URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2468647775
I'm going to recommend a 43.1.0 release after we get all these resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #13330: Add support for string view to a few functions
URL: https://github.com/apache/datafusion/issues/13330
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #13330:
URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2475308405
crypto PR - https://github.com/apache/datafusion/pull/13407
array_to_string/string_to_array PR - https://github.com/apache/datafusion/pull/13403
nvl PR - https://github.com/apache/datafusion/pull/13382
nullif PR - https://github.com/apache/datafusion/pull/13380
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Add support for string view to a few functions [datafusion]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on issue #13330:
URL: https://github.com/apache/datafusion/issues/13330#issuecomment-2466977633
Related: https://github.com/apache/datafusion/issues/13347
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org