You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2025/01/30 21:40:01 UTC
[PR] Fully support LIKE/NLIKE with Utf8View [datafusion]
alamb opened a new pull request, #14379:
URL: https://github.com/apache/datafusion/pull/14379
## Which issue does this PR close?
- Related to https://github.com/apache/datafusion/issues/13361
## Rationale for this change
I wanted to ensure that we can fully support `LIKE` / `ILIKE` with `UTF8View`
It works today except for if the pattern was `Utf8View` but that turns out to be an over eager error in the planner -- the kernels work just fine
## What changes are included in this PR?
1. Update tests to show `LIKE`/ `ILIKE` work with `Utf8View` (and `LargeUtf8` and `Dictionary`)
2. Remove check in planner
## Are these changes tested?
Yes
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
3. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
## Are there any user-facing changes?
You can now run `... LIKE <Utf8View>` column
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fully support LIKE/ILIKE with Utf8View [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #14379:
URL: https://github.com/apache/datafusion/pull/14379
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fully support LIKE/NLIKE with Utf8View [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #14379:
URL: https://github.com/apache/datafusion/pull/14379#discussion_r1936332144
##########
datafusion/sql/src/expr/mod.rs:
##########
@@ -819,10 +820,6 @@ impl<S: ContextProvider> SqlToRel<'_, S> {
return not_impl_err!("ANY in LIKE expression");
}
let pattern = self.sql_expr_to_logical_expr(pattern, schema, planner_context)?;
- let pattern_type = pattern.get_type(schema)?;
Review Comment:
this is the code change -- don't do type checking in the planner
##########
datafusion/sqllogictest/test_files/string/string.slt:
##########
@@ -41,141 +41,6 @@ select arrow_cast(col1, 'Utf8') as c1 from test_substr_base;
#
include ./string_query.slt.part
-# TODO support all String types in sql_like_to_expr and move this test to `string_query.slt.part`
Review Comment:
By moving these tests to `string_query.slt.part` they now run for strings, large strings, stringview and dictionary arrays
FYI @goldmedal
##########
datafusion/sqllogictest/test_files/scalar.slt:
##########
@@ -1689,6 +1689,15 @@ true true false false true true
statement ok
drop table t1
+# can't use like with non stirngs
Review Comment:
this shows what happens when you try to run `like` on an integer 😆
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fully support LIKE/ILIKE with Utf8View [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #14379:
URL: https://github.com/apache/datafusion/pull/14379#issuecomment-2630800635
Thank you @2010YOUY01 -- I have
> The change makes sense to me, thank you
>
> I have one suggestion for additional test: now tests for StringView arrays are included, there are also tests for StringView literals can be added in https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/test_files/string/string_literal.slt, like `select 'foo' like arrow_cast('foo', 'Utf8View')`
Great idea -- thank you @2010YOUY01 -- done in f138f606e
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Fully support LIKE/ILIKE with Utf8View [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #14379:
URL: https://github.com/apache/datafusion/pull/14379#issuecomment-2631220679
Thanks again @2010YOUY01
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org