You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "ajita-asthana (via GitHub)" <gi...@apache.org> on 2025/05/17 21:43:56 UTC
[PR] hex function [datafusion]
ajita-asthana opened a new pull request, #16077:
URL: https://github.com/apache/datafusion/pull/16077
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->
- Closes #15986
## Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
## Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Optimize Hex Function [datafusion]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #16077:
URL: https://github.com/apache/datafusion/pull/16077#discussion_r2365946163
##########
datafusion/sqllogictest/test_files/spark/math/hex.slt:
##########
@@ -16,24 +16,24 @@
# under the License.
query T
-SELECT hex('Spark SQL');
+SELECT CAST(hex('Spark SQL') AS VARCHAR);
----
537061726B2053514C
query T
-SELECT hex(1234::INT);
+SELECT CAST(hex(1234::INT) AS VARCHAR);
----
4D2
query T
-SELECT hex(a) from VALUES (1234::INT), (NULL), (456::INT) AS t(a);
+SELECT CAST(hex(a) AS VARCHAR) from VALUES (1234::INT), (NULL), (456::INT) AS t(a);
----
4D2
NULL
1C8
query T
-SELECT hex(a) from VALUES ('foo'), (NULL), ('foobarbaz') AS t(a);
+SELECT CAST(hex(a) AS VARCHAR) from VALUES ('foo'), (NULL), ('foobarbaz') AS t(a);
Review Comment:
These tests aren't sufficient to demonstrate the optimization, as they are only casting the output of the hex function to view type.
##########
datafusion/spark/src/function/math/hex.rs:
##########
@@ -75,7 +75,7 @@ impl ScalarUDFImpl for SparkHex {
&self,
_arg_types: &[DataType],
) -> datafusion_common::Result<DataType> {
- Ok(DataType::Utf8)
+ Ok(DataType::Utf8View)
Review Comment:
The implementation must be changed to accomodate this changed DataType
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Optimize Hex Function [datafusion]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey closed pull request #16077: Optimize Hex Function
URL: https://github.com/apache/datafusion/pull/16077
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Optimize Hex Function [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #16077:
URL: https://github.com/apache/datafusion/pull/16077#issuecomment-2971062970
Marking as draft as I think this PR is no longer waiting on feedback and I am trying to make it easier to find PRs in need of review. Please mark it as ready for review when it is ready for another look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Optimize Hex Function [datafusion]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on PR #16077:
URL: https://github.com/apache/datafusion/pull/16077#issuecomment-3541755906
Closing this as stale; feel free to reopen it when it becomes active again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Optimize Hex Function [datafusion]
Posted by "ajita-asthana (via GitHub)" <gi...@apache.org>.
ajita-asthana commented on PR #16077:
URL: https://github.com/apache/datafusion/pull/16077#issuecomment-3066987604
Would it be feasible to add a benchmark for the new spark compatible hex function as well if it doesn't exist ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Optimize Hex Function [datafusion]
Posted by "jfahne (via GitHub)" <gi...@apache.org>.
jfahne commented on PR #16077:
URL: https://github.com/apache/datafusion/pull/16077#issuecomment-2889503095
You should document why using the view struct is optimal on this PR or the linked issue. If no tests are being added, you document why no additional tests are needed. Seems like this is referencing the Java implementation but it's not clear whether that is beneficial.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org