You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2025/12/23 21:17:55 UTC
[I] Improve performance of cast from string to integral types [datafusion-comet]
andygrove opened a new issue, #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981
### What is the problem the feature request solves?
`CometCastStringToNumericBenchmarkSuite` shows that casts from string to integral types are slower than Spark:
```
OpenJDK 64-Bit Server VM 11.0.22+7-LTS on Mac OS X 14.6.1
Apple M3 Max
CAST String to INT: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Spark 56 64 11 18.7 53.4 1.0X
Comet (Scan) 56 68 16 18.9 53.0 1.0X
Comet (Scan + Exec) 70 88 21 15.0 66.6 0.8X
```
I would expect these to be much faster in Comet.
### Describe the potential solution
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve performance of cast from string to integral types [datafusion-comet]
Posted by "s6r6k (via GitHub)" <gi...@apache.org>.
s6r6k commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688577681
Hello. Im new to this codebase and interested in learning more about performance work like this. Could you please help me by pointing out where the string -> integral cast logic currently lives? and also which code paths are most relevant for this? I guess im gonna start with smaller issues to understand this project first. thankss lot
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve performance of cast from string to integral types [datafusion-comet]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #2981: Improve performance of cast from string to integral types
URL: https://github.com/apache/datafusion-comet/issues/2981
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve performance of cast from string to integral types [datafusion-comet]
Posted by "SushantDevHub (via GitHub)" <gi...@apache.org>.
SushantDevHub commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688817996
ok thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve performance of cast from string to integral types [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688816239
The link should be in main readme.md
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve performance of cast from string to integral types [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3695453863
looking into this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve performance of cast from string to integral types [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688885129
https://datafusion.apache.org/comet/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve performance of cast from string to integral types [datafusion-comet]
Posted by "SushantDevHub (via GitHub)" <gi...@apache.org>.
SushantDevHub commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688814737
@coderfender Thanks a lot. sure, I will read the steps to understand the current flow of this project.
could you also please mention the link to setup and also the required stuffs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve performance of cast from string to integral types [datafusion-comet]
Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688791809
Hello . I would love to help you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve performance of cast from string to integral types [datafusion-comet]
Posted by "SushantDevHub (via GitHub)" <gi...@apache.org>.
SushantDevHub commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3695532299
@coderfender problem[#2981] I think the performance gap in the Scan + Exec path is likely due to per-row string parsing and additional overhead from native execution and Arrow interop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve performance of cast from string to integral types [datafusion-comet]
Posted by "SushantDevHub (via GitHub)" <gi...@apache.org>.
SushantDevHub commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688727250
Hi! I’m new to this codebase and exploring performance optimizations.
Could someone please help me locate the string-to-integer cast logic and the most relevant code paths involved?
I’ll start with some smaller issues to get familiar with the project.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org