You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2025/12/23 21:17:55 UTC

[I] Improve performance of cast from string to integral types [datafusion-comet]

andygrove opened a new issue, #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981

   ### What is the problem the feature request solves?
   
   `CometCastStringToNumericBenchmarkSuite` shows that casts from string to integral types are slower than Spark:
   
   ```
   OpenJDK 64-Bit Server VM 11.0.22+7-LTS on Mac OS X 14.6.1
   Apple M3 Max
   CAST String to INT:                       Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
   ------------------------------------------------------------------------------------------------------------------------
   Spark                                                56             64          11         18.7          53.4       1.0X
   Comet (Scan)                                         56             68          16         18.9          53.0       1.0X
   Comet (Scan + Exec)                                  70             88          21         15.0          66.6       0.8X
   ```
   
   I would expect these to be much faster in Comet.
   
   ### Describe the potential solution
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Improve performance of cast from string to integral types [datafusion-comet]

Posted by "s6r6k (via GitHub)" <gi...@apache.org>.
s6r6k commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688577681

   Hello. Im new to this codebase and interested in learning more about performance work like this. Could you please help me by pointing out where the string -> integral cast logic currently lives? and also which code paths are most relevant for this? I guess im gonna start with smaller issues to understand this project first. thankss lot


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Improve performance of cast from string to integral types [datafusion-comet]

Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove closed issue #2981: Improve performance of cast from string to integral types
URL: https://github.com/apache/datafusion-comet/issues/2981


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Improve performance of cast from string to integral types [datafusion-comet]

Posted by "SushantDevHub (via GitHub)" <gi...@apache.org>.
SushantDevHub commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688817996

   ok thanks.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Improve performance of cast from string to integral types [datafusion-comet]

Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688816239

   The link should be in main readme.md


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Improve performance of cast from string to integral types [datafusion-comet]

Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3695453863

   looking into this 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Improve performance of cast from string to integral types [datafusion-comet]

Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688885129

   https://datafusion.apache.org/comet/


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Improve performance of cast from string to integral types [datafusion-comet]

Posted by "SushantDevHub (via GitHub)" <gi...@apache.org>.
SushantDevHub commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688814737

   @coderfender Thanks a lot. sure, I will read the steps to understand the current flow of this project.
   could you also please mention the link to setup and also the required stuffs?
    


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Improve performance of cast from string to integral types [datafusion-comet]

Posted by "coderfender (via GitHub)" <gi...@apache.org>.
coderfender commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688791809

   Hello . I would love to help you 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Improve performance of cast from string to integral types [datafusion-comet]

Posted by "SushantDevHub (via GitHub)" <gi...@apache.org>.
SushantDevHub commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3695532299

   @coderfender problem[#2981] I think the performance gap in the Scan + Exec path is likely due to per-row string parsing and additional overhead from native execution and Arrow interop.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Improve performance of cast from string to integral types [datafusion-comet]

Posted by "SushantDevHub (via GitHub)" <gi...@apache.org>.
SushantDevHub commented on issue #2981:
URL: https://github.com/apache/datafusion-comet/issues/2981#issuecomment-3688727250

   Hi! I’m new to this codebase and exploring performance optimizations.
   Could someone please help me locate the string-to-integer cast logic and the most relevant code paths involved?
   I’ll start with some smaller issues to get familiar with the project.
   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org