You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "neilconway (via GitHub)" <gi...@apache.org> on 2026/02/11 17:48:51 UTC
[I] Optimize strpos() for ASCII inputs [datafusion]
neilconway opened a new issue, #20294:
URL: https://github.com/apache/datafusion/issues/20294
### Is your feature request related to a problem or challenge?
strpos() in DataFusion is roughly 1.8x slower than in DuckDB in the string microbenchmark [here](https://github.com/apache/datafusion-benchmarks).
### Describe the solution you'd like
_No response_
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Optimize strpos() for ASCII inputs [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #20294: Optimize strpos() for ASCII inputs
URL: https://github.com/apache/datafusion/issues/20294
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Optimize strpos() for ASCII inputs [datafusion]
Posted by "neilconway (via GitHub)" <gi...@apache.org>.
neilconway commented on issue #20294:
URL: https://github.com/apache/datafusion/issues/20294#issuecomment-3886061938
* The previous ASCII fastpath avoids `string::find` because it is slow. Indeed, it is -- but it is mainly slow because it has a lot of per-call overhead. The previous fast-path algorithm (slide a window using `[T]::windows` and check each window for an exact match) doesn't have per-call overhead but the algorithm itself is slow.
• memchr::memchr is considerably faster because of SIMD and other chicanery. It also has very low per-call overhead (unlike `string::find` or `memmem::find`).
### Possible Further Improvements
• For the common case that the needle is a scalar, we could explore building a `memmem::Finder` for the needle once and then applying it to the haystack.
• For UTF inputs, we could still use `memchr` or a similar byte-based search, but once we find a match, we'd need to iterate over `char_indexes` to find the right character (not byte) offset. This would be a clear win when the needle isn't found in the haystack, but the net performance change for successful matches would be less clear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Optimize strpos() for ASCII inputs [datafusion]
Posted by "neilconway (via GitHub)" <gi...@apache.org>.
neilconway commented on issue #20294:
URL: https://github.com/apache/datafusion/issues/20294#issuecomment-3885976143
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org