You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alexander-beedie (via GitHub)" <gi...@apache.org> on 2026/02/20 20:18:37 UTC
[PR] Avoid string allocations in keyword lookup [datafusion-sqlparser-rs]
alexander-beedie opened a new pull request, #2226:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2226
Follow-up to #2225.
## Description
* Found another ~25 or so places where we can make `peek_token` โ `peek_token_ref` updates.
* Keyword lookups were a _significant_ hotspot, so the following optimisations were made:
* Added a `Token::make_word_owned` variant to avoid `to_string()` copies, factoring out the keyword lookup.
* The keyword lookup itself was then optimised to eliminated `to_ascii_uppercase()` heap allocations.
## Results
I ran `sqlparser_bench` several times I (and rebaselined) as the results are better than expected ๐
```
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโฌโโโโโโโโโโโโฌโโโโโโโโโ
โ benchmark โ baseline โ this pr โ change โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโค
โ sqlparser::select โ 2.691 ยตs โ 2.477 ยตs โ โ8.0% โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโค
โ sqlparser::with_select โ 12.944 ยตs โ 11.652 ยตs โ โ10.4% โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโค
โ parse_large_statement โ 4.467 ms โ 3.880 ms โ โ13.1% โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโค
โ format_large_statement โ 242.62 ยตs โ 232.23 ยตs โ โ4.1% โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโค
โ select_100_columns โ 68.56 ยตs โ 63.94 ยตs โ โ6.7% โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโโโโผโโโโโโโโโค
โ select_100_qualified_columns โ 142.47 ยตs โ 133.75 ยตs โ โ6.1% โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโดโโโโโโโโโโโโดโโโโโโโโโ
```
_(Benchmarking run on: Apple M3 Max)_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Optimise out string allocations and copies in keyword lookup [datafusion-sqlparser-rs]
Posted by "iffyio (via GitHub)" <gi...@apache.org>.
iffyio merged PR #2226:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2226
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org