You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "echou (via GitHub)" <gi...@apache.org> on 2025/10/11 01:13:02 UTC
[I] High compile time of crates using sqlparser(codegen phase) - any way to reduce generated code? [datafusion-sqlparser-rs]
echou opened a new issue, #2066:
URL: https://github.com/apache/datafusion-sqlparser-rs/issues/2066
In my project, some crates have very long compilation times (e.g., 40-50 seconds, with the codegen phase accounting for 70%-90% of that time). I suspect these crates are slow to compile due to their dependency on sqlparser.
I analyzed the compilation of one such crate using cargo llvm-lines. The top functions by lines are listed below:
```
Lines Copies Function name
----- ------ -------------
2180708 31532 (TOTAL)
220326 (10.1%, 10.1%) 21 (0.1%, 0.1%) <sqlparser::ast::Statement as sqlparser::ast::visitor::VisitMut>::visit
163970 (7.5%, 17.6%) 1995 (6.3%, 6.4%) <alloc::vec::Vec<T> as sqlparser::ast::visitor::VisitMut>::visit
117960 (5.4%, 23.0%) 3654 (11.6%, 18.0%) <core::option::Option<T> as sqlparser::ast::visitor::VisitMut>::visit
76507 (3.5%, 26.5%) 21 (0.1%, 18.0%) <sqlparser::ast::Expr as sqlparser::ast::visitor::VisitMut>::visit
```
As shown, the visitor trait implementations for the AST types generate a significant amount of code. Is there any way to reduce the code generation footprint of sqlparser?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] High compile time of crates using sqlparser(codegen phase) - any way to reduce generated code? [datafusion-sqlparser-rs]
Posted by "echou (via GitHub)" <gi...@apache.org>.
echou commented on issue #2066:
URL: https://github.com/apache/datafusion-sqlparser-rs/issues/2066#issuecomment-3396632032
yes, my project uses vistor feature heavily. The crates not using sqlparser are quite fast to compile.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] High compile time of crates using sqlparser(codegen phase) - any way to reduce generated code? [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on issue #2066:
URL: https://github.com/apache/datafusion-sqlparser-rs/issues/2066#issuecomment-3396645613
There was recently an update in the sqlparser derive crate - have you already tried whether the new version has improved / worsened these time requirements?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] High compile time of crates using sqlparser(codegen phase) - any way to reduce generated code? [datafusion-sqlparser-rs]
Posted by "LucaCappelletti94 (via GitHub)" <gi...@apache.org>.
LucaCappelletti94 commented on issue #2066:
URL: https://github.com/apache/datafusion-sqlparser-rs/issues/2066#issuecomment-3393057558
Hi, which compilation flags are you using? I assume you are using the visitor-generated code, otherwise a trivial solution is not to enable that feature. How much does the compilation time change with and without the visitor feature enabled?
In my own projects, where admittedly I have no use of the visitor feature, the time requirement of compiling sqlparser is rather negligible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org