[GitHub] [arrow-datafusion] TennyZhuang commented on issue #4248: Make a data driven SQL testing tool (so we can reuse duckdb test suite, example)

[GitBox <gi...@apache.org>]

TennyZhuang commented on issue #4248:
URL: https://github.com/apache/arrow-datafusion/issues/4248#issuecomment-1321193056

   Hi, I'm one of the maintainers in [sqllogictest-rs](https://github.com/risinglightdb/sqllogictest-rs). I'm very happy that arrow-datafusion and databend are interested in the project.
   
   Currently, the project is only used by [risinglight](https://github.com/risinglightdb/risinglight) and [risingwave](https://github.com/risingwavelabs/risingwave), while the maintainers for these three projects overlap highly. However, we really hope to increase the diversity of the sqllogictest project.
   
   ## Components
   
   There are several components in sqllogictest-rs:
   
   1. The core slt file parser.
   2. The driver trait `AsyncDB`.
   3. A cli tool to visit files by glob paths, parse, run and compare results in the different logical namespaces.
   4. (WIP/Experimental) A subprocess-based plugin framework to test the compatibility for non-rust drivers, e.g. [JDBC](https://github.com/risinglightdb/sqllogictest-driver-jdbc).
   
   
   ### Parser
   
   Currently, the parser part is generally stabilized and we add very few extensions to the original syntax, the most important thing is the `include` macro, which is useful for reusing some test codes.
   
   ### AsyncDB trait
   
   Currently, the trait is easy and general enough, it only accepts a SQL string and returns a concatenated result. This means that it's compatible with different SQL syntaxes (Postgres, MySQL, or even Spanner), but know very few things about the data type information. We even can't check whether the result types are correct for queries.
   
   We'll improve that, but I'm not sure if should we generalize the data types between different databases, it looks like hard work.
   
   ### Runner
   
   A simple and general part.
   
   ## Conclusion
   
   We have several ways of working together.
   
   1. Just fork the project, customize them by yourselves and contributions are always welcome!
   2. Use the sqllogictest-rs parser and customize other things by yourselves.
   3. Use the same project, and work together to make the `AsyncDB` and the `Postgres` implementations better.
   4. Make the project even more general, and open to every database interface. (Likely not a short-term goal).
   
   We would like to donate the project to some org if it would help our collaboration.
   
   Thanks for your interest in sqllogictest-rs!
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org