You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by "zhuruixuan (via GitHub)" <gi...@apache.org> on 2024/04/19 06:35:51 UTC
[I] Support parsing MySQL CREATE sql [shardingsphere]
zhuruixuan opened a new issue, #30958:
URL: https://github.com/apache/shardingsphere/issues/30958
## Background
Hi community.\
The ShardingSphere SQL parser engine helps users to parse SQL to create the AST (Abstract Syntax Tree) and visit the AST to get SQLStatement (Java Object). Currently, we are planning to enhance the support for MySQL SQL parsing in ShardingSphere.
More details:
<https://shardingsphere.apache.org/document/current/en/reference/sharding/parse/>
## Issue Background Explanation
The current issue involves using a custom web scraping script to fetch SQL cases from the official website of a corresponding database. These SQL cases are then passed to ShardingSphere's parsing engine for analysis. For SQL cases that fail to be parsed successfully, every 3 to 5 SQL cases are grouped together as an issue.
1. Since SQL cases are obtained through web scraping, it cannot be guaranteed that all SQL cases are correct. Please follow the following process to handle this pull request (PR).
2. Some SQL cases may have already been fixed in other PRs. For cases that can already be executed successfully, simply leave a comment to ignore them.
3. If a SQL case can be executed successfully without any code changes, there is no need to add a corresponding test assert file.
## Task
This issue is to support more MySQL sql parsing, as follows:
```
CREATE FUNCTION GTID_IS_DISJOINT(gs1 LONGTEXT, gs2 LONGTEXT)
RETURNS INT
RETURN GTID_SUBSET(gs1, GTID_SUBTRACT(gs1, gs2));
```
[link](
https://dev.mysql.com/doc/refman/8.0/en/replication-gtids-functions.html)
```
CREATE FUNCTION GTID_IS_DISJOINT_UNION(gs1 LONGTEXT, gs2 LONGTEXT, sum LONGTEXT)
RETURNS INT
RETURN GTID_IS_EQUAL(GTID_SUBTRACT(sum, gs1), gs2) AND
GTID_IS_EQUAL(GTID_SUBTRACT(sum, gs2), gs1);
```
[link](https://dev.mysql.com/doc/refman/8.0/en/replication-gtids-functions.html)
```
CREATE FUNCTION GTID_INTERSECTION(gs1 LONGTEXT, gs2 LONGTEXT)
RETURNS LONGTEXT
RETURN GTID_SUBTRACT(gs1, GTID_SUBTRACT(gs1, gs2));
```
[link](https://dev.mysql.com/doc/refman/8.0/en/replication-gtids-functions.html)
```
CREATE EVENT e_insert
ON SCHEDULE
EVERY 7 SECOND
DO
INSERT INTO myschema.mytable;
```
[link](https://dev.mysql.com/doc/refman/8.0/en/events-privileges.html)
```
CREATE FUNCTION GTID_INTERSECTION_WITH_UUID(gs LONGTEXT, uuid TEXT)
RETURNS LONGTEXT
RETURN GTID_SUBTRACT(gs, GTID_SUBTRACT_UUID(gs, uuid));
```
[link](https://dev.mysql.com/doc/refman/8.0/en/replication-gtids-functions.html)
## Overall Procedure
If you intend to participate in fixing this issue, please feel free to leave a comment below the issue. Community members will assign the issue accordingly.
><u>For example, you can leave a comment like this: "Hi, please assign this issue to me. Thank you!"</u>
Once you have claimed the issue, please review the syntax of the SQL on the official website of the corresponding database. Execute the SQL on the respective database to ensure the correctness of the SQL syntax.
><u>You can check the corresponding source of each SQL case on the official database website by clicking on the link provided below each case.</u>
Next, execute the problematic SQL cases mentioned above in the database (you can quickly start the corresponding database using the Docker image for that database, and then connect to it using
a client you are familiar with), to ensure that the SQL syntax itself is correct.
## Fixing ANTLR Grammar Parsing Issue
Once you have confirmed the correctness of the SQL syntax, you can validate and fix the grammar parsing issue in ShardingSphere.
>If you are using IntelliJ IDEA, you will need to install the ANTLR plugin before proceeding.
If it is an ANTLR parsing error message, try to repair the `.g4` file by comparing it with the official database syntax until the SQL can be correctly parsed by ANTLR.
When there is no error message in the ANTLR Preview window, it means that ANTLR can correctly parse the SQL.
## Visitor problem fix
After ANTLR parses SQL into an abstract syntax tree, ShardingSphere will access the abstract syntax tree through Visitor and extract the required information.
If you need to extract Segments, you need to first execute:
```
mvn -T 2C clean install -DskipTests
```
Under the shardingsphere-parser module to compile the entire parser module.\
Then rewrite the corresponding visit method in SQLStatementVisitorr as needed to extract the corresponding Segment.
## Add assertion test file
After the above SQL parsing problem is repaired, the corresponding Test needs to be added.
The steps are as follows:
1. Add the corresponding `sql-case` in the `sql/supported` directory.
2. Add case assertions in the case directory of the `shardingsphere-test-it-parser` module.
3. Run `org.apache.shardingsphere.test.it.sql.parser.internal.InternalSQLParserIT`\
After SQL Parser IT runs successfully, you can submit a PR.
## Relevant Skills
1. Master JAVA language
2. Have a basic understanding of Antlr g4 file
3. Be familiar with MySQL SQLs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support parsing MySQL CREATE sql [shardingsphere]
Posted by "strongduanmu (via GitHub)" <gi...@apache.org>.
strongduanmu closed issue #30958: Support parsing MySQL CREATE sql
URL: https://github.com/apache/shardingsphere/issues/30958
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support parsing MySQL CREATE sql [shardingsphere]
Posted by "PDGGK (via GitHub)" <gi...@apache.org>.
PDGGK commented on issue #30958:
URL: https://github.com/apache/shardingsphere/issues/30958#issuecomment-3726806419
Hi @zhuruixuan @terrymanu,
I noticed this issue was unassigned recently. I'd like to work on it.
**Background:**
- I'm a student preparing for GSoC 2026
- I have experience with Java, ANTLR4, and MySQL SQL syntax
- I've studied the ShardingSphere parser module and understand the development workflow
**Plan:**
- Validate SQL syntax against MySQL official documentation
- Fix ANTLR grammar files for CREATE FUNCTION with RETURNS clause
- Add support for CREATE EVENT with ON SCHEDULE syntax
- Implement corresponding visitor methods
- Add test cases and run InternalSQLParserIT
Could you please assign this issue to me? Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support parsing MySQL CREATE sql [shardingsphere]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30958:
URL: https://github.com/apache/shardingsphere/issues/30958#issuecomment-2125655120
There hasn't been any activity on this issue recently, and in order to prioritize active issues, it will be marked as stale.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support parsing MySQL CREATE sql [shardingsphere]
Posted by "strongduanmu (via GitHub)" <gi...@apache.org>.
strongduanmu commented on issue #30958:
URL: https://github.com/apache/shardingsphere/issues/30958#issuecomment-3976207765
Fixed by https://github.com/apache/shardingsphere/pull/38237
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support parsing MySQL CREATE sql [shardingsphere]
Posted by "zhuruixuan (via GitHub)" <gi...@apache.org>.
zhuruixuan commented on issue #30958:
URL: https://github.com/apache/shardingsphere/issues/30958#issuecomment-2069486389
> Hi, please assign this issue to me. Thank you! btw,I find the content of issue#30959 is exactly the same as this issue.I just wonder if you guys just add the issue by mistake
Yes, I accidentally generated the same one, I will close the other one. Anyway, I'll assign you this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support parsing MySQL CREATE sql [shardingsphere]
Posted by "lujx98 (via GitHub)" <gi...@apache.org>.
lujx98 commented on issue #30958:
URL: https://github.com/apache/shardingsphere/issues/30958#issuecomment-2069469499
Hi, please assign this issue to me. Thank you!
btw,I find the content of issue#30959 is exactly the same as this issue.I just wonder if you guys just add the issue by mistake
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org