You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/10/09 15:14:53 UTC
[GitHub] [shardingsphere] Jacob953 opened a new pull request, #21431: Dynamic Loading SQL Parser Parameterized Test
Jacob953 opened a new pull request, #21431:
URL: https://github.com/apache/shardingsphere/pull/21431
Fixes #16383.
Changes proposed in this pull request:
- create DynamicLoadingSQLParserParameterizedTest
- define the test case as IT, and run it in GitHub Action only.
---
Before committing this PR, I'm sure that I have checked the following options:
- [x] My code follows the [code of conduct](https://shardingsphere.apache.org/community/en/involved/conduct/code/) of this project.
- [x] I have self-reviewed the commit code.
- [x] I have passed maven check: `mvn clean install -B -T2C -DskipTests -Dmaven.javadoc.skip=true -e`.
- [ ] I have made corresponding changes to the documentation.
- [ ] I have added corresponding unit tests for my changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] Jacob953 commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
Jacob953 commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1013149735
##########
pom.xml:
##########
@@ -691,6 +693,19 @@
<artifactId>caffeine</artifactId>
<version>${caffeine.version}</version>
</dependency>
+
+ <dependency>
+ <groupId>org.jsoup</groupId>
+ <artifactId>jsoup</artifactId>
+ <version>${jsoup.version}</version>
+ </dependency>
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] Jacob953 commented on pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
Jacob953 commented on PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#issuecomment-1295841634
Implement steps:
- Get all `download_url`s of SQL case files via github-api at once through getTestParameters.
- Append each SQL ended up with `;` and ID, which is generated by the SQL case file and case number, to an array.
- Pass the array to assertDynamicLoadingSQL at DynamicLoading(MySQL/PostgreSQL)ParserParameterizedTest.
- Each SQL will go through the parsing engine
- if successful, the case would pass at `new SQLVisitorEngine`.
- else, print id, value, and db-type of SQL and throw non-RuntimeException to make the test case pass with @SingletonSPI
<img width="1338" alt="image" src="https://user-images.githubusercontent.com/64075563/198834803-094a3b35-330c-431a-aa22-6a75777e716f.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] Jacob953 commented on pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
Jacob953 commented on PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#issuecomment-1302369039
test comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] Jacob953 commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
Jacob953 commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1013149417
##########
pom.xml:
##########
@@ -691,6 +693,19 @@
<artifactId>caffeine</artifactId>
<version>${caffeine.version}</version>
</dependency>
+
+ <dependency>
+ <groupId>org.jsoup</groupId>
+ <artifactId>jsoup</artifactId>
+ <version>${jsoup.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.testng</groupId>
+ <artifactId>testng</artifactId>
+ <version>${testng.version}</version>
+ <scope>compile</scope>
+ </dependency>
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
terrymanu commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1013158266
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+import org.springframework.web.client.RestTemplate;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.ArrayList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static ArrayList<Map<String, String>> getResponse(final String sqlCaseURL) {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String sqlCasesOwner = patches[3];
+ String sqlCasesRepo = patches[4];
+ String sqlCasesDirectory = patches[7];
+ return new RestTemplate().getForObject(
+ "https://api.github.com/repos/{owner}/{repo}/contents/{directory}", ArrayList.class,
+ sqlCasesOwner, sqlCasesRepo, sqlCasesDirectory);
+ }
+
+ protected static Collection<Object[]> getTestParameters(final String sqlCaseURL) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new ArrayList<>();
+ ArrayList<Map<String, String>> response = getResponse(sqlCaseURL);
+ for (Map<String, String> each : response) {
+ result.addAll(getSqlCases(each));
+ }
+ return result;
+ }
+
+ private static Collection<Object[]> getSqlCases(final Map<String, String> elements) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new ArrayList<>();
+ String sqlCaseFileName = elements.get("name");
+ String sqlCaseFileContent = IOUtils.toString(new URI(elements.get("download_url")), StandardCharsets.UTF_8);
+ String[] lines = sqlCaseFileContent.split("\n");
+ int sqlCaseEnum = 1;
+ for (String each : lines) {
+ if (each.isEmpty()) {
+ continue;
+ }
+ if (Character.isLetter(each.charAt(0)) && each.charAt(each.length() - 1) == ';') {
+ String sqlCaseId = sqlCaseFileName.split("\\.")[0] + sqlCaseEnum;
+ result.add(new Object[]{
+ sqlCaseId, each,
+ });
+ sqlCaseEnum++;
+ }
+ }
+ return result;
+ }
+
+ @Test(expected = Exception.class)
+ public final void assertDynamicLoadingSQL() {
+ CacheOption cacheOption = new CacheOption(128, 1024L);
+ ParseASTNode parseContext = new SQLParserEngine(databaseType, cacheOption).parse(sqlCaseValue, false);
+ new SQLVisitorEngine(databaseType, "STATEMENT", true, new Properties()).visit(parseContext);
+ System.out.println("ParserError: " + sqlCaseId + " value: " + sqlCaseValue + " db-type: " + databaseType);
+ throw new RuntimeException();
Review Comment:
Maybe you misunderstand the usage of @SingletonSPI.
@SingletonSPI is for create SPI with singleton way only.
I still wonder why always throw exception, it should be record failed SQL without any exception thrown
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,118 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import com.google.common.collect.ImmutableMap;
+import lombok.RequiredArgsConstructor;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.net.URL;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.List;
+import java.util.LinkedList;
+import java.util.stream.Collectors;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static String[] getContentLines(final String url) throws IOException {
+ InputStreamReader in = new InputStreamReader(new URL(url).openStream());
+ String content = new BufferedReader(in).lines().collect(Collectors.joining(System.lineSeparator()));
+ return content.split("\n");
+ }
+
+ private static LinkedList<Map<String, String>> getResponse(final String sqlCaseURL) throws IOException {
Review Comment:
Please use interface (List) for return type declaration
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+import org.springframework.web.client.RestTemplate;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.ArrayList;
+
+@RequiredArgsConstructor
+@SingletonSPI
Review Comment:
@SingletonSPI is for SPI only. obviously, DynamicLoadingSQLParserParameterizedTest is not SPI. So the annotation is useless here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu merged pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
terrymanu merged PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] Jacob953 commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
Jacob953 commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1013148500
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,145 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.apache.shardingsphere.sql.parser.exception.SQLParsingException;
+import org.apache.shardingsphere.sql.parser.sql.common.statement.SQLStatement;
+import org.jsoup.Jsoup;
+import org.jsoup.nodes.Document;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.testng.asserts.IAssert;
+import org.testng.asserts.SoftAssert;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.LinkedList;
+import java.util.Objects;
+import java.util.Properties;
+
+@RunWith(Parameterized.class)
+public final class DynamicLoadingSQLParserParameterizedTest extends SoftAssert {
+
+ private static final String SQL_CASE_FILE_MARKER = "js-navigation-open Link--primary";
+
+ private static final String SQL_CASE_MARKER = "blob-code blob-code-inner js-file-line";
+
+ private static boolean inBracket;
+
+ private static StringBuilder sqlCase;
+
+ private static int sqlCaseId;
+
+ private static String sqlCaseURL;
+
+ private static String databaseType;
+
+ public DynamicLoadingSQLParserParameterizedTest() {
+ sqlCaseURL = System.getProperty("url");
+ databaseType = System.getProperty("database");
+ }
+
+ private static String[] getSqlCaseFiles(final String sqlCaseURL) throws IOException {
+ Document document = Jsoup.connect(sqlCaseURL).get();
+ return document.getElementsByClass(SQL_CASE_FILE_MARKER).text().split("\\s+");
+ }
+
+ private static Collection<Object[]> getTestParameters(final String databaseType) throws IOException {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] Jacob953 commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
Jacob953 commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1013147915
##########
.github/workflows/it.yml:
##########
@@ -226,3 +226,34 @@ jobs:
run: |
./mvnw -B clean install -f test/integration-agent-test/plugins/metrics/pom.xml -Dspotless.apply.skip=true -Pit.env.metrics
./mvnw -B clean install -f test/integration-agent-test/plugins/opentelemetry/pom.xml -Dspotless.apply.skip=true -Pit.env.opentelemetry
+
+ dynamic-loading-test-case:
+ name: Dynamic Loading SQL Parser
+ runs-on: ubuntu-latest
+ strategy:
+ matrix:
+ database: [ MySQL, PostgreSQL ]
+ include:
+ - database: MySQL
+ url: https://github.com/mysql/mysql-server/tree/8.0/mysql-test/t
+ - database: PostgreSQL
+ url: https://github.com/postgres/postgres/tree/master/src/test/regress/sql
+ steps:
+ - uses: actions/checkout@v2
+ - name: Maven Resolve Ranges
+ run: ./mvnw versions:resolve-ranges -ntp -Dincludes='org.springframework:*,org.springframework.boot:*'
+ - uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ ${{ runner.os }}-maven-
+ - uses: actions/setup-java@v2
+ with:
+ distribution: 'temurin'
+ java-version: 8
+ - name: Build Project
+ run: ./mvnw -B clean install -DskipITs -DskipTests
+ - name: Run Parameterized Test
+ run: |
+ ./mvnw -B clean install -f test/parser/pom.xml -Pit.dynamic.loading -Dspotless.apply.skip=true -Ddatabase=${{ matrix.database }} -Durl=${{ matrix.url }}
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
terrymanu commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1012664144
##########
test/parser/pom.xml:
##########
@@ -117,5 +117,17 @@
<artifactId>shardingsphere-test-common</artifactId>
<version>${project.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-starter</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>commons-io</groupId>
+ <artifactId>commons-io</artifactId>
+ </dependency>
Review Comment:
Is it necessary to import the dependencies?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] Jacob953 commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
Jacob953 commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1012723527
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.List;
+import java.util.LinkedList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static LinkedList<Map<String, Object>> getResponse(final String sqlCaseURL) throws IOException {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String url = "https://api.github.com/repos/" + patches[3] + "/" + patches[4] + "/contents/" + patches[7];
+ return new JsonMapper().readValue(new URL(url), new TypeReference<LinkedList<Map<String, Object>>>() {
Review Comment:
Call api-github to get information of files in the case directory
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
terrymanu commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1012453045
##########
pom.xml:
##########
@@ -153,7 +153,8 @@
<git-commit-id-plugin.version>4.9.10</git-commit-id-plugin.version>
<freemarker.version>2.3.31</freemarker.version>
<spotless-maven-plugin.version>2.22.1</spotless-maven-plugin.version>
- <caffeine.version>2.9.3</caffeine.version>
+ <caffeine.version>2.9.2</caffeine.version>
Review Comment:
Is it necessary to downgrade caffeine?
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+import org.springframework.web.client.RestTemplate;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.ArrayList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static ArrayList<Map<String, String>> getResponse(final String sqlCaseURL) {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String sqlCasesOwner = patches[3];
+ String sqlCasesRepo = patches[4];
+ String sqlCasesDirectory = patches[7];
+ return new RestTemplate().getForObject(
+ "https://api.github.com/repos/{owner}/{repo}/contents/{directory}", ArrayList.class,
+ sqlCasesOwner, sqlCasesRepo, sqlCasesDirectory);
+ }
+
+ protected static Collection<Object[]> getTestParameters(final String sqlCaseURL) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new ArrayList<>();
+ ArrayList<Map<String, String>> response = getResponse(sqlCaseURL);
Review Comment:
Please use List to instead of ArrayList here
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+import org.springframework.web.client.RestTemplate;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.ArrayList;
+
+@RequiredArgsConstructor
+@SingletonSPI
Review Comment:
What is the useful?
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+import org.springframework.web.client.RestTemplate;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.ArrayList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static ArrayList<Map<String, String>> getResponse(final String sqlCaseURL) {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String sqlCasesOwner = patches[3];
+ String sqlCasesRepo = patches[4];
+ String sqlCasesDirectory = patches[7];
+ return new RestTemplate().getForObject(
+ "https://api.github.com/repos/{owner}/{repo}/contents/{directory}", ArrayList.class,
+ sqlCasesOwner, sqlCasesRepo, sqlCasesDirectory);
+ }
+
+ protected static Collection<Object[]> getTestParameters(final String sqlCaseURL) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new ArrayList<>();
+ ArrayList<Map<String, String>> response = getResponse(sqlCaseURL);
+ for (Map<String, String> each : response) {
+ result.addAll(getSqlCases(each));
+ }
+ return result;
+ }
+
+ private static Collection<Object[]> getSqlCases(final Map<String, String> elements) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new ArrayList<>();
+ String sqlCaseFileName = elements.get("name");
+ String sqlCaseFileContent = IOUtils.toString(new URI(elements.get("download_url")), StandardCharsets.UTF_8);
+ String[] lines = sqlCaseFileContent.split("\n");
+ int sqlCaseEnum = 1;
+ for (String each : lines) {
+ if (each.isEmpty()) {
+ continue;
+ }
+ if (Character.isLetter(each.charAt(0)) && each.charAt(each.length() - 1) == ';') {
+ String sqlCaseId = sqlCaseFileName.split("\\.")[0] + sqlCaseEnum;
+ result.add(new Object[]{
+ sqlCaseId, each,
+ });
+ sqlCaseEnum++;
+ }
+ }
+ return result;
+ }
+
+ @Test(expected = Exception.class)
+ public final void assertDynamicLoadingSQL() {
+ CacheOption cacheOption = new CacheOption(128, 1024L);
+ ParseASTNode parseContext = new SQLParserEngine(databaseType, cacheOption).parse(sqlCaseValue, false);
+ new SQLVisitorEngine(databaseType, "STATEMENT", true, new Properties()).visit(parseContext);
+ System.out.println("ParserError: " + sqlCaseId + " value: " + sqlCaseValue + " db-type: " + databaseType);
+ throw new RuntimeException();
Review Comment:
Why always throw exception?
##########
.mvn/jvm.config:
##########
@@ -1 +1 @@
--Xmx2g -XX:MaxMetaspaceSize=512m
Review Comment:
Why need to increase so much mem here?
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+import org.springframework.web.client.RestTemplate;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.ArrayList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static ArrayList<Map<String, String>> getResponse(final String sqlCaseURL) {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String sqlCasesOwner = patches[3];
+ String sqlCasesRepo = patches[4];
+ String sqlCasesDirectory = patches[7];
+ return new RestTemplate().getForObject(
+ "https://api.github.com/repos/{owner}/{repo}/contents/{directory}", ArrayList.class,
+ sqlCasesOwner, sqlCasesRepo, sqlCasesDirectory);
+ }
+
+ protected static Collection<Object[]> getTestParameters(final String sqlCaseURL) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new ArrayList<>();
Review Comment:
Please use LinkedList to instead of ArrayList here
##########
pom.xml:
##########
@@ -688,6 +689,11 @@
<version>${springframework.version}</version>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.springframework</groupId>
+ <artifactId>spring-web</artifactId>
+ <version>${springframework.version}</version>
+ </dependency>
Review Comment:
ShardingSphere is a backend project, please do not involve spring-web dependencies
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] Jacob953 commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
Jacob953 commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1012778267
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+import org.springframework.web.client.RestTemplate;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.ArrayList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static ArrayList<Map<String, String>> getResponse(final String sqlCaseURL) {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String sqlCasesOwner = patches[3];
+ String sqlCasesRepo = patches[4];
+ String sqlCasesDirectory = patches[7];
+ return new RestTemplate().getForObject(
+ "https://api.github.com/repos/{owner}/{repo}/contents/{directory}", ArrayList.class,
+ sqlCasesOwner, sqlCasesRepo, sqlCasesDirectory);
+ }
+
+ protected static Collection<Object[]> getTestParameters(final String sqlCaseURL) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new ArrayList<>();
+ ArrayList<Map<String, String>> response = getResponse(sqlCaseURL);
Review Comment:
fixed
##########
pom.xml:
##########
@@ -688,6 +689,11 @@
<version>${springframework.version}</version>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.springframework</groupId>
+ <artifactId>spring-web</artifactId>
+ <version>${springframework.version}</version>
+ </dependency>
Review Comment:
fixed
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.List;
+import java.util.LinkedList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static LinkedList<Map<String, Object>> getResponse(final String sqlCaseURL) throws IOException {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String url = "https://api.github.com/repos/" + patches[3] + "/" + patches[4] + "/contents/" + patches[7];
+ return new JsonMapper().readValue(new URL(url), new TypeReference<LinkedList<Map<String, Object>>>() {
+ });
+ }
+
+ protected static Collection<Object[]> getTestParameters(final String sqlCaseURL) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new LinkedList<>();
+ List<Map<String, Object>> response = getResponse(sqlCaseURL);
Review Comment:
Information of each file, such as `name`, `download_url`. Actually, those two are all we need. Using `Object` is to avoid convert failure caused by the complex item `_links`.
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.List;
+import java.util.LinkedList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static LinkedList<Map<String, Object>> getResponse(final String sqlCaseURL) throws IOException {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String url = "https://api.github.com/repos/" + patches[3] + "/" + patches[4] + "/contents/" + patches[7];
+ return new JsonMapper().readValue(new URL(url), new TypeReference<LinkedList<Map<String, Object>>>() {
+ });
+ }
+
+ protected static Collection<Object[]> getTestParameters(final String sqlCaseURL) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new LinkedList<>();
+ List<Map<String, Object>> response = getResponse(sqlCaseURL);
Review Comment:
Change to Map<String, String> now
##########
test/parser/pom.xml:
##########
@@ -117,5 +117,17 @@
<artifactId>shardingsphere-test-common</artifactId>
<version>${project.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-starter</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>commons-io</groupId>
+ <artifactId>commons-io</artifactId>
+ </dependency>
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
terrymanu commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1012669427
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.List;
+import java.util.LinkedList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static LinkedList<Map<String, Object>> getResponse(final String sqlCaseURL) throws IOException {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String url = "https://api.github.com/repos/" + patches[3] + "/" + patches[4] + "/contents/" + patches[7];
+ return new JsonMapper().readValue(new URL(url), new TypeReference<LinkedList<Map<String, Object>>>() {
+ });
+ }
+
+ protected static Collection<Object[]> getTestParameters(final String sqlCaseURL) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new LinkedList<>();
+ List<Map<String, Object>> response = getResponse(sqlCaseURL);
Review Comment:
What the content in `Map<String, Object>` ?
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.net.URL;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.List;
+import java.util.LinkedList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static LinkedList<Map<String, Object>> getResponse(final String sqlCaseURL) throws IOException {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String url = "https://api.github.com/repos/" + patches[3] + "/" + patches[4] + "/contents/" + patches[7];
+ return new JsonMapper().readValue(new URL(url), new TypeReference<LinkedList<Map<String, Object>>>() {
Review Comment:
Why need to spilt url, how about use the whole url?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] codecov-commenter commented on pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#issuecomment-1272580527
# [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/21431?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#21431](https://codecov.io/gh/apache/shardingsphere/pull/21431?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0bbe13d) into [master](https://codecov.io/gh/apache/shardingsphere/commit/9b9b6e0802d1d8f8c7fa95f1e0c4bf978bbcfae7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9b9b6e0) will **decrease** coverage by `0.05%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #21431 +/- ##
============================================
- Coverage 61.49% 61.44% -0.06%
Complexity 2509 2509
============================================
Files 4097 4098 +1
Lines 56761 56820 +59
Branches 7501 7509 +8
============================================
+ Hits 34908 34911 +3
- Misses 18927 18983 +56
Partials 2926 2926
```
| [Impacted Files](https://codecov.io/gh/apache/shardingsphere/pull/21431?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...gine/DynamicLoadingSQLParserParameterizedTest.java](https://codecov.io/gh/apache/shardingsphere/pull/21431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-dGVzdC9wYXJzZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3Rlc3Qvc3FsL3BhcnNlci9wYXJhbWV0ZXJpemVkL2VuZ2luZS9EeW5hbWljTG9hZGluZ1NRTFBhcnNlclBhcmFtZXRlcml6ZWRUZXN0LmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...handler/distsql/ral/hint/enums/HintSourceType.java](https://codecov.io/gh/apache/shardingsphere/pull/21431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cHJveHkvYmFja2VuZC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvcHJveHkvYmFja2VuZC9oYW5kbGVyL2Rpc3RzcWwvcmFsL2hpbnQvZW51bXMvSGludFNvdXJjZVR5cGUuamF2YQ==) | `42.85% <0.00%> (+42.85%)` | :arrow_up: |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
terrymanu commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r990811558
##########
.github/workflows/it.yml:
##########
@@ -226,3 +226,34 @@ jobs:
run: |
./mvnw -B clean install -f test/integration-agent-test/plugins/metrics/pom.xml -Dspotless.apply.skip=true -Pit.env.metrics
./mvnw -B clean install -f test/integration-agent-test/plugins/opentelemetry/pom.xml -Dspotless.apply.skip=true -Pit.env.opentelemetry
+
+ dynamic-loading-test-case:
+ name: Dynamic Loading SQL Parser
+ runs-on: ubuntu-latest
+ strategy:
+ matrix:
+ database: [ MySQL, PostgreSQL ]
+ include:
+ - database: MySQL
+ url: https://github.com/mysql/mysql-server/tree/8.0/mysql-test/t
+ - database: PostgreSQL
+ url: https://github.com/postgres/postgres/tree/master/src/test/regress/sql
+ steps:
+ - uses: actions/checkout@v2
+ - name: Maven Resolve Ranges
+ run: ./mvnw versions:resolve-ranges -ntp -Dincludes='org.springframework:*,org.springframework.boot:*'
+ - uses: actions/cache@v2
+ with:
+ path: ~/.m2/repository
+ key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ ${{ runner.os }}-maven-
+ - uses: actions/setup-java@v2
+ with:
+ distribution: 'temurin'
+ java-version: 8
+ - name: Build Project
+ run: ./mvnw -B clean install -DskipITs -DskipTests
+ - name: Run Parameterized Test
+ run: |
+ ./mvnw -B clean install -f test/parser/pom.xml -Pit.dynamic.loading -Dspotless.apply.skip=true -Ddatabase=${{ matrix.database }} -Durl=${{ matrix.url }}
Review Comment:
It is better to put business logic into java codes with junit, just keep GitHub action simple.
It could be better to run in local `mvn install` too.
##########
pom.xml:
##########
@@ -691,6 +693,19 @@
<artifactId>caffeine</artifactId>
<version>${caffeine.version}</version>
</dependency>
+
+ <dependency>
+ <groupId>org.jsoup</groupId>
+ <artifactId>jsoup</artifactId>
+ <version>${jsoup.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.testng</groupId>
+ <artifactId>testng</artifactId>
+ <version>${testng.version}</version>
+ <scope>compile</scope>
+ </dependency>
Review Comment:
Is it necessary to import new testing framework?
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,145 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.apache.shardingsphere.sql.parser.exception.SQLParsingException;
+import org.apache.shardingsphere.sql.parser.sql.common.statement.SQLStatement;
+import org.jsoup.Jsoup;
+import org.jsoup.nodes.Document;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+import org.testng.asserts.IAssert;
+import org.testng.asserts.SoftAssert;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.LinkedList;
+import java.util.Objects;
+import java.util.Properties;
+
+@RunWith(Parameterized.class)
+public final class DynamicLoadingSQLParserParameterizedTest extends SoftAssert {
+
+ private static final String SQL_CASE_FILE_MARKER = "js-navigation-open Link--primary";
+
+ private static final String SQL_CASE_MARKER = "blob-code blob-code-inner js-file-line";
+
+ private static boolean inBracket;
+
+ private static StringBuilder sqlCase;
+
+ private static int sqlCaseId;
+
+ private static String sqlCaseURL;
+
+ private static String databaseType;
+
+ public DynamicLoadingSQLParserParameterizedTest() {
+ sqlCaseURL = System.getProperty("url");
+ databaseType = System.getProperty("database");
+ }
+
+ private static String[] getSqlCaseFiles(final String sqlCaseURL) throws IOException {
+ Document document = Jsoup.connect(sqlCaseURL).get();
+ return document.getElementsByClass(SQL_CASE_FILE_MARKER).text().split("\\s+");
+ }
+
+ private static Collection<Object[]> getTestParameters(final String databaseType) throws IOException {
Review Comment:
What is the useful of parameter `databaseType`?
##########
pom.xml:
##########
@@ -691,6 +693,19 @@
<artifactId>caffeine</artifactId>
<version>${caffeine.version}</version>
</dependency>
+
+ <dependency>
+ <groupId>org.jsoup</groupId>
+ <artifactId>jsoup</artifactId>
+ <version>${jsoup.version}</version>
+ </dependency>
Review Comment:
How about using GitHub API to instead of jsoup to fetch sql text?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
terrymanu commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1012752487
##########
test/parser/pom.xml:
##########
@@ -117,5 +117,9 @@
<artifactId>shardingsphere-test-common</artifactId>
<version>${project.version}</version>
</dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ </dependency>
Review Comment:
Is it necessary to import it？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] Jacob953 commented on a diff in pull request #21431: Dynamic Loading SQL Parser Parameterized Test
Posted by GitBox <gi...@apache.org>.
Jacob953 commented on code in PR #21431:
URL: https://github.com/apache/shardingsphere/pull/21431#discussion_r1012778197
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+import org.springframework.web.client.RestTemplate;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.ArrayList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static ArrayList<Map<String, String>> getResponse(final String sqlCaseURL) {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String sqlCasesOwner = patches[3];
+ String sqlCasesRepo = patches[4];
+ String sqlCasesDirectory = patches[7];
+ return new RestTemplate().getForObject(
+ "https://api.github.com/repos/{owner}/{repo}/contents/{directory}", ArrayList.class,
+ sqlCasesOwner, sqlCasesRepo, sqlCasesDirectory);
+ }
+
+ protected static Collection<Object[]> getTestParameters(final String sqlCaseURL) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new ArrayList<>();
Review Comment:
fixed
##########
.mvn/jvm.config:
##########
@@ -1 +1 @@
--Xmx2g -XX:MaxMetaspaceSize=512m
Review Comment:
Otherwise, it will fail in the progress of `mvn install`, since GC overhead limit is exceeded when running CI/IT.
##########
pom.xml:
##########
@@ -153,7 +153,8 @@
<git-commit-id-plugin.version>4.9.10</git-commit-id-plugin.version>
<freemarker.version>2.3.31</freemarker.version>
<spotless-maven-plugin.version>2.22.1</spotless-maven-plugin.version>
- <caffeine.version>2.9.3</caffeine.version>
+ <caffeine.version>2.9.2</caffeine.version>
Review Comment:
fixed
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+import org.springframework.web.client.RestTemplate;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.ArrayList;
+
+@RequiredArgsConstructor
+@SingletonSPI
Review Comment:
For passing failed test cases.
##########
test/parser/src/main/java/org/apache/shardingsphere/test/sql/parser/parameterized/engine/DynamicLoadingSQLParserParameterizedTest.java:
##########
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.sql.parser.parameterized.engine;
+
+import lombok.RequiredArgsConstructor;
+import org.apache.commons.io.IOUtils;
+import org.apache.shardingsphere.infra.util.spi.annotation.SingletonSPI;
+import org.apache.shardingsphere.sql.parser.api.CacheOption;
+import org.apache.shardingsphere.sql.parser.api.SQLParserEngine;
+import org.apache.shardingsphere.sql.parser.api.SQLVisitorEngine;
+import org.apache.shardingsphere.sql.parser.core.ParseASTNode;
+import org.junit.Test;
+import org.springframework.web.client.RestTemplate;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+import java.util.ArrayList;
+
+@RequiredArgsConstructor
+@SingletonSPI
+public abstract class DynamicLoadingSQLParserParameterizedTest {
+
+ private final String sqlCaseId;
+
+ private final String sqlCaseValue;
+
+ private final String databaseType;
+
+ private static ArrayList<Map<String, String>> getResponse(final String sqlCaseURL) {
+ String[] patches = sqlCaseURL.split("/", 8);
+ String sqlCasesOwner = patches[3];
+ String sqlCasesRepo = patches[4];
+ String sqlCasesDirectory = patches[7];
+ return new RestTemplate().getForObject(
+ "https://api.github.com/repos/{owner}/{repo}/contents/{directory}", ArrayList.class,
+ sqlCasesOwner, sqlCasesRepo, sqlCasesDirectory);
+ }
+
+ protected static Collection<Object[]> getTestParameters(final String sqlCaseURL) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new ArrayList<>();
+ ArrayList<Map<String, String>> response = getResponse(sqlCaseURL);
+ for (Map<String, String> each : response) {
+ result.addAll(getSqlCases(each));
+ }
+ return result;
+ }
+
+ private static Collection<Object[]> getSqlCases(final Map<String, String> elements) throws IOException, URISyntaxException {
+ Collection<Object[]> result = new ArrayList<>();
+ String sqlCaseFileName = elements.get("name");
+ String sqlCaseFileContent = IOUtils.toString(new URI(elements.get("download_url")), StandardCharsets.UTF_8);
+ String[] lines = sqlCaseFileContent.split("\n");
+ int sqlCaseEnum = 1;
+ for (String each : lines) {
+ if (each.isEmpty()) {
+ continue;
+ }
+ if (Character.isLetter(each.charAt(0)) && each.charAt(each.length() - 1) == ';') {
+ String sqlCaseId = sqlCaseFileName.split("\\.")[0] + sqlCaseEnum;
+ result.add(new Object[]{
+ sqlCaseId, each,
+ });
+ sqlCaseEnum++;
+ }
+ }
+ return result;
+ }
+
+ @Test(expected = Exception.class)
+ public final void assertDynamicLoadingSQL() {
+ CacheOption cacheOption = new CacheOption(128, 1024L);
+ ParseASTNode parseContext = new SQLParserEngine(databaseType, cacheOption).parse(sqlCaseValue, false);
+ new SQLVisitorEngine(databaseType, "STATEMENT", true, new Properties()).visit(parseContext);
+ System.out.println("ParserError: " + sqlCaseId + " value: " + sqlCaseValue + " db-type: " + databaseType);
+ throw new RuntimeException();
Review Comment:
This exception will be caught by @SingletonSPI to ignore and pass the failed test case. (The successful case would pass at SQLVisitorEngine().visit().)
##########
test/parser/pom.xml:
##########
@@ -117,5 +117,9 @@
<artifactId>shardingsphere-test-common</artifactId>
<version>${project.version}</version>
</dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ </dependency>
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org