You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/09/05 04:29:53 UTC
[GitHub] [incubator-seatunnel] liugddx opened a new pull request, #2646: [Feature][Connector-V2] add sqlserver connector
liugddx opened a new pull request, #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646
Signed-off-by: gdliu3 <gd...@iflytek.com>
<!--
Thank you for contributing to SeaTunnel! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
## Contribution Checklist
- Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/incubator-seatunnel/issues).
- Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
- Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
-->
## Purpose of this pull request
<!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
support sqlserver connector
## Check list
* [ ] Code changed are covered with tests, or it does not need tests for reason:
* [ ] If any new Jar binary package adding in your PR, please add License Notice according
[New License Guide](https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/contribution/new-license.md)
* [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1263418683
> > rerun CI? I don't think this mistake affects @TyrantLucifer
>
> CI had been optimized, so some steps will be enabled, before your pr is merged CI must run successfully. It's a community norm.
>
> 
>
> There are some problems with the dependency checking step, pls check again.
At now the dependency have some problem because st-engine bring some dependency but not update LICENSE file, we will update it later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1253526599
@TyrantLucifer please rerun the CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
hailin0 commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1261703976
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
TyrantLucifer commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1263376915
> rerun CI? I don't think this mistake affects @TyrantLucifer
CI had been optimized, so some steps will be enabled, before your pr is merged CI must run successfully. It's a community norm.
There are some problems with the dependency checking step, pls check again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1262382955
@TyrantLucifer @Hisoka-X PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
TyrantLucifer commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1263437481
> > > rerun CI? I don't think this mistake affects @TyrantLucifer
> >
> >
> > CI had been optimized, so some steps will be enabled, before your pr is merged CI must run successfully. It's a community norm.
> > 
> > There are some problems with the dependency checking step, pls check again.
>
> At now the dependency have some problem because st-engine bring some dependency but not update LICENSE file, we will update it later.
Good, let's waiting other steps of CI. I had restarted the failed steps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on a diff in pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
EricJoy2048 commented on code in PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#discussion_r964420300
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcSqlserverIT.java:
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.flink.v2.jdbc;
+
+import org.apache.seatunnel.e2e.flink.FlinkContainer;
+
+import lombok.extern.slf4j.Slf4j;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.MSSQLServerContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.shaded.com.google.common.collect.Lists;
+import org.testcontainers.shaded.org.awaitility.Awaitility;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.Duration;
+import java.util.List;
+import java.util.stream.Stream;
+
+@Slf4j
+public class JdbcSqlserverIT extends FlinkContainer {
+
+ private MSSQLServerContainer<?> mssqlServerContainer;
+
+ @SuppressWarnings("checkstyle:MagicNumber")
+ @BeforeEach
+ public void startSqlserverContainer() throws ClassNotFoundException, SQLException {
+ mssqlServerContainer = new MSSQLServerContainer<>(DockerImageName.parse("mcr.microsoft.com/mssql/server:2022-latest"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("sqlserver")
+ .withLogConsumer(new Slf4jLogConsumer(log));
+ Startables.deepStart(Stream.of(mssqlServerContainer)).join();
+ log.info("Sqlserver container started");
+ Class.forName(mssqlServerContainer.getDriverClassName());
+ Awaitility.given().ignoreExceptions()
+ .await()
+ .atMost(Duration.ofMinutes(1))
+ .untilAsserted(this::initializeJdbcTable);
+ batchInsertData();
+ }
+
+ private void initializeJdbcTable() {
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ String sourceSql = "CREATE TABLE [source] (\n" +
+ " [ids] bigint NOT NULL,\n" +
+ " [name] text COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sfzh] varchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sort] int NULL,\n" +
+ " [dz] nvarchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xchar] char(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xdecimal] decimal(18) NULL,\n" +
+ " [xfloat] float(53) NULL,\n" +
+ " [xnumeric] numeric(18) NULL,\n" +
+ " [xsmall] smallint NULL,\n" +
+ " [xbit] bit NULL,\n" +
+ " [rq] datetime DEFAULT NULL NULL,\n" +
+ " [xrq] smalldatetime NULL,\n" +
+ " [xreal] real NULL,\n" +
+ " [ximage] image NULL\n" +
+ ")";
+ String sinkSql = "CREATE TABLE [sink] (\n" +
+ " [ids] bigint NOT NULL,\n" +
+ " [name] text COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sfzh] varchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sort] int NULL,\n" +
+ " [dz] nvarchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xchar] char(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xdecimal] decimal(18) NULL,\n" +
+ " [xfloat] float(53) NULL,\n" +
+ " [xnumeric] numeric(18) NULL,\n" +
+ " [xsmall] smallint NULL,\n" +
+ " [xbit] bit NULL,\n" +
+ " [rq] datetime DEFAULT NULL NULL,\n" +
+ " [xrq] smalldatetime NULL,\n" +
+ " [xreal] real NULL,\n" +
+ " [ximage] image NULL\n" +
+ ")";
+ statement.execute(sourceSql);
+ statement.execute(sinkSql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Initializing Sqlserver table failed!", e);
+ }
+ }
+
+ @SuppressWarnings("checkstyle:RegexpSingleline")
+ private void batchInsertData() {
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ String sql =
+ "INSERT INTO [source] ([ids], [name], [sfzh], [sort], [dz], [xchar], [xdecimal], [xfloat], [xnumeric], [xsmall], [xbit], [rq], [xrq], [xreal], [ximage]) " +
+ "VALUES (1504057, '张三', '3ee98c990e2011eda8fd00ff27b3340d', 1, N'3232', 'qwq', 1, 19.1, 2, 1, '0', '2022-07-26 11:58:46.000', '2022-07-26 13:49:00', 2, 0x)";
+ Statement statement = connection.createStatement();
+ statement.execute(sql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Batch insert data failed!", e);
+ }
+ }
+
+ @Test
+ public void tesSqlserverSourceAndSink() throws SQLException, IOException, InterruptedException {
+ Container.ExecResult execResult = executeSeaTunnelFlinkJob("/jdbc/jdbc_sqlserver_source_to_sink.conf");
+ Assertions.assertEquals(0, execResult.getExitCode());
+ // query result
+ String sql = "select * from sink";
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ ResultSet resultSet = statement.executeQuery(sql);
+ List<String> result = Lists.newArrayList();
+ while (resultSet.next()) {
+ result.add(resultSet.getString("ids"));
+ }
+ Assertions.assertFalse(result.isEmpty());
Review Comment:
same as above
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-jdbc-flink-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcSqlserverIT.java:
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.flink.v2.jdbc;
+
+import org.apache.seatunnel.e2e.flink.FlinkContainer;
+
+import lombok.extern.slf4j.Slf4j;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.MSSQLServerContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.shaded.com.google.common.collect.Lists;
+import org.testcontainers.shaded.org.awaitility.Awaitility;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.Duration;
+import java.util.List;
+import java.util.stream.Stream;
+
+@Slf4j
+public class JdbcSqlserverIT extends FlinkContainer {
+
+ private MSSQLServerContainer<?> mssqlServerContainer;
+
+ @SuppressWarnings("checkstyle:MagicNumber")
+ @BeforeEach
+ public void startSqlserverContainer() throws ClassNotFoundException, SQLException {
+ mssqlServerContainer = new MSSQLServerContainer<>(DockerImageName.parse("mcr.microsoft.com/mssql/server:2022-latest"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("sqlserver")
+ .withLogConsumer(new Slf4jLogConsumer(log));
+ Startables.deepStart(Stream.of(mssqlServerContainer)).join();
+ log.info("Sqlserver container started");
+ Class.forName(mssqlServerContainer.getDriverClassName());
+ Awaitility.given().ignoreExceptions()
+ .await()
+ .atMost(Duration.ofMinutes(1))
+ .untilAsserted(this::initializeJdbcTable);
+ batchInsertData();
+ }
+
+ private void initializeJdbcTable() {
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ String sourceSql = "CREATE TABLE [source] (\n" +
+ " [ids] bigint NOT NULL,\n" +
+ " [name] text COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sfzh] varchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sort] int NULL,\n" +
+ " [dz] nvarchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xchar] char(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xdecimal] decimal(18) NULL,\n" +
+ " [xfloat] float(53) NULL,\n" +
+ " [xnumeric] numeric(18) NULL,\n" +
+ " [xsmall] smallint NULL,\n" +
+ " [xbit] bit NULL,\n" +
+ " [rq] datetime DEFAULT NULL NULL,\n" +
+ " [xrq] smalldatetime NULL,\n" +
+ " [xreal] real NULL,\n" +
+ " [ximage] image NULL\n" +
+ ")";
+ String sinkSql = "CREATE TABLE [sink] (\n" +
+ " [ids] bigint NOT NULL,\n" +
+ " [name] text COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sfzh] varchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sort] int NULL,\n" +
+ " [dz] nvarchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xchar] char(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xdecimal] decimal(18) NULL,\n" +
+ " [xfloat] float(53) NULL,\n" +
+ " [xnumeric] numeric(18) NULL,\n" +
+ " [xsmall] smallint NULL,\n" +
+ " [xbit] bit NULL,\n" +
+ " [rq] datetime DEFAULT NULL NULL,\n" +
+ " [xrq] smalldatetime NULL,\n" +
+ " [xreal] real NULL,\n" +
+ " [ximage] image NULL\n" +
+ ")";
+ statement.execute(sourceSql);
+ statement.execute(sinkSql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Initializing Sqlserver table failed!", e);
+ }
+ }
+
+ @SuppressWarnings("checkstyle:RegexpSingleline")
+ private void batchInsertData() {
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ String sql =
+ "INSERT INTO [source] ([ids], [name], [sfzh], [sort], [dz], [xchar], [xdecimal], [xfloat], [xnumeric], [xsmall], [xbit], [rq], [xrq], [xreal], [ximage]) " +
+ "VALUES (1504057, '张三', '3ee98c990e2011eda8fd00ff27b3340d', 1, N'3232', 'qwq', 1, 19.1, 2, 1, '0', '2022-07-26 11:58:46.000', '2022-07-26 13:49:00', 2, 0x)";
+ Statement statement = connection.createStatement();
+ statement.execute(sql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Batch insert data failed!", e);
+ }
+ }
+
+ @Test
+ public void tesSqlserverSourceAndSink() throws SQLException, IOException, InterruptedException {
+ Container.ExecResult execResult = executeSeaTunnelFlinkJob("/jdbc/jdbc_sqlserver_source_to_sink.conf");
+ Assertions.assertEquals(0, execResult.getExitCode());
+ // query result
+ String sql = "select * from sink";
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ ResultSet resultSet = statement.executeQuery(sql);
+ List<String> result = Lists.newArrayList();
+ while (resultSet.next()) {
+ result.add(resultSet.getString("ids"));
+ }
+ Assertions.assertFalse(result.isEmpty());
Review Comment:
Can you assert all the columns of the data?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ic4y commented on a diff in pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
ic4y commented on code in PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#discussion_r974849230
##########
seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-jdbc-spark-e2e/src/test/resources/container-license-acceptance.txt:
##########
@@ -0,0 +1 @@
+mcr.microsoft.com/mssql/server:2022-latest
Review Comment:
And this file.
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-jdbc-flink-e2e/src/test/resources/container-license-acceptance.txt:
##########
@@ -0,0 +1 @@
+mcr.microsoft.com/mssql/server:2022-latest
Review Comment:
Why do you need to add this file？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1261803876
please rerun CI thanks @TyrantLucifer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ic4y commented on a diff in pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
ic4y commented on code in PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#discussion_r964413514
##########
pom.xml:
##########
@@ -344,14 +344,22 @@
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4j.version}</version>
</dependency>
-
+
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>${commons.logging.version}</version>
</dependency>
<!--springboot-->
+ <dependency>
+ <groupId>org.springframework.boot</groupId>
+ <artifactId>spring-boot-dependencies</artifactId>
+ <version>${spring-boot.version}</version>
+ <type>pom</type>
+ <scope>import</scope>
+ </dependency>
+
Review Comment:
Why add this dependency and where is it used?
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/resources/container-license-acceptance.txt:
##########
@@ -0,0 +1 @@
+mcr.microsoft.com/mssql/server:2022-latest
Review Comment:
Why do you need to add this file in this location
##########
seatunnel-connectors-v2/connector-jdbc/pom.xml:
##########
@@ -50,11 +50,17 @@
<version>${pg.version}</version>
</dependency>
+ <dependency>
+ <groupId>com.microsoft.sqlserver</groupId>
+ <artifactId>mssql-jdbc</artifactId>
+ <scope>provided</scope>
Review Comment:
add version
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/resources/jdbc/jdbc_sqlserver_source_to_sink.conf:
##########
@@ -0,0 +1,60 @@
+#
Review Comment:
duplicate files
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-jdbc-flink-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcSqlserverIT.java:
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.flink.v2.jdbc;
+
+import org.apache.seatunnel.e2e.flink.FlinkContainer;
+
+import lombok.extern.slf4j.Slf4j;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.MSSQLServerContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.shaded.com.google.common.collect.Lists;
+import org.testcontainers.shaded.org.awaitility.Awaitility;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.Duration;
+import java.util.List;
+import java.util.stream.Stream;
+
+@Slf4j
+public class JdbcSqlserverIT extends FlinkContainer {
+
+ private MSSQLServerContainer<?> mssqlServerContainer;
+
+ @SuppressWarnings("checkstyle:MagicNumber")
+ @BeforeEach
+ public void startSqlserverContainer() throws ClassNotFoundException, SQLException {
+ mssqlServerContainer = new MSSQLServerContainer<>(DockerImageName.parse("mcr.microsoft.com/mssql/server:2022-latest"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("sqlserver")
+ .withLogConsumer(new Slf4jLogConsumer(log));
+ Startables.deepStart(Stream.of(mssqlServerContainer)).join();
+ log.info("Sqlserver container started");
+ Class.forName(mssqlServerContainer.getDriverClassName());
+ Awaitility.given().ignoreExceptions()
+ .await()
+ .atMost(Duration.ofMinutes(1))
+ .untilAsserted(this::initializeJdbcTable);
+ batchInsertData();
+ }
+
+ private void initializeJdbcTable() {
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ String sourceSql = "CREATE TABLE [source] (\n" +
+ " [ids] bigint NOT NULL,\n" +
+ " [name] text COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sfzh] varchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sort] int NULL,\n" +
+ " [dz] nvarchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xchar] char(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xdecimal] decimal(18) NULL,\n" +
+ " [xfloat] float(53) NULL,\n" +
+ " [xnumeric] numeric(18) NULL,\n" +
+ " [xsmall] smallint NULL,\n" +
+ " [xbit] bit NULL,\n" +
+ " [rq] datetime DEFAULT NULL NULL,\n" +
+ " [xrq] smalldatetime NULL,\n" +
+ " [xreal] real NULL,\n" +
+ " [ximage] image NULL\n" +
+ ")";
+ String sinkSql = "CREATE TABLE [sink] (\n" +
+ " [ids] bigint NOT NULL,\n" +
+ " [name] text COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sfzh] varchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sort] int NULL,\n" +
+ " [dz] nvarchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xchar] char(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xdecimal] decimal(18) NULL,\n" +
+ " [xfloat] float(53) NULL,\n" +
+ " [xnumeric] numeric(18) NULL,\n" +
+ " [xsmall] smallint NULL,\n" +
+ " [xbit] bit NULL,\n" +
+ " [rq] datetime DEFAULT NULL NULL,\n" +
+ " [xrq] smalldatetime NULL,\n" +
+ " [xreal] real NULL,\n" +
+ " [ximage] image NULL\n" +
+ ")";
+ statement.execute(sourceSql);
+ statement.execute(sinkSql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Initializing Sqlserver table failed!", e);
+ }
+ }
+
+ @SuppressWarnings("checkstyle:RegexpSingleline")
+ private void batchInsertData() {
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ String sql =
+ "INSERT INTO [source] ([ids], [name], [sfzh], [sort], [dz], [xchar], [xdecimal], [xfloat], [xnumeric], [xsmall], [xbit], [rq], [xrq], [xreal], [ximage]) " +
+ "VALUES (1504057, '张三', '3ee98c990e2011eda8fd00ff27b3340d', 1, N'3232', 'qwq', 1, 19.1, 2, 1, '0', '2022-07-26 11:58:46.000', '2022-07-26 13:49:00', 2, 0x)";
+ Statement statement = connection.createStatement();
+ statement.execute(sql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Batch insert data failed!", e);
+ }
+ }
+
+ @Test
+ public void tesSqlserverSourceAndSink() throws SQLException, IOException, InterruptedException {
+ Container.ExecResult execResult = executeSeaTunnelFlinkJob("/jdbc/jdbc_sqlserver_source_to_sink.conf");
+ Assertions.assertEquals(0, execResult.getExitCode());
+ // query result
+ String sql = "select * from sink";
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ ResultSet resultSet = statement.executeQuery(sql);
+ List<String> result = Lists.newArrayList();
+ while (resultSet.next()) {
+ result.add(resultSet.getString("ids"));
+ }
+ Assertions.assertFalse(result.isEmpty());
Review Comment:
Check whether the data in the source table and sink table are consistent, just checking the amount of data is not accurate enough
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcSqlserverIT.java:
##########
@@ -0,0 +1,148 @@
+/*
Review Comment:
This file is duplicated and in the wrong location, delete it
##########
seatunnel-e2e/seatunnel-spark-connector-v2-e2e/src/test/resources/jdbc/jdbc_sqlserver_source_to_sink.conf:
##########
@@ -0,0 +1,62 @@
+#
Review Comment:
duplicate files
##########
seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-jdbc-spark-e2e/src/test/java/org/apache/seatunnel/e2e/spark/v2/jdbc/JdbcSqlserverIT.java:
##########
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.spark.v2.jdbc;
+
+import org.apache.seatunnel.e2e.spark.SparkContainer;
+
+import lombok.extern.slf4j.Slf4j;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.MSSQLServerContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.shaded.com.google.common.collect.Lists;
+import org.testcontainers.shaded.org.awaitility.Awaitility;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.Duration;
+import java.util.List;
+import java.util.stream.Stream;
+
+@Slf4j
+public class JdbcSqlserverIT extends SparkContainer {
+
+ private MSSQLServerContainer<?> mssqlServerContainer;
+
+ @SuppressWarnings("checkstyle:MagicNumber")
+ @BeforeEach
+ public void startSqlServerContainer() throws ClassNotFoundException, SQLException {
+ mssqlServerContainer = new MSSQLServerContainer<>(DockerImageName.parse("mcr.microsoft.com/mssql/server:2022-latest"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("sqlserver")
+ .withLogConsumer(new Slf4jLogConsumer(log));
+ Startables.deepStart(Stream.of(mssqlServerContainer)).join();
+ log.info("Sqlserver container started");
+ Class.forName(mssqlServerContainer.getDriverClassName());
+ Awaitility.given().ignoreExceptions()
+ .await()
+ .atMost(Duration.ofMinutes(1))
+ .untilAsserted(this::initializeJdbcTable);
+ batchInsertData();
+ }
+
+ private void initializeJdbcTable() {
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ String sourceSql = "CREATE TABLE [source] (\n" +
+ " [ids] bigint NOT NULL,\n" +
+ " [name] text COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sfzh] varchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sort] int NULL,\n" +
+ " [dz] nvarchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xchar] char(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xdecimal] decimal(18) NULL,\n" +
+ " [xfloat] float(53) NULL,\n" +
+ " [xnumeric] numeric(18) NULL,\n" +
+ " [xsmall] smallint NULL,\n" +
+ " [xbit] bit NULL,\n" +
+ " [rq] datetime DEFAULT NULL NULL,\n" +
+ " [xrq] smalldatetime NULL,\n" +
+ " [xreal] real NULL,\n" +
+ " [ximage] image NULL\n" +
+ ")";
+ String sinkSql = "CREATE TABLE [sink] (\n" +
+ " [ids] bigint NOT NULL,\n" +
+ " [name] text COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sfzh] varchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [sort] int NULL,\n" +
+ " [dz] nvarchar(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xchar] char(255) COLLATE Chinese_PRC_CI_AS NULL,\n" +
+ " [xdecimal] decimal(18) NULL,\n" +
+ " [xfloat] float(53) NULL,\n" +
+ " [xnumeric] numeric(18) NULL,\n" +
+ " [xsmall] smallint NULL,\n" +
+ " [xbit] bit NULL,\n" +
+ " [rq] datetime DEFAULT NULL NULL,\n" +
+ " [xrq] smalldatetime NULL,\n" +
+ " [xreal] real NULL,\n" +
+ " [ximage] image NULL\n" +
+ ")";
+ statement.execute(sourceSql);
+ statement.execute(sinkSql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Initializing Sqlserver table failed!", e);
+ }
+ }
+
+ @SuppressWarnings("checkstyle:RegexpSingleline")
+ private void batchInsertData() {
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ String sql =
+ "INSERT INTO [source] ([ids], [name], [sfzh], [sort], [dz], [xchar], [xdecimal], [xfloat], [xnumeric], [xsmall], [xbit], [rq], [xrq], [xreal], [ximage]) " +
+ "VALUES (1504057, '张三', '3ee98c990e2011eda8fd00ff27b3340d', 1, N'3232', 'qwq', 1, 19.1, 2, 1, '0', '2022-07-26 11:58:46.000', '2022-07-26 13:49:00', 2, 0x)";
+ Statement statement = connection.createStatement();
+ statement.execute(sql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Batch insert data failed!", e);
+ }
+ }
+
+ @Test
+ public void tesSqlserverSourceAndSink() throws SQLException, IOException, InterruptedException {
+ Container.ExecResult execResult = executeSeaTunnelSparkJob("/jdbc/jdbc_sqlserver_source_to_sink.conf");
+ Assertions.assertEquals(0, execResult.getExitCode());
+ // query result
+ String sql = "select * from sink";
+ try (Connection connection = DriverManager.getConnection(mssqlServerContainer.getJdbcUrl(), mssqlServerContainer.getUsername(), mssqlServerContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ ResultSet resultSet = statement.executeQuery(sql);
+ List<String> result = Lists.newArrayList();
+ while (resultSet.next()) {
+ result.add(resultSet.getString("ids"));
+ }
+ Assertions.assertFalse(result.isEmpty());
Review Comment:
same as flink
##########
seatunnel-e2e/seatunnel-spark-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/spark/v2/jdbc/JdbcSqlserverIT.java:
##########
@@ -0,0 +1,148 @@
+/*
Review Comment:
duplicate files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1250250673
PTAL @ic4y @EricJoy2048 @CalvinKirs thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1253157411
@Hisoka-X @ashulin @TyrantLucifer @ic4y change `/tmp/spark/seatunnel/` to `/tmp/seatunnel/` PTAL ,thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1263418702
> > rerun CI? I don't think this mistake affects @TyrantLucifer
>
> CI had been optimized, so some steps will be enabled, before your pr is merged CI must run successfully. It's a community norm.
>
> 
>
> There are some problems with the dependency checking step, pls check again.
At now the dependency have some problem because st-engine bring some dependency but not update LICENSE file, we will update it later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on a diff in pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on code in PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#discussion_r967180744
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/resources/container-license-acceptance.txt:
##########
@@ -0,0 +1 @@
+mcr.microsoft.com/mssql/server:2022-latest
Review Comment:
the sqlserver container in testcontainers need this file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on code in PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#discussion_r974914305
##########
docs/en/connector-v2/sink/Jdbc.md:
##########
@@ -32,61 +35,85 @@ Use `Xa transactions` to ensure `exactly-once`. So only support `exactly-once` f
| transaction_timeout_sec | Int | No | -1 |
### driver [string]
+
The jdbc class name used to connect to the remote data source, if you use MySQL the value is com.mysql.cj.jdbc.Driver.
-Warn: for license compliance, you have to provide MySQL JDBC driver yourself, e.g. copy mysql-connector-java-xxx.jar to $SEATNUNNEL_HOME/lib for Standalone.
+Warn: for license compliance, you have to provide MySQL JDBC driver yourself, e.g. copy mysql-connector-java-xxx.jar to
Review Comment:
```suggestion
Warn: for license compliance, you have to provide any driver yourself like MySQL JDBC Driver, e.g. copy mysql-connector-java-xxx.jar to
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on a diff in pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
TyrantLucifer commented on code in PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#discussion_r984252829
##########
seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/AbstractFlinkContainer.java:
##########
@@ -32,22 +31,19 @@
* The before method will create a Flink cluster, and after method will close the Flink cluster.
* You can use {@link AbstractFlinkContainer#executeJob} to submit a seatunnel config and run a seatunnel job.
*/
-@Slf4j
Review Comment:
> > I found out that all the integration tests had failed,and the reason is log be inherited.So i fix it provisionally.
> > @hailin0 @TyrantLucifer @ashulin
>
> @hailin0 PTAL
#2946 has resolved this problem, these files should be reverted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1250250850
> PTAL @ic4y @EricJoy2048 @CalvinKirs thanks.
e2e test pass
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1256964174
PTAL @CalvinKirs @TyrantLucifer @Hisoka-X .thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on a diff in pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on code in PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#discussion_r983085312
##########
seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/AbstractFlinkContainer.java:
##########
@@ -32,22 +31,19 @@
* The before method will create a Flink cluster, and after method will close the Flink cluster.
* You can use {@link AbstractFlinkContainer#executeJob} to submit a seatunnel config and run a seatunnel job.
*/
-@Slf4j
Review Comment:
I found out that all the integration tests had failed,and the reason is log be inherited.So i fix it provisionally.
@hailin0 @TyrantLucifer @ashulin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ic4y commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
ic4y commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1251799479
LGTM @Hisoka-X
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1254445133
@ashulin @TyrantLucifer @CalvinKirs @Hisoka-X @EricJoy2048 PTAL thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on a diff in pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on code in PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#discussion_r974856648
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-jdbc-flink-e2e/src/test/resources/container-license-acceptance.txt:
##########
@@ -0,0 +1 @@
+mcr.microsoft.com/mssql/server:2022-latest
Review Comment:
[https://www.testcontainers.org/modules/databases/mssqlserver/](mssqlserver)
[https://hub.docker.com/_/microsoft-mssql-server#environment-variables](microsoft-mssql-server)
the image `mcr.microsoft.com/mssql/server` need to accept an EULA .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1263367723
rerun CI? I don't think this mistake affects @TyrantLucifer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on a diff in pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
EricJoy2048 commented on code in PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#discussion_r984244658
##########
seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/AbstractFlinkContainer.java:
##########
@@ -32,22 +31,19 @@
* The before method will create a Flink cluster, and after method will close the Flink cluster.
* You can use {@link AbstractFlinkContainer#executeJob} to submit a seatunnel config and run a seatunnel job.
*/
-@Slf4j
Review Comment:
> I found out that all the integration tests had failed,and the reason is log be inherited.So i fix it provisionally.
>
> @hailin0 @TyrantLucifer @ashulin
@hailin0 PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1252148771
PTAL @CalvinKirs @TyrantLucifer @EricJoy2048 .thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] githublaohu commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
githublaohu commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1237392688
是否应该是jdbc-connent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on a diff in pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
TyrantLucifer commented on code in PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#discussion_r984248511
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/connector-jdbc-flink-e2e/pom.xml:
##########
@@ -97,6 +97,17 @@
<artifactId>ali-phoenix-shaded-thin-client</artifactId>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.testcontainers</groupId>
+ <artifactId>mssqlserver</artifactId>
+ <version>1.17.3</version>
Review Comment:
Using `${testcontainer.version}` instead of it.
##########
seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/AbstractSparkContainer.java:
##########
@@ -19,29 +19,25 @@
import org.apache.seatunnel.e2e.common.container.spark.AbstractTestSparkContainer;
-import lombok.extern.slf4j.Slf4j;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.TestInstance;
import org.testcontainers.containers.Container;
import java.io.IOException;
-@Slf4j
Review Comment:
Please revert this file because pr #2946 has resolved problems of e2e test.
##########
seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/AbstractFlinkContainer.java:
##########
@@ -32,22 +31,19 @@
* The before method will create a Flink cluster, and after method will close the Flink cluster.
* You can use {@link AbstractFlinkContainer#executeJob} to submit a seatunnel config and run a seatunnel job.
*/
-@Slf4j
Review Comment:
Please revert this file because pr #2946 has resolved problems of e2e test.
##########
seatunnel-e2e/seatunnel-spark-connector-v2-e2e/connector-jdbc-spark-e2e/pom.xml:
##########
@@ -86,6 +86,16 @@
<artifactId>ali-phoenix-shaded-thin-client</artifactId>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.testcontainers</groupId>
+ <artifactId>mssqlserver</artifactId>
+ <version>1.17.3</version>
Review Comment:
Use `${testcontainer.version}` instead of it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer merged pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
TyrantLucifer merged PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on a diff in pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on code in PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#discussion_r984262635
##########
seatunnel-e2e/seatunnel-e2e-common/src/test/java/org/apache/seatunnel/e2e/common/AbstractFlinkContainer.java:
##########
@@ -32,22 +31,19 @@
* The before method will create a Flink cluster, and after method will close the Flink cluster.
* You can use {@link AbstractFlinkContainer#executeJob} to submit a seatunnel config and run a seatunnel job.
*/
-@Slf4j
Review Comment:
> > > I found out that all the integration tests had failed,and the reason is log be inherited.So i fix it provisionally.
> > > @hailin0 @TyrantLucifer @ashulin
> >
> >
> > @hailin0 PTAL
>
> #2946 has resolved this problem, these files should be reverted.
yes. I've revert.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on pull request #2646: [Feature][Connector-V2] add sqlserver connector
Posted by GitBox <gi...@apache.org>.
TyrantLucifer commented on PR #2646:
URL: https://github.com/apache/incubator-seatunnel/pull/2646#issuecomment-1261806085
> please rerun CI thanks @TyrantLucifer
Wait until the running workflow is finished before restarting. Please be patient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org