You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/08/28 10:53:11 UTC
[GitHub] [incubator-seatunnel] liugddx opened a new pull request, #2550: [Feature][Connector-V2] oracle connector
liugddx opened a new pull request, #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550
<!--
Thank you for contributing to SeaTunnel! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
## Contribution Checklist
- Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/incubator-seatunnel/issues).
- Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
- Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
-->
## Purpose of this pull request
<!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
new v2 connector for oracle
## Check list
* [ ] Code changed are covered with tests, or it does not need tests for reason:
* [ ] If any new Jar binary package adding in your PR, please add License Notice according
[New License Guide](https://github.com/apache/incubator-seatunnel/blob/dev/docs/en/contribution/new-license.md)
* [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1272250682
Already add oracle doc. @EricJoy2048
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on a diff in pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
hailin0 commented on code in PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#discussion_r964598711
##########
seatunnel-connectors-v2/connector-jdbc/pom.xml:
##########
@@ -55,6 +56,13 @@
<artifactId>ali-phoenix-shaded-thin-client</artifactId>
<version>${phoenix.version}</version>
</dependency>
+
+ <dependency>
+ <groupId>com.oracle.database.jdbc</groupId>
+ <artifactId>ojdbc8</artifactId>
+ <version>${oracle.version}</version>
+ <scope>test</scope>
Review Comment:
use `provided`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1232517646
CI is not stable. @CalvinKirs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
EricJoy2048 commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1272342230
Please merge the dev branch to your branch and push again because https://github.com/apache/incubator-seatunnel/pull/3022
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on a diff in pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on code in PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#discussion_r957938758
##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/oracle/OracleTypeMapper.java:
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.connectors.seatunnel.jdbc.internal.dialect.oracle;
+
+import org.apache.seatunnel.api.table.type.BasicType;
+import org.apache.seatunnel.api.table.type.DecimalType;
+import org.apache.seatunnel.api.table.type.LocalTimeType;
+import org.apache.seatunnel.api.table.type.PrimitiveByteArrayType;
+import org.apache.seatunnel.api.table.type.SeaTunnelDataType;
+import org.apache.seatunnel.connectors.seatunnel.jdbc.internal.dialect.JdbcDialect;
+import org.apache.seatunnel.connectors.seatunnel.jdbc.internal.dialect.JdbcDialectTypeMapper;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.sql.ResultSetMetaData;
+import java.sql.SQLException;
+
+public class OracleTypeMapper implements JdbcDialectTypeMapper {
+
+ private static final Logger LOG = LoggerFactory.getLogger(JdbcDialect.class);
Review Comment:
> We used @slf4j logging annotation.
i will fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1232371917
Hi, rerun, due to its large size
On 08/31/2022 ***@***.***> wrote：
CI is error ,and the error is JdbcPhoenixIT .@CalvinKirs
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs commented on a diff in pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on code in PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#discussion_r957932043
##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/oracle/OracleTypeMapper.java:
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.connectors.seatunnel.jdbc.internal.dialect.oracle;
+
+import org.apache.seatunnel.api.table.type.BasicType;
+import org.apache.seatunnel.api.table.type.DecimalType;
+import org.apache.seatunnel.api.table.type.LocalTimeType;
+import org.apache.seatunnel.api.table.type.PrimitiveByteArrayType;
+import org.apache.seatunnel.api.table.type.SeaTunnelDataType;
+import org.apache.seatunnel.connectors.seatunnel.jdbc.internal.dialect.JdbcDialect;
+import org.apache.seatunnel.connectors.seatunnel.jdbc.internal.dialect.JdbcDialectTypeMapper;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.sql.ResultSetMetaData;
+import java.sql.SQLException;
+
+public class OracleTypeMapper implements JdbcDialectTypeMapper {
+
+ private static final Logger LOG = LoggerFactory.getLogger(JdbcDialect.class);
Review Comment:
We used @Slf4j logging annotation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1231047038
> CI is error. how do i fix it?
maven network connection error, I rerun again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1229431793
#2550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on a diff in pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
hailin0 commented on code in PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#discussion_r960162084
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcOracleIT.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.flink.v2.jdbc;
+
+import org.apache.seatunnel.e2e.flink.FlinkContainer;
+
+import com.google.common.collect.Lists;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.OracleContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.List;
+import java.util.stream.Stream;
+
+public class JdbcOracleIT extends FlinkContainer {
+ private static final Logger LOGGER = LoggerFactory.getLogger(JdbcSourceToConsoleIT.class);
+ private OracleContainer oracleContainer;
+
+ @BeforeEach
+ public void startOracleContainer() throws InterruptedException, ClassNotFoundException, SQLException {
+ oracleContainer = new OracleContainer(DockerImageName.parse("gvenzl/oracle-xe:18.4.0-slim"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("oracle")
+ .withLogConsumer(new Slf4jLogConsumer(LOGGER));
+ Startables.deepStart(Stream.of(oracleContainer)).join();
+ LOGGER.info("Oracle container started");
+ Thread.sleep(5000L);
+ Class.forName(oracleContainer.getDriverClassName());
+ initializeJdbcTable();
+ batchInsertData();
+ }
+
+ private void initializeJdbcTable() {
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ String sourceSql = "CREATE TABLE source (\n" +
+ " name varchar(255) NOT NULL\n" +
Review Comment:
Test all oracle data types
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcOracleIT.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.flink.v2.jdbc;
+
+import org.apache.seatunnel.e2e.flink.FlinkContainer;
+
+import com.google.common.collect.Lists;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.OracleContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.List;
+import java.util.stream.Stream;
+
+public class JdbcOracleIT extends FlinkContainer {
+ private static final Logger LOGGER = LoggerFactory.getLogger(JdbcSourceToConsoleIT.class);
+ private OracleContainer oracleContainer;
+
+ @BeforeEach
+ public void startOracleContainer() throws InterruptedException, ClassNotFoundException, SQLException {
+ oracleContainer = new OracleContainer(DockerImageName.parse("gvenzl/oracle-xe:18.4.0-slim"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("oracle")
+ .withLogConsumer(new Slf4jLogConsumer(LOGGER));
+ Startables.deepStart(Stream.of(oracleContainer)).join();
+ LOGGER.info("Oracle container started");
+ Thread.sleep(5000L);
+ Class.forName(oracleContainer.getDriverClassName());
+ initializeJdbcTable();
+ batchInsertData();
+ }
+
+ private void initializeJdbcTable() {
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ String sourceSql = "CREATE TABLE source (\n" +
+ " name varchar(255) NOT NULL\n" +
+ ")";
+ String sinkSql = "CREATE TABLE sink (\n" +
+ " name varchar(255) NOT NULL\n" +
+ ")";
+ statement.execute(sourceSql);
+ statement.execute(sinkSql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Initializing Oracle table failed!", e);
+ }
+ }
+
+ @SuppressWarnings("checkstyle:MagicNumber")
+ private void batchInsertData() {
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ String sql = "insert into source(name) values(?)";
+ connection.setAutoCommit(false);
+ PreparedStatement preparedStatement = connection.prepareStatement(sql);
+ for (int i = 0; i < 10; i++) {
+ preparedStatement.setString(1, "Mike_" + i);
+ preparedStatement.addBatch();
+ }
+ preparedStatement.executeBatch();
+ connection.commit();
+ } catch (SQLException e) {
+ throw new RuntimeException("Batch insert data failed!", e);
+ }
+ }
+
+ @Test
+ public void testFakeSourceToJdbcSink() throws SQLException, IOException, InterruptedException {
+ Container.ExecResult execResult = executeSeaTunnelFlinkJob("/jdbc/jdbc_oracle_source_to_sink.conf");
+ Assertions.assertEquals(0, execResult.getExitCode());
+ // query result
+ String sql = "select * from sink";
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ ResultSet resultSet = statement.executeQuery(sql);
+ List<String> result = Lists.newArrayList();
+ while (resultSet.next()) {
+ result.add(resultSet.getString("name"));
+ }
+ Assertions.assertFalse(result.isEmpty());
+ }
+ }
+
+ @AfterEach
+ public void closePostgreSqlContainer() {
Review Comment:
rename `PostgreSql` to `Oracle`
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcOracleIT.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.flink.v2.jdbc;
+
+import org.apache.seatunnel.e2e.flink.FlinkContainer;
+
+import com.google.common.collect.Lists;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.OracleContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.List;
+import java.util.stream.Stream;
+
+public class JdbcOracleIT extends FlinkContainer {
+ private static final Logger LOGGER = LoggerFactory.getLogger(JdbcSourceToConsoleIT.class);
Review Comment:
replace `JdbcSourceToConsoleIT` to `JdbcOracleIT`
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcOracleIT.java:
##########
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.flink.v2.jdbc;
+
+import org.apache.seatunnel.e2e.flink.FlinkContainer;
+
+import com.google.common.collect.Lists;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.OracleContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.List;
+import java.util.stream.Stream;
+
+public class JdbcOracleIT extends FlinkContainer {
+ private static final Logger LOGGER = LoggerFactory.getLogger(JdbcSourceToConsoleIT.class);
+ private OracleContainer oracleContainer;
+
+ @BeforeEach
+ public void startOracleContainer() throws InterruptedException, ClassNotFoundException, SQLException {
+ oracleContainer = new OracleContainer(DockerImageName.parse("gvenzl/oracle-xe:18.4.0-slim"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("oracle")
+ .withLogConsumer(new Slf4jLogConsumer(LOGGER));
+ Startables.deepStart(Stream.of(oracleContainer)).join();
+ LOGGER.info("Oracle container started");
+ Thread.sleep(5000L);
+ Class.forName(oracleContainer.getDriverClassName());
+ initializeJdbcTable();
Review Comment:
Use `Awaitility`?
example
```java
Class.forName(oracleContainer.getDriverClassName())
Awaitility.given().ignoreExceptions()
.await()
.atMost(xxx, TimeUnit.SECONDS)
.untilAsserted(this::initializeJdbcTable);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1229433462
Can you give a quick review of #2250 review? Some places have not been modified
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ic4y commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
ic4y commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1239183357
Please add connector doc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1232363107
CI is error ,and the error is JdbcPhoenixIT .@CalvinKirs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on a diff in pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on code in PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#discussion_r957938417
##########
pom.xml:
##########
@@ -358,6 +359,12 @@
<version>${postgresql.version}</version>
</dependency>
+ <dependency>
+ <groupId>com.oracle.database.jdbc</groupId>
+ <artifactId>ojdbc8</artifactId>
+ <version>${oracle.version}</version>
Review Comment:
ok. i will fix it and explain in the document
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
hailin0 commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1239128897
Need to merge #2648
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1272223180
@ic4y @EricJoy2048 @Hisoka-X @hailin0 please help to review ,and rerun CI.Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs commented on a diff in pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on code in PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#discussion_r957929981
##########
pom.xml:
##########
@@ -358,6 +359,12 @@
<version>${postgresql.version}</version>
</dependency>
+ <dependency>
+ <groupId>com.oracle.database.jdbc</groupId>
+ <artifactId>ojdbc8</artifactId>
+ <version>${oracle.version}</version>
Review Comment:
you need add scope `<scope>test</scope>`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1230347086
@CalvinKirs flink e2e fake2oracle is ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1236028600
> LGTM @CalvinKirs
@CalvinKirs pls review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on a diff in pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
hailin0 commented on code in PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#discussion_r961302457
##########
seatunnel-e2e/seatunnel-spark-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/spark/v2/jdbc/JdbcOracleIT.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.spark.v2.jdbc;
+
+import org.apache.seatunnel.e2e.spark.SparkContainer;
+
+import com.google.common.collect.Lists;
+import lombok.extern.slf4j.Slf4j;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.OracleContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.shaded.org.awaitility.Awaitility;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.Duration;
+import java.util.List;
+import java.util.stream.Stream;
+
+@Slf4j
+public class JdbcOracleIT extends SparkContainer {
+
+ private OracleContainer oracleContainer;
+
+ @BeforeEach
+ public void startOracleContainer() throws ClassNotFoundException {
+ oracleContainer = new OracleContainer(DockerImageName.parse("gvenzl/oracle-xe:18.4.0-slim"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("oracle")
+ .withLogConsumer(new Slf4jLogConsumer(log));
+ Startables.deepStart(Stream.of(oracleContainer)).join();
+ log.info("Oracle container started");
+ Class.forName(oracleContainer.getDriverClassName());
+ Awaitility.given().ignoreExceptions()
+ .await()
+ .atMost(Duration.ofSeconds(5))
Review Comment:
Is 5 seconds too small?
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcOracleIT.java:
##########
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.flink.v2.jdbc;
+
+import org.apache.seatunnel.e2e.flink.FlinkContainer;
+
+import com.google.common.collect.Lists;
+import lombok.extern.slf4j.Slf4j;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.OracleContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.shaded.org.awaitility.Awaitility;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.Duration;
+import java.util.List;
+import java.util.stream.Stream;
+
+@Slf4j
+public class JdbcOracleIT extends FlinkContainer {
+ private OracleContainer oracleContainer;
+
+ @BeforeEach
+ public void startOracleContainer() throws ClassNotFoundException {
+ oracleContainer = new OracleContainer(DockerImageName.parse("gvenzl/oracle-xe:18.4.0-slim"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("oracle")
+ .withLogConsumer(new Slf4jLogConsumer(log));
+ Startables.deepStart(Stream.of(oracleContainer)).join();
+ log.info("Oracle container started");
+ Class.forName(oracleContainer.getDriverClassName());
+ Awaitility.given().ignoreExceptions()
+ .await()
+ .atMost(Duration.ofSeconds(5))
+ .untilAsserted(this::initializeJdbcTable);
+ batchInsertData();
+ }
+
+ private void initializeJdbcTable() {
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ String sourceSql = "CREATE TABLE source\n" +
+ " (\"ID\" NUMBER(20,0) NOT NULL ENABLE, \n" +
+ "\t\"BINARY_DOUBLE1\" BINARY_DOUBLE, \n" +
+ "\t\"BINARY_FLOAT1\" BINARY_FLOAT, \n" +
+ "\t\"BLOB1\" BLOB, \n" +
+ "\t\"CHAR1\" CHAR(100), \n" +
+ "\t\"CLOB1\" CLOB, \n" +
+ "\t\"DATE1\" DATE, \n" +
+ "\t\"DOUBLE_PRECISION1\" FLOAT(126), \n" +
+ "\t\"FLOAT1\" FLOAT(126), \n" +
+ "\t\"INT1\" NUMBER(10,0), \n" +
+ "\t\"INTEGER1\" NUMBER(20,0), \n" +
+ "\t\"NCLOB1\" NCLOB, \n" +
+ "\t\"NUMBER1\" NUMBER(30,0), \n" +
+ "\t\"SMALLINT1\" NUMBER(10,0), \n" +
+ "\t\"TIMESTAMP1\" TIMESTAMP (6) DEFAULT NULL, \n" +
+ "\t\"VAR11\" VARCHAR2(255) DEFAULT NULL\n" +
+ " )";
+ String sinkSql = "CREATE TABLE sink\n" +
+ " (\"ID\" NUMBER(20,0) NOT NULL ENABLE, \n" +
+ "\t\"BINARY_DOUBLE1\" BINARY_DOUBLE, \n" +
+ "\t\"BINARY_FLOAT1\" BINARY_FLOAT, \n" +
+ "\t\"BLOB1\" BLOB, \n" +
+ "\t\"CHAR1\" CHAR(100), \n" +
+ "\t\"CLOB1\" CLOB, \n" +
+ "\t\"DATE1\" DATE, \n" +
+ "\t\"DOUBLE_PRECISION1\" FLOAT(126), \n" +
+ "\t\"FLOAT1\" FLOAT(126), \n" +
+ "\t\"INT1\" NUMBER(10,0), \n" +
+ "\t\"INTEGER1\" NUMBER(20,0), \n" +
+ "\t\"NCLOB1\" NCLOB, \n" +
+ "\t\"NUMBER1\" NUMBER(30,0), \n" +
+ "\t\"SMALLINT1\" NUMBER(10,0), \n" +
+ "\t\"TIMESTAMP1\" TIMESTAMP (6) DEFAULT NULL, \n" +
+ "\t\"VAR11\" VARCHAR2(255) DEFAULT NULL\n" +
+ " )";
+ statement.execute(sourceSql);
+ statement.execute(sinkSql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Initializing Oracle table failed!", e);
+ }
+ }
+
+ @SuppressWarnings("checkstyle:RegexpSingleline")
+ private void batchInsertData() {
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ String sql =
+ "INSERT INTO source (ID, BINARY_DOUBLE1, BINARY_FLOAT1, BLOB1, CHAR1, CLOB1, DATE1, DOUBLE_PRECISION1, FLOAT1, INT1, INTEGER1, NCLOB1, NUMBER1, SMALLINT1, TIMESTAMP1, VAR11) " +
+ "VALUES(1, 12.12, 123.123, '56415243484152', 'char1中', 'clob中文1234', TIMESTAMP '2022-06-01 10:57:52.000000', 123.1234, 12.123, 12, 1234, 'clob中文1234阿斯顿', -1344, 11, TIMESTAMP '2022-06-01 10:59:15.000000', 'varchar1234和总共')";
+ Statement statement = connection.createStatement();
+ statement.execute(sql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Batch insert data failed!", e);
+ }
+ }
+
+ @Test
+ public void testFakeSourceToJdbcSink() throws SQLException, IOException, InterruptedException {
Review Comment:
rename to `testOracleSourceAndSink`?
##########
seatunnel-e2e/seatunnel-spark-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/spark/v2/jdbc/JdbcOracleIT.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.spark.v2.jdbc;
+
+import org.apache.seatunnel.e2e.spark.SparkContainer;
+
+import com.google.common.collect.Lists;
+import lombok.extern.slf4j.Slf4j;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.OracleContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.shaded.org.awaitility.Awaitility;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.Duration;
+import java.util.List;
+import java.util.stream.Stream;
+
+@Slf4j
+public class JdbcOracleIT extends SparkContainer {
+
+ private OracleContainer oracleContainer;
+
+ @BeforeEach
+ public void startOracleContainer() throws ClassNotFoundException {
+ oracleContainer = new OracleContainer(DockerImageName.parse("gvenzl/oracle-xe:18.4.0-slim"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("oracle")
+ .withLogConsumer(new Slf4jLogConsumer(log));
+ Startables.deepStart(Stream.of(oracleContainer)).join();
+ log.info("Oracle container started");
+ Class.forName(oracleContainer.getDriverClassName());
+ Awaitility.given().ignoreExceptions()
+ .await()
+ .atMost(Duration.ofSeconds(5))
+ .untilAsserted(this::initializeJdbcTable);
+ batchInsertData();
+ }
+
+ private void initializeJdbcTable() {
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ String sourceSql = "CREATE TABLE source\n" +
+ " (\"ID\" NUMBER(20,0) NOT NULL ENABLE, \n" +
+ "\t\"BINARY_DOUBLE1\" BINARY_DOUBLE, \n" +
+ "\t\"BINARY_FLOAT1\" BINARY_FLOAT, \n" +
+ "\t\"BLOB1\" BLOB, \n" +
+ "\t\"CHAR1\" CHAR(100), \n" +
+ "\t\"CLOB1\" CLOB, \n" +
+ "\t\"DATE1\" DATE, \n" +
+ "\t\"DOUBLE_PRECISION1\" FLOAT(126), \n" +
+ "\t\"FLOAT1\" FLOAT(126), \n" +
+ "\t\"INT1\" NUMBER(10,0), \n" +
+ "\t\"INTEGER1\" NUMBER(20,0), \n" +
+ "\t\"NCLOB1\" NCLOB, \n" +
+ "\t\"NUMBER1\" NUMBER(30,0), \n" +
+ "\t\"SMALLINT1\" NUMBER(10,0), \n" +
+ "\t\"TIMESTAMP1\" TIMESTAMP (6) DEFAULT NULL, \n" +
+ "\t\"VAR11\" VARCHAR2(255) DEFAULT NULL\n" +
+ " )";
+ String sinkSql = "CREATE TABLE sink\n" +
+ " (\"ID\" NUMBER(20,0) NOT NULL ENABLE, \n" +
+ "\t\"BINARY_DOUBLE1\" BINARY_DOUBLE, \n" +
+ "\t\"BINARY_FLOAT1\" BINARY_FLOAT, \n" +
+ "\t\"BLOB1\" BLOB, \n" +
+ "\t\"CHAR1\" CHAR(100), \n" +
+ "\t\"CLOB1\" CLOB, \n" +
+ "\t\"DATE1\" DATE, \n" +
+ "\t\"DOUBLE_PRECISION1\" FLOAT(126), \n" +
+ "\t\"FLOAT1\" FLOAT(126), \n" +
+ "\t\"INT1\" NUMBER(10,0), \n" +
+ "\t\"INTEGER1\" NUMBER(20,0), \n" +
+ "\t\"NCLOB1\" NCLOB, \n" +
+ "\t\"NUMBER1\" NUMBER(30,0), \n" +
+ "\t\"SMALLINT1\" NUMBER(10,0), \n" +
+ "\t\"TIMESTAMP1\" TIMESTAMP (6) DEFAULT NULL, \n" +
+ "\t\"VAR11\" VARCHAR2(255) DEFAULT NULL\n" +
+ " )";
+ statement.execute(sourceSql);
+ statement.execute(sinkSql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Initializing Oracle table failed!", e);
+ }
+ }
+
+ @SuppressWarnings("checkstyle:RegexpSingleline")
+ private void batchInsertData() {
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ String sql =
+ "INSERT INTO source (ID, BINARY_DOUBLE1, BINARY_FLOAT1, BLOB1, CHAR1, CLOB1, DATE1, DOUBLE_PRECISION1, FLOAT1, INT1, INTEGER1, NCLOB1, NUMBER1, SMALLINT1, TIMESTAMP1, VAR11) " +
+ "VALUES(1, 12.12, 123.123, '56415243484152', 'char1中', 'clob中文1234', TIMESTAMP '2022-06-01 10:57:52.000000', 123.1234, 12.123, 12, 1234, 'clob中文1234阿斯顿', -1344, 11, TIMESTAMP '2022-06-01 10:59:15.000000', 'varchar1234和总共')";
+ Statement statement = connection.createStatement();
+ statement.execute(sql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Batch insert data failed!", e);
+ }
+ }
+
+ @Test
+ public void testFakeSourceToJdbcSink() throws SQLException, IOException, InterruptedException {
Review Comment:
rename to `testOracleSourceAndSink`?
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcOracleIT.java:
##########
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.flink.v2.jdbc;
+
+import org.apache.seatunnel.e2e.flink.FlinkContainer;
+
+import com.google.common.collect.Lists;
+import lombok.extern.slf4j.Slf4j;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.OracleContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.shaded.org.awaitility.Awaitility;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.Duration;
+import java.util.List;
+import java.util.stream.Stream;
+
+@Slf4j
+public class JdbcOracleIT extends FlinkContainer {
+ private OracleContainer oracleContainer;
+
+ @BeforeEach
+ public void startOracleContainer() throws ClassNotFoundException {
+ oracleContainer = new OracleContainer(DockerImageName.parse("gvenzl/oracle-xe:18.4.0-slim"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("oracle")
+ .withLogConsumer(new Slf4jLogConsumer(log));
+ Startables.deepStart(Stream.of(oracleContainer)).join();
+ log.info("Oracle container started");
+ Class.forName(oracleContainer.getDriverClassName());
+ Awaitility.given().ignoreExceptions()
+ .await()
+ .atMost(Duration.ofSeconds(5))
Review Comment:
Is 5 seconds too small?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1231036787
CI is error. how do i fix it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on a diff in pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on code in PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#discussion_r962525732
##########
pom.xml:
##########
@@ -361,6 +362,13 @@
<version>${postgresql.version}</version>
</dependency>
+ <dependency>
Review Comment:
Don't add the connector's dependency in root pom, just use it in connector's pom.xml, reference: https://github.com/apache/incubator-seatunnel/pull/2630/files
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] ic4y commented on a diff in pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
ic4y commented on code in PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#discussion_r964640561
##########
seatunnel-e2e/seatunnel-spark-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/spark/v2/jdbc/JdbcOracleIT.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.spark.v2.jdbc;
+
+import org.apache.seatunnel.e2e.spark.SparkContainer;
+
+import com.google.common.collect.Lists;
+import lombok.extern.slf4j.Slf4j;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.OracleContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.shaded.org.awaitility.Awaitility;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.Duration;
+import java.util.List;
+import java.util.stream.Stream;
+
+@Slf4j
+public class JdbcOracleIT extends SparkContainer {
+
+ private OracleContainer oracleContainer;
+
+ @BeforeEach
+ public void startOracleContainer() throws ClassNotFoundException {
+ oracleContainer = new OracleContainer(DockerImageName.parse("gvenzl/oracle-xe:18.4.0-slim"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("oracle")
+ .withLogConsumer(new Slf4jLogConsumer(log));
+ Startables.deepStart(Stream.of(oracleContainer)).join();
+ log.info("Oracle container started");
+ Class.forName(oracleContainer.getDriverClassName());
+ Awaitility.given().ignoreExceptions()
+ .await()
+ .atMost(Duration.ofSeconds(60))
+ .untilAsserted(this::initializeJdbcTable);
+ batchInsertData();
+ }
+
+ private void initializeJdbcTable() {
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ String sourceSql = "CREATE TABLE source\n" +
+ " (\"ID\" NUMBER(20,0) NOT NULL ENABLE, \n" +
+ "\t\"BINARY_DOUBLE1\" BINARY_DOUBLE, \n" +
+ "\t\"BINARY_FLOAT1\" BINARY_FLOAT, \n" +
+ "\t\"BLOB1\" BLOB, \n" +
+ "\t\"CHAR1\" CHAR(100), \n" +
+ "\t\"CLOB1\" CLOB, \n" +
+ "\t\"DATE1\" DATE, \n" +
+ "\t\"DOUBLE_PRECISION1\" FLOAT(10), \n" +
+ "\t\"FLOAT1\" FLOAT(10), \n" +
+ "\t\"INT1\" NUMBER(10,0), \n" +
+ "\t\"INTEGER1\" NUMBER(20,0), \n" +
+ "\t\"NCLOB1\" NCLOB, \n" +
+ "\t\"NUMBER1\" NUMBER(30,0), \n" +
+ "\t\"SMALLINT1\" NUMBER(10,0), \n" +
+ "\t\"TIMESTAMP1\" TIMESTAMP (6) DEFAULT NULL, \n" +
+ "\t\"VAR11\" VARCHAR2(255) DEFAULT NULL\n" +
+ " )";
+ String sinkSql = "CREATE TABLE sink\n" +
+ " (\"ID\" NUMBER(20,0) NOT NULL ENABLE, \n" +
+ "\t\"BINARY_DOUBLE1\" BINARY_DOUBLE, \n" +
+ "\t\"BINARY_FLOAT1\" BINARY_FLOAT, \n" +
+ "\t\"BLOB1\" BLOB, \n" +
+ "\t\"CHAR1\" CHAR(100), \n" +
+ "\t\"CLOB1\" CLOB, \n" +
+ "\t\"DATE1\" DATE, \n" +
+ "\t\"DOUBLE_PRECISION1\" FLOAT(10), \n" +
+ "\t\"FLOAT1\" FLOAT(10), \n" +
+ "\t\"INT1\" NUMBER(10,0), \n" +
+ "\t\"INTEGER1\" NUMBER(20,0), \n" +
+ "\t\"NCLOB1\" NCLOB, \n" +
+ "\t\"NUMBER1\" NUMBER(30,0), \n" +
+ "\t\"SMALLINT1\" NUMBER(10,0), \n" +
+ "\t\"TIMESTAMP1\" TIMESTAMP (6) DEFAULT NULL, \n" +
+ "\t\"VAR11\" VARCHAR2(255) DEFAULT NULL\n" +
+ " )";
+ statement.execute(sourceSql);
+ statement.execute(sinkSql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Initializing Oracle table failed!", e);
+ }
+ }
+
+ @SuppressWarnings("checkstyle:RegexpSingleline")
+ private void batchInsertData() {
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ String sql =
+ "INSERT INTO source (ID, BINARY_DOUBLE1, BINARY_FLOAT1, BLOB1, CHAR1, CLOB1, DATE1, DOUBLE_PRECISION1, FLOAT1, INT1, INTEGER1, NCLOB1, NUMBER1, SMALLINT1, TIMESTAMP1, VAR11) " +
+ "VALUES(1, 12.12, 123.123, '56415243484152', 'char1中', 'clob中文1234', TIMESTAMP '2022-06-01 10:57:52.000000', 123.1234, 12.123, 12, 1234, 'clob中文1234阿斯顿', -1344, 11, TIMESTAMP '2022-06-01 10:59:15.000000', 'varchar1234和总共')";
+ Statement statement = connection.createStatement();
+ statement.execute(sql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Batch insert data failed!", e);
+ }
+ }
+
+ @Test
+ public void testOracleSourceAndSink() throws SQLException, IOException, InterruptedException {
+ Container.ExecResult execResult = executeSeaTunnelSparkJob("/jdbc/jdbc_oracle_source_to_sink.conf");
+ Assertions.assertEquals(0, execResult.getExitCode());
+ // query result
+ String sql = "select * from sink";
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ ResultSet resultSet = statement.executeQuery(sql);
+ List<String> result = Lists.newArrayList();
+ while (resultSet.next()) {
+ result.add(resultSet.getString("ID"));
+ }
+ Assertions.assertFalse(result.isEmpty());
Review Comment:
Suggest ,check if inserted data and result data are equal
##########
seatunnel-e2e/seatunnel-flink-connector-v2-e2e/src/test/java/org/apache/seatunnel/e2e/flink/v2/jdbc/JdbcOracleIT.java:
##########
@@ -0,0 +1,147 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.e2e.flink.v2.jdbc;
+
+import org.apache.seatunnel.e2e.flink.FlinkContainer;
+
+import com.google.common.collect.Lists;
+import lombok.extern.slf4j.Slf4j;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.OracleContainer;
+import org.testcontainers.containers.output.Slf4jLogConsumer;
+import org.testcontainers.lifecycle.Startables;
+import org.testcontainers.shaded.org.awaitility.Awaitility;
+import org.testcontainers.utility.DockerImageName;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.Duration;
+import java.util.List;
+import java.util.stream.Stream;
+
+@Slf4j
+public class JdbcOracleIT extends FlinkContainer {
+ private OracleContainer oracleContainer;
+
+ @BeforeEach
+ public void startOracleContainer() throws ClassNotFoundException {
+ oracleContainer = new OracleContainer(DockerImageName.parse("gvenzl/oracle-xe:18.4.0-slim"))
+ .withNetwork(NETWORK)
+ .withNetworkAliases("oracle")
+ .withLogConsumer(new Slf4jLogConsumer(log));
+ Startables.deepStart(Stream.of(oracleContainer)).join();
+ log.info("Oracle container started");
+ Class.forName(oracleContainer.getDriverClassName());
+ Awaitility.given().ignoreExceptions()
+ .await()
+ .atMost(Duration.ofSeconds(5))
+ .untilAsserted(this::initializeJdbcTable);
+ batchInsertData();
+ }
+
+ private void initializeJdbcTable() {
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ String sourceSql = "CREATE TABLE source\n" +
+ " (\"ID\" NUMBER(20,0) NOT NULL ENABLE, \n" +
+ "\t\"BINARY_DOUBLE1\" BINARY_DOUBLE, \n" +
+ "\t\"BINARY_FLOAT1\" BINARY_FLOAT, \n" +
+ "\t\"BLOB1\" BLOB, \n" +
+ "\t\"CHAR1\" CHAR(100), \n" +
+ "\t\"CLOB1\" CLOB, \n" +
+ "\t\"DATE1\" DATE, \n" +
+ "\t\"DOUBLE_PRECISION1\" FLOAT(126), \n" +
+ "\t\"FLOAT1\" FLOAT(126), \n" +
+ "\t\"INT1\" NUMBER(10,0), \n" +
+ "\t\"INTEGER1\" NUMBER(20,0), \n" +
+ "\t\"NCLOB1\" NCLOB, \n" +
+ "\t\"NUMBER1\" NUMBER(30,0), \n" +
+ "\t\"SMALLINT1\" NUMBER(10,0), \n" +
+ "\t\"TIMESTAMP1\" TIMESTAMP (6) DEFAULT NULL, \n" +
+ "\t\"VAR11\" VARCHAR2(255) DEFAULT NULL\n" +
+ " )";
+ String sinkSql = "CREATE TABLE sink\n" +
+ " (\"ID\" NUMBER(20,0) NOT NULL ENABLE, \n" +
+ "\t\"BINARY_DOUBLE1\" BINARY_DOUBLE, \n" +
+ "\t\"BINARY_FLOAT1\" BINARY_FLOAT, \n" +
+ "\t\"BLOB1\" BLOB, \n" +
+ "\t\"CHAR1\" CHAR(100), \n" +
+ "\t\"CLOB1\" CLOB, \n" +
+ "\t\"DATE1\" DATE, \n" +
+ "\t\"DOUBLE_PRECISION1\" FLOAT(126), \n" +
+ "\t\"FLOAT1\" FLOAT(126), \n" +
+ "\t\"INT1\" NUMBER(10,0), \n" +
+ "\t\"INTEGER1\" NUMBER(20,0), \n" +
+ "\t\"NCLOB1\" NCLOB, \n" +
+ "\t\"NUMBER1\" NUMBER(30,0), \n" +
+ "\t\"SMALLINT1\" NUMBER(10,0), \n" +
+ "\t\"TIMESTAMP1\" TIMESTAMP (6) DEFAULT NULL, \n" +
+ "\t\"VAR11\" VARCHAR2(255) DEFAULT NULL\n" +
+ " )";
+ statement.execute(sourceSql);
+ statement.execute(sinkSql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Initializing Oracle table failed!", e);
+ }
+ }
+
+ @SuppressWarnings("checkstyle:RegexpSingleline")
+ private void batchInsertData() {
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ String sql =
+ "INSERT INTO source (ID, BINARY_DOUBLE1, BINARY_FLOAT1, BLOB1, CHAR1, CLOB1, DATE1, DOUBLE_PRECISION1, FLOAT1, INT1, INTEGER1, NCLOB1, NUMBER1, SMALLINT1, TIMESTAMP1, VAR11) " +
+ "VALUES(1, 12.12, 123.123, '56415243484152', 'char1中', 'clob中文1234', TIMESTAMP '2022-06-01 10:57:52.000000', 123.1234, 12.123, 12, 1234, 'clob中文1234阿斯顿', -1344, 11, TIMESTAMP '2022-06-01 10:59:15.000000', 'varchar1234和总共')";
+ Statement statement = connection.createStatement();
+ statement.execute(sql);
+ } catch (SQLException e) {
+ throw new RuntimeException("Batch insert data failed!", e);
+ }
+ }
+
+ @Test
+ public void testOracleSourceAndSink() throws SQLException, IOException, InterruptedException {
+ Container.ExecResult execResult = executeSeaTunnelFlinkJob("/jdbc/jdbc_oracle_source_to_sink.conf");
+ Assertions.assertEquals(0, execResult.getExitCode());
+ // query result
+ String sql = "select * from sink";
+ try (Connection connection = DriverManager.getConnection(oracleContainer.getJdbcUrl(), oracleContainer.getUsername(), oracleContainer.getPassword())) {
+ Statement statement = connection.createStatement();
+ ResultSet resultSet = statement.executeQuery(sql);
+ List<String> result = Lists.newArrayList();
+ while (resultSet.next()) {
+ result.add(resultSet.getString("ID"));
+ }
+ Assertions.assertFalse(result.isEmpty());
Review Comment:
as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hailin0 commented on a diff in pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
hailin0 commented on code in PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#discussion_r990632935
##########
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcOracledbIT.java:
##########
@@ -0,0 +1,165 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.seatunnel.connectors.seatunnel.jdbc;
+
+import org.apache.seatunnel.api.table.type.SeaTunnelRow;
+
+import org.junit.jupiter.api.Assertions;
+import org.testcontainers.shaded.com.google.common.collect.Lists;
+import org.testcontainers.shaded.org.apache.commons.io.IOUtils;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.LocalDate;
+import java.time.LocalDateTime;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+
+public class JdbcOracledbIT extends AbstractJdbcIT {
+
+ private static final String DOCKER_IMAGE = "gvenzl/oracle-xe:18.4.0-slim";
+ private static final String NETWORK_ALIASES = "e2e_oracleDb";
+ private static final String DRIVER_CLASS = "oracle.jdbc.OracleDriver";
+ private static final int PORT = 1521;
+ private static final String URL = "jdbc:oracle:thin:@" + HOST + ":%s/%s";
+ private static final String USERNAME = "test";
+ private static final String PASSWORD = "test";
+ private static final String DATABASE = "xepdb1";
+ private static final String SOURCE_TABLE = "e2e_table_source";
+ private static final String SINK_TABLE = "e2e_table_sink";
+ private static final String DRIVER_JAR = "https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/12.2.0.1/ojdbc8-12.2.0.1.jar";
+ private static final String CONFIG_FILE = "/jdbc_oracle_source_to_sink.conf";
+ private static final String DDL_SOURCE = "create table " + SOURCE_TABLE + " (\n" +
+ " varchar_10_col varchar2(10),\n" +
+ " char_10_col char(10),\n" +
+ " clob_col clob,\n" +
+ " number_3_sf_2_dp number(3, 2),\n" +
+ " integer_col integer,\n" +
+ " float_col float(10),\n" +
+ " real_col real,\n" +
+ " binary_float_col binary_float,\n" +
+ " binary_double_col binary_double,\n" +
+ " date_col date,\n" +
+ " timestamp_with_3_frac_sec_col timestamp(3),\n" +
+ " timestamp_with_local_tz timestamp with local time zone,\n" +
+ " raw_col raw(1000),\n" +
+ " blob_col blob\n" +
+ ")";
+ private static final String DDL_SINK = "create table " + SINK_TABLE + "(\n" +
+ " varchar_10_col varchar2(10),\n" +
+ " char_10_col char(10),\n" +
+ " clob_col clob,\n" +
+ " number_3_sf_2_dp number(3, 2),\n" +
+ " integer_col integer,\n" +
+ " float_col float(10),\n" +
+ " real_col real,\n" +
+ " binary_float_col binary_float,\n" +
+ " binary_double_col binary_double,\n" +
+ " date_col date,\n" +
+ " timestamp_with_3_frac_sec_col timestamp(3),\n" +
+ " timestamp_with_local_tz timestamp with local time zone,\n" +
+ " raw_col raw(1000),\n" +
+ " blob_col blob\n" +
+ ")";
+ private static final String INIT_DATA_SQL = "insert into " + SOURCE_TABLE + " (\n" +
+ " varchar_10_col,\n" +
+ " char_10_col,\n" +
+ " clob_col,\n" +
+ " number_3_sf_2_dp,\n" +
+ " integer_col,\n" +
+ " float_col,\n" +
+ " real_col,\n" +
+ " binary_float_col,\n" +
+ " binary_double_col,\n" +
+ " date_col,\n" +
+ " timestamp_with_3_frac_sec_col,\n" +
+ " timestamp_with_local_tz,\n" +
+ " raw_col,\n" +
+ " blob_col\n" +
+ ")values(\n" +
+ "\t?,?,?,?,?,?,?,?,?,?,?,?,?,rawtohex(?),rawtohex(?)\n" +
+ ")";
+
+ @Override
+ JdbcCase getJdbcCase() {
+ Map<String, String> containerEnv = new HashMap<>();
+ containerEnv.put("ORACLE_PASSWORD", PASSWORD);
+ containerEnv.put("APP_USER", USERNAME);
+ containerEnv.put("APP_USER_PASSWORD", PASSWORD);
+ String jdbcUrl = String.format(URL, PORT, DATABASE);
+ return JdbcCase.builder().dockerImage(DOCKER_IMAGE).networkAliases(NETWORK_ALIASES).containerEnv(containerEnv).driverClass(DRIVER_CLASS)
+ .host(HOST).port(PORT).jdbcUrl(jdbcUrl).userName(USERNAME).password(PASSWORD).dataBase(DATABASE)
+ .sourceTable(SOURCE_TABLE).sinkTable(SINK_TABLE).driverJar(DRIVER_JAR)
+ .ddlSource(DDL_SOURCE).ddlSink(DDL_SINK).initDataSql(INIT_DATA_SQL).configFile(CONFIG_FILE).seaTunnelRow(initTestData()).build();
+ }
+
+ @Override
+ void compareResult() throws SQLException, IOException {
+ String sourceSql = "select * from " + SOURCE_TABLE;
+ String sinkSql = "select * from " + SINK_TABLE;
+ List<String> columns = Lists.newArrayList("varchar_10_col", "char_10_col", "clob_col", "number_3_sf_2_dp", "integer_col", "float_col", "real_col", "binary_float_col", "binary_double_col", "date_col", "timestamp_with_3_frac_sec_col", "timestamp_with_local_tz", "raw_col", "blob_col");
+ Statement sourceStatement = jdbcConnection.createStatement();
+ Statement sinkStatement = jdbcConnection.createStatement();
+ ResultSet sourceResultSet = sourceStatement.executeQuery(sourceSql);
+ ResultSet sinkResultSet = sinkStatement.executeQuery(sinkSql);
+ while (sourceResultSet.next()) {
+ if (sinkResultSet.next()) {
+ for (String column : columns) {
+ Object source = sourceResultSet.getObject(column);
+ Object sink = sinkResultSet.getObject(column);
+ if (!Objects.deepEquals(source, sink)) {
+
+ InputStream sourceAsciiStream = sourceResultSet.getBinaryStream(column);
+ InputStream sinkAsciiStream = sourceResultSet.getBinaryStream(column);
Review Comment:
```suggestion
InputStream sinkAsciiStream = sinkResultSet.getBinaryStream(column);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1272342507
> Please merge the dev branch to your branch and push again because #3022
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] liugddx commented on pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
liugddx commented on PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550#issuecomment-1272454490
merge #3031. fix the e2e error. @EricJoy2048 help to rerun CI,and i need the common part of the jdbc e2e for gbase 8a,thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs merged pull request #2550: [Feature][Connector-V2] oracle connector
Posted by GitBox <gi...@apache.org>.
CalvinKirs merged PR #2550:
URL: https://github.com/apache/incubator-seatunnel/pull/2550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org