You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Hongshun Wang (Jira)" <ji...@apache.org> on 2024/05/14 06:13:00 UTC
[jira] [Commented] (FLINK-35349) Use connection in openJdbcConnection of SqlServerDialect/Db2Dialect/OracleDialect
[ https://issues.apache.org/jira/browse/FLINK-35349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17846183#comment-17846183 ]
Hongshun Wang commented on FLINK-35349:
---------------------------------------
I'd like to do it.
> Use connection in openJdbcConnection of SqlServerDialect/Db2Dialect/OracleDialect
> ---------------------------------------------------------------------------------
>
> Key: FLINK-35349
> URL: https://issues.apache.org/jira/browse/FLINK-35349
> Project: Flink
> Issue Type: Improvement
> Components: Flink CDC
> Affects Versions: cdc-3.1.0
> Reporter: Hongshun Wang
> Priority: Minor
> Fix For: cdc-3.2.0
>
>
> Current, some dialect's `openJdbcConnection` create connection without connection pool. It means that will create a new connection each time.
> Howver , openJdbcConnection is used in generateSplits now, which means that enumerator will create a new connection for once split. A big table will create connection again and again.
> {code:java}
> public Collection<SnapshotSplit> generateSplits(TableId tableId) {
> try (JdbcConnection jdbc = dialect.openJdbcConnection(sourceConfig)) {
> LOG.info("Start splitting table {} into chunks...", tableId);
> long start = System.currentTimeMillis();
> Table table =
> Objects.requireNonNull(dialect.queryTableSchema(jdbc, tableId)).getTable();
> Column splitColumn = getSplitColumn(table, sourceConfig.getChunkKeyColumn());
> final List<ChunkRange> chunks; {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)