You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by ki...@apache.org on 2022/07/01 08:26:51 UTC
[incubator-seatunnel] branch dev updated (770bade75 -> df251bf47)
This is an automated email from the ASF dual-hosted git repository.
kirs pushed a change to branch dev
in repository https://gitbox.apache.org/repos/asf/incubator-seatunnel.git
from 770bade75 Update .asf.yaml (#2103)
add b7d2d23f5 [Feature][core] base interface.
add c7f36e403 [Feature][core] base logic
add f844d8c31 fix build error
add 4639ba16a [api-draft] Add license header (#1777)
add c3407957f Add seatunnel datatype and convert origin value into seatunnel data type (#1797)
add 1935d7124 Add flink datatype converter (#1801)
add 3753ed3ef Add complex seatunnel datatype (#1807)
add 06e74eb5b Add flink type converter UT (#1813)
add 609f58716 Rename Row to SeaTunnelRow (#1832)
add 8f710f76a Add spark basic type converter (#1837)
add 0db318bc9 [Api-Draft] Add Flink Sink Converter to support SeaTunnel transfer to Flink engine. (#1839)
add 6b5136b7b [Api-Draft] SeaTunnel Source support Flink engine. (#1842)
add 21e159a85 Add generics type about flink sink (#1844)
add 8ff90fdcf Add SeatunnelTaskExecuteCommand (#1847)
add b063bd564 Add basic fake source (#1864)
add 48423a921 [Api-draft] Support Spark Sink convert to SeaTunnel (#1863)
add 96bd7031f Add SeaTunnelAPIExample (#1867)
add af7f6e019 [Api-draft] Fix Spark Sink Can't support batch mode. (#1868)
add 5f79f45d2 Add plugin discovery module (#1881)
add 3abf728a7 Add boundedness to source reader context, so that we can control to stop this reader when produce data (#1885)
add 661f8d506 Add license header (#1886)
add fbeab83a5 [API-Draft] Add comment on common interface. (#1888)
add 1844a7a17 Add SeaTunnelPluginDiscovery to load seatunnel new api plugin (#1889)
add dce9ec9c3 [Api-Draft] Fix Flink sink type convert error. (#1890)
add 862c31e4c [Api-Draft] SeaTunnel Source support Spark engine. (#1871)
add b16f382c3 fix seatunnel source parallelism (#1892)
add b23266e0e Add SeaTunnelRowTypeInfo to support source return record type (#1894)
add 62b304ad9 Add Transform to new SeaTunnel API (#1900)
add 273121d5a Add plugin processor for SeaTunnel API (#1902)
add 727c7b261 Add SeaTunnelRowTypeInfo to SeaTunnelSink (#1904)
add c08573826 Add SeaTunnel plugin lifecycle (#1914)
add d26605c03 [Api-Draft] Support Spark MicroBatch Reader & Add CheckpointLock (#1907)
add b3ca59a09 [Api-draft] Add spark example (#1913)
add 7eebd8d1b Support set batch mode in flink (#1916)
add 5ecdebf86 [Api-Draft] Spark InternalRow Serialization (#1918)
add 38d5c6881 fix spark example can't run error. (#1921)
add ad5869625 Add Transform to SeaTunnel Spark API (#1925)
add 62b922b61 Add new architecture (#1928)
add 6d5ef6dab fix spark batch can't stop (#1927)
add 51fd0f7fe Add SeaTunnel runtime environment (#1933)
add 34b37aa3c [Api-Draft] Remove useless code. (#1932)
add afff61947 [API-DRAFT]refactor api base (#1939)
add 59e3ca533 [Api-draft] Add seatunnel kafka connector (#1940)
add a508e29e9 [API-DRAFT]Rename Flink-stater package name (#1942)
add a71e77796 Spark micro-batch reader state (#1943)
add 62d3468ae [API-DRAFT]Add spark-core-starter module Code splitting for old and new engines (#1945)
add b2cd4c02f Add SeaTunnel kafka sink (#1952)
add 206f4d02d Fix Kafka Sink on flink cannot serialize element (#1955)
add 694bbdd70 [Api-draft] Add seatunnel kafka source connectors (#1949)
add 691aabcb8 [api-draft] Support coordinated source & Fix checkpoint lock implementation (#1951)
add ffb2d86ee [api-draft][flink] non-key operator can't get the keyed state store (#1961)
add 3601f4294 Clean code for new API starter (#1962)
add 6979cfc30 Add new api starter to distribution (#1964)
add 75c91fc05 Add seatunnel new connector example (#1967)
add 8fc708317 [Api-draft] add README.zh.md for new api (#1966)
add 9a0ae7271 [Api-draft] kafka support commit on checkpoint (#1974)
add a9d54fdd0 Add e2e for new connector (#1973)
add ede29370f fix checkpoint and commit error (#1987)
add dc81c3167 [api-draft#1990][Common] The DeployMode code cleanup (#1991)
add 0bfd62593 [Improve]Use Jackson replace Fastjson (#1971) (#1985)
add f389cd791 [api-draft][connector] New hive sink connector (#1997)
add 805f88cbc [Api-draft] add jobId and checkpointId (#1998)
add 80794cf34 Fix the kafka consumer still getting consumption data when the client is closed (#2000)
add 9f1b474f8 [Api-Draft] improve class comment and fix spark row StringType convert error (#2002)
add 3e7d82bd7 [api-draft][connector] Fix hive sink dist package in new connector. (#2001)
add b6ed4ce5b [api-draft][connector] new socket source (#1999)
add 890211c23 [api-draft][connector] Add SeaTunnel jdbc sink (#1946) (#2009)
add 62c8a1753 [Api-draft] change kafka sink transaction (#2010)
add ca6c67707 [api-draft] Improved conversion of data types (#2003)
add 1673dcebb [api-draft][formats] json format (#2014)
add ebf9489f2 [Api-draft] Add clickhouse source support (#2013)
add 1113cb9c3 [api-draft][connector] Add new http source (#2012)
add a8d47c97f [api-draft][connector] apache pulsar source (#1984)
add 5ce65e298 [Feature][connector common] Add Hadoop2 and Hadoop3 shade jar (#2030)
add 67e9f6fbe [hotfix][api-draft] fix flink batch mode (#2038)
add 605a0861c [hotfix][api-draft] Coordinated source cannot be stopped in offline job (#2040)
add f19555cd8 update windows ci timeout to 120min (#2046)
add 24b4d624e [API-Draft] Fix hadoop shade can't be imported problem (#2045)
add f35fdc8d5 add new api doc English version (#2050)
add 500799d8b [bugfix][api-draft] Fix KafkaSource parallel mode failure (#2039)
add 20077eb04 [API-Draft] Remove hadoop shade module (#2057)
add a549df646 [api-draft][api] Improve SeaTunnel's data types & Mapping engine data types (#2052)
add 97f7b432d [api-draft][connector] fix ThreadLocalRandom use (#2059) (#2060)
add 463cafe38 [api-draft][connector] Add SeaTunnel jdbc source (#2048)
add a15ab7fa6 fix VariablesSubstituteTest create time now twice (#2063)
add a7357b6a9 [api-draft][connector] Add simplified connector api (#2041)
add 94fe1e081 [api-draft][catalog] jdbc catalog (#2042)
add 682c58db5 [bugfix] Change the JDBC data type (#2065)
add da0949926 [feature] Add postgres support to jdbcSource (#2066)
add def84d9e9 [API-Draft][DOC] Add jdbc connector doc (#2069)
add 02ff9608b [Cherry-pick#2029]Improve CI jobs to reduce waiting time (#2070)
add 3057ba205 [API-Draft] [Connector] Add Clickhouse source and sink connector (#2051)
add f79e3112b [api-draft][Optimize] Optimize module name (#2062)
add 5dae9c1ef [API-DRAFT] [MERGE] Fix obvious bugs that don't work properly before merge (#2082)
add 02a41902a [api-draft][connector] support Rsync to transfer clickhouse data file (#2080)
add 369bb0e89 [api-draft][flink] The FinkCommitter's commit info class could not be inferred. (#2086)
add fc640b52b add assert sink to Api draft (#2071)
add 032906fdd add clickhouse source and sink docs (#2072)
add d265597c6 merge dev to api-draft
add 736ac01c8 [API-DRAFT] [MERGE] fix merge error
add 3c0e98464 [API-DRAFT] [MERGE] fix merge error
add 0656b5025 [API-Draft] [Doc]add common option (#2095)
add 6a492d79b Update Clickhouse.md (#2096)
add 10ac17e8c Merge remote-tracking branch 'origin/api-draft' into api-draft
add 2bad99d35 [api-draft][doc] move Assert.md to new-connector file https://github.com/apache/incubator-seatunnel/issues/2097 (#2098)
add 5ae8865b7 [API-DRAFT] [MERGE] update license and pom.xml
add 1b706c16a Merge branch 'dev' into api-draft
new df251bf47 Merge pull request #2083 from apache/api-draft
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/backend.yml | 2 +-
.licenserc.yaml | 2 +-
LICENSE | 1 +
config/flink.batch.conf.template | 2 +
config/spark.streaming.conf.template | 2 +-
docs/en/connector/sink/Assert.md | 47 +-
docs/en/images/seatunnel_architecture.png | Bin 0 -> 778394 bytes
docs/en/images/seatunnel_starter.png | Bin 0 -> 423840 bytes
.../en/{connector => new-connector}/sink/Assert.md | 51 +-
docs/en/new-connector/sink/Clickhouse.md | 104 ++++
docs/en/new-connector/sink/ClickhouseFile.md | 106 ++++
docs/en/new-connector/sink/Jdbc.md | 96 ++++
.../sink/common-options.md | 0
docs/en/new-connector/source/Clickhouse.md | 63 ++
docs/en/new-connector/source/Jdbc.md | 75 +++
docs/en/new-connector/source/common-options.md | 33 ++
...mapping.properties => plugin-mapping.properties | 17 +
pom.xml | 105 ++--
{seatunnel-core => seatunnel-api}/pom.xml | 23 +-
.../api/common/PluginIdentifierInterface.java | 21 +-
.../seatunnel/api/common/PrepareFailException.java | 30 +-
.../seatunnel/api/common/SeaTunnelContext.java | 87 +++
.../api/common/SeaTunnelPluginLifeCycle.java | 35 +-
.../api/serialization/DefaultSerializer.java | 31 +-
.../api/serialization/DeserializationSchema.java | 36 +-
.../api/serialization/SerializationSchema.java | 21 +-
.../seatunnel/api/serialization/Serializer.java | 29 +-
.../api/sink/DefaultSinkWriterContext.java | 36 +-
.../apache/seatunnel/api/sink/SeaTunnelSink.java | 122 ++++
.../api/sink/SinkAggregatedCommitter.java | 66 +++
.../apache/seatunnel/api/sink/SinkCommitter.java | 50 ++
.../org/apache/seatunnel/api/sink/SinkWriter.java | 95 +++
.../apache/seatunnel/api/source/Boundedness.java | 22 +-
.../org/apache/seatunnel/api/source/Collector.java | 20 +-
.../api/source/SeaTunnelRuntimeEnvironment.java | 19 +-
.../seatunnel/api/source/SeaTunnelSource.java | 99 ++++
.../apache/seatunnel/api/source/SourceEvent.java | 18 +-
.../apache/seatunnel/api/source/SourceReader.java | 119 ++++
.../apache/seatunnel/api/source/SourceSplit.java | 18 +-
.../api/source/SourceSplitEnumerator.java | 124 ++++
.../seatunnel/api/source/SupportCoordinate.java | 18 +-
.../seatunnel/api/state/CheckpointListener.java | 18 +-
.../seatunnel/api/table/catalog/Catalog.java | 116 ++++
.../seatunnel/api/table/catalog/CatalogTable.java | 93 +++
.../apache/seatunnel/api/table/catalog/Column.java | 223 ++++++++
.../api/table/catalog/TableIdentifier.java | 81 +++
.../seatunnel/api/table/catalog/TablePath.java | 86 +++
.../seatunnel/api/table/catalog/TableSchema.java | 132 +++++
.../table/catalog/exception/CatalogException.java | 31 +-
.../exception/DatabaseNotExistException.java | 32 ++
.../catalog/exception/TableNotExistException.java | 35 ++
.../api/table/connector/DeserializationFormat.java | 28 +-
.../api/table/connector/SerializationFormat.java | 15 +-
.../table/connector/SupportReadingMetadata.java | 21 +-
.../seatunnel/api/table/connector/TableSink.java | 15 +-
.../seatunnel/api/table/connector/TableSource.java | 16 +-
.../api/table/factory/CatalogFactory.java | 19 +-
.../factory/DeserializationFormatFactory.java | 16 +-
.../seatunnel/api/table/factory/Factory.java | 21 +-
.../api/table/factory/FactoryException.java | 15 +-
.../seatunnel/api/table/factory/FactoryUtil.java | 155 +++++
.../table/factory/SerializationFormatFactory.java | 16 +-
.../api/table/factory/SupportMultipleTable.java | 60 ++
.../api/table/factory/TableFactoryContext.java | 58 ++
.../api/table/factory/TableSinkFactory.java | 20 +-
.../api/table/factory/TableSourceFactory.java | 31 +-
.../apache/seatunnel/api/table/type/ArrayType.java | 87 +++
.../apache/seatunnel/api/table/type/BasicType.java | 79 +++
.../seatunnel/api/table/type/CompositeType.java | 15 +-
.../seatunnel/api/table/type/DecimalType.java | 65 +++
.../seatunnel/api/table/type/LocalTimeType.java | 72 +++
.../apache/seatunnel/api/table/type/MapType.java | 104 ++++
.../api/table/type/PrimitiveByteArrayType.java | 38 +-
.../apache/seatunnel/api/table/type/RowKind.java | 117 ++++
.../api/table/type/SeaTunnelDataType.java | 19 +-
.../seatunnel/api/table/type/SeaTunnelRow.java | 106 ++++
.../seatunnel/api/table/type/SeaTunnelRowType.java | 121 ++++
.../apache/seatunnel/api/table/type/SqlType.java | 34 +-
.../apache/seatunnel/apis/base/env/RuntimeEnv.java | 4 +-
.../apache/seatunnel/apis/base/plugin/Plugin.java | 14 +-
.../apache/seatunnel/flink/FlinkEnvironment.java | 18 +-
.../apache/seatunnel/flink/util/SchemaUtil.java | 2 +
.../apache/seatunnel/spark/SparkEnvironment.java | 4 +-
seatunnel-common/pom.xml | 9 +
.../apache/seatunnel/common/PropertiesUtil.java | 37 ++
.../org/apache/seatunnel/common/config/Common.java | 33 +-
.../common/constants/CollectionConstants.java | 15 +-
.../seatunnel/common/constants}/PluginType.java | 6 +-
.../seatunnel/common/utils/ReflectionUtils.java | 49 ++
.../seatunnel/common/utils/SerializationUtils.java | 49 ++
.../apache/seatunnel/common/config/CommonTest.java | 2 +-
.../common/utils/VariablesSubstituteTest.java | 7 +-
seatunnel-connectors-v2-dist/pom.xml | 108 ++++
seatunnel-connectors-v2/README.md | 84 +++
seatunnel-connectors-v2/README.zh.md | 64 +++
.../connector-assert}/pom.xml | 33 +-
.../assertion/excecutor/AssertExecutor.java | 108 ++++
.../seatunnel/assertion/rule/AssertFieldRule.java | 65 +++
.../seatunnel/assertion/rule/AssertRuleParser.java | 83 +++
.../seatunnel/assertion/sink/AssertSink.java | 85 +++
.../seatunnel/assertion/sink/AssertSinkWriter.java | 55 ++
.../flink/assertion/AssertExecutorTest.java | 110 ++++
.../flink/assertion/rule/AssertRuleParserTest.java | 85 +++
.../connector-clickhouse/pom.xml | 70 +++
.../config/ClickhouseFileCopyMethod.java | 34 +-
.../seatunnel/clickhouse/config/Config.java | 96 ++++
.../clickhouse/config/FileReaderOption.java | 113 ++++
.../seatunnel/clickhouse/config/ReaderOption.java | 94 +++
.../seatunnel/clickhouse/shard/Shard.java | 98 ++++
.../seatunnel/clickhouse/shard/ShardMetadata.java | 145 +++++
.../clickhouse/sink/DistributedEngine.java | 58 ++
.../sink/client/ClickhouseBatchStatement.java | 52 ++
.../clickhouse/sink/client/ClickhouseProxy.java | 212 +++++++
.../clickhouse/sink/client/ClickhouseSink.java | 172 ++++++
.../sink/client/ClickhouseSinkWriter.java | 230 ++++++++
.../clickhouse/sink/client/ShardRouter.java | 97 ++++
.../clickhouse/sink/file/ClickhouseFileSink.java | 145 +++++
.../sink/file/ClickhouseFileSinkWriter.java | 237 ++++++++
.../clickhouse/sink/file/ClickhouseTable.java | 117 ++++
.../clickhouse/sink/file/FileTransfer.java | 19 +-
.../clickhouse/sink/file/FileTransferFactory.java | 36 +-
.../clickhouse/sink/file/RsyncFileTransfer.java | 144 +++++
.../clickhouse/sink/file/ScpFileTransfer.java | 124 ++++
.../sink/inject/ArrayInjectFunction.java | 28 +-
.../sink/inject/BigDecimalInjectFunction.java | 28 +-
.../sink/inject/ClickhouseFieldInjectFunction.java | 46 ++
.../clickhouse/sink/inject/DateInjectFunction.java | 37 +-
.../sink/inject/DateTimeInjectFunction.java | 34 +-
.../sink/inject/DoubleInjectFunction.java | 31 +-
.../sink/inject/FloatInjectFunction.java | 34 +-
.../clickhouse/sink/inject/IntInjectFunction.java | 47 ++
.../clickhouse/sink/inject/LongInjectFunction.java | 29 +-
.../sink/inject/StringInjectFunction.java | 18 +-
.../clickhouse/source/ClickhouseSource.java | 132 +++++
.../clickhouse/source/ClickhouseSourceReader.java | 108 ++++
.../clickhouse/source/ClickhouseSourceSplit.java | 17 +-
.../source/ClickhouseSourceSplitEnumerator.java | 100 ++++
.../clickhouse/state/CKAggCommitInfo.java | 15 +-
.../seatunnel/clickhouse/state/CKCommitInfo.java | 15 +-
.../clickhouse/state/ClickhouseSinkState.java | 15 +-
.../clickhouse/state/ClickhouseSourceState.java | 15 +-
.../seatunnel/clickhouse/tool/IntHolder.java | 19 +-
.../seatunnel/clickhouse/util/ClickhouseUtil.java | 40 ++
.../seatunnel/clickhouse/util/TypeConvertUtil.java | 100 ++++
.../connector-common}/pom.xml | 23 +-
.../seatunnel/common/sink/AbstractSimpleSink.java | 59 ++
.../seatunnel/common/sink/AbstractSinkWriter.java | 19 +-
.../common/source/AbstractSingleSplitReader.java | 67 +++
.../common/source/AbstractSingleSplitSource.java | 58 ++
.../seatunnel/common/source/SingleSplit.java | 21 +-
.../common/source/SingleSplitEnumerator.java | 94 +++
.../common/source/SingleSplitEnumeratorState.java | 15 +-
.../common/source/SingleSplitReaderContext.java | 26 +-
.../connector-console}/pom.xml | 23 +-
.../seatunnel/console/sink/ConsoleSink.java | 63 ++
.../seatunnel/console/sink/ConsoleSinkWriter.java | 37 +-
.../connector-fake}/pom.xml | 21 +-
.../seatunnel/fake/source/FakeSource.java | 73 +++
.../seatunnel/fake/source/FakeSourceReader.java | 73 +++
.../connector-hive}/pom.xml | 25 +-
.../connectors/seatunnel/hive/config/Config.java | 15 +-
.../hive/sink/HiveAggregatedCommitInfo.java | 25 +-
.../seatunnel/hive/sink/HiveCommitInfo.java | 25 +-
.../connectors/seatunnel/hive/sink/HiveSink.java | 94 +++
.../hive/sink/HiveSinkAggregatedCommitter.java | 88 +++
.../seatunnel/hive/sink/HiveSinkConfig.java | 106 ++++
.../seatunnel/hive/sink/HiveSinkState.java | 19 +-
.../seatunnel/hive/sink/HiveSinkWriter.java | 99 ++++
.../hive/sink/file/writer/AbstractFileWriter.java | 155 +++++
.../hive/sink/file/writer/FileWriter.java | 49 ++
.../hive/sink/file/writer/HdfsTxtFileWriter.java | 151 +++++
.../seatunnel/hive/sink/file/writer/HdfsUtils.java | 96 ++++
.../connector-http}/pom.xml | 23 +-
.../seatunnel/http/client/HttpClientProvider.java | 283 +++++++++
.../seatunnel/http/client/HttpResponse.java | 74 +++
.../connectors/seatunnel/http/config/Config.java | 18 +-
.../seatunnel/http/source/HttpSource.java | 109 ++++
.../seatunnel/http/source/HttpSourceParameter.java | 70 +++
.../seatunnel/http/source/HttpSourceReader.java | 79 +++
.../connector-jdbc}/pom.xml | 26 +-
.../jdbc/catalog/AbstractJdbcCatalog.java | 211 +++++++
.../seatunnel/jdbc/catalog/MySqlCatalog.java | 230 ++++++++
.../seatunnel/jdbc/config/JdbcConfig.java | 102 ++++
.../seatunnel/jdbc/config/JdbcSinkOptions.java | 35 +-
.../seatunnel/jdbc/config/JdbcSourceOptions.java | 77 +++
.../seatunnel/jdbc/internal/JdbcInputFormat.java | 220 +++++++
.../seatunnel/jdbc/internal/JdbcOutputFormat.java | 257 +++++++++
.../jdbc/internal/connection/DataSourceUtils.java | 104 ++++
.../connection/JdbcConnectionProvider.java | 70 +++
.../connection/SimpleJdbcConnectionProvider.java | 154 +++++
.../converter/AbstractJdbcRowConverter.java | 97 ++++
.../jdbc/internal/converter/JdbcRowConverter.java | 25 +-
.../jdbc/internal/dialect/JdbcDialect.java | 52 ++
.../jdbc/internal/dialect/JdbcDialectFactory.java | 25 +-
.../jdbc/internal/dialect/JdbcDialectLoader.java | 102 ++++
.../internal/dialect/JdbcDialectTypeMapper.java | 23 +-
.../dialect/mysql/MySqlDialectFactory.java | 29 +-
.../internal/dialect/mysql/MySqlTypeMapper.java | 170 ++++++
.../jdbc/internal/dialect/mysql/MysqlDialect.java | 30 +-
.../dialect/mysql/MysqlJdbcRowConverter.java | 31 +-
.../internal/dialect/psql/PostgresDialect.java | 30 +-
.../dialect/psql/PostgresDialectFactory.java | 29 +-
.../dialect/psql/PostgresJdbcRowConverter.java | 31 +-
.../internal/dialect/psql/PostgresTypeMapper.java | 159 +++++
.../internal/executor/BiConsumerWithException.java | 61 ++
.../executor/JdbcBatchStatementExecutor.java | 29 +-
.../internal/executor/JdbcStatementBuilder.java | 22 +-
.../executor/SimpleBatchStatementExecutor.java | 79 +++
.../jdbc/internal/options/JdbcConfig.java | 55 ++
.../internal/options/JdbcConnectionOptions.java | 200 +++++++
.../internal/options/JdbcConnectorOptions.java | 253 ++++++++
.../split/JdbcGenericParameterValuesProvider.java | 30 +-
.../JdbcNumericBetweenParametersProvider.java | 116 ++++
.../split/JdbcParameterValuesProvider.java | 32 +-
.../jdbc/internal/xa/GroupXaOperationResult.java | 80 +++
.../jdbc/internal/xa/SemanticXidGenerator.java | 113 ++++
.../seatunnel/jdbc/internal/xa/XaFacade.java | 113 ++++
.../jdbc/internal/xa/XaFacadeImplAutoLoad.java | 477 +++++++++++++++
.../seatunnel/jdbc/internal/xa/XaGroupOps.java | 43 ++
.../seatunnel/jdbc/internal/xa/XaGroupOpsImpl.java | 151 +++++
.../seatunnel/jdbc/internal/xa/XidGenerator.java | 62 ++
.../seatunnel/jdbc/internal/xa/XidImpl.java | 136 +++++
.../jdbc/sink/JdbcExactlyOnceSinkWriter.java | 190 ++++++
.../connectors/seatunnel/jdbc/sink/JdbcSink.java | 143 +++++
.../jdbc/sink/JdbcSinkAggregatedCommitter.java | 91 +++
.../seatunnel/jdbc/sink/JdbcSinkCommitter.java | 70 +++
.../seatunnel/jdbc/sink/JdbcSinkWriter.java | 96 ++++
.../seatunnel/jdbc/source/JdbcSource.java | 216 +++++++
.../seatunnel/jdbc/source/JdbcSourceReader.java | 97 ++++
.../seatunnel/jdbc/source/JdbcSourceSplit.java | 22 +-
.../jdbc/source/JdbcSourceSplitEnumerator.java | 100 ++++
.../seatunnel/jdbc/source/PartitionParameter.java | 20 +-
.../jdbc/state/JdbcAggregatedCommitInfo.java | 20 +-
.../seatunnel/jdbc/state/JdbcSinkState.java | 19 +-
.../seatunnel/jdbc/state/JdbcSourceState.java | 15 +-
.../connectors/seatunnel/jdbc/state/XidInfo.java | 23 +-
.../seatunnel/jdbc/utils/ExceptionUtils.java | 53 ++
.../connectors/seatunnel/jdbc/utils/JdbcUtils.java | 158 +++++
.../seatunnel/jdbc/utils/ThrowingRunnable.java | 52 ++
.../jdbc/catalog/AbstractJdbcCatalogTest.java | 48 ++
.../connector-kafka}/pom.xml | 25 +-
.../connectors/seatunnel/kafka/config/Config.java | 53 ++
.../seatunnel/kafka/config/KafkaSemantics.java | 27 +-
.../serialize/DefaultSeaTunnelRowSerializer.java | 50 ++
.../kafka/serialize/SeaTunnelRowSerializer.java | 22 +-
.../kafka/sink/KafkaInternalProducer.java | 164 ++++++
.../kafka/sink/KafkaNoTransactionSender.java | 83 +++
.../seatunnel/kafka/sink/KafkaProduceSender.java | 64 +++
.../connectors/seatunnel/kafka/sink/KafkaSink.java | 95 +++
.../seatunnel/kafka/sink/KafkaSinkCommitter.java | 85 +++
.../seatunnel/kafka/sink/KafkaSinkWriter.java | 161 ++++++
.../kafka/sink/KafkaTransactionSender.java | 126 ++++
.../seatunnel/kafka/source/ConsumerMetadata.java | 82 +++
.../seatunnel/kafka/source/KafkaSource.java | 132 +++++
.../seatunnel/kafka/source/KafkaSourceReader.java | 171 ++++++
.../seatunnel/kafka/source/KafkaSourceSplit.java | 72 +++
.../kafka/source/KafkaSourceSplitEnumerator.java | 167 ++++++
.../kafka/state/KafkaAggregatedCommitInfo.java | 19 +-
.../seatunnel/kafka/state/KafkaCommitInfo.java | 23 +-
.../seatunnel/kafka/state/KafkaSinkState.java | 23 +-
.../seatunnel/kafka/state/KafkaSourceState.java | 35 +-
seatunnel-connectors-v2/connector-pulsar/pom.xml | 99 ++++
.../seatunnel/pulsar/config/BasePulsarConfig.java | 27 +-
.../seatunnel/pulsar/config/PulsarAdminConfig.java | 76 +++
.../pulsar/config/PulsarClientConfig.java | 80 +++
.../seatunnel/pulsar/config/PulsarConfigUtil.java | 81 +++
.../pulsar/config/PulsarConsumerConfig.java | 60 ++
.../seatunnel/pulsar/config/SourceProperties.java | 119 ++++
.../seatunnel/pulsar/source/PulsarSource.java | 281 +++++++++
.../source/enumerator/PulsarSplitEnumerator.java | 281 +++++++++
.../enumerator/PulsarSplitEnumeratorState.java | 36 ++
.../cursor/start/MessageIdStartCursor.java | 65 +++
.../enumerator/cursor/start/StartCursor.java | 78 +++
.../cursor/start/SubscriptionStartCursor.java | 63 ++
.../cursor/start/TimestampStartCursor.java | 40 ++
.../cursor/stop/LatestMessageStopCursor.java | 59 ++
.../cursor/stop/MessageIdStopCursor.java | 58 ++
.../enumerator/cursor/stop/NeverStopCursor.java | 49 ++
.../source/enumerator/cursor/stop/StopCursor.java | 63 ++
.../cursor/stop/TimestampStopCursor.java | 44 ++
.../enumerator/discoverer/PulsarDiscoverer.java | 48 ++
.../enumerator/discoverer/TopicListDiscoverer.java | 63 ++
.../discoverer/TopicPatternDiscoverer.java | 98 ++++
.../source/enumerator/topic/TopicPartition.java | 91 +++
.../seatunnel/pulsar/source/reader/Handover.java | 88 +++
.../pulsar/source/reader/PulsarSourceReader.java | 221 +++++++
.../source/reader/PulsarSplitReaderThread.java | 143 +++++
.../pulsar/source/reader/RecordWithSplitId.java | 22 +-
.../pulsar/source/split/PulsarPartitionSplit.java | 94 +++
.../connector-socket}/pom.xml | 21 +-
.../seatunnel/socket/source/SocketSource.java | 72 +++
.../socket/source/SocketSourceParameter.java | 32 +-
.../socket/source/SocketSourceReader.java | 91 +++
.../seatunnel/socket/state/SocketState.java | 15 +-
.../pom.xml | 40 +-
.../seatunnel-connector-flink-clickhouse/pom.xml | 5 -
.../flink/clickhouse/sink/ClickhouseBatchSink.java | 3 -
.../clickhouse/sink/ClickhouseOutputFormat.java | 2 +-
.../org/apache/seatunnel/flink/fake/Config.java | 2 +-
.../flink/fake/source/FakeSourceStream.java | 2 +-
seatunnel-core/README.md | 8 +
seatunnel-core/pom.xml | 3 +
seatunnel-core/seatunnel-core-base/pom.xml | 6 +
.../org/apache/seatunnel/core/base/Seatunnel.java | 3 +-
.../core/base/command/AbstractCommandArgs.java | 1 +
.../core/base/command/BaseTaskExecuteCommand.java | 5 +-
.../seatunnel/core/base/command/Command.java | 3 +-
...nContext.java => AbstractExecutionContext.java} | 43 +-
.../seatunnel/core/base/config/ConfigBuilder.java | 21 +-
.../seatunnel/core/base/config/ConfigChecker.java | 36 +-
.../core/base/config/EnvironmentFactory.java | 2 +
.../core/base/config/ExecutionFactory.java | 4 +-
.../seatunnel/core/base/config/PluginFactory.java | 237 --------
.../CommandException.java} | 16 +-
.../CommandExecuteException.java} | 16 +-
.../ConfigCheckException.java} | 16 +-
.../core/base/config/PluginFactoryTest.java | 65 ---
.../seatunnel/core/base/utils/FileUtilsTest.java | 1 +
.../apache/seatunnel/core/sql/FlinkSqlStarter.java | 6 +-
.../apache/seatunnel/core/sql/job/Executor.java | 16 +-
seatunnel-core/seatunnel-core-flink/pom.xml | 24 +-
.../apache/seatunnel/core/flink/FlinkStarter.java | 4 +-
.../seatunnel/core/flink/SeatunnelFlink.java | 3 +-
.../core/flink/args/FlinkCommandArgs.java | 4 +-
...mmand.java => FlinkApiConfValidateCommand.java} | 14 +-
...ommand.java => FlinkApiTaskExecuteCommand.java} | 37 +-
.../core/flink/command/FlinkCommandBuilder.java | 9 +-
.../core/flink/config/FlinkApiConfigChecker.java | 44 ++
.../core/flink/config/FlinkExecutionContext.java | 104 ++++
.../core/flink/utils/CommandLineUtils.java | 17 +-
.../flink/command/FlinkTaskExecuteCommandTest.java | 2 +-
.../core/flink/utils/CommandLineUtilsTest.java | 17 +-
seatunnel-core/seatunnel-core-spark/pom.xml | 26 +
.../seatunnel/core/spark/SeatunnelSpark.java | 3 +-
.../apache/seatunnel/core/spark/SparkStarter.java | 36 +-
.../core/spark/command/SparkCommandBuilder.java | 5 +-
.../spark/command/SparkConfValidateCommand.java | 8 +-
.../spark/command/SparkTaskExecuteCommand.java | 59 +-
.../core/spark/config/SparkApiConfigChecker.java | 44 ++
.../core/spark/config/SparkExecutionContext.java | 103 ++++
seatunnel-core/seatunnel-core-starter/README.md | 12 +
.../pom.xml | 7 +-
.../apache/seatunnel/core/starter}/Seatunnel.java | 9 +-
.../apache/seatunnel/core/starter/Starter.java} | 16 +-
.../core/starter}/command/AbstractCommandArgs.java | 6 +-
.../seatunnel/core/starter}/command/Command.java | 8 +-
.../core/starter/command/CommandArgs.java} | 18 +-
.../core/starter/command/CommandBuilder.java} | 16 +-
.../core/starter/command/DeployModeConverter.java} | 20 +-
.../starter/config/AbstractExecutionContext.java} | 45 +-
.../core/starter}/config/ConfigBuilder.java | 22 +-
.../core/starter/config/ConfigChecker.java | 36 +-
.../seatunnel/core/starter/config/EngineType.java} | 18 +-
.../core/starter}/config/EnvironmentFactory.java | 3 +-
.../core/starter}/config/ExecutionFactory.java | 6 +-
.../seatunnel/core/starter}/config/PluginType.java | 2 +-
.../starter/constants/CommonParamConstants.java} | 16 +-
.../core/starter/exception/CommandException.java} | 16 +-
.../exception/CommandExecuteException.java} | 16 +-
.../starter/exception/ConfigCheckException.java} | 16 +-
.../starter/exception/TaskExecuteException.java} | 15 +-
.../core/starter/execution/TaskExecution.java} | 15 +-
.../core/starter/utils/AsciiArtUtils.java | 72 +++
.../core/starter/utils/CompressionUtils.java | 158 +++++
.../seatunnel/core/starter/utils/FileUtils.java | 70 +++
.../core/starter/utils/CompressionUtilsTest.java | 50 ++
.../core/starter}/utils/FileUtilsTest.java | 4 +-
.../src/test/resources/flink.batch.conf | 0
.../pom.xml | 64 ++-
.../bin/start-seatunnel-flink-new-connector.sh | 46 ++
.../src/main/docker/Dockerfile | 0
.../core/starter}/flink/FlinkStarter.java | 16 +-
.../core/starter}/flink/SeatunnelFlink.java | 17 +-
.../core/starter}/flink/args/FlinkCommandArgs.java | 9 +-
.../command/FlinkApiConfValidateCommand.java} | 27 +-
.../flink/command/FlinkApiTaskExecuteCommand.java | 61 ++
.../flink/command/FlinkCommandBuilder.java | 18 +-
.../flink/config/FlinkApiConfigChecker.java} | 17 +-
.../starter/flink/config/FlinkApiEnvironment.java | 75 +++
.../core/starter/flink/config/FlinkJobType.java} | 12 +-
.../core/starter/flink/config/FlinkRunMode.java} | 22 +-
.../starter/flink/constant/FlinkConstant.java} | 16 +-
.../core/starter/flink/env}/FlinkEnvironment.java | 63 +-
.../execution/AbstractPluginExecuteProcessor.java | 76 +++
.../starter/flink/execution/FlinkExecution.java | 73 +++
.../flink/execution/PluginExecuteProcessor.java} | 25 +-
.../flink/execution/SinkExecuteProcessor.java | 84 +++
.../flink/execution/SourceExecuteProcessor.java | 121 ++++
.../flink/execution/TransformExecuteProcessor.java | 86 +++
.../starter}/flink/utils/CommandLineUtils.java | 28 +-
.../core/starter/flink/FlinkStarterTest.java | 65 +++
.../starter/flink/args/FlinkCommandArgsTest.java | 30 +-
.../pom.xml | 58 +-
.../bin/start-seatunnel-spark-new-connector.sh | 45 ++
.../src/main/docker/Dockerfile | 0
.../core/starter}/spark/SeatunnelSpark.java | 15 +-
.../core/starter}/spark/SparkStarter.java | 68 ++-
.../core/starter/spark/args/SparkCommandArgs.java | 62 ++
.../command/SparkApiConfValidateCommand.java} | 29 +-
.../spark/command/SparkApiTaskExecuteCommand.java | 60 ++
.../spark/command/SparkCommandBuilder.java | 26 +-
.../spark/config/SparkApiConfigChecker.java} | 17 +-
.../starter/spark/config/SparkEnvironment.java | 45 +-
.../execution/AbstractPluginExecuteProcessor.java | 68 +++
.../spark/execution/PluginExecuteProcessor.java} | 25 +-
.../spark/execution/SinkExecuteProcessor.java | 84 +++
.../spark/execution/SourceExecuteProcessor.java | 81 +++
.../starter/spark/execution/SparkExecution.java | 63 ++
.../spark/execution/TransformExecuteProcessor.java | 80 +++
.../core/starter/spark/utils/CommandLineUtils.java | 29 +-
.../core/starter/spark/SparkStarterTest.java | 25 +-
.../starter/spark/args/SparkCommandArgsTest.java | 67 +++
.../starter/spark/utils/CommandLineUtilsTest.java | 27 +-
.../src/test/resources/spark_application.conf | 0
seatunnel-dist/release-docs/LICENSE | 52 +-
seatunnel-dist/release-docs/NOTICE | 20 +
.../release-docs/licenses/LICENSE-bouncycastle.txt | 7 +
.../release-docs/licenses/LICENSE-jaxrs-api.txt | 637 +++++++++++++++++++++
seatunnel-dist/src/main/assembly/assembly-bin.xml | 48 +-
seatunnel-e2e/pom.xml | 4 +-
.../apache/seatunnel/e2e/flink/FlinkContainer.java | 2 +-
.../seatunnel-flink-new-connector-e2e}/pom.xml | 27 +-
.../apache/seatunnel/e2e/flink/FlinkContainer.java | 44 +-
.../e2e/flink/assertion/FakeSourceToAssertIT.java | 0
.../e2e/flink/fake/FakeSourceToConsoleIT.java | 25 +-
.../resources/assertion/fakesource_to_assert.conf | 88 +++
.../test/resources/fake/fakesource_to_console.conf | 35 +-
.../src/test/resources/log4j.properties | 0
.../apache/seatunnel/e2e/spark/SparkContainer.java | 2 +-
.../seatunnel-spark-new-connector-e2e}/pom.xml | 28 +-
.../apache/seatunnel/e2e/spark/SparkContainer.java | 21 +-
.../e2e/spark/fake/FakeSourceToConsoleIT.java | 29 +-
.../test/resources/fake/fakesource_to_console.conf | 34 +-
.../src/test/resources/log4j.properties | 0
seatunnel-examples/pom.xml | 2 +
.../seatunnel-flink-examples/pom.xml | 6 +
.../seatunnel/example/flink/LocalFlinkExample.java | 3 +-
.../pom.xml | 22 +-
.../example/flink/SeaTunnelApiExample.java} | 15 +-
.../main/resources/examples/fake_to_console.conf | 35 +-
.../src/main/resources/log4j.properties | 0
.../seatunnel-flink-sql-examples/pom.xml | 2 +-
.../seatunnel/example/spark/LocalSparkExample.java | 3 +-
.../seatunnel-spark-new-connector-example}/pom.xml | 54 +-
.../example/spark/SeaTunnelApiExample.java} | 33 +-
.../src/main/resources/examples/spark.batch.conf | 33 +-
{seatunnel-e2e => seatunnel-formats}/pom.xml | 8 +-
.../seatunnel-format-json}/pom.xml | 28 +-
.../format/json/JsonDeserializationSchema.java | 122 ++++
.../seatunnel/format/json/JsonToRowConverters.java | 281 +++++++++
.../pom.xml | 28 +-
.../plugin/discovery/AbstractPluginDiscovery.java | 181 ++++++
.../plugin/discovery/PluginDiscovery.java | 70 +++
.../plugin/discovery/PluginIdentifier.java | 89 +++
.../discovery/flink/FlinkSinkPluginDiscovery.java | 17 +-
.../flink/FlinkSourcePluginDiscovery.java | 18 +-
.../flink/FlinkTransformPluginDiscovery.java | 40 +-
.../SeaTunnelFlinkTransformPluginDiscovery.java | 29 +-
.../seatunnel/SeaTunnelSinkPluginDiscovery.java | 17 +-
.../seatunnel/SeaTunnelSourcePluginDiscovery.java | 18 +-
.../SeaTunnelSparkTransformPluginDiscovery.java | 18 +-
.../discovery/spark/SparkSinkPluginDiscovery.java | 17 +-
.../spark/SparkSourcePluginDiscovery.java | 17 +-
.../spark/SparkTransformPluginDiscovery.java | 39 +-
{seatunnel-e2e => seatunnel-translation}/pom.xml | 8 +-
.../seatunnel-translation-base}/pom.xml | 18 +-
.../translation/serialization/RowConverter.java | 128 +++++
.../serialization/SerializerConverter.java | 20 +-
.../sink/SinkAggregatedCommitterConverter.java | 19 +-
.../translation/sink/SinkCommitterConverter.java | 20 +-
.../seatunnel/translation/sink/SinkConverter.java | 22 +-
.../translation/sink/SinkWriterConverter.java | 14 +-
.../translation/source/BaseSourceFunction.java | 24 +-
.../source/CoordinatedEnumeratorContext.java | 60 ++
.../source/CoordinatedReaderContext.java | 62 ++
.../translation/source/CoordinatedSource.java | 273 +++++++++
.../source/ParallelEnumeratorContext.java | 77 +++
.../translation/source/ParallelReaderContext.java | 63 ++
.../translation/source/ParallelSource.java | 201 +++++++
.../util/ThreadPoolExecutorFactory.java | 31 +-
.../seatunnel-translation-flink}/pom.xml | 45 +-
.../serialization/CommitWrapperSerializer.java | 66 +++
.../flink/serialization/FlinkRowConverter.java | 115 ++++
.../FlinkSimpleVersionedSerializer.java | 33 +-
.../serialization/FlinkWriterStateSerializer.java | 68 +++
.../flink/serialization/KryoTypeInfo.java | 41 ++
.../flink/serialization/WrappedRow.java | 40 +-
.../translation/flink/sink/CommitWrapper.java | 16 +-
.../translation/flink/sink/FlinkCommitter.java | 56 ++
.../flink/sink/FlinkGlobalCommitter.java | 72 +++
.../translation/flink/sink/FlinkSink.java | 90 +++
.../translation/flink/sink/FlinkSinkWriter.java | 75 +++
.../translation/flink/sink/FlinkWriterState.java | 33 +-
.../flink/source/BaseSeaTunnelSourceFunction.java | 136 +++++
.../translation/flink/source/RowCollector.java | 55 ++
.../flink/source/SeaTunnelCoordinatedSource.java | 43 ++
.../flink/source/SeaTunnelParallelSource.java | 49 ++
.../flink/utils/TypeConverterUtils.java | 146 +++++
.../flink/utils/TypeConverterUtilsTest.java | 129 +++++
.../seatunnel-translation-spark}/pom.xml | 44 +-
.../spark/serialization/InternalRowConverter.java | 178 ++++++
.../spark/sink/SparkDataSourceWriter.java | 95 +++
.../translation/spark/sink/SparkDataWriter.java | 101 ++++
.../spark/sink/SparkDataWriterFactory.java | 62 ++
.../translation/spark/sink/SparkSink.java | 76 +++
.../translation/spark/sink/SparkSinkInjector.java | 49 ++
.../translation/spark/sink/SparkStreamWriter.java | 73 +++
.../spark/sink/SparkWriterCommitMessage.java | 23 +-
.../translation/spark/source/Handover.java | 88 +++
.../spark/source/InternalRowCollector.java | 53 ++
.../translation/spark/source/ReaderState.java | 32 +-
.../spark/source/SeaTunnelSourceSupport.java | 92 +++
.../spark/source/batch/BatchPartition.java | 47 ++
.../spark/source/batch/BatchSourceReader.java | 62 ++
.../batch/CoordinatedBatchPartitionReader.java | 97 ++++
.../source/batch/ParallelBatchPartitionReader.java | 147 +++++
.../source/continnous/ContinuousPartition.java | 53 ++
.../source/continnous/ContinuousSourceReader.java | 116 ++++
.../spark/source/continnous/CoordinationState.java | 49 ++
.../ParallelContinuousPartitionReader.java | 78 +++
.../CoordinatedMicroBatchPartitionReader.java | 138 +++++
.../spark/source/micro/MicroBatchPartition.java | 64 +++
.../spark/source/micro/MicroBatchSourceReader.java | 111 ++++
.../spark/source/micro/MicroBatchState.java | 27 +-
.../micro/ParallelMicroBatchPartitionReader.java | 182 ++++++
.../spark/utils/TypeConverterUtils.java | 173 ++++++
...org.apache.spark.sql.sources.DataSourceRegister | 18 +
tools/dependencies/known-dependencies.txt | 30 +-
528 files changed, 28605 insertions(+), 3192 deletions(-)
create mode 100644 docs/en/images/seatunnel_architecture.png
create mode 100644 docs/en/images/seatunnel_starter.png
copy docs/en/{connector => new-connector}/sink/Assert.md (58%)
create mode 100644 docs/en/new-connector/sink/Clickhouse.md
create mode 100644 docs/en/new-connector/sink/ClickhouseFile.md
create mode 100644 docs/en/new-connector/sink/Jdbc.md
copy docs/en/{connector => new-connector}/sink/common-options.md (100%)
create mode 100644 docs/en/new-connector/source/Clickhouse.md
create mode 100644 docs/en/new-connector/source/Jdbc.md
create mode 100644 docs/en/new-connector/source/common-options.md
rename seatunnel-connectors/plugin-mapping.properties => plugin-mapping.properties (86%)
copy {seatunnel-core => seatunnel-api}/pom.xml (79%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/common/PluginIdentifierInterface.java (64%)
copy seatunnel-apis/seatunnel-api-base/src/main/java/org/apache/seatunnel/apis/base/env/RuntimeEnv.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/common/PrepareFailException.java (63%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/common/SeaTunnelContext.java
copy seatunnel-apis/seatunnel-api-base/src/main/java/org/apache/seatunnel/apis/base/env/RuntimeEnv.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/common/SeaTunnelPluginLifeCycle.java (62%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/serialization/DefaultSerializer.java (57%)
copy seatunnel-common/src/main/java/org/apache/seatunnel/common/utils/ReflectionUtils.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/serialization/DeserializationSchema.java (50%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/serialization/SerializationSchema.java (69%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/serialization/Serializer.java (55%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/DefaultSinkWriterContext.java (50%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SeaTunnelSink.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkAggregatedCommitter.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkCommitter.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/SinkWriter.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/source/Boundedness.java (67%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/source/Collector.java (72%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SeaTunnelRuntimeEnvironment.java (71%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SeaTunnelSource.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceEvent.java (73%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceReader.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceSplit.java (74%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SourceSplitEnumerator.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/source/SupportCoordinate.java (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/state/CheckpointListener.java (73%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/Catalog.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/CatalogTable.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/Column.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/TableIdentifier.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/TablePath.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/TableSchema.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/exception/CatalogException.java (58%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/exception/DatabaseNotExistException.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/catalog/exception/TableNotExistException.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/connector/DeserializationFormat.java (54%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/connector/SerializationFormat.java (74%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/connector/SupportReadingMetadata.java (60%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/connector/TableSink.java (74%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/connector/TableSource.java (74%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/CatalogFactory.java (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/DeserializationFormatFactory.java (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/Factory.java (63%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/FactoryException.java (75%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/FactoryUtil.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/SerializationFormatFactory.java (73%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/SupportMultipleTable.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/TableFactoryContext.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/TableSinkFactory.java (51%)
copy seatunnel-apis/seatunnel-api-base/src/main/java/org/apache/seatunnel/apis/base/env/RuntimeEnv.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/factory/TableSourceFactory.java (58%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/ArrayType.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/BasicType.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/CompositeType.java (74%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/DecimalType.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/LocalTimeType.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/MapType.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/PrimitiveByteArrayType.java (51%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/RowKind.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/SeaTunnelDataType.java (68%)
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/SeaTunnelRow.java
create mode 100644 seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/SeaTunnelRowType.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-api/src/main/java/org/apache/seatunnel/api/table/type/SqlType.java (72%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-common/src/main/java/org/apache/seatunnel/common/constants/CollectionConstants.java (73%)
copy {seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config => seatunnel-common/src/main/java/org/apache/seatunnel/common/constants}/PluginType.java (92%)
create mode 100644 seatunnel-common/src/main/java/org/apache/seatunnel/common/utils/SerializationUtils.java
create mode 100644 seatunnel-connectors-v2-dist/pom.xml
create mode 100644 seatunnel-connectors-v2/README.md
create mode 100644 seatunnel-connectors-v2/README.zh.md
copy {seatunnel-e2e => seatunnel-connectors-v2/connector-assert}/pom.xml (59%)
create mode 100644 seatunnel-connectors-v2/connector-assert/src/main/java/org/apache/seatunnel/connectors/seatunnel/assertion/excecutor/AssertExecutor.java
create mode 100644 seatunnel-connectors-v2/connector-assert/src/main/java/org/apache/seatunnel/connectors/seatunnel/assertion/rule/AssertFieldRule.java
create mode 100644 seatunnel-connectors-v2/connector-assert/src/main/java/org/apache/seatunnel/connectors/seatunnel/assertion/rule/AssertRuleParser.java
create mode 100644 seatunnel-connectors-v2/connector-assert/src/main/java/org/apache/seatunnel/connectors/seatunnel/assertion/sink/AssertSink.java
create mode 100644 seatunnel-connectors-v2/connector-assert/src/main/java/org/apache/seatunnel/connectors/seatunnel/assertion/sink/AssertSinkWriter.java
create mode 100644 seatunnel-connectors-v2/connector-assert/src/test/java/org/apache/seatunnel/flink/assertion/AssertExecutorTest.java
create mode 100644 seatunnel-connectors-v2/connector-assert/src/test/java/org/apache/seatunnel/flink/assertion/rule/AssertRuleParserTest.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/pom.xml
copy seatunnel-common/src/main/java/org/apache/seatunnel/common/utils/ReflectionUtils.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/config/ClickhouseFileCopyMethod.java (54%)
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/config/Config.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/config/FileReaderOption.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/config/ReaderOption.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/shard/Shard.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/shard/ShardMetadata.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/DistributedEngine.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/client/ClickhouseBatchStatement.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/client/ClickhouseProxy.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/client/ClickhouseSink.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/client/ClickhouseSinkWriter.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/client/ShardRouter.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/file/ClickhouseFileSink.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/file/ClickhouseFileSinkWriter.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/file/ClickhouseTable.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/file/FileTransfer.java (72%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/file/FileTransferFactory.java (56%)
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/file/RsyncFileTransfer.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/file/ScpFileTransfer.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/inject/ArrayInjectFunction.java (56%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/inject/BigDecimalInjectFunction.java (56%)
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/inject/ClickhouseFieldInjectFunction.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/inject/DateInjectFunction.java (55%)
copy seatunnel-common/src/main/java/org/apache/seatunnel/common/utils/ReflectionUtils.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/inject/DateTimeInjectFunction.java (54%)
copy seatunnel-core/seatunnel-core-flink/src/main/java/org/apache/seatunnel/core/flink/command/FlinkCommandBuilder.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/inject/DoubleInjectFunction.java (51%)
copy seatunnel-common/src/main/java/org/apache/seatunnel/common/utils/ReflectionUtils.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/inject/FloatInjectFunction.java (54%)
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/inject/IntInjectFunction.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/inject/LongInjectFunction.java (58%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/sink/inject/StringInjectFunction.java (61%)
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/source/ClickhouseSource.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/source/ClickhouseSourceReader.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/source/ClickhouseSourceSplit.java (74%)
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/source/ClickhouseSourceSplitEnumerator.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/state/CKAggCommitInfo.java (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/state/CKCommitInfo.java (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/state/ClickhouseSinkState.java (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/state/ClickhouseSourceState.java (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/tool/IntHolder.java (70%)
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/util/ClickhouseUtil.java
create mode 100644 seatunnel-connectors-v2/connector-clickhouse/src/main/java/org/apache/seatunnel/connectors/seatunnel/clickhouse/util/TypeConvertUtil.java
copy {seatunnel-core => seatunnel-connectors-v2/connector-common}/pom.xml (76%)
create mode 100644 seatunnel-connectors-v2/connector-common/src/main/java/org/apache/seatunnel/connectors/seatunnel/common/sink/AbstractSimpleSink.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-common/src/main/java/org/apache/seatunnel/connectors/seatunnel/common/sink/AbstractSinkWriter.java (67%)
create mode 100644 seatunnel-connectors-v2/connector-common/src/main/java/org/apache/seatunnel/connectors/seatunnel/common/source/AbstractSingleSplitReader.java
create mode 100644 seatunnel-connectors-v2/connector-common/src/main/java/org/apache/seatunnel/connectors/seatunnel/common/source/AbstractSingleSplitSource.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-common/src/main/java/org/apache/seatunnel/connectors/seatunnel/common/source/SingleSplit.java (67%)
create mode 100644 seatunnel-connectors-v2/connector-common/src/main/java/org/apache/seatunnel/connectors/seatunnel/common/source/SingleSplitEnumerator.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-common/src/main/java/org/apache/seatunnel/connectors/seatunnel/common/source/SingleSplitEnumeratorState.java (74%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-common/src/main/java/org/apache/seatunnel/connectors/seatunnel/common/source/SingleSplitReaderContext.java (60%)
copy {seatunnel-core => seatunnel-connectors-v2/connector-console}/pom.xml (77%)
create mode 100644 seatunnel-connectors-v2/connector-console/src/main/java/org/apache/seatunnel/connectors/seatunnel/console/sink/ConsoleSink.java
copy seatunnel-core/seatunnel-core-flink/src/main/java/org/apache/seatunnel/core/flink/command/FlinkConfValidateCommand.java => seatunnel-connectors-v2/connector-console/src/main/java/org/apache/seatunnel/connectors/seatunnel/console/sink/ConsoleSinkWriter.java (50%)
copy {seatunnel-core => seatunnel-connectors-v2/connector-fake}/pom.xml (77%)
create mode 100644 seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeSource.java
create mode 100644 seatunnel-connectors-v2/connector-fake/src/main/java/org/apache/seatunnel/connectors/seatunnel/fake/source/FakeSourceReader.java
copy {seatunnel-common => seatunnel-connectors-v2/connector-hive}/pom.xml (76%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/config/Config.java (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/sink/HiveAggregatedCommitInfo.java (64%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/sink/HiveCommitInfo.java (64%)
create mode 100644 seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/sink/HiveSink.java
create mode 100644 seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/sink/HiveSinkAggregatedCommitter.java
create mode 100644 seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/sink/HiveSinkConfig.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/sink/HiveSinkState.java (74%)
create mode 100644 seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/sink/HiveSinkWriter.java
create mode 100644 seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/sink/file/writer/AbstractFileWriter.java
create mode 100644 seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/sink/file/writer/FileWriter.java
create mode 100644 seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/sink/file/writer/HdfsTxtFileWriter.java
create mode 100644 seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/sink/file/writer/HdfsUtils.java
copy {seatunnel-common => seatunnel-connectors-v2/connector-http}/pom.xml (74%)
create mode 100644 seatunnel-connectors-v2/connector-http/src/main/java/org/apache/seatunnel/connectors/seatunnel/http/client/HttpClientProvider.java
create mode 100644 seatunnel-connectors-v2/connector-http/src/main/java/org/apache/seatunnel/connectors/seatunnel/http/client/HttpResponse.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-http/src/main/java/org/apache/seatunnel/connectors/seatunnel/http/config/Config.java (67%)
create mode 100644 seatunnel-connectors-v2/connector-http/src/main/java/org/apache/seatunnel/connectors/seatunnel/http/source/HttpSource.java
create mode 100644 seatunnel-connectors-v2/connector-http/src/main/java/org/apache/seatunnel/connectors/seatunnel/http/source/HttpSourceParameter.java
create mode 100644 seatunnel-connectors-v2/connector-http/src/main/java/org/apache/seatunnel/connectors/seatunnel/http/source/HttpSourceReader.java
copy {seatunnel-common => seatunnel-connectors-v2/connector-jdbc}/pom.xml (74%)
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/AbstractJdbcCatalog.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/MySqlCatalog.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/config/JdbcConfig.java
copy seatunnel-common/src/main/java/org/apache/seatunnel/common/PropertiesUtil.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/config/JdbcSinkOptions.java (52%)
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/config/JdbcSourceOptions.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/JdbcInputFormat.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/JdbcOutputFormat.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/connection/DataSourceUtils.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/connection/JdbcConnectionProvider.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/connection/SimpleJdbcConnectionProvider.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/converter/AbstractJdbcRowConverter.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/converter/JdbcRowConverter.java (52%)
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/JdbcDialect.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/JdbcDialectFactory.java (53%)
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/JdbcDialectLoader.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/JdbcDialectTypeMapper.java (59%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/mysql/MySqlDialectFactory.java (56%)
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/mysql/MySqlTypeMapper.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/mysql/MysqlDialect.java (54%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/mysql/MysqlJdbcRowConverter.java (52%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/psql/PostgresDialect.java (53%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/psql/PostgresDialectFactory.java (57%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/psql/PostgresJdbcRowConverter.java (52%)
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/psql/PostgresTypeMapper.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/executor/BiConsumerWithException.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/executor/JdbcBatchStatementExecutor.java (56%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/executor/JdbcStatementBuilder.java (66%)
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/executor/SimpleBatchStatementExecutor.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/options/JdbcConfig.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/options/JdbcConnectionOptions.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/options/JdbcConnectorOptions.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/split/JdbcGenericParameterValuesProvider.java (53%)
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/split/JdbcNumericBetweenParametersProvider.java
copy seatunnel-apis/seatunnel-api-base/src/main/java/org/apache/seatunnel/apis/base/env/RuntimeEnv.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/split/JdbcParameterValuesProvider.java (58%)
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/xa/GroupXaOperationResult.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/xa/SemanticXidGenerator.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/xa/XaFacade.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/xa/XaFacadeImplAutoLoad.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/xa/XaGroupOps.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/xa/XaGroupOpsImpl.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/xa/XidGenerator.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/xa/XidImpl.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/sink/JdbcExactlyOnceSinkWriter.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/sink/JdbcSink.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/sink/JdbcSinkAggregatedCommitter.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/sink/JdbcSinkCommitter.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/sink/JdbcSinkWriter.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/JdbcSource.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/JdbcSourceReader.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/JdbcSourceSplit.java (68%)
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/JdbcSourceSplitEnumerator.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/PartitionParameter.java (72%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/state/JdbcAggregatedCommitInfo.java (72%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/state/JdbcSinkState.java (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/state/JdbcSourceState.java (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/state/XidInfo.java (67%)
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/ExceptionUtils.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/JdbcUtils.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/utils/ThrowingRunnable.java
create mode 100644 seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/AbstractJdbcCatalogTest.java
copy {seatunnel-common => seatunnel-connectors-v2/connector-kafka}/pom.xml (71%)
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/config/Config.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/config/KafkaSemantics.java (61%)
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/serialize/DefaultSeaTunnelRowSerializer.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/serialize/SeaTunnelRowSerializer.java (65%)
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/sink/KafkaInternalProducer.java
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/sink/KafkaNoTransactionSender.java
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/sink/KafkaProduceSender.java
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/sink/KafkaSink.java
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/sink/KafkaSinkCommitter.java
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/sink/KafkaSinkWriter.java
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/sink/KafkaTransactionSender.java
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/source/ConsumerMetadata.java
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/source/KafkaSource.java
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/source/KafkaSourceReader.java
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/source/KafkaSourceSplit.java
create mode 100644 seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/source/KafkaSourceSplitEnumerator.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/state/KafkaAggregatedCommitInfo.java (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/state/KafkaCommitInfo.java (66%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/state/KafkaSinkState.java (65%)
copy seatunnel-apis/seatunnel-api-base/src/main/java/org/apache/seatunnel/apis/base/env/RuntimeEnv.java => seatunnel-connectors-v2/connector-kafka/src/main/java/org/apache/seatunnel/connectors/seatunnel/kafka/state/KafkaSourceState.java (57%)
create mode 100644 seatunnel-connectors-v2/connector-pulsar/pom.xml
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/config/BasePulsarConfig.java (59%)
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/config/PulsarAdminConfig.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/config/PulsarClientConfig.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/config/PulsarConfigUtil.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/config/PulsarConsumerConfig.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/config/SourceProperties.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/PulsarSource.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/PulsarSplitEnumerator.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/PulsarSplitEnumeratorState.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/cursor/start/MessageIdStartCursor.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/cursor/start/StartCursor.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/cursor/start/SubscriptionStartCursor.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/cursor/start/TimestampStartCursor.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/cursor/stop/LatestMessageStopCursor.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/cursor/stop/MessageIdStopCursor.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/cursor/stop/NeverStopCursor.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/cursor/stop/StopCursor.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/cursor/stop/TimestampStopCursor.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/discoverer/PulsarDiscoverer.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/discoverer/TopicListDiscoverer.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/discoverer/TopicPatternDiscoverer.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/enumerator/topic/TopicPartition.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/reader/Handover.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/reader/PulsarSourceReader.java
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/reader/PulsarSplitReaderThread.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/reader/RecordWithSplitId.java (62%)
create mode 100644 seatunnel-connectors-v2/connector-pulsar/src/main/java/org/apache/seatunnel/connectors/seatunnel/pulsar/source/split/PulsarPartitionSplit.java
copy {seatunnel-core => seatunnel-connectors-v2/connector-socket}/pom.xml (77%)
create mode 100644 seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSource.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceParameter.java (53%)
create mode 100644 seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/source/SocketSourceReader.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-connectors-v2/connector-socket/src/main/java/org/apache/seatunnel/connectors/seatunnel/socket/state/SocketState.java (73%)
copy {seatunnel-common => seatunnel-connectors-v2}/pom.xml (63%)
create mode 100644 seatunnel-core/README.md
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/{ExecutionContext.java => AbstractExecutionContext.java} (60%)
copy seatunnel-apis/seatunnel-api-base/src/main/java/org/apache/seatunnel/apis/base/env/RuntimeEnv.java => seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/ConfigChecker.java (63%)
delete mode 100644 seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginFactory.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/{config/PluginType.java => exception/CommandException.java} (75%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/{config/PluginType.java => exception/CommandExecuteException.java} (73%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/{config/PluginType.java => exception/ConfigCheckException.java} (74%)
delete mode 100644 seatunnel-core/seatunnel-core-base/src/test/java/org/apache/seatunnel/core/base/config/PluginFactoryTest.java
copy seatunnel-core/seatunnel-core-flink/src/main/java/org/apache/seatunnel/core/flink/command/{FlinkConfValidateCommand.java => FlinkApiConfValidateCommand.java} (74%)
rename seatunnel-core/seatunnel-core-flink/src/main/java/org/apache/seatunnel/core/flink/command/{FlinkTaskExecuteCommand.java => FlinkApiTaskExecuteCommand.java} (75%)
create mode 100644 seatunnel-core/seatunnel-core-flink/src/main/java/org/apache/seatunnel/core/flink/config/FlinkApiConfigChecker.java
create mode 100644 seatunnel-core/seatunnel-core-flink/src/main/java/org/apache/seatunnel/core/flink/config/FlinkExecutionContext.java
create mode 100644 seatunnel-core/seatunnel-core-spark/src/main/java/org/apache/seatunnel/core/spark/config/SparkApiConfigChecker.java
create mode 100644 seatunnel-core/seatunnel-core-spark/src/main/java/org/apache/seatunnel/core/spark/config/SparkExecutionContext.java
create mode 100644 seatunnel-core/seatunnel-core-starter/README.md
copy seatunnel-core/{seatunnel-core-base => seatunnel-core-starter}/pom.xml (89%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter}/Seatunnel.java (91%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/Starter.java} (72%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter}/command/AbstractCommandArgs.java (94%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter}/command/Command.java (77%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/command/CommandArgs.java} (73%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/command/CommandBuilder.java} (75%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/command/DeployModeConverter.java} (59%)
rename seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/ExecutionContext.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/config/AbstractExecutionContext.java} (58%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter}/config/ConfigBuilder.java (74%)
copy seatunnel-apis/seatunnel-api-base/src/main/java/org/apache/seatunnel/apis/base/env/RuntimeEnv.java => seatunnel-core/seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/config/ConfigChecker.java (63%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/config/EngineType.java} (75%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter}/config/EnvironmentFactory.java (96%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter}/config/ExecutionFactory.java (94%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter}/config/PluginType.java (95%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/constants/CommonParamConstants.java} (73%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/exception/CommandException.java} (75%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/exception/CommandExecuteException.java} (73%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/exception/ConfigCheckException.java} (74%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/exception/TaskExecuteException.java} (74%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/execution/TaskExecution.java} (74%)
create mode 100644 seatunnel-core/seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/utils/AsciiArtUtils.java
create mode 100644 seatunnel-core/seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/utils/CompressionUtils.java
create mode 100644 seatunnel-core/seatunnel-core-starter/src/main/java/org/apache/seatunnel/core/starter/utils/FileUtils.java
create mode 100644 seatunnel-core/seatunnel-core-starter/src/test/java/org/apache/seatunnel/core/starter/utils/CompressionUtilsTest.java
copy seatunnel-core/{seatunnel-core-base/src/test/java/org/apache/seatunnel/core/base => seatunnel-core-starter/src/test/java/org/apache/seatunnel/core/starter}/utils/FileUtilsTest.java (94%)
copy seatunnel-core/{seatunnel-core-base => seatunnel-core-starter}/src/test/resources/flink.batch.conf (100%)
copy seatunnel-core/{seatunnel-core-flink => seatunnel-flink-starter}/pom.xml (80%)
create mode 100755 seatunnel-core/seatunnel-flink-starter/src/main/bin/start-seatunnel-flink-new-connector.sh
copy seatunnel-core/{seatunnel-core-flink => seatunnel-flink-starter}/src/main/docker/Dockerfile (100%)
copy seatunnel-core/{seatunnel-core-flink/src/main/java/org/apache/seatunnel/core => seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter}/flink/FlinkStarter.java (79%)
copy seatunnel-core/{seatunnel-core-flink/src/main/java/org/apache/seatunnel/core => seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter}/flink/SeatunnelFlink.java (65%)
copy seatunnel-core/{seatunnel-core-flink/src/main/java/org/apache/seatunnel/core => seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter}/flink/args/FlinkCommandArgs.java (91%)
rename seatunnel-core/{seatunnel-core-flink/src/main/java/org/apache/seatunnel/core/flink/command/FlinkConfValidateCommand.java => seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/command/FlinkApiConfValidateCommand.java} (55%)
create mode 100644 seatunnel-core/seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/command/FlinkApiTaskExecuteCommand.java
copy seatunnel-core/{seatunnel-core-flink/src/main/java/org/apache/seatunnel/core => seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter}/flink/command/FlinkCommandBuilder.java (63%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/config/FlinkApiConfigChecker.java} (64%)
create mode 100644 seatunnel-core/seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/config/FlinkApiEnvironment.java
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/config/FlinkJobType.java} (80%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/config/FlinkRunMode.java} (63%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/constant/FlinkConstant.java} (73%)
copy {seatunnel-apis/seatunnel-api-flink/src/main/java/org/apache/seatunnel/flink => seatunnel-core/seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/env}/FlinkEnvironment.java (84%)
create mode 100644 seatunnel-core/seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/execution/AbstractPluginExecuteProcessor.java
create mode 100644 seatunnel-core/seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/execution/FlinkExecution.java
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/execution/PluginExecuteProcessor.java} (58%)
create mode 100644 seatunnel-core/seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/execution/SinkExecuteProcessor.java
create mode 100644 seatunnel-core/seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/execution/SourceExecuteProcessor.java
create mode 100644 seatunnel-core/seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter/flink/execution/TransformExecuteProcessor.java
copy seatunnel-core/{seatunnel-core-flink/src/main/java/org/apache/seatunnel/core => seatunnel-flink-starter/src/main/java/org/apache/seatunnel/core/starter}/flink/utils/CommandLineUtils.java (74%)
create mode 100644 seatunnel-core/seatunnel-flink-starter/src/test/java/org/apache/seatunnel/core/starter/flink/FlinkStarterTest.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-core/seatunnel-flink-starter/src/test/java/org/apache/seatunnel/core/starter/flink/args/FlinkCommandArgsTest.java (54%)
copy seatunnel-core/{seatunnel-core-spark => seatunnel-spark-starter}/pom.xml (80%)
create mode 100755 seatunnel-core/seatunnel-spark-starter/src/main/bin/start-seatunnel-spark-new-connector.sh
copy seatunnel-core/{seatunnel-core-spark => seatunnel-spark-starter}/src/main/docker/Dockerfile (100%)
copy seatunnel-core/{seatunnel-core-spark/src/main/java/org/apache/seatunnel/core => seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter}/spark/SeatunnelSpark.java (67%)
copy seatunnel-core/{seatunnel-core-spark/src/main/java/org/apache/seatunnel/core => seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter}/spark/SparkStarter.java (80%)
create mode 100644 seatunnel-core/seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/args/SparkCommandArgs.java
copy seatunnel-core/{seatunnel-core-spark/src/main/java/org/apache/seatunnel/core/spark/command/SparkConfValidateCommand.java => seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/command/SparkApiConfValidateCommand.java} (52%)
create mode 100644 seatunnel-core/seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/command/SparkApiTaskExecuteCommand.java
copy seatunnel-core/{seatunnel-core-spark/src/main/java/org/apache/seatunnel/core => seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter}/spark/command/SparkCommandBuilder.java (56%)
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/config/SparkApiConfigChecker.java} (64%)
copy seatunnel-apis/seatunnel-api-base/src/main/java/org/apache/seatunnel/apis/base/env/RuntimeEnv.java => seatunnel-core/seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/config/SparkEnvironment.java (56%)
create mode 100644 seatunnel-core/seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/execution/AbstractPluginExecuteProcessor.java
copy seatunnel-core/{seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/execution/PluginExecuteProcessor.java} (60%)
create mode 100644 seatunnel-core/seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/execution/SinkExecuteProcessor.java
create mode 100644 seatunnel-core/seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/execution/SourceExecuteProcessor.java
create mode 100644 seatunnel-core/seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/execution/SparkExecution.java
create mode 100644 seatunnel-core/seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/execution/TransformExecuteProcessor.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-core/seatunnel-spark-starter/src/main/java/org/apache/seatunnel/core/starter/spark/utils/CommandLineUtils.java (56%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-core/seatunnel-spark-starter/src/test/java/org/apache/seatunnel/core/starter/spark/SparkStarterTest.java (59%)
create mode 100644 seatunnel-core/seatunnel-spark-starter/src/test/java/org/apache/seatunnel/core/starter/spark/args/SparkCommandArgsTest.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-core/seatunnel-spark-starter/src/test/java/org/apache/seatunnel/core/starter/spark/utils/CommandLineUtilsTest.java (51%)
copy seatunnel-core/{seatunnel-core-spark => seatunnel-spark-starter}/src/test/resources/spark_application.conf (100%)
create mode 100644 seatunnel-dist/release-docs/licenses/LICENSE-bouncycastle.txt
create mode 100644 seatunnel-dist/release-docs/licenses/LICENSE-jaxrs-api.txt
copy {seatunnel-common => seatunnel-e2e/seatunnel-flink-new-connector-e2e}/pom.xml (71%)
copy seatunnel-e2e/{seatunnel-flink-e2e => seatunnel-flink-new-connector-e2e}/src/test/java/org/apache/seatunnel/e2e/flink/FlinkContainer.java (83%)
copy seatunnel-e2e/{seatunnel-flink-e2e => seatunnel-flink-new-connector-e2e}/src/test/java/org/apache/seatunnel/e2e/flink/assertion/FakeSourceToAssertIT.java (100%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-e2e/seatunnel-flink-new-connector-e2e/src/test/java/org/apache/seatunnel/e2e/flink/fake/FakeSourceToConsoleIT.java (60%)
create mode 100644 seatunnel-e2e/seatunnel-flink-new-connector-e2e/src/test/resources/assertion/fakesource_to_assert.conf
copy config/flink.batch.conf.template => seatunnel-e2e/seatunnel-flink-new-connector-e2e/src/test/resources/fake/fakesource_to_console.conf (66%)
copy seatunnel-e2e/{seatunnel-flink-e2e => seatunnel-flink-new-connector-e2e}/src/test/resources/log4j.properties (100%)
copy {seatunnel-common => seatunnel-e2e/seatunnel-spark-new-connector-e2e}/pom.xml (71%)
copy seatunnel-e2e/{seatunnel-spark-e2e => seatunnel-spark-new-connector-e2e}/src/test/java/org/apache/seatunnel/e2e/spark/SparkContainer.java (88%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-e2e/seatunnel-spark-new-connector-e2e/src/test/java/org/apache/seatunnel/e2e/spark/fake/FakeSourceToConsoleIT.java (54%)
copy config/spark.streaming.conf.template => seatunnel-e2e/seatunnel-spark-new-connector-e2e/src/test/resources/fake/fakesource_to_console.conf (78%)
copy seatunnel-e2e/{seatunnel-spark-e2e => seatunnel-spark-new-connector-e2e}/src/test/resources/log4j.properties (100%)
copy seatunnel-examples/{seatunnel-flink-examples => seatunnel-flink-new-connector-example}/pom.xml (87%)
copy seatunnel-examples/{seatunnel-flink-examples/src/main/java/org/apache/seatunnel/example/flink/LocalFlinkExample.java => seatunnel-flink-new-connector-example/src/main/java/org/apache/seatunnel/example/flink/SeaTunnelApiExample.java} (78%)
copy config/flink.batch.conf.template => seatunnel-examples/seatunnel-flink-new-connector-example/src/main/resources/examples/fake_to_console.conf (66%)
copy seatunnel-examples/{seatunnel-flink-examples => seatunnel-flink-new-connector-example}/src/main/resources/log4j.properties (100%)
copy {seatunnel-connectors/seatunnel-connectors-flink/seatunnel-connector-flink-clickhouse => seatunnel-examples/seatunnel-spark-new-connector-example}/pom.xml (53%)
copy seatunnel-examples/{seatunnel-flink-examples/src/main/java/org/apache/seatunnel/example/flink/LocalFlinkExample.java => seatunnel-spark-new-connector-example/src/main/java/org/apache/seatunnel/example/spark/SeaTunnelApiExample.java} (56%)
copy config/spark.streaming.conf.template => seatunnel-examples/seatunnel-spark-new-connector-example/src/main/resources/examples/spark.batch.conf (78%)
copy {seatunnel-e2e => seatunnel-formats}/pom.xml (87%)
copy {seatunnel-core => seatunnel-formats/seatunnel-format-json}/pom.xml (76%)
create mode 100644 seatunnel-formats/seatunnel-format-json/src/main/java/org/apache/seatunnel/format/json/JsonDeserializationSchema.java
create mode 100644 seatunnel-formats/seatunnel-format-json/src/main/java/org/apache/seatunnel/format/json/JsonToRowConverters.java
copy {seatunnel-core/seatunnel-core-base => seatunnel-plugin-discovery}/pom.xml (81%)
create mode 100644 seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/AbstractPluginDiscovery.java
create mode 100644 seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/PluginDiscovery.java
create mode 100644 seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/PluginIdentifier.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/flink/FlinkSinkPluginDiscovery.java (65%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/flink/FlinkSourcePluginDiscovery.java (64%)
copy seatunnel-apis/seatunnel-api-base/src/main/java/org/apache/seatunnel/apis/base/env/RuntimeEnv.java => seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/flink/FlinkTransformPluginDiscovery.java (53%)
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/seatunnel/SeaTunnelFlinkTransformPluginDiscovery.java (60%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/seatunnel/SeaTunnelSinkPluginDiscovery.java (64%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/seatunnel/SeaTunnelSourcePluginDiscovery.java (63%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/seatunnel/SeaTunnelSparkTransformPluginDiscovery.java (62%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/spark/SparkSinkPluginDiscovery.java (65%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/spark/SparkSourcePluginDiscovery.java (64%)
copy seatunnel-apis/seatunnel-api-base/src/main/java/org/apache/seatunnel/apis/base/env/RuntimeEnv.java => seatunnel-plugin-discovery/src/main/java/org/apache/seatunnel/plugin/discovery/spark/SparkTransformPluginDiscovery.java (51%)
copy {seatunnel-e2e => seatunnel-translation}/pom.xml (85%)
copy {seatunnel-e2e => seatunnel-translation/seatunnel-translation-base}/pom.xml (77%)
create mode 100644 seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/serialization/RowConverter.java
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/serialization/SerializerConverter.java (67%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/sink/SinkAggregatedCommitterConverter.java (61%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/sink/SinkCommitterConverter.java (67%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/sink/SinkConverter.java (63%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/sink/SinkWriterConverter.java (74%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/command/Command.java => seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/source/BaseSourceFunction.java (63%)
create mode 100644 seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/source/CoordinatedEnumeratorContext.java
create mode 100644 seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/source/CoordinatedReaderContext.java
create mode 100644 seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/source/CoordinatedSource.java
create mode 100644 seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/source/ParallelEnumeratorContext.java
create mode 100644 seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/source/ParallelReaderContext.java
create mode 100644 seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/source/ParallelSource.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-translation/seatunnel-translation-base/src/main/java/org/apache/seatunnel/translation/util/ThreadPoolExecutorFactory.java (55%)
copy {seatunnel-connectors/seatunnel-connectors-flink/seatunnel-connector-flink-clickhouse => seatunnel-translation/seatunnel-translation-flink}/pom.xml (71%)
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/serialization/CommitWrapperSerializer.java
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/serialization/FlinkRowConverter.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/serialization/FlinkSimpleVersionedSerializer.java (51%)
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/serialization/FlinkWriterStateSerializer.java
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/serialization/KryoTypeInfo.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/serialization/WrappedRow.java (51%)
copy seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/sink/CommitWrapper.java (75%)
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/sink/FlinkCommitter.java
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/sink/FlinkGlobalCommitter.java
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/sink/FlinkSink.java
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/sink/FlinkSinkWriter.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/sink/FlinkWriterState.java (56%)
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/source/BaseSeaTunnelSourceFunction.java
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/source/RowCollector.java
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/source/SeaTunnelCoordinatedSource.java
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/source/SeaTunnelParallelSource.java
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/main/java/org/apache/seatunnel/translation/flink/utils/TypeConverterUtils.java
create mode 100644 seatunnel-translation/seatunnel-translation-flink/src/test/java/org/apache/seatunnel/translation/flink/utils/TypeConverterUtilsTest.java
copy {seatunnel-core/seatunnel-core-base => seatunnel-translation/seatunnel-translation-spark}/pom.xml (61%)
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/serialization/InternalRowConverter.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/sink/SparkDataSourceWriter.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/sink/SparkDataWriter.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/sink/SparkDataWriterFactory.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/sink/SparkSink.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/sink/SparkSinkInjector.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/sink/SparkStreamWriter.java
rename seatunnel-core/seatunnel-core-base/src/main/java/org/apache/seatunnel/core/base/config/PluginType.java => seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/sink/SparkWriterCommitMessage.java (63%)
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/Handover.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/InternalRowCollector.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/ReaderState.java (51%)
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/SeaTunnelSourceSupport.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/batch/BatchPartition.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/batch/BatchSourceReader.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/batch/CoordinatedBatchPartitionReader.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/batch/ParallelBatchPartitionReader.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/continnous/ContinuousPartition.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/continnous/ContinuousSourceReader.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/continnous/CoordinationState.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/continnous/ParallelContinuousPartitionReader.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/micro/CoordinatedMicroBatchPartitionReader.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/micro/MicroBatchPartition.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/micro/MicroBatchSourceReader.java
copy seatunnel-common/src/test/java/org/apache/seatunnel/common/config/CommonTest.java => seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/micro/MicroBatchState.java (58%)
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/source/micro/ParallelMicroBatchPartitionReader.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/java/org/apache/seatunnel/translation/spark/utils/TypeConverterUtils.java
create mode 100644 seatunnel-translation/seatunnel-translation-spark/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister
[incubator-seatunnel] 01/01: Merge pull request #2083 from apache/api-draft
Posted by ki...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kirs pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/incubator-seatunnel.git
commit df251bf47773fa2f80c2d7008d9ff94b88c8c912
Merge: 770bade75 1b706c16a
Author: Kirs <ki...@apache.org>
AuthorDate: Fri Jul 1 16:26:43 2022 +0800
Merge pull request #2083 from apache/api-draft
[API-DRAFT] [MERGE] Merge api-draft to dev branch
.github/workflows/backend.yml | 2 +-
.licenserc.yaml | 2 +-
LICENSE | 1 +
config/flink.batch.conf.template | 2 +
config/spark.streaming.conf.template | 2 +-
docs/en/connector/sink/Assert.md | 47 +-
docs/en/images/seatunnel_architecture.png | Bin 0 -> 778394 bytes
docs/en/images/seatunnel_starter.png | Bin 0 -> 423840 bytes
.../en/{connector => new-connector}/sink/Assert.md | 51 +-
docs/en/new-connector/sink/Clickhouse.md | 104 ++++
docs/en/new-connector/sink/ClickhouseFile.md | 106 ++++
docs/en/new-connector/sink/Jdbc.md | 96 ++++
docs/en/new-connector/sink/common-options.md | 45 ++
docs/en/new-connector/source/Clickhouse.md | 63 ++
docs/en/new-connector/source/Jdbc.md | 75 +++
docs/en/new-connector/source/common-options.md | 33 ++
...mapping.properties => plugin-mapping.properties | 17 +
pom.xml | 105 ++--
{seatunnel-core => seatunnel-api}/pom.xml | 23 +-
.../api/common/PluginIdentifierInterface.java | 21 +-
.../seatunnel/api/common/PrepareFailException.java | 30 +-
.../seatunnel/api/common/SeaTunnelContext.java | 87 +++
.../api/common/SeaTunnelPluginLifeCycle.java | 35 +-
.../api/serialization/DefaultSerializer.java | 31 +-
.../api/serialization/DeserializationSchema.java | 36 +-
.../api/serialization/SerializationSchema.java | 21 +-
.../seatunnel/api/serialization/Serializer.java | 29 +-
.../api/sink/DefaultSinkWriterContext.java | 36 +-
.../apache/seatunnel/api/sink/SeaTunnelSink.java | 122 ++++
.../api/sink/SinkAggregatedCommitter.java | 66 +++
.../apache/seatunnel/api/sink/SinkCommitter.java | 50 ++
.../org/apache/seatunnel/api/sink/SinkWriter.java | 95 +++
.../apache/seatunnel/api/source/Boundedness.java | 22 +-
.../org/apache/seatunnel/api/source/Collector.java | 20 +-
.../api/source/SeaTunnelRuntimeEnvironment.java | 19 +-
.../seatunnel/api/source/SeaTunnelSource.java | 99 ++++
.../apache/seatunnel/api/source/SourceEvent.java | 18 +-
.../apache/seatunnel/api/source/SourceReader.java | 119 ++++
.../apache/seatunnel/api/source/SourceSplit.java | 18 +-
.../api/source/SourceSplitEnumerator.java | 124 ++++
.../seatunnel/api/source/SupportCoordinate.java | 18 +-
.../seatunnel/api/state/CheckpointListener.java | 18 +-
.../seatunnel/api/table/catalog/Catalog.java | 116 ++++
.../seatunnel/api/table/catalog/CatalogTable.java | 93 +++
.../apache/seatunnel/api/table/catalog/Column.java | 223 ++++++++
.../api/table/catalog/TableIdentifier.java | 81 +++
.../seatunnel/api/table/catalog/TablePath.java | 86 +++
.../seatunnel/api/table/catalog/TableSchema.java | 132 +++++
.../table/catalog/exception/CatalogException.java | 31 +-
.../exception/DatabaseNotExistException.java | 32 ++
.../catalog/exception/TableNotExistException.java | 35 ++
.../api/table/connector/DeserializationFormat.java | 28 +-
.../api/table/connector/SerializationFormat.java | 15 +-
.../table/connector/SupportReadingMetadata.java | 21 +-
.../seatunnel/api/table/connector/TableSink.java | 15 +-
.../seatunnel/api/table/connector/TableSource.java | 16 +-
.../api/table/factory/CatalogFactory.java | 19 +-
.../factory/DeserializationFormatFactory.java | 16 +-
.../seatunnel/api/table/factory/Factory.java | 21 +-
.../api/table/factory/FactoryException.java | 15 +-
.../seatunnel/api/table/factory/FactoryUtil.java | 155 +++++
.../table/factory/SerializationFormatFactory.java | 16 +-
.../api/table/factory/SupportMultipleTable.java | 60 ++
.../api/table/factory/TableFactoryContext.java | 58 ++
.../api/table/factory/TableSinkFactory.java | 20 +-
.../api/table/factory/TableSourceFactory.java | 31 +-
.../apache/seatunnel/api/table/type/ArrayType.java | 87 +++
.../apache/seatunnel/api/table/type/BasicType.java | 79 +++
.../seatunnel/api/table/type/CompositeType.java | 15 +-
.../seatunnel/api/table/type/DecimalType.java | 65 +++
.../seatunnel/api/table/type/LocalTimeType.java | 72 +++
.../apache/seatunnel/api/table/type/MapType.java | 104 ++++
.../api/table/type/PrimitiveByteArrayType.java | 38 +-
.../apache/seatunnel/api/table/type/RowKind.java | 117 ++++
.../api/table/type/SeaTunnelDataType.java | 19 +-
.../seatunnel/api/table/type/SeaTunnelRow.java | 106 ++++
.../seatunnel/api/table/type/SeaTunnelRowType.java | 121 ++++
.../apache/seatunnel/api/table/type/SqlType.java | 34 +-
.../apache/seatunnel/apis/base/env/RuntimeEnv.java | 4 +-
.../apache/seatunnel/apis/base/plugin/Plugin.java | 14 +-
.../apache/seatunnel/flink/FlinkEnvironment.java | 18 +-
.../apache/seatunnel/flink/util/SchemaUtil.java | 2 +
.../apache/seatunnel/spark/SparkEnvironment.java | 4 +-
seatunnel-common/pom.xml | 9 +
.../apache/seatunnel/common/PropertiesUtil.java | 37 ++
.../org/apache/seatunnel/common/config/Common.java | 33 +-
.../common/constants/CollectionConstants.java | 15 +-
.../seatunnel/common/constants}/PluginType.java | 6 +-
.../seatunnel/common/utils/ReflectionUtils.java | 49 ++
.../seatunnel/common/utils/SerializationUtils.java | 49 ++
.../apache/seatunnel/common/config/CommonTest.java | 2 +-
.../common/utils/VariablesSubstituteTest.java | 7 +-
seatunnel-connectors-v2-dist/pom.xml | 108 ++++
seatunnel-connectors-v2/README.md | 84 +++
seatunnel-connectors-v2/README.zh.md | 64 +++
.../connector-assert}/pom.xml | 33 +-
.../assertion/excecutor/AssertExecutor.java | 108 ++++
.../seatunnel/assertion/rule/AssertFieldRule.java | 65 +++
.../seatunnel/assertion/rule/AssertRuleParser.java | 83 +++
.../seatunnel/assertion/sink/AssertSink.java | 85 +++
.../seatunnel/assertion/sink/AssertSinkWriter.java | 55 ++
.../flink/assertion/AssertExecutorTest.java | 110 ++++
.../flink/assertion/rule/AssertRuleParserTest.java | 85 +++
.../connector-clickhouse/pom.xml | 70 +++
.../config/ClickhouseFileCopyMethod.java | 34 +-
.../seatunnel/clickhouse/config/Config.java | 96 ++++
.../clickhouse/config/FileReaderOption.java | 113 ++++
.../seatunnel/clickhouse/config/ReaderOption.java | 94 +++
.../seatunnel/clickhouse/shard/Shard.java | 98 ++++
.../seatunnel/clickhouse/shard/ShardMetadata.java | 145 +++++
.../clickhouse/sink/DistributedEngine.java | 58 ++
.../sink/client/ClickhouseBatchStatement.java | 52 ++
.../clickhouse/sink/client/ClickhouseProxy.java | 212 +++++++
.../clickhouse/sink/client/ClickhouseSink.java | 172 ++++++
.../sink/client/ClickhouseSinkWriter.java | 230 ++++++++
.../clickhouse/sink/client/ShardRouter.java | 97 ++++
.../clickhouse/sink/file/ClickhouseFileSink.java | 145 +++++
.../sink/file/ClickhouseFileSinkWriter.java | 237 ++++++++
.../clickhouse/sink/file/ClickhouseTable.java | 117 ++++
.../clickhouse/sink/file/FileTransfer.java | 19 +-
.../clickhouse/sink/file/FileTransferFactory.java | 36 +-
.../clickhouse/sink/file/RsyncFileTransfer.java | 144 +++++
.../clickhouse/sink/file/ScpFileTransfer.java | 124 ++++
.../sink/inject/ArrayInjectFunction.java | 28 +-
.../sink/inject/BigDecimalInjectFunction.java | 28 +-
.../sink/inject/ClickhouseFieldInjectFunction.java | 46 ++
.../clickhouse/sink/inject/DateInjectFunction.java | 37 +-
.../sink/inject/DateTimeInjectFunction.java | 34 +-
.../sink/inject/DoubleInjectFunction.java | 31 +-
.../sink/inject/FloatInjectFunction.java | 34 +-
.../clickhouse/sink/inject/IntInjectFunction.java | 47 ++
.../clickhouse/sink/inject/LongInjectFunction.java | 29 +-
.../sink/inject/StringInjectFunction.java | 18 +-
.../clickhouse/source/ClickhouseSource.java | 132 +++++
.../clickhouse/source/ClickhouseSourceReader.java | 108 ++++
.../clickhouse/source/ClickhouseSourceSplit.java | 17 +-
.../source/ClickhouseSourceSplitEnumerator.java | 100 ++++
.../clickhouse/state/CKAggCommitInfo.java | 15 +-
.../seatunnel/clickhouse/state/CKCommitInfo.java | 15 +-
.../clickhouse/state/ClickhouseSinkState.java | 15 +-
.../clickhouse/state/ClickhouseSourceState.java | 15 +-
.../seatunnel/clickhouse/tool/IntHolder.java | 19 +-
.../seatunnel/clickhouse/util/ClickhouseUtil.java | 40 ++
.../seatunnel/clickhouse/util/TypeConvertUtil.java | 100 ++++
.../connector-common}/pom.xml | 23 +-
.../seatunnel/common/sink/AbstractSimpleSink.java | 59 ++
.../seatunnel/common/sink/AbstractSinkWriter.java | 19 +-
.../common/source/AbstractSingleSplitReader.java | 67 +++
.../common/source/AbstractSingleSplitSource.java | 58 ++
.../seatunnel/common/source/SingleSplit.java | 21 +-
.../common/source/SingleSplitEnumerator.java | 94 +++
.../common/source/SingleSplitEnumeratorState.java | 15 +-
.../common/source/SingleSplitReaderContext.java | 26 +-
.../connector-console}/pom.xml | 23 +-
.../seatunnel/console/sink/ConsoleSink.java | 63 ++
.../seatunnel/console/sink/ConsoleSinkWriter.java | 37 +-
.../connector-fake}/pom.xml | 21 +-
.../seatunnel/fake/source/FakeSource.java | 73 +++
.../seatunnel/fake/source/FakeSourceReader.java | 73 +++
.../connector-hive}/pom.xml | 25 +-
.../connectors/seatunnel/hive/config/Config.java | 15 +-
.../hive/sink/HiveAggregatedCommitInfo.java | 25 +-
.../seatunnel/hive/sink/HiveCommitInfo.java | 25 +-
.../connectors/seatunnel/hive/sink/HiveSink.java | 94 +++
.../hive/sink/HiveSinkAggregatedCommitter.java | 88 +++
.../seatunnel/hive/sink/HiveSinkConfig.java | 106 ++++
.../seatunnel/hive/sink/HiveSinkState.java | 19 +-
.../seatunnel/hive/sink/HiveSinkWriter.java | 99 ++++
.../hive/sink/file/writer/AbstractFileWriter.java | 155 +++++
.../hive/sink/file/writer/FileWriter.java | 49 ++
.../hive/sink/file/writer/HdfsTxtFileWriter.java | 151 +++++
.../seatunnel/hive/sink/file/writer/HdfsUtils.java | 96 ++++
.../connector-http}/pom.xml | 23 +-
.../seatunnel/http/client/HttpClientProvider.java | 283 +++++++++
.../seatunnel/http/client/HttpResponse.java | 74 +++
.../connectors/seatunnel/http/config/Config.java | 18 +-
.../seatunnel/http/source/HttpSource.java | 109 ++++
.../seatunnel/http/source/HttpSourceParameter.java | 70 +++
.../seatunnel/http/source/HttpSourceReader.java | 79 +++
.../connector-jdbc}/pom.xml | 26 +-
.../jdbc/catalog/AbstractJdbcCatalog.java | 211 +++++++
.../seatunnel/jdbc/catalog/MySqlCatalog.java | 230 ++++++++
.../seatunnel/jdbc/config/JdbcConfig.java | 102 ++++
.../seatunnel/jdbc/config/JdbcSinkOptions.java | 35 +-
.../seatunnel/jdbc/config/JdbcSourceOptions.java | 77 +++
.../seatunnel/jdbc/internal/JdbcInputFormat.java | 220 +++++++
.../seatunnel/jdbc/internal/JdbcOutputFormat.java | 257 +++++++++
.../jdbc/internal/connection/DataSourceUtils.java | 104 ++++
.../connection/JdbcConnectionProvider.java | 70 +++
.../connection/SimpleJdbcConnectionProvider.java | 154 +++++
.../converter/AbstractJdbcRowConverter.java | 97 ++++
.../jdbc/internal/converter/JdbcRowConverter.java | 25 +-
.../jdbc/internal/dialect/JdbcDialect.java | 52 ++
.../jdbc/internal/dialect/JdbcDialectFactory.java | 25 +-
.../jdbc/internal/dialect/JdbcDialectLoader.java | 102 ++++
.../internal/dialect/JdbcDialectTypeMapper.java | 23 +-
.../dialect/mysql/MySqlDialectFactory.java | 29 +-
.../internal/dialect/mysql/MySqlTypeMapper.java | 170 ++++++
.../jdbc/internal/dialect/mysql/MysqlDialect.java | 30 +-
.../dialect/mysql/MysqlJdbcRowConverter.java | 31 +-
.../internal/dialect/psql/PostgresDialect.java | 30 +-
.../dialect/psql/PostgresDialectFactory.java | 29 +-
.../dialect/psql/PostgresJdbcRowConverter.java | 31 +-
.../internal/dialect/psql/PostgresTypeMapper.java | 159 +++++
.../internal/executor/BiConsumerWithException.java | 61 ++
.../executor/JdbcBatchStatementExecutor.java | 29 +-
.../internal/executor/JdbcStatementBuilder.java | 22 +-
.../executor/SimpleBatchStatementExecutor.java | 79 +++
.../jdbc/internal/options/JdbcConfig.java | 55 ++
.../internal/options/JdbcConnectionOptions.java | 200 +++++++
.../internal/options/JdbcConnectorOptions.java | 253 ++++++++
.../split/JdbcGenericParameterValuesProvider.java | 30 +-
.../JdbcNumericBetweenParametersProvider.java | 116 ++++
.../split/JdbcParameterValuesProvider.java | 32 +-
.../jdbc/internal/xa/GroupXaOperationResult.java | 80 +++
.../jdbc/internal/xa/SemanticXidGenerator.java | 113 ++++
.../seatunnel/jdbc/internal/xa/XaFacade.java | 113 ++++
.../jdbc/internal/xa/XaFacadeImplAutoLoad.java | 477 +++++++++++++++
.../seatunnel/jdbc/internal/xa/XaGroupOps.java | 43 ++
.../seatunnel/jdbc/internal/xa/XaGroupOpsImpl.java | 151 +++++
.../seatunnel/jdbc/internal/xa/XidGenerator.java | 62 ++
.../seatunnel/jdbc/internal/xa/XidImpl.java | 136 +++++
.../jdbc/sink/JdbcExactlyOnceSinkWriter.java | 190 ++++++
.../connectors/seatunnel/jdbc/sink/JdbcSink.java | 143 +++++
.../jdbc/sink/JdbcSinkAggregatedCommitter.java | 91 +++
.../seatunnel/jdbc/sink/JdbcSinkCommitter.java | 70 +++
.../seatunnel/jdbc/sink/JdbcSinkWriter.java | 96 ++++
.../seatunnel/jdbc/source/JdbcSource.java | 216 +++++++
.../seatunnel/jdbc/source/JdbcSourceReader.java | 97 ++++
.../seatunnel/jdbc/source/JdbcSourceSplit.java | 22 +-
.../jdbc/source/JdbcSourceSplitEnumerator.java | 100 ++++
.../seatunnel/jdbc/source/PartitionParameter.java | 20 +-
.../jdbc/state/JdbcAggregatedCommitInfo.java | 20 +-
.../seatunnel/jdbc/state/JdbcSinkState.java | 19 +-
.../seatunnel/jdbc/state/JdbcSourceState.java | 15 +-
.../connectors/seatunnel/jdbc/state/XidInfo.java | 23 +-
.../seatunnel/jdbc/utils/ExceptionUtils.java | 53 ++
.../connectors/seatunnel/jdbc/utils/JdbcUtils.java | 158 +++++
.../seatunnel/jdbc/utils/ThrowingRunnable.java | 52 ++
.../jdbc/catalog/AbstractJdbcCatalogTest.java | 48 ++
.../connector-kafka}/pom.xml | 25 +-
.../connectors/seatunnel/kafka/config/Config.java | 53 ++
.../seatunnel/kafka/config/KafkaSemantics.java | 27 +-
.../serialize/DefaultSeaTunnelRowSerializer.java | 50 ++
.../kafka/serialize/SeaTunnelRowSerializer.java | 22 +-
.../kafka/sink/KafkaInternalProducer.java | 164 ++++++
.../kafka/sink/KafkaNoTransactionSender.java | 83 +++
.../seatunnel/kafka/sink/KafkaProduceSender.java | 64 +++
.../connectors/seatunnel/kafka/sink/KafkaSink.java | 95 +++
.../seatunnel/kafka/sink/KafkaSinkCommitter.java | 85 +++
.../seatunnel/kafka/sink/KafkaSinkWriter.java | 161 ++++++
.../kafka/sink/KafkaTransactionSender.java | 126 ++++
.../seatunnel/kafka/source/ConsumerMetadata.java | 82 +++
.../seatunnel/kafka/source/KafkaSource.java | 132 +++++
.../seatunnel/kafka/source/KafkaSourceReader.java | 171 ++++++
.../seatunnel/kafka/source/KafkaSourceSplit.java | 72 +++
.../kafka/source/KafkaSourceSplitEnumerator.java | 167 ++++++
.../kafka/state/KafkaAggregatedCommitInfo.java | 19 +-
.../seatunnel/kafka/state/KafkaCommitInfo.java | 23 +-
.../seatunnel/kafka/state/KafkaSinkState.java | 23 +-
.../seatunnel/kafka/state/KafkaSourceState.java | 35 +-
seatunnel-connectors-v2/connector-pulsar/pom.xml | 99 ++++
.../seatunnel/pulsar/config/BasePulsarConfig.java | 27 +-
.../seatunnel/pulsar/config/PulsarAdminConfig.java | 76 +++
.../pulsar/config/PulsarClientConfig.java | 80 +++
.../seatunnel/pulsar/config/PulsarConfigUtil.java | 81 +++
.../pulsar/config/PulsarConsumerConfig.java | 60 ++
.../seatunnel/pulsar/config/SourceProperties.java | 119 ++++
.../seatunnel/pulsar/source/PulsarSource.java | 281 +++++++++
.../source/enumerator/PulsarSplitEnumerator.java | 281 +++++++++
.../enumerator/PulsarSplitEnumeratorState.java | 36 ++
.../cursor/start/MessageIdStartCursor.java | 65 +++
.../enumerator/cursor/start/StartCursor.java | 78 +++
.../cursor/start/SubscriptionStartCursor.java | 63 ++
.../cursor/start/TimestampStartCursor.java | 40 ++
.../cursor/stop/LatestMessageStopCursor.java | 59 ++
.../cursor/stop/MessageIdStopCursor.java | 58 ++
.../enumerator/cursor/stop/NeverStopCursor.java | 49 ++
.../source/enumerator/cursor/stop/StopCursor.java | 63 ++
.../cursor/stop/TimestampStopCursor.java | 44 ++
.../enumerator/discoverer/PulsarDiscoverer.java | 48 ++
.../enumerator/discoverer/TopicListDiscoverer.java | 63 ++
.../discoverer/TopicPatternDiscoverer.java | 98 ++++
.../source/enumerator/topic/TopicPartition.java | 91 +++
.../seatunnel/pulsar/source/reader/Handover.java | 88 +++
.../pulsar/source/reader/PulsarSourceReader.java | 221 +++++++
.../source/reader/PulsarSplitReaderThread.java | 143 +++++
.../pulsar/source/reader/RecordWithSplitId.java | 22 +-
.../pulsar/source/split/PulsarPartitionSplit.java | 94 +++
.../connector-socket}/pom.xml | 21 +-
.../seatunnel/socket/source/SocketSource.java | 72 +++
.../socket/source/SocketSourceParameter.java | 32 +-
.../socket/source/SocketSourceReader.java | 91 +++
.../seatunnel/socket/state/SocketState.java | 15 +-
.../pom.xml | 40 +-
.../seatunnel-connector-flink-clickhouse/pom.xml | 5 -
.../flink/clickhouse/sink/ClickhouseBatchSink.java | 3 -
.../clickhouse/sink/ClickhouseOutputFormat.java | 2 +-
.../org/apache/seatunnel/flink/fake/Config.java | 2 +-
.../flink/fake/source/FakeSourceStream.java | 2 +-
seatunnel-core/README.md | 8 +
seatunnel-core/pom.xml | 3 +
seatunnel-core/seatunnel-core-base/pom.xml | 6 +
.../org/apache/seatunnel/core/base/Seatunnel.java | 3 +-
.../core/base/command/AbstractCommandArgs.java | 1 +
.../core/base/command/BaseTaskExecuteCommand.java | 5 +-
.../seatunnel/core/base/command/Command.java | 3 +-
...nContext.java => AbstractExecutionContext.java} | 43 +-
.../seatunnel/core/base/config/ConfigBuilder.java | 21 +-
.../seatunnel/core/base/config/ConfigChecker.java | 36 +-
.../core/base/config/EnvironmentFactory.java | 2 +
.../core/base/config/ExecutionFactory.java | 4 +-
.../seatunnel/core/base/config/PluginFactory.java | 237 --------
.../CommandException.java} | 16 +-
.../CommandExecuteException.java} | 16 +-
.../ConfigCheckException.java} | 16 +-
.../core/base/config/PluginFactoryTest.java | 65 ---
.../seatunnel/core/base/utils/FileUtilsTest.java | 1 +
.../apache/seatunnel/core/sql/FlinkSqlStarter.java | 6 +-
.../apache/seatunnel/core/sql/job/Executor.java | 16 +-
seatunnel-core/seatunnel-core-flink/pom.xml | 24 +-
.../apache/seatunnel/core/flink/FlinkStarter.java | 4 +-
.../seatunnel/core/flink/SeatunnelFlink.java | 3 +-
.../core/flink/args/FlinkCommandArgs.java | 4 +-
...mmand.java => FlinkApiConfValidateCommand.java} | 14 +-
...ommand.java => FlinkApiTaskExecuteCommand.java} | 37 +-
.../core/flink/command/FlinkCommandBuilder.java | 9 +-
.../core/flink/config/FlinkApiConfigChecker.java | 44 ++
.../core/flink/config/FlinkExecutionContext.java | 104 ++++
.../core/flink/utils/CommandLineUtils.java | 17 +-
.../flink/command/FlinkTaskExecuteCommandTest.java | 2 +-
.../core/flink/utils/CommandLineUtilsTest.java | 17 +-
seatunnel-core/seatunnel-core-spark/pom.xml | 26 +
.../seatunnel/core/spark/SeatunnelSpark.java | 3 +-
.../apache/seatunnel/core/spark/SparkStarter.java | 36 +-
.../core/spark/command/SparkCommandBuilder.java | 5 +-
.../spark/command/SparkConfValidateCommand.java | 8 +-
.../spark/command/SparkTaskExecuteCommand.java | 59 +-
.../core/spark/config/SparkApiConfigChecker.java | 44 ++
.../core/spark/config/SparkExecutionContext.java | 103 ++++
seatunnel-core/seatunnel-core-starter/README.md | 12 +
.../pom.xml | 7 +-
.../apache/seatunnel/core/starter}/Seatunnel.java | 9 +-
.../apache/seatunnel/core/starter/Starter.java} | 16 +-
.../core/starter}/command/AbstractCommandArgs.java | 6 +-
.../seatunnel/core/starter}/command/Command.java | 8 +-
.../core/starter/command/CommandArgs.java} | 18 +-
.../core/starter/command/CommandBuilder.java} | 16 +-
.../core/starter/command/DeployModeConverter.java} | 20 +-
.../starter/config/AbstractExecutionContext.java} | 45 +-
.../core/starter}/config/ConfigBuilder.java | 22 +-
.../core/starter/config/ConfigChecker.java | 36 +-
.../seatunnel/core/starter/config/EngineType.java} | 18 +-
.../core/starter}/config/EnvironmentFactory.java | 3 +-
.../core/starter}/config/ExecutionFactory.java | 6 +-
.../seatunnel/core/starter}/config/PluginType.java | 2 +-
.../starter/constants/CommonParamConstants.java} | 16 +-
.../core/starter/exception/CommandException.java} | 16 +-
.../exception/CommandExecuteException.java} | 16 +-
.../starter/exception/ConfigCheckException.java} | 16 +-
.../starter/exception/TaskExecuteException.java} | 15 +-
.../core/starter/execution/TaskExecution.java} | 15 +-
.../core/starter/utils/AsciiArtUtils.java | 72 +++
.../core/starter/utils/CompressionUtils.java | 158 +++++
.../seatunnel/core/starter/utils/FileUtils.java | 70 +++
.../core/starter/utils/CompressionUtilsTest.java | 50 ++
.../core/starter}/utils/FileUtilsTest.java | 4 +-
.../src/test/resources/flink.batch.conf | 5 +-
.../pom.xml | 64 ++-
.../bin/start-seatunnel-flink-new-connector.sh | 46 ++
.../src/main/docker/Dockerfile | 37 ++
.../core/starter}/flink/FlinkStarter.java | 16 +-
.../core/starter}/flink/SeatunnelFlink.java | 17 +-
.../core/starter}/flink/args/FlinkCommandArgs.java | 9 +-
.../command/FlinkApiConfValidateCommand.java} | 27 +-
.../flink/command/FlinkApiTaskExecuteCommand.java | 61 ++
.../flink/command/FlinkCommandBuilder.java | 18 +-
.../flink/config/FlinkApiConfigChecker.java} | 17 +-
.../starter/flink/config/FlinkApiEnvironment.java | 75 +++
.../core/starter/flink/config/FlinkJobType.java} | 12 +-
.../core/starter/flink/config/FlinkRunMode.java} | 22 +-
.../starter/flink/constant/FlinkConstant.java} | 16 +-
.../core/starter/flink/env}/FlinkEnvironment.java | 63 +-
.../execution/AbstractPluginExecuteProcessor.java | 76 +++
.../starter/flink/execution/FlinkExecution.java | 73 +++
.../flink/execution/PluginExecuteProcessor.java} | 25 +-
.../flink/execution/SinkExecuteProcessor.java | 84 +++
.../flink/execution/SourceExecuteProcessor.java | 121 ++++
.../flink/execution/TransformExecuteProcessor.java | 86 +++
.../starter}/flink/utils/CommandLineUtils.java | 28 +-
.../core/starter/flink/FlinkStarterTest.java | 65 +++
.../starter/flink/args/FlinkCommandArgsTest.java | 30 +-
.../pom.xml | 58 +-
.../bin/start-seatunnel-spark-new-connector.sh | 45 ++
.../src/main/docker/Dockerfile | 37 ++
.../core/starter}/spark/SeatunnelSpark.java | 15 +-
.../core/starter}/spark/SparkStarter.java | 68 ++-
.../core/starter/spark/args/SparkCommandArgs.java | 62 ++
.../command/SparkApiConfValidateCommand.java} | 29 +-
.../spark/command/SparkApiTaskExecuteCommand.java | 60 ++
.../spark/command/SparkCommandBuilder.java | 26 +-
.../spark/config/SparkApiConfigChecker.java} | 17 +-
.../starter/spark/config/SparkEnvironment.java | 45 +-
.../execution/AbstractPluginExecuteProcessor.java | 68 +++
.../spark/execution/PluginExecuteProcessor.java} | 25 +-
.../spark/execution/SinkExecuteProcessor.java | 84 +++
.../spark/execution/SourceExecuteProcessor.java | 81 +++
.../starter/spark/execution/SparkExecution.java | 63 ++
.../spark/execution/TransformExecuteProcessor.java | 80 +++
.../core/starter/spark/utils/CommandLineUtils.java | 29 +-
.../core/starter/spark/SparkStarterTest.java | 25 +-
.../starter/spark/args/SparkCommandArgsTest.java | 67 +++
.../starter/spark/utils/CommandLineUtilsTest.java | 27 +-
.../src/test/resources/spark_application.conf | 66 +++
seatunnel-dist/release-docs/LICENSE | 52 +-
seatunnel-dist/release-docs/NOTICE | 20 +
.../release-docs/licenses/LICENSE-bouncycastle.txt | 7 +
.../release-docs/licenses/LICENSE-jaxrs-api.txt | 637 +++++++++++++++++++++
seatunnel-dist/src/main/assembly/assembly-bin.xml | 48 +-
seatunnel-e2e/pom.xml | 4 +-
.../apache/seatunnel/e2e/flink/FlinkContainer.java | 2 +-
.../seatunnel-flink-new-connector-e2e}/pom.xml | 27 +-
.../apache/seatunnel/e2e/flink/FlinkContainer.java | 44 +-
.../e2e/flink/assertion/FakeSourceToAssertIT.java | 24 +-
.../e2e/flink/fake/FakeSourceToConsoleIT.java | 25 +-
.../resources/assertion/fakesource_to_assert.conf | 88 +++
.../test/resources/fake/fakesource_to_console.conf | 35 +-
.../src/test/resources/log4j.properties | 23 +
.../apache/seatunnel/e2e/spark/SparkContainer.java | 2 +-
.../seatunnel-spark-new-connector-e2e}/pom.xml | 28 +-
.../apache/seatunnel/e2e/spark/SparkContainer.java | 21 +-
.../e2e/spark/fake/FakeSourceToConsoleIT.java | 29 +-
.../test/resources/fake/fakesource_to_console.conf | 34 +-
.../src/test/resources/log4j.properties | 22 +
seatunnel-examples/pom.xml | 2 +
.../seatunnel-flink-examples/pom.xml | 6 +
.../seatunnel/example/flink/LocalFlinkExample.java | 3 +-
.../pom.xml | 22 +-
.../example/flink/SeaTunnelApiExample.java} | 15 +-
.../main/resources/examples/fake_to_console.conf | 35 +-
.../src/main/resources/log4j.properties | 22 +
.../seatunnel-flink-sql-examples/pom.xml | 2 +-
.../seatunnel/example/spark/LocalSparkExample.java | 3 +-
.../seatunnel-spark-new-connector-example}/pom.xml | 54 +-
.../example/spark/SeaTunnelApiExample.java} | 33 +-
.../src/main/resources/examples/spark.batch.conf | 33 +-
{seatunnel-e2e => seatunnel-formats}/pom.xml | 8 +-
.../seatunnel-format-json}/pom.xml | 28 +-
.../format/json/JsonDeserializationSchema.java | 122 ++++
.../seatunnel/format/json/JsonToRowConverters.java | 281 +++++++++
.../pom.xml | 28 +-
.../plugin/discovery/AbstractPluginDiscovery.java | 181 ++++++
.../plugin/discovery/PluginDiscovery.java | 70 +++
.../plugin/discovery/PluginIdentifier.java | 89 +++
.../discovery/flink/FlinkSinkPluginDiscovery.java | 17 +-
.../flink/FlinkSourcePluginDiscovery.java | 18 +-
.../flink/FlinkTransformPluginDiscovery.java | 40 +-
.../SeaTunnelFlinkTransformPluginDiscovery.java | 29 +-
.../seatunnel/SeaTunnelSinkPluginDiscovery.java | 17 +-
.../seatunnel/SeaTunnelSourcePluginDiscovery.java | 18 +-
.../SeaTunnelSparkTransformPluginDiscovery.java | 18 +-
.../discovery/spark/SparkSinkPluginDiscovery.java | 17 +-
.../spark/SparkSourcePluginDiscovery.java | 17 +-
.../spark/SparkTransformPluginDiscovery.java | 39 +-
{seatunnel-e2e => seatunnel-translation}/pom.xml | 8 +-
.../seatunnel-translation-base}/pom.xml | 18 +-
.../translation/serialization/RowConverter.java | 128 +++++
.../serialization/SerializerConverter.java | 20 +-
.../sink/SinkAggregatedCommitterConverter.java | 19 +-
.../translation/sink/SinkCommitterConverter.java | 20 +-
.../seatunnel/translation/sink/SinkConverter.java | 22 +-
.../translation/sink/SinkWriterConverter.java | 14 +-
.../translation/source/BaseSourceFunction.java | 24 +-
.../source/CoordinatedEnumeratorContext.java | 60 ++
.../source/CoordinatedReaderContext.java | 62 ++
.../translation/source/CoordinatedSource.java | 273 +++++++++
.../source/ParallelEnumeratorContext.java | 77 +++
.../translation/source/ParallelReaderContext.java | 63 ++
.../translation/source/ParallelSource.java | 201 +++++++
.../util/ThreadPoolExecutorFactory.java | 31 +-
.../seatunnel-translation-flink}/pom.xml | 45 +-
.../serialization/CommitWrapperSerializer.java | 66 +++
.../flink/serialization/FlinkRowConverter.java | 115 ++++
.../FlinkSimpleVersionedSerializer.java | 33 +-
.../serialization/FlinkWriterStateSerializer.java | 68 +++
.../flink/serialization/KryoTypeInfo.java | 41 ++
.../flink/serialization/WrappedRow.java | 40 +-
.../translation/flink/sink/CommitWrapper.java | 16 +-
.../translation/flink/sink/FlinkCommitter.java | 56 ++
.../flink/sink/FlinkGlobalCommitter.java | 72 +++
.../translation/flink/sink/FlinkSink.java | 90 +++
.../translation/flink/sink/FlinkSinkWriter.java | 75 +++
.../translation/flink/sink/FlinkWriterState.java | 33 +-
.../flink/source/BaseSeaTunnelSourceFunction.java | 136 +++++
.../translation/flink/source/RowCollector.java | 55 ++
.../flink/source/SeaTunnelCoordinatedSource.java | 43 ++
.../flink/source/SeaTunnelParallelSource.java | 49 ++
.../flink/utils/TypeConverterUtils.java | 146 +++++
.../flink/utils/TypeConverterUtilsTest.java | 129 +++++
.../seatunnel-translation-spark}/pom.xml | 44 +-
.../spark/serialization/InternalRowConverter.java | 178 ++++++
.../spark/sink/SparkDataSourceWriter.java | 95 +++
.../translation/spark/sink/SparkDataWriter.java | 101 ++++
.../spark/sink/SparkDataWriterFactory.java | 62 ++
.../translation/spark/sink/SparkSink.java | 76 +++
.../translation/spark/sink/SparkSinkInjector.java | 49 ++
.../translation/spark/sink/SparkStreamWriter.java | 73 +++
.../spark/sink/SparkWriterCommitMessage.java | 23 +-
.../translation/spark/source/Handover.java | 88 +++
.../spark/source/InternalRowCollector.java | 53 ++
.../translation/spark/source/ReaderState.java | 32 +-
.../spark/source/SeaTunnelSourceSupport.java | 92 +++
.../spark/source/batch/BatchPartition.java | 47 ++
.../spark/source/batch/BatchSourceReader.java | 62 ++
.../batch/CoordinatedBatchPartitionReader.java | 97 ++++
.../source/batch/ParallelBatchPartitionReader.java | 147 +++++
.../source/continnous/ContinuousPartition.java | 53 ++
.../source/continnous/ContinuousSourceReader.java | 116 ++++
.../spark/source/continnous/CoordinationState.java | 49 ++
.../ParallelContinuousPartitionReader.java | 78 +++
.../CoordinatedMicroBatchPartitionReader.java | 138 +++++
.../spark/source/micro/MicroBatchPartition.java | 64 +++
.../spark/source/micro/MicroBatchSourceReader.java | 111 ++++
.../spark/source/micro/MicroBatchState.java | 27 +-
.../micro/ParallelMicroBatchPartitionReader.java | 182 ++++++
.../spark/utils/TypeConverterUtils.java | 173 ++++++
...org.apache.spark.sql.sources.DataSourceRegister | 18 +
tools/dependencies/known-dependencies.txt | 30 +-
528 files changed, 28870 insertions(+), 3208 deletions(-)