[GitHub] [incubator-seatunnel] gaojun2048 commented on issue #1608: [Umbrella][Feature][Core] Decoupling connectors from compute engines

[GitBox <gi...@apache.org>]

gaojun2048 commented on issue #1608:
URL: https://github.com/apache/incubator-seatunnel/issues/1608#issuecomment-1123146072

   > > > We do translation work for different engines, so if faced with inconsistent semantics or inconsistent functions how to solve the situation? For example, Spark and Flink both have Checkpoint, but their mechanisms and functions are completely different. If we provide Checkpoint capability externally, how can we unify Checkpoint in the two engines?
   > > 
   > > 
   > > @lonelyGhostisdog Because the checkpoint mechanism is not completely consistent, we are still discussing how to adapt spark's micro-batch and continuous reader.
   > 
   > Why don't we design a new engine that doesn't dependency on Spark or Flink
   
   The new engine is being designed. However, because a large number of SeaTunnel users are using Flink and spark, we hope to be compatible with Flink and spark engines as much as possible. In the future, if our own engine is good enough, we will discuss again whether to continue to rely on Flink and spark.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org