[GitHub] [iotdb] chengjianyun commented on issue #3954: Integrate Apache Ratis to help manage Raft status

[GitBox <gi...@apache.org>]

chengjianyun commented on issue #3954:
URL: https://github.com/apache/iotdb/issues/3954#issuecomment-918015757


   Reply https://github.com/apache/iotdb/pull/3939#issuecomment-917983405, @LebronAl 
   
   > Production-level raft algorithms generally guarantee linearizability, which is the strictest consistency in distributed systems. Our current Raft algorithm may not be complete, so strictly speaking it does not guarantee linearizability, but at the same time it's performance may be better. What would we say if we migrated raft implementation and performance dropped? Furthermore, is linearizability really necessary for OLAP databases like time-series databases?
   
   What do you mean by "strictly speaking it does not guarantee linearizability"? In which case, IoTDB doesn't guarantee the linearizability? In my opinion, it doesn't matter IoTDB or other products, if you are using Raft, that means you need the linearizability which is what raft provided. If IoTDB doesn't guarantee the linearizability, I think it's a bug. Right now, I didn't find any of the case. 
   
   I agree that we can simply some aspects of Raft implementation but which won't affect performance much. The main performance improvement relays on the multi-raft support.
   
   > At the moment we have Raft implementation and business logic mixed together, but there is a certain amount of performance improvement. For example, we currently put data from multiple storage groups into one Raft group to be executed synchronously. In fact, at the bottom, they can be executed in parallel. So we implemented parallel asynchronous apply optimization so that plans from the same Raft group but different storage groups can be applied in parallel. The performance gains from this optimization are significant. How should we handle this case after we migrate raft implementation? Add parallel apply feature to raft library Or do we change our partitioning pattern so that a Raft group serves only one storage group? Or just let them execute inefficiently?
   
   > "For example, we currently put data from multiple storage groups into one Raft group to be executed synchronously."
   Could you explain this one more detail? 
   
   > Add parallel apply feature to raft library Or do we change our partitioning pattern so that a Raft group serves only one storage group?
   
   Application(Raft user) takes care of log apply, so it doesn't matter how Raft implemented.
   
   Finally, in my opinion, current performance main gains from multi-raft support. So it won't affect performance too much after integrate Ratis.
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org