[GitHub] [zookeeper] vladimirivic opened a new pull request #1172: ZOOKEEPER-3594: Ability to skip proposing requests with error transactions

[GitBox <gi...@apache.org>]

vladimirivic opened a new pull request #1172: ZOOKEEPER-3594: Ability to skip proposing requests with error transactions
URL: https://github.com/apache/zookeeper/pull/1172
 
 
   Ensembles that have a high write request rate could be skipping proposing requests that contain errors instead of having the quorum vote on those transactions.
   
   For example, having a sizable write traffic that results in error transactions would be creating additional network and log disk space overhead for each of the requests that would only return errors to the client in the end (e.g. delete non-existing path, version mismatch, trying to create a node that already exists etc). 
   
   Currently, there is no such logic in the ProposalRequestProcessor, every request that comes out of PrepRequestProcessor will be proposed to the quorum.  
   
   Proposed solution workflow:
   
   A client sends a new write request trying to setData on an non-existing node
   The server accepts the request and sends it through the PrepRequestProcessor
   PrepRequestProcessor detects the error and assigns the error transaction to the request
   Between PrepRequestProcessor and ProposalRequestProcessor there is another processor named SkipRequestProcessor which sole responsibility is to decide will the request be forwarded or returned to the originating quorum peer (Leader or Learner).
   The quorum peer waits for all previous requests to complete before the error request proceeds with echoing the error back to the client.
   Requirements: 
   
   We should be conservative about the use of ZXID. If the request generates an error transaction we don't want to increment the last proposed ZXID and cause any gaps in the log.
   Request that are found to be invalid should be sent directly to the origin (either to the corresponding Learner or to the Leader directly) so there is exactly one roundtrip for each request with error transaction.
   All the requests must preserve its order, the changes must be backwards compatible with the Zookeeper ordering guarantees. 
    Challenges:
   
   Skipping request without having them go through the proposal pipeline poses a challenge in preserving Zookeeper transaction order.
   Avoiding any additional changes inside CommitProcessor if possible.
   Have unified logic for all three different paths in which write requests can come through:
   Via Leader placed directly into the PrepRequestProcessor
   Via Follower where the request is forwarded to the leader and also passed to CommitProcessor to wait for COMMIT packet
   Via Observer, where the request is forwarded to the Leader and the Observer waits for the INFORM packet
   
   https://issues.apache.org/jira/browse/ZOOKEEPER-3594 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services