You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2020/02/24 20:58:09 UTC
[GitHub] [helix] NealSun96 opened a new issue #800: Make
ZkMetadataStoreDirectory singleton
NealSun96 opened a new issue #800: Make ZkMetadataStoreDirectory singleton
URL: https://github.com/apache/helix/issues/800
During the leader request forwarding implementation of `ZkRoutingDataWriter`, it is uncovered that with the current implementation, the request will be forwarded indefinitely and eventually it will cause an overflow. This is due to the per-request lifecycle of Jersey - when a request is sent to a follower host, in its attempt to forward the request to the leader host, the following would happen:
1. The request is forwarded from host B to the current leader host A.
2. Host A creates a new resource class object (`MetadataStoreDirectoryAccessor`) to serve the request.
3. `MetadataStoreDirectoryAccessor` creates a new `ZkMetadataStoreDirectory`, which in turn creates a new `ZkRoutingDataWriter`.
4. Since the `ZkRoutingDataWriter` on host B is blocked by this operation, the newly created `ZkRoutingDataWriter` on host A will never be a leader.
5. The request is forwarded to host A again. Back to step 1.
A couple of potential fixes were discussed, including making `MetadataStoreDirectoryAccessor` singleton, or modify the leader election behavior. Ultimately, we decided to make `ZkMetadataStoreDirectory`. `ZkMetadataStoreDirectory` serves all namespaces while `MetadataStoreDirectoryAccessor` only serves one namespace; it makes sense for all instances of `MetadataStoreDirectoryAccessor` to rely on one single instance of `ZkMetadataStoreDirectory`.
Converting it to singleton ensures that on one host, there will always only be one `ZkRoutingDataWriter` that gets called over and over, avoiding the infinite loop described above. After modification, the operations above will now become:
1. The request is forwarded from host B to the current leader host A.
2. Host A creates a new resource class object (`MetadataStoreDirectoryAccessor`) to serve the request.
3. `MetadataStoreDirectoryAccessor` **connects to the singleton** `ZkMetadataStoreDirectory`, which has a leader `ZkRoutingDataWriter`.
4. The leader `ZkRoutingDataWriter` resolves the request.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] narendly closed issue #800: Make ZkMetadataStoreDirectory
singleton
Posted by GitBox <gi...@apache.org>.
narendly closed issue #800: Make ZkMetadataStoreDirectory singleton
URL: https://github.com/apache/helix/issues/800
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org