You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/06/10 21:58:55 UTC
[GitHub] [pulsar-helm-chart] sdwerwed opened a new pull request #127: Antiaffinity with zone topology for high availability
sdwerwed opened a new pull request #127:
URL: https://github.com/apache/pulsar-helm-chart/pull/127
### Motivation
There is no option in the current helm chart to make the pulsar high available. With this pull request, we can make some of the pulsar components highly available (multiAZ support). This is usually a requirement by many companies and I believe it will help many people that are facing this limitation. This version is backwards compatible, the default values are the same. It is tested and works as expected, pods can be scheduled in different availability zones if `<component>.affinity.anti_affinity_topology.zone.enabled: true`
### Modifications
I have added an extra option in the anti-affinity section for multiAZ support that is disabled by default. Made the weight from static value to variable (default is same), so users can adjust the weight in case they want to use both `host` and `zone` anti-affinity
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-helm-chart] michaeljmarshall commented on pull request #127: Antiaffinity with zone topology for high availability
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on pull request #127:
URL: https://github.com/apache/pulsar-helm-chart/pull/127#issuecomment-1017883950
> @sdwerwed: I think the zone affinity would work for stateless components. I don't think it will work well for stateful components. Because the location of bookie and zookeeper pods will be bound to the location of EBS volumes.
To add some AWS context, EBS volumes are restricted to a single availability zone within a region. Once a bookie gets an EBS volume, it cannot be migrated to another AZ. I haven't looked closely at this PR, but we'll need to make sure that it works for upgrading existing clusters that do not comply with the anti-affinity policy.
> Also, even you are able to place bookies across different zones. But you don't configure the rack information, so bookkeeper can potentially place two same copies in the pods running in the same zone.
I agree that this could be misleading. We could mitigate the shortcoming with documentation, though. Also, this feature can work without configuring rack awareness. For example, if your ack quorum is greater than the number of bookies in a single AZ, you'll know that your writes are spread across multiple AZs and are therefore HA. This isn't as good as configuring rack awareness, but it is probably better than random bookie scheduling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-helm-chart] sdwerwed commented on pull request #127: Antiaffinity with zone topology for high availability
Posted by GitBox <gi...@apache.org>.
sdwerwed commented on pull request #127:
URL: https://github.com/apache/pulsar-helm-chart/pull/127#issuecomment-1074108404
@sijie
> I think the zone affinity would work for stateless components. I don't think it will work well for stateful components. Because the location of bookie and zookeeper pods will be bound to the location of EBS volumes.
This is a known limitation of AWS and probably Azure too, however, the default value is not multi az, if an enginner would like to make it multi az can do it manually with additional steps, not sure if we can handle that in the helm chart anyway, we could document that or if you have another idea how we could make it fully functional with helm let me know.
> But you don't configure the rack information, so bookkeeper can potentially place two same copies in the pods running in the same zone.
I think I miss some context here as I did not get what is the issue, what do you mean to configure rack information? Could you give me some sources to read about it so I can suggest a solution?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-helm-chart] sdwerwed edited a comment on pull request #127: Antiaffinity with zone topology for high availability
Posted by GitBox <gi...@apache.org>.
sdwerwed edited a comment on pull request #127:
URL: https://github.com/apache/pulsar-helm-chart/pull/127#issuecomment-859123386
Hello @sijie, can you have a look at this PR, let me know if any other action is required from my side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-helm-chart] sijie commented on pull request #127: Antiaffinity with zone topology for high availability
Posted by GitBox <gi...@apache.org>.
sijie commented on pull request #127:
URL: https://github.com/apache/pulsar-helm-chart/pull/127#issuecomment-1017163937
@sdwerwed: I think the zone affinity would work for stateless components. I don't think it will work well for stateful components. Because the location of bookie and zookeeper pods will be bound to the location of EBS volumes. Also, even you are able to place bookies across different zones. But you don't configure the rack information, so bookkeeper can potentially place two same copies in the pods running in the same zone.
Hence I believe the solution is incomplete. If we merge this, it will cause confusion to people and let them believe it is a "complete" solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-helm-chart] sdwerwed commented on pull request #127: Antiaffinity with zone topology for high availability
Posted by GitBox <gi...@apache.org>.
sdwerwed commented on pull request #127:
URL: https://github.com/apache/pulsar-helm-chart/pull/127#issuecomment-999727380
Hello @wolfstudy, @sijie, can you have a look at this PR, let me know if any other action is required from my side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-helm-chart] sdwerwed edited a comment on pull request #127: Antiaffinity with zone topology for high availability
Posted by GitBox <gi...@apache.org>.
sdwerwed edited a comment on pull request #127:
URL: https://github.com/apache/pulsar-helm-chart/pull/127#issuecomment-1074108404
@sijie
> I think the zone affinity would work for stateless components. I don't think it will work well for stateful components. Because the location of bookie and zookeeper pods will be bound to the location of EBS volumes.
This is a known limitation of AWS and probably Azure too, however, the default value is not multi az, if an enginner would like to make it multi az can do it manually with additional steps, not sure if we can handle that in the helm chart anyway, we could document that or if you have another idea how we could make it fully functional with helm let me know.
> But you don't configure the rack information, so bookkeeper can potentially place two same copies in the pods running in the same zone.
I think I miss some context here as I did not get what is the issue, what do you mean to configure rack information? Could you give me some sources to read about it so I can suggest a solution? Do you mean that there is the possibility that data will not be 100% highly available?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-helm-chart] sdwerwed commented on pull request #127: Antiaffinity with zone topology for high availability
Posted by GitBox <gi...@apache.org>.
sdwerwed commented on pull request #127:
URL: https://github.com/apache/pulsar-helm-chart/pull/127#issuecomment-859123386
As I am a first-time contributor, can a maintainer run the pipelines, please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-helm-chart] sdwerwed edited a comment on pull request #127: Antiaffinity with zone topology for high availability
Posted by GitBox <gi...@apache.org>.
sdwerwed edited a comment on pull request #127:
URL: https://github.com/apache/pulsar-helm-chart/pull/127#issuecomment-1074108404
@sijie
> I think the zone affinity would work for stateless components. I don't think it will work well for stateful components. Because the location of bookie and zookeeper pods will be bound to the location of EBS volumes.
This is a known limitation of AWS and probably Azure too, however, the default value is not multi az, if an enginner would like to make it multi az can do it manually with additional steps, not sure if we can handle that in the helm chart anyway, we could document that or if you have another idea how we could make it fully functional with helm let me know.
> But you don't configure the rack information, so bookkeeper can potentially place two same copies in the pods running in the same zone.
I think I miss some context here as I did not get what is the issue, what do you mean to configure rack information?
Do you mean that there is the possibility that data will not be 100% highly available?
Feel free to add some more details...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-helm-chart] sdwerwed removed a comment on pull request #127: Antiaffinity with zone topology for high availability
Posted by GitBox <gi...@apache.org>.
sdwerwed removed a comment on pull request #127:
URL: https://github.com/apache/pulsar-helm-chart/pull/127#issuecomment-859123386
Hello @wolfstudy, @sijie, can you have a look at this PR, let me know if any other action is required from my side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-helm-chart] sdwerwed edited a comment on pull request #127: Antiaffinity with zone topology for high availability
Posted by GitBox <gi...@apache.org>.
sdwerwed edited a comment on pull request #127:
URL: https://github.com/apache/pulsar-helm-chart/pull/127#issuecomment-859123386
Hello @wolfstudy, @sijie, can you have a look at this PR, let me know if any other action is required from my side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org