You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/07/26 08:57:07 UTC
[GitHub] [druid] rohangarg commented on pull request #12817: Fix hash calcuation in RendezvousHasher
rohangarg commented on PR #12817:
URL: https://github.com/apache/druid/pull/12817#issuecomment-1195202133
Thanks for the review @abhishekagarwal87. I have added a test for verifying distribution in 100 request keys amongst 5 brokers.
While testing, I found that as the number of request keys decrease the skewness increases in the new algorithm as well. I ran some tests with old and new rendezvous hashing algorithm along with consistent hashing. The test includes 8 brokers and computes the max skew for (100, 1000, 10_000) request keys. max skew is calculated as the difference in request between lightest and heaviest broker. Each request key count config is run 1000 times and the result is the maximum skew amongst all the runs.
Please find below the results :
| hashing algorithm | max skew with 100 request keys | max skew with 1000 request keys | max skew with 10_000 request keys|
|-------------------|------------------|-------------------|----------------------|
| rendezvous old | 20 (20%) | 67 (6.7%) | 193 (1.93%) |
| rendezvous new | 34 (34%) | 238 (23.8%) | 2090 (21%) |
| consistent hash | 22 (22%) | 72 (7.2%) | 395 (3.95%) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org