You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by lyy4j <gi...@git.apache.org> on 2017/07/27 07:01:22 UTC
[GitHub] incubator-rocketmq pull request #137: Develop
GitHub user lyy4j opened a pull request:
https://github.com/apache/incubator-rocketmq/pull/137
Develop
producer client of ConsistentHash selector:
here the code:
public class SelectMessageQueueByConsistentHash implements MessageQueueSelector {
private volatile SortedMap<Integer, String> virtualNodes =
new TreeMap<Integer, String>();
private static final int VIRTUAL_NODES = 100;
private volatile HashMap<String, MessageQueue> idToQueueMap = new HashMap<String, MessageQueue>();
private Object idToQueueMapMonitor = new Object();
@Override
public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) {
if (queueChange(mqs)) {
synchronized (this.idToQueueMapMonitor) {
if (queueChange(mqs)) {
reloadConsistentHash(mqs);
}
}
}
String uniqueQueueId = getMsgQueue(arg.toString());
MessageQueue messageQueue = idToQueueMap.get(uniqueQueueId);
return messageQueue;
}
private boolean queueChange(List<MessageQueue> mqs) {
if (mqs.size() != this.idToQueueMap.size()) {
return true;
}
for (MessageQueue queue : mqs) {
String id = queue.getTopic() + "_" + queue.getBrokerName() + "_" + queue.getQueueId();
if (!this.idToQueueMap.containsKey(id)) {
return true;
}
}
return false;
}
private String getMsgQueue(String node) {
int hash = getHash(node);
SortedMap<Integer, String> subMap =
virtualNodes.tailMap(hash);
Integer i;
String virtualNode;
if (subMap.size() == 0) {
i = virtualNodes.firstKey();
virtualNode = virtualNodes.get(i);
} else {
i = subMap.firstKey();
virtualNode = subMap.get(i);
}
String result = virtualNode.substring(0, virtualNode.indexOf("&&"));
return result;
}
private int getHash(String str) {
final int p = 16777619;
int hash = (int) 2166136261L;
for (int i = 0; i < str.length(); i++)
hash = (hash ^ str.charAt(i)) * p;
hash += hash << 13;
hash ^= hash >> 7;
hash += hash << 3;
hash ^= hash >> 17;
hash += hash << 5;
if (hash < 0)
hash = Math.abs(hash);
return hash;
}
private void reloadConsistentHash(List<MessageQueue> mqs) {
idToQueueMap.clear();
for (MessageQueue messageQueue : mqs) {
String id = messageQueue.getTopic() + "_" + messageQueue.getBrokerName() + "_" + messageQueue.getQueueId();
idToQueueMap.put(id, messageQueue);
}
virtualNodes.clear();
for (String id : idToQueueMap.keySet()) {
for (int i = 0; i < VIRTUAL_NODES; i++) {
String virtualNodeName = id + "&&VN" + String.valueOf(i);
int hash = getHash(virtualNodeName);
virtualNodes.put(hash, virtualNodeName);
}
}
}
}
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-rocketmq develop
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-rocketmq/pull/137.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #137
----
commit d545f86f66e51bf87f837d707feeb7cabc76de8d
Author: yilingfeng <yi...@aliyun.com>
Date: 2017-02-28T11:22:00Z
[ROCKETMQ-111] Fix possible MQClientException when query message before today, closes apache/incubator-rocketmq#69
commit 3940924561810f889888ba6fb898886dfb125696
Author: shroman <rs...@yahoo.com>
Date: 2017-02-28T11:33:15Z
[ROCKETMQ-104] Make MQAdmin commands throw exceptions, closes apache/incubator-rocketmq#65
commit b692a73ebb2ca2e37a9da08cc2abc3005b6ce8b9
Author: lizhanhui <li...@gmail.com>
Date: 2017-02-28T11:38:47Z
[ROCKETMQ-99] Add scripts for Windows, closes apache/incubator-rocketmq#62
commit 42a393188e32a3b7d5704e6f6f9b146f5459ab96
Author: Zhanhui Li <li...@apache.org>
Date: 2017-03-03T08:51:09Z
Fix Windows script to handle return code properly
commit 53b98d0d8c1016c5b29c7f14293cb740c843d296
Author: Zhanhui Li <li...@apache.org>
Date: 2017-03-06T06:17:29Z
Trivial Changes: Specify maven-assembly-plugin version to 2.8 for release-client profile similar to release-all profile, reason of performing this change is that latest version of maven-assembly-plugin has removed finalName property.
commit e3f4251c91a73f4e51732bcb1690554ac5fb3096
Author: yukon <yu...@apache.org>
Date: 2017-03-06T12:01:10Z
[ROCKETMQ-119] Add ThreadUtils and shutdown PullMessageService properly
commit c5d9fcb548ac80ab32cbb8dccd121de874c2e6d7
Author: shtykh_roman <rs...@yahoo.com>
Date: 2017-03-09T09:07:54Z
Corrected spellings.
commit a146646b27af75540b7691e6dd9b1227d6aaf59b
Author: shtykh_roman <rs...@yahoo.com>
Date: 2017-03-10T06:50:04Z
[ROCKETMQ-75] Logging when RemotingCommand header decoding swallows exceptions. closes #51
commit d7decc84abc32dab63ee423d4d904f28d18cb1d7
Author: yukon <yu...@apache.org>
Date: 2017-03-10T08:41:58Z
[ROCKETMQ-139] Degrade the client related modules' JDK version to 1.6
commit e05a445b213a52d9b45196bf3e56716b45e9853a
Author: shtykh_roman <rs...@yahoo.com>
Date: 2017-03-14T05:23:25Z
Logging on exception when producing a response.
commit 6b7d206f09b928ea58fb3a62413a053f008b8c1c
Author: yukon <yu...@apache.org>
Date: 2017-03-15T03:23:56Z
[ROCKETMQ-143][HOTFIX] Update fastjson from 1.2.12 to 1.2.28
commit 087d989fccccd0c5e6f1d8c89741171592e62c47
Author: shtykh_roman <rs...@yahoo.com>
Date: 2017-03-17T06:09:13Z
Corrected string check for emptiness.
commit 11653ce24c72189f4e6121c4babe709b33bc0230
Author: dongeforever <do...@apache.org>
Date: 2017-03-17T10:59:43Z
ROCKETMQ-80 Add batch feature closes apache/incubator-rocketmq#53
commit 0b39fcadfa2950f1dd3975e1262ccd544f350750
Author: yukon <yu...@apache.org>
Date: 2017-03-18T07:34:39Z
[ROCKETMQ-145][HOTFIX] Resolve concureent issue in HAService and GroupCommitService
commit 19f358cb1d0fec7dd4c97efc4221199d175d731d
Author: vongosling <vo...@apache.org>
Date: 2017-03-16T01:43:54Z
Update community projects illustration
commit 72e6def1441083c6c5a8c4b831eebefcbb618b56
Author: yukon <yu...@apache.org>
Date: 2017-03-19T14:57:24Z
[ROCKETMQ-148] Polish the contributing guide.
commit 3fe7535bc27fa21b2bc945b8bda273f29b31c131
Author: yukon <yu...@apache.org>
Date: 2017-03-23T10:15:51Z
[ROCKETMQ-143][HOTFIX] Update fastjson to 1.2.29
commit 15af63e23b8649537cce56a831c7f965451ce399
Author: yukon <yu...@apache.org>
Date: 2017-03-24T03:05:19Z
[ROCKETMQ-153][HOTFIX] Fetch name server address dynamically.
commit 01a0eb0088860b10ae0cff257e4f25a0c59cd44c
Author: Jaskey <li...@gmail.com>
Date: 2017-02-15T14:17:47Z
Fix possible NullPointerException when retry in send Async way
commit d933eeb5ac46eb8887dd49975d838aadba700fa1
Author: vesense <be...@163.com>
Date: 2017-03-24T05:38:55Z
fix typo in ClientConfig
commit 08e70ee21061d15d40f3d714d3fda329fc268224
Author: vesense <be...@163.com>
Date: 2017-03-28T01:09:47Z
[ROCKETMQ-154] Added a newline after help info. closes #83
Signed-off-by: shtykh_roman <rs...@yahoo.com>
commit 203cb30a906a77f41b0e5ba09fc351434862d408
Author: shtykh_roman <rs...@yahoo.com>
Date: 2017-03-28T01:22:53Z
Trivial simplification of broker initialization by removing unnecessary assignment.
commit 7e37799e822cda30b0607f859fc5574f468a49f3
Author: Zhanhui Li <li...@apache.org>
Date: 2017-03-28T12:34:30Z
Aggregate packaging specific files to a new sub-module: distribution
commit ab013861cd488f535d9cc03c21481922ba21ed9d
Author: shtykh_roman <rs...@yahoo.com>
Date: 2017-03-29T08:13:51Z
[ROCKETMQ-76] Expose IntegrationTestBase to be used by other integration projects closes apache/incubator-rocketmq#52
commit 45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9
Author: Zhanhui Li <li...@apache.org>
Date: 2017-03-29T13:50:59Z
Include client IP per message queue of consumer progress command output
commit a8fa05e8c0b0e57b6e2278747a5f6a8111447a8e
Author: Jaskey <li...@gmail.com>
Date: 2017-04-11T06:38:05Z
[ROCKETMQ-138] Remove hard coded Aliyun authentication code, closes apache/incubator-rocketmq#75
commit 40a233aa05424cfddf1c2922c40cffe5446cf5c6
Author: Zhanhui Li <li...@apache.org>
Date: 2017-04-12T13:46:45Z
Guard MQVesion methods.
commit 7c7374e38c0c1d6e8ee9d704717b2c94b7170a61
Author: Li Zhanhui <li...@apache.org>
Date: 2017-04-06T08:43:52Z
Fix issue 165
commit 7bcb3b3eae1e3c441861f2a3cd79ff54a8e691b9
Author: Zhanhui Li <li...@apache.org>
Date: 2017-04-12T14:25:11Z
BugFix: WS_DOMAIN_NAME, SUBGROUP default values override custom values passed by java options options
commit c183e0d4026770a68bedce02507446431cdf6265
Author: Jaskey <li...@gmail.com>
Date: 2017-04-17T11:28:26Z
[ROCKETMQ-172]log improvement for rocketmq client closes apache/incubator-rocketmq#90
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #137: Develop
Posted by coveralls <gi...@git.apache.org>.
Github user coveralls commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/137
[](https://coveralls.io/builds/12635031)
Coverage decreased (-0.03%) to 39.09% when pulling **9bb6eae4bd35697808174a2ff9195e393450ec91 on develop** into **d4149207e27ed3516f1f06407b55986790b806ae on master**.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #137: Develop
Posted by vsair <gi...@git.apache.org>.
Github user vsair commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/137
I think you may have a look at the contributor's guideline.
http://rocketmq.apache.org/docs/pull-request/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #137: Develop
Posted by coveralls <gi...@git.apache.org>.
Github user coveralls commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/137
[](https://coveralls.io/builds/12635031)
Coverage decreased (-0.03%) to 39.09% when pulling **9bb6eae4bd35697808174a2ff9195e393450ec91 on develop** into **d4149207e27ed3516f1f06407b55986790b806ae on master**.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq pull request #137: Develop
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-rocketmq/pull/137
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---