You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/08/18 13:30:24 UTC
[GitHub] [pulsar] zhanghaou opened a new pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
zhanghaou opened a new pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839
Link [https://github.com/apache/pulsar/issues/7758](https://github.com/apache/pulsar/issues/7758) and master issue [https://github.com/apache/pulsar/issues/2688](https://github.com/apache/pulsar/issues/2688)
### Motivation
Support set/get/remove maxProducers on a topic level.
### Verifying this change
new unit test added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on a change in pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
jiazhai commented on a change in pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#discussion_r474724409
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
##########
@@ -2352,6 +2352,50 @@ protected void internalGetPersistence(AsyncResponse asyncResponse){
return pulsar().getTopicPoliciesService().updateTopicPoliciesAsync(topicName, topicPolicies.get());
}
+ protected void internalGetMaxProducers(AsyncResponse asyncResponse){
Review comment:
How about make the method name more clear and align with namespace policies?
e.g. add "PerTopic" at the end: "xxxMaxProducersPerTopic"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on a change in pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on a change in pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#discussion_r475030514
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
##########
@@ -2352,6 +2352,50 @@ protected void internalGetPersistence(AsyncResponse asyncResponse){
return pulsar().getTopicPoliciesService().updateTopicPoliciesAsync(topicName, topicPolicies.get());
}
+ protected void internalGetMaxProducers(AsyncResponse asyncResponse){
Review comment:
Now it is the policies for a single topic, namespaces policies can take control of all the topics of the namespace, so 'perTopic' is suitable, but at the topic level, just set to only one topic, so I remove the 'perTopic' in all methods.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#issuecomment-675774838
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on a change in pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on a change in pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#discussion_r475030592
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/TopicPoliciesTest.java
##########
@@ -337,4 +338,80 @@ public void testRemovePersistence() throws Exception {
admin.topics().deletePartitionedTopic(testTopic, true);
}
+
+ @Test
+ public void testCheckMaxProducers() throws Exception {
+ Integer maxProducers = new Integer(-1);
+ log.info("MaxProducers: {} will set to the topic: {}", maxProducers, testTopic);
+ try {
+ admin.topics().setMaxProducers(testTopic, maxProducers);
+ Assert.fail();
+ } catch (PulsarAdminException e) {
+ Assert.assertEquals(e.getStatusCode(), 412);
+ }
+
+ admin.topics().deletePartitionedTopic(testTopic, true);
+ }
+
+ @Test
+ public void testSetMaxProducers() throws Exception {
+ Integer maxProducers = 2;
+ log.info("MaxProducers: {} will set to the topic: {}", maxProducers, persistenceTopic);
+ admin.topics().setMaxProducers(persistenceTopic, maxProducers);
+ Thread.sleep(3000);
+
+ admin.topics().createPartitionedTopic(persistenceTopic, 2);
+ Producer producer1 = null;
+ Producer producer2 = null;
+ Producer producer3 = null;
+ try {
+ producer1 = pulsarClient.newProducer().topic(persistenceTopic).create();
+ } catch (PulsarClientException e) {
+ Assert.fail();
+ }
+ try {
+ producer2 = pulsarClient.newProducer().topic(persistenceTopic).create();
+ } catch (PulsarClientException e) {
+ Assert.fail();
+ }
+ try {
+ producer3 = pulsarClient.newProducer().topic(persistenceTopic).create();
+ Assert.fail();
+ } catch (PulsarClientException e) {
+ log.info("Topic reached max producers limit");
+ }
+ Assert.assertNotNull(producer1);
+ Assert.assertNotNull(producer2);
+ Assert.assertNull(producer3);
+ producer1.close();
+ producer2.close();
+
+ Integer getMaxProducers = admin.topics().getMaxProducers(persistenceTopic);
+ log.info("MaxProducers {} get on topic: {}", getMaxProducers, persistenceTopic);
+ Assert.assertEquals(getMaxProducers, maxProducers);
+
+ admin.topics().deletePartitionedTopic(persistenceTopic);
+ admin.topics().deletePartitionedTopic(testTopic, true);
+ }
+
+ @Test
+ public void testRemoveMaxProducers() throws Exception {
+ Integer maxProducers = 10;
+ log.info("MaxProducers: {} will set to the topic: {}", maxProducers, testTopic);
+
+ admin.topics().setMaxProducers(testTopic, maxProducers);
+ Thread.sleep(3000);
+ Integer getMaxProducers = admin.topics().getMaxProducers(testTopic);
+
+ log.info("MaxProducers: {} get on topic: {}", getMaxProducers, testTopic);
+ Assert.assertEquals(getMaxProducers, maxProducers);
+
+ admin.topics().removeMaxProducers(testTopic);
Review comment:
OK. I will change it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou closed pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou closed pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#issuecomment-678605368
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on a change in pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on a change in pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#discussion_r475030539
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
##########
@@ -2352,6 +2352,50 @@ protected void internalGetPersistence(AsyncResponse asyncResponse){
return pulsar().getTopicPoliciesService().updateTopicPoliciesAsync(topicName, topicPolicies.get());
}
+ protected void internalGetMaxProducers(AsyncResponse asyncResponse){
+ validateAdminAccessForTenant(namespaceName.getTenant());
Review comment:
OK.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on a change in pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
jiazhai commented on a change in pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#discussion_r474723433
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/TopicPoliciesTest.java
##########
@@ -337,4 +338,80 @@ public void testRemovePersistence() throws Exception {
admin.topics().deletePartitionedTopic(testTopic, true);
}
+
Review comment:
Maybe we should also consider add test along with policies in namespace?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#issuecomment-675830637
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#issuecomment-678931282
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou removed a comment on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou removed a comment on pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#issuecomment-675774838
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on a change in pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
jiazhai commented on a change in pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#discussion_r474718765
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/TopicPoliciesTest.java
##########
@@ -337,4 +338,80 @@ public void testRemovePersistence() throws Exception {
admin.topics().deletePartitionedTopic(testTopic, true);
}
+
+ @Test
+ public void testCheckMaxProducers() throws Exception {
+ Integer maxProducers = new Integer(-1);
+ log.info("MaxProducers: {} will set to the topic: {}", maxProducers, testTopic);
+ try {
+ admin.topics().setMaxProducers(testTopic, maxProducers);
+ Assert.fail();
+ } catch (PulsarAdminException e) {
+ Assert.assertEquals(e.getStatusCode(), 412);
+ }
+
+ admin.topics().deletePartitionedTopic(testTopic, true);
+ }
+
+ @Test
+ public void testSetMaxProducers() throws Exception {
+ Integer maxProducers = 2;
+ log.info("MaxProducers: {} will set to the topic: {}", maxProducers, persistenceTopic);
+ admin.topics().setMaxProducers(persistenceTopic, maxProducers);
+ Thread.sleep(3000);
+
+ admin.topics().createPartitionedTopic(persistenceTopic, 2);
+ Producer producer1 = null;
+ Producer producer2 = null;
+ Producer producer3 = null;
+ try {
+ producer1 = pulsarClient.newProducer().topic(persistenceTopic).create();
+ } catch (PulsarClientException e) {
+ Assert.fail();
+ }
+ try {
+ producer2 = pulsarClient.newProducer().topic(persistenceTopic).create();
+ } catch (PulsarClientException e) {
+ Assert.fail();
+ }
+ try {
+ producer3 = pulsarClient.newProducer().topic(persistenceTopic).create();
+ Assert.fail();
+ } catch (PulsarClientException e) {
+ log.info("Topic reached max producers limit");
+ }
+ Assert.assertNotNull(producer1);
+ Assert.assertNotNull(producer2);
+ Assert.assertNull(producer3);
+ producer1.close();
+ producer2.close();
+
+ Integer getMaxProducers = admin.topics().getMaxProducers(persistenceTopic);
+ log.info("MaxProducers {} get on topic: {}", getMaxProducers, persistenceTopic);
+ Assert.assertEquals(getMaxProducers, maxProducers);
+
+ admin.topics().deletePartitionedTopic(persistenceTopic);
+ admin.topics().deletePartitionedTopic(testTopic, true);
+ }
+
+ @Test
+ public void testRemoveMaxProducers() throws Exception {
+ Integer maxProducers = 10;
+ log.info("MaxProducers: {} will set to the topic: {}", maxProducers, testTopic);
+
+ admin.topics().setMaxProducers(testTopic, maxProducers);
+ Thread.sleep(3000);
+ Integer getMaxProducers = admin.topics().getMaxProducers(testTopic);
+
+ log.info("MaxProducers: {} get on topic: {}", getMaxProducers, testTopic);
+ Assert.assertEquals(getMaxProducers, maxProducers);
+
+ admin.topics().removeMaxProducers(testTopic);
Review comment:
How about change this test into this logic:
- maxProducers = 2;
- after set, can only create 2 producers;
- after remove, can create more than 2 producers;
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#issuecomment-678721349
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
sijie commented on pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#issuecomment-679395943
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#issuecomment-675789050
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#issuecomment-678599094
/pulsarbot run-failure-checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on a change in pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on a change in pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#discussion_r474723733
##########
File path: pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
##########
@@ -2352,6 +2352,50 @@ protected void internalGetPersistence(AsyncResponse asyncResponse){
return pulsar().getTopicPoliciesService().updateTopicPoliciesAsync(topicName, topicPolicies.get());
}
+ protected void internalGetMaxProducers(AsyncResponse asyncResponse){
+ validateAdminAccessForTenant(namespaceName.getTenant());
Review comment:
Call `checkTopicLevelPolicyEnable()` before use topic policy API, please check all.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zhanghaou commented on a change in pull request #7839: [ISSUE 7758]Support set Max Producer on topic level
Posted by GitBox <gi...@apache.org>.
zhanghaou commented on a change in pull request #7839:
URL: https://github.com/apache/pulsar/pull/7839#discussion_r475030632
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/TopicPoliciesTest.java
##########
@@ -337,4 +338,80 @@ public void testRemovePersistence() throws Exception {
admin.topics().deletePartitionedTopic(testTopic, true);
}
+
Review comment:
OK.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org