You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/05/16 14:52:44 UTC
[GitHub] [pulsar] BewareMyPower opened a new pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
BewareMyPower opened a new pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601
Fixes #8813
### Motivation
Actually #8818 didn't fix the issue. The reason is when `PulsarAdmin` tries to get partition metadata, the `checkAllowAutoCreation` query param is false, so the topic existence check will never be performed.
### Modifications
- Check if the topic exists when the partition's metadata is 0 and `checkAllowAutoCreation` is false. See comments for detail explanations.
- Change the current tests to make sure the check works.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
This change is already covered by existing tests, such as *AdminApiTest#partitionedTopics*.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower edited a comment on pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
BewareMyPower edited a comment on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-841920405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-841920405
Now it's ready for review. It looks like there're still two flaky tests that can always pass in my local env.
- LoadBalancerTest.testBrokerRanking
- MessagePublishBufferThrottleTest.testBlockByPublishRateLimiting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-841936355
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-842252443
@eolivelli I'll push a commit to fix the `getPartitionedTopicMetadata` for non-persistent topics in this PR. The other problems are subscription related admin operations. I may not have time to fix them recently, maybe we can open an issue to track it first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-842450161
@codelipenghui Please include this fix in 2.8.0 if it's merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-842234107
Thanks @eolivelli for your advice. I tried to add tests for non-persistent topics and found some difference between persistent topics and non-persistent topics. In this PR, I'll only fix the `getPartitionedTopicMetadata` implementation of non-persistent topics and mark the difference as comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower edited a comment on pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
BewareMyPower edited a comment on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-842252443
@eolivelli I'll push a commit to fix the `getPartitionedTopicMetadata` for non-persistent topics in this PR soon. The other problems are subscription related admin operations. I may not have time to fix them recently, maybe we can open an issue to track it first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-842963603
@eolivelli PTAL again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on a change in pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#discussion_r633360745
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
##########
@@ -873,6 +872,13 @@ public void persistentTopics(String topicName) throws Exception {
public void partitionedTopics(String topicName) throws Exception {
assertEquals(admin.topics().getPartitionedTopicList("prop-xyz/ns1"), Lists.newArrayList());
final String partitionedTopicName = "persistent://prop-xyz/ns1/" + topicName;
+
+ try {
+ admin.topics().getPartitionedTopicMetadata(partitionedTopicName);
+ fail("getPartitionedTopicMetadata of " + partitionedTopicName + " should not succeed");
+ } catch (NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
##########
@@ -882,8 +888,11 @@ public void partitionedTopics(String topicName) throws Exception {
List<String> topics = admin.topics().getList("prop-xyz/ns1");
assertEquals(topics.size(), 4);
- assertEquals(admin.topics().getPartitionedTopicMetadata("persistent://prop-xyz/ns1/ds2").partitions,
- 0);
+ try {
+ admin.topics().getPartitionedTopicMetadata("persistent://prop-xyz/ns1/ds2");
+ fail("getPartitionedTopicMetadata of persistent://prop-xyz/ns1/ds2 should not succeed");
+ } catch (NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
##########
@@ -1003,7 +1012,11 @@ public void partitionedTopics(String topicName) throws Exception {
admin.topics().deletePartitionedTopic(partitionedTopicName);
- assertEquals(admin.topics().getPartitionedTopicMetadata(partitionedTopicName).partitions, 0);
+ try {
+ admin.topics().getPartitionedTopicMetadata(partitionedTopicName);
+ fail("getPartitionedTopicMetadata of " + partitionedTopicName + " should not succeed");
+ } catch (NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v1/V1_AdminApiTest.java
##########
@@ -908,7 +910,11 @@ public void partitionedTopics(String topicName) throws Exception {
admin.topics().deletePartitionedTopic(partitionedTopicName);
- assertEquals(admin.topics().getPartitionedTopicMetadata(partitionedTopicName).partitions, 0);
+ try {
+ admin.topics().getPartitionedTopicMetadata(partitionedTopicName);
+ fail("getPartitionedTopicMetadata of " + partitionedTopicName + " should not succeed");
+ } catch (NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceAutoTopicCreationTest.java
##########
@@ -154,8 +154,10 @@ public void testGetPartitionedMetadataWithoutCheckAllowAutoCreation() throws Exc
pulsar.getConfiguration().setDefaultNumPartitions(3);
final String topicString = "persistent://prop/ns-abc/test-topic-3";
- int partitions = admin.topics().getPartitionedTopicMetadata(topicString).partitions;
- assertEquals(partitions, 0);
+ try {
+ admin.topics().getPartitionedTopicMetadata(topicString);
+ } catch (PulsarAdminException.NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v1/V1_AdminApiTest.java
##########
@@ -792,9 +792,11 @@ public void partitionedTopics(String topicName) throws Exception {
List<String> topics = admin.topics().getList("prop-xyz/use/ns1");
assertEquals(topics.size(), 4);
- assertEquals(
- admin.topics().getPartitionedTopicMetadata("persistent://prop-xyz/use/ns1/ds2").partitions,
- 0);
+ try {
+ admin.topics().getPartitionedTopicMetadata("persistent://prop-xyz/use/ns1/ds2");
+ fail("getPartitionedTopicMetadata of persistent://prop-xyz/use/ns1/ds2 should not succeed");
+ } catch (NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-842246664
@BewareMyPower can you please follow up with a new PR for non persistent topics ?
I am fine with merging this patch (but please check my comments)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #10601: [WIP] Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-841834142
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on a change in pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#discussion_r633360745
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
##########
@@ -873,6 +872,13 @@ public void persistentTopics(String topicName) throws Exception {
public void partitionedTopics(String topicName) throws Exception {
assertEquals(admin.topics().getPartitionedTopicList("prop-xyz/ns1"), Lists.newArrayList());
final String partitionedTopicName = "persistent://prop-xyz/ns1/" + topicName;
+
+ try {
+ admin.topics().getPartitionedTopicMetadata(partitionedTopicName);
+ fail("getPartitionedTopicMetadata of " + partitionedTopicName + " should not succeed");
+ } catch (NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
##########
@@ -882,8 +888,11 @@ public void partitionedTopics(String topicName) throws Exception {
List<String> topics = admin.topics().getList("prop-xyz/ns1");
assertEquals(topics.size(), 4);
- assertEquals(admin.topics().getPartitionedTopicMetadata("persistent://prop-xyz/ns1/ds2").partitions,
- 0);
+ try {
+ admin.topics().getPartitionedTopicMetadata("persistent://prop-xyz/ns1/ds2");
+ fail("getPartitionedTopicMetadata of persistent://prop-xyz/ns1/ds2 should not succeed");
+ } catch (NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/AdminApiTest.java
##########
@@ -1003,7 +1012,11 @@ public void partitionedTopics(String topicName) throws Exception {
admin.topics().deletePartitionedTopic(partitionedTopicName);
- assertEquals(admin.topics().getPartitionedTopicMetadata(partitionedTopicName).partitions, 0);
+ try {
+ admin.topics().getPartitionedTopicMetadata(partitionedTopicName);
+ fail("getPartitionedTopicMetadata of " + partitionedTopicName + " should not succeed");
+ } catch (NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v1/V1_AdminApiTest.java
##########
@@ -908,7 +910,11 @@ public void partitionedTopics(String topicName) throws Exception {
admin.topics().deletePartitionedTopic(partitionedTopicName);
- assertEquals(admin.topics().getPartitionedTopicMetadata(partitionedTopicName).partitions, 0);
+ try {
+ admin.topics().getPartitionedTopicMetadata(partitionedTopicName);
+ fail("getPartitionedTopicMetadata of " + partitionedTopicName + " should not succeed");
+ } catch (NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/service/BrokerServiceAutoTopicCreationTest.java
##########
@@ -154,8 +154,10 @@ public void testGetPartitionedMetadataWithoutCheckAllowAutoCreation() throws Exc
pulsar.getConfiguration().setDefaultNumPartitions(3);
final String topicString = "persistent://prop/ns-abc/test-topic-3";
- int partitions = admin.topics().getPartitionedTopicMetadata(topicString).partitions;
- assertEquals(partitions, 0);
+ try {
+ admin.topics().getPartitionedTopicMetadata(topicString);
+ } catch (PulsarAdminException.NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
##########
File path: pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v1/V1_AdminApiTest.java
##########
@@ -792,9 +792,11 @@ public void partitionedTopics(String topicName) throws Exception {
List<String> topics = admin.topics().getList("prop-xyz/use/ns1");
assertEquals(topics.size(), 4);
- assertEquals(
- admin.topics().getPartitionedTopicMetadata("persistent://prop-xyz/use/ns1/ds2").partitions,
- 0);
+ try {
+ admin.topics().getPartitionedTopicMetadata("persistent://prop-xyz/use/ns1/ds2");
+ fail("getPartitionedTopicMetadata of persistent://prop-xyz/use/ns1/ds2 should not succeed");
+ } catch (NotFoundException ignored) {
Review comment:
nit: rename `ignored` to `expected`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #10601: [WIP] Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-841834142
Mark it as WIP first because there may be still some failed tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower edited a comment on pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
BewareMyPower edited a comment on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-841920405
It looks like there're still two flaky tests that can always pass in my local env.
- LoadBalancerTest.testBrokerRanking
- MessagePublishBufferThrottleTest.testBlockByPublishRateLimiting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli merged pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
eolivelli merged pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #10601: Fix getting partition metadata of a nonexistent topic returns 0
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #10601:
URL: https://github.com/apache/pulsar/pull/10601#issuecomment-842246664
@BewareMyPower can you please follow up with a new PR for non persistent topics ?
I am fine with merging this patch (but please check my comments)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org