You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/12/21 19:58:30 UTC
[GitHub] [pulsar] zbentley opened a new issue #13439: Regression creating a namespace with "null" in "backlog_quota_map" causes a NullPointerException on 2.9.1, but not 2.9.0 or earlier
zbentley opened a new issue #13439:
URL: https://github.com/apache/pulsar/issues/13439
**Describe the bug**
A stacktrace is raised when creating a namespace with a backlog quota map containing null, e.g.:
```
"backlog_quota_map": {"destination_storage": null, "message_age": {"policy": "producer_exception", "limitTime": 7200.0}}
```
**To Reproduce**
1. Start a standalone 2.9.1 broker, I used `brew install apache-pulsar` on an updated Homebrew on 12/21/21.
2. Create a new Namespace via the admin API. When POSTing to create it, set the backlog quota map field to the example value above.
3. See error
**Expected behavior**
No error should be raised. Behavior should be consistent with prior broker versions.
**Desktop (please complete the following information):**
- OSX and Homebrew; MacOS 11.6.2.
**Stacktrace**
```
Response code: 500
URL: PUT http://127.0.0.1:8080/admin/v2/namespaces/chariot3/chariot_namespace_test_topic_str
Headers: Headers({'host': '127.0.0.1:8080', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'user-agent': 'python-httpx/0.21.1', 'content-length': '1938', 'content-type': 'application/json'})
Payload: b'{"anti_affinity_group": null, "backlog_quota_map": {"destination_storage": null, "message_age": {"policy": "producer_exception", "limitTime": 7200.0}}, "retention_policies": {"retentionTimeInMinutes": 720, "retentionSizeInMB": -1}, "persistence": {"bookkeeperEnsemble": 1, "bookkeeperWriteQuorum": 1, "bookkeeperAckQuorum": 1, "managedLedgerMaxMarkDeleteRate": 0}, "deduplicationEnabled": true, "deduplicationSnapshotIntervalSeconds": 60.0, "autoSubscriptionCreationOverride": {"allowAutoSubscriptionCreation": false}, "autoTopicCreationOverride": {"allowAutoTopicCreation": false, "defaultNumPartitions": 0, "topicType": "non-partitioned"}, "message_ttl_in_seconds": 0, "subscription_expiration_time_minutes": 0, "encryption_required": false, "inactive_topic_policies": {"maxInactiveDurationSeconds": -1, "inactiveTopicDeleteMode": "delete_when_no_subscriptions", "deleteWhileInactive": false}, "max_producers_per_topic": 100, "max_consumers_per_topic": 10, "max_consumers_per_subscr
iption": 0, "max_unacked_messages_per_subscription": 10000, "max_unacked_messages_per_consumer": 100, "properties": {"chariot_primary": true, "chariot_topic_config": "{\\"consumer_groups\\": [{\\"name\\": \\"testsub\\", \\"type\\": 3}], \\"durability\\": 2, \\"max_consumers\\": 10, \\"max_message_size\\": 1024, \\"max_producers\\": 100, \\"payload\\": {\\"primitive_type\\": \\"STRING\\"}, \\"rate_limits\\": [{\\"actions\\": [2], \\"bytes\\": null, \\"messages\\": 1000, \\"period\\": 60.0}, {\\"actions\\": [1], \\"bytes\\": null, \\"messages\\": 500, \\"period\\": 60.0}, {\\"actions\\": [0], \\"bytes\\": null, \\"messages\\": 10, \\"period\\": 3600.0}], \\"send_timeout\\": 1.0, \\"team\\": \\"test_team\\", \\"thresholds\\": [{\\"actions\\": [1], \\"age\\": 43200.0, \\"kind\\": 0, \\"size\\": null}, {\\"actions\\": [3], \\"age\\": 7200.0, \\"kind\\": 1, \\"size\\": null}, {\\"actions\\": [0], \\"age\\": 1800.0, \\"kind\\": 1, \\"size\\": null}]}"}, "is_allow_auto_update_schema": true,
"schema_validation_enforced": false}'
--- An unexpected error occurred in the server ---
Message: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.NullPointerException) (through reference chain: org.apache.pulsar.common.policies.data.Policies["backlog_quota_map"]->java.util.LinkedHashMap["message_age"]->org.apache.pulsar.common.policies.data.impl.BacklogQuotaImpl["limit"])
Stacktrace:
org.apache.pulsar.metadata.api.MetadataStoreException: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.NullPointerException) (through reference chain: org.apache.pulsar.common.policies.data.Policies["backlog_quota_map"]->java.util.LinkedHashMap["message_age"]->org.apache.pulsar.common.policies.data.impl.BacklogQuotaImpl["limit"])
at org.apache.pulsar.broker.resources.BaseResources.create(BaseResources.java:134)
at org.apache.pulsar.broker.resources.NamespaceResources.createPolicies(NamespaceResources.java:75)
at org.apache.pulsar.broker.admin.impl.NamespacesBase.internalCreateNamespace(NamespacesBase.java:146)
at org.apache.pulsar.broker.admin.v2.Namespaces.createNamespace(Namespaces.java:138)
at jdk.internal.reflect.GeneratedMethodAccessor171.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
at org.apache.pulsar.broker.web.ResponseHandlerFilter.doFilter(ResponseHandlerFilter.java:67)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.NullPointerException) (through reference chain: org.apache.pulsar.common.policies.data.Policies["backlog_quota_map"]->java.util.LinkedHashMap["message_age"]->org.apache.pulsar.common.policies.data.impl.BacklogQuotaImpl["limit"])
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:390)
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:349)
at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:316)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:778)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeOptionalFields(MapSerializer.java:869)
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeWithoutTypeInfo(MapSerializer.java:760)
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:720)
at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:35)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:770)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4487)
at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsBytes(ObjectMapper.java:3765)
at org.apache.pulsar.metadata.cache.impl.JSONMetadataSerdeSimpleType.serialize(JSONMetadataSerdeSimpleType.java:37)
at org.apache.pulsar.metadata.cache.impl.MetadataCacheImpl.create(MetadataCacheImpl.java:211)
at org.apache.pulsar.broker.resources.BaseResources.createAsync(BaseResources.java:141)
at org.apache.pulsar.broker.resources.BaseResources.create(BaseResources.java:131)
... 65 more
Caused by: java.lang.NullPointerException
at org.apache.pulsar.common.policies.data.impl.BacklogQuotaImpl.getLimit(BacklogQuotaImpl.java:61)
at jdk.internal.reflect.GeneratedMethodAccessor181.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:689)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:770)
... 81 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] leizhiyuan commented on issue #13439: Regression creating a namespace with "null" in "backlog_quota_map" causes a NullPointerException on 2.9.1, but not 2.9.0 or earlier
Posted by GitBox <gi...@apache.org>.
leizhiyuan commented on issue #13439:
URL: https://github.com/apache/pulsar/issues/13439#issuecomment-999937196
Null of Long transalte to long
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zbentley commented on issue #13439: Regression creating a namespace with "null" in "backlog_quota_map" causes a NullPointerException on 2.9.1, but not 2.9.0 or earlier
Posted by GitBox <gi...@apache.org>.
zbentley commented on issue #13439:
URL: https://github.com/apache/pulsar/issues/13439#issuecomment-1002262691
Some additional conversation related to my above questions is here: https://apache-pulsar.slack.com/archives/C5Z4T36F7/p1640719936008200
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Shoothzj commented on issue #13439: Regression creating a namespace with "null" in "backlog_quota_map" causes a NullPointerException on 2.9.1, but not 2.9.0 or earlier
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on issue #13439:
URL: https://github.com/apache/pulsar/issues/13439#issuecomment-1002337049
@zbentley Thanks for your report. Today I will prepare a PR fix this issue. I can't read slack message now, I will check slack message six hours after
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #13439: Regression creating a namespace with "null" in "backlog_quota_map" causes a NullPointerException on 2.9.1, but not 2.9.0 or earlier
Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #13439:
URL: https://github.com/apache/pulsar/issues/13439
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] leizhiyuan commented on issue #13439: Regression creating a namespace with "null" in "backlog_quota_map" causes a NullPointerException on 2.9.1, but not 2.9.0 or earlier
Posted by GitBox <gi...@apache.org>.
leizhiyuan commented on issue #13439:
URL: https://github.com/apache/pulsar/issues/13439#issuecomment-999941262
@Shoothzj
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #13439: Regression creating a namespace with "null" in "backlog_quota_map" causes a NullPointerException on 2.9.1, but not 2.9.0 or earlier
Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #13439:
URL: https://github.com/apache/pulsar/issues/13439
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] leizhiyuan commented on issue #13439: Regression creating a namespace with "null" in "backlog_quota_map" causes a NullPointerException on 2.9.1, but not 2.9.0 or earlier
Posted by GitBox <gi...@apache.org>.
leizhiyuan commented on issue #13439:
URL: https://github.com/apache/pulsar/issues/13439#issuecomment-999940754
{"policy": "producer_exception", "limitTime": 7200.0}
you has not limit or limitSize provided,
limit and limitType will be null both
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zbentley commented on issue #13439: Regression creating a namespace with "null" in "backlog_quota_map" causes a NullPointerException on 2.9.1, but not 2.9.0 or earlier
Posted by GitBox <gi...@apache.org>.
zbentley commented on issue #13439:
URL: https://github.com/apache/pulsar/issues/13439#issuecomment-1001789124
I'm happy to forward fix (I don't need back compat for multiple broker versions, though I understand if others do). Could you provide the ideal/desired way to specify these limits? Specifically, looking for examples of JSON payloads for:
- Size and time-based limits.
- No limits.
- Size limits but no time limits.
- Time limits but no size limits.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Shoothzj commented on issue #13439: Regression creating a namespace with "null" in "backlog_quota_map" causes a NullPointerException on 2.9.1, but not 2.9.0 or earlier
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on issue #13439:
URL: https://github.com/apache/pulsar/issues/13439#issuecomment-999951844
@leizhiyuan @zbentley #13353 was written based on the `pulsar-admin` tools, in `pulsar-admin` tools, the limitStr, aka limitSize, is a required parameter, that's why I think `the limitSize and limit can't be both null`. Sorry for my fault.
Now, I think we need to compat situation that the user uses rest API to set `backlogQuotaMap` without any limit set.
@codelipenghui WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org