You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/12 08:39:44 UTC
[GitHub] [pulsar] naveenkusakula opened a new issue #11646: [Pulsar Java client] Not able to load "deadLetterPolicy" using loadConf(Map config) in ConsumerBuilder.
naveenkusakula opened a new issue #11646:
URL: https://github.com/apache/pulsar/issues/11646
Not able to load "deadLetterPolicy" when using loadConf(Map<String,Object> config) and not able to loadConf(Map<String,Object> config), after deadLetterPolicy() is called in ConsumerBuilder<T>.
**Steps to Reporoduce:**
1.
```
final Map<String, Object> consumerConfiguration = new HashMap<String, Object>();
configuration.put("ackTimeoutMillis", 1000);
consumerConfiguration.put("receiverQueueSize", 2000);
configuration.put("deadLetterPolicy", DeadLetterPolicy.builder()
.maxRedeliverCount(1)
.build());
Consumer<byte[]> builder = client.newConsumer()
.loadConf(config)
.subscribe();
```
2.
```
final Map<String, Object> consumerConfiguration = new HashMap<String, Object>();
configuration.put("ackTimeoutMillis", 1000);
consumerConfiguration.put("receiverQueueSize", 2000);
Consumer<byte[]> builder = client.newConsumer()
.deadLetterPolicy(
DeadLetterPolicy.builder()
.maxRedeliverCount(1)
.build())
.loadConf(config)
.subscribe();
```
**Expected behavior**
Should load config using loadConf(Map<String,Object> config) in consumerBuilder<T> with deadLetterPolicy.
**Errors**
```
java.lang.RuntimeException: Failed to load config into existing configuration data
at org.apache.pulsar.client.impl.conf.ConfigurationDataUtils.loadData(ConfigurationDataUtils.java:69)
at org.apache.pulsar.client.impl.ConsumerBuilderImpl.loadConf(ConsumerBuilderImpl.java:83)
....................
....................
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:210)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:206)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:131)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:65)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:69)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
Caused by: org.apache.pulsar.shade.com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.apache.pulsar.client.api.DeadLetterPolicy` (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
at [Source: (String)"{"autoUpdatePartitionsIntervalSeconds":60,"deadLetterPolicy":{"maxRedeliverCount":1},"tickDurationMillis":1000,"replicateSubscriptionState":true,"topicsPattern":"persistent://public/default","negativeAckRedeliveryDelayMicros":2000,"expireTimeOfIncompleteChunkedMessageMillis":60000,"maxPendingChuckedMessage":10,"topicNames":[],"priorityLevel":0,"ackReceiptEnabled":false,"acknowledgementsGroupTimeMicros":100000,"receiverQueueSize":2000,"autoUpdatePartitions":true,"subscriptionMode":"Durable",""[truncated 499 chars]; line: 1, column: 63] (through reference chain: org.apache.pulsar.client.impl.conf.ConsumerConfigurationData["deadLetterPolicy"])
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1764)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:400)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1209)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1415)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:129)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:324)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3548)
at org.apache.pulsar.shade.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3516)
at org.apache.pulsar.client.impl.conf.ConfigurationDataUtils.loadData(ConfigurationDataUtils.java:67)
... 73 more
```
**Desktop (please complete the following information):**
- OS: Ubuntu 18.04.5 LTS
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] mgrenonville commented on issue #11646: [Pulsar Java client] Not able to load "deadLetterPolicy" using loadConf(Map config) in ConsumerBuilder.
Posted by GitBox <gi...@apache.org>.
mgrenonville commented on issue #11646:
URL: https://github.com/apache/pulsar/issues/11646#issuecomment-993542046
I've also have this issue with client 2.8.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] naveenkusakula commented on issue #11646: [Pulsar Java client] Not able to load "deadLetterPolicy" using loadConf(Map config) in ConsumerBuilder.
Posted by GitBox <gi...@apache.org>.
naveenkusakula commented on issue #11646:
URL: https://github.com/apache/pulsar/issues/11646#issuecomment-897492852
I loaded deadLetterPolicy() using normal builder and than loadConfi() without deadLetterPolicy still not able to load configuration like I mentioned in option 2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] mgrenonville commented on issue #11646: [Pulsar Java client] Not able to load "deadLetterPolicy" using loadConf(Map config) in ConsumerBuilder.
Posted by GitBox <gi...@apache.org>.
mgrenonville commented on issue #11646:
URL: https://github.com/apache/pulsar/issues/11646#issuecomment-993562776
@naveenkusakula I've found a workarround, that may help you :
You should loadConf before setting deadLetterPolicy.
```java
final Map<String, Object> consumerConfiguration = new HashMap<String, Object>();
configuration.put("ackTimeoutMillis", 1000);
consumerConfiguration.put("receiverQueueSize", 2000);
Consumer<byte[]> builder = client.newConsumer()
.loadConf(config)
.deadLetterPolicy(
DeadLetterPolicy.builder()
.maxRedeliverCount(1)
.build())
.subscribe();
```
@eolivelli I've reproduced the bug in `org.apache.pulsar.client.impl.conf.ConfigurationDataUtilsTest`.
```java
@Test
public void testLoadConsumerConfigurationData() {
ConsumerConfigurationData confData = new ConsumerConfigurationData();
confData.setSubscriptionName("unknown-subscription");
confData.setPriorityLevel(10000);
confData.setConsumerName("unknown-consumer");
confData.setAutoUpdatePartitionsIntervalSeconds(1, TimeUnit.MINUTES);
Map<String, Object> config = new HashMap<>();
config.put("subscriptionName", "test-subscription");
config.put("priorityLevel", 100);
confData.setDeadLetterPolicy(DeadLetterPolicy.builder().deadLetterTopic("dlt").build());
confData = ConfigurationDataUtils.loadData(config, confData, ConsumerConfigurationData.class);
assertEquals("test-subscription", confData.getSubscriptionName());
assertEquals(100, confData.getPriorityLevel());
assertEquals("unknown-consumer", confData.getConsumerName());
assertEquals(60,confData.getAutoUpdatePartitionsIntervalSeconds());
}
```
This is due to serialization of dead letter policy in JSON by ConfigurationDataUtils, and jackson inability to create DeadLetterPolicy, since there is no default constructor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #11646: [Pulsar Java client] Not able to load "deadLetterPolicy" using loadConf(Map config) in ConsumerBuilder.
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #11646:
URL: https://github.com/apache/pulsar/issues/11646#issuecomment-1058887179
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on issue #11646: [Pulsar Java client] Not able to load "deadLetterPolicy" using loadConf(Map config) in ConsumerBuilder.
Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #11646:
URL: https://github.com/apache/pulsar/issues/11646#issuecomment-897490867
Unfortunately there are some other configuration options that are not configurable via loadConf, for instance Authentication in the PulsarClientBuilder.
I would like to see a consistent way of configuring the client, the consumer and the producer using a Map
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org