You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/12 20:31:30 UTC
[GitHub] [pulsar] acjohnson opened a new issue #11654: [Functions worker] Topic has an existing exclusive producer - self-deadlock issue on Pulsar 2.8.0
acjohnson opened a new issue #11654:
URL: https://github.com/apache/pulsar/issues/11654
**Describe the bug**
After upgrading to Official Apache Pulsar 2.8.0 docker image we are now seeing a flood of messages like this:
```
{"line":"22:48:42.086 [pulsar-client-io-51-1] ERROR org.apache.pulsar.client.impl.ProducerImpl - [persistent://public/functions/metadata] [c-iauto-poc-fw-10.13.2.236-8080-leader] Failed to create producer: org.apache.pulsar.broker.service.BrokerServiceException$ProducerFencedException: Topic has an existing exclusive producer: c-iauto-poc-fw-10.13.2.198-8080-leader\r","source":"stdout","tag":"81a71b1e87da"}
```
When this happens we are unable to deploy functions without restarting the function workers/broker nodes
**To Reproduce**
```
docker run -d \
--name=pulsar-broker \
--env='PULSAR_MEM= -Xms10g -Xmx10g -XX:MaxDirectMemorySize=10g' \
--volume=/docker/pulsar-broker/pulsar-broker.conf:/pulsar/conf/broker.conf:rw \
--volume=/docker/pulsar-broker/pulsar-functions-worker.yml:/pulsar/conf/functions_worker.yml:rw \
-p 6650:6650 \
-p 8080:8080 \
--restart=unless-stopped \
docker.io/apachepulsar/pulsar-all:2.8.0 bin/pulsar
```
Load it up with a bunch of go functions on 3 x i3.xlarge instances and watch the exclusive producers messages flood your screen
**Expected behavior**
This exclusive producer message and deadlock issue did not appear on Pulsar 2.7.x
**Desktop (please complete the following information):**
Ubuntu 20.04 amd64 + Docker CE 20.10.7
**Additional context**
We've been working with several StreamNative people to diagnose this bug via Slack. Please reach out to us there if you need any more info. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] acjohnson commented on issue #11654: [Functions worker] Topic has an existing exclusive producer - self-deadlock issue on Pulsar 2.8.0
Posted by GitBox <gi...@apache.org>.
acjohnson commented on issue #11654:
URL: https://github.com/apache/pulsar/issues/11654#issuecomment-1051366421
Upgraded to 2.8.2 and this problem appears to be resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] acjohnson closed issue #11654: [Functions worker] Topic has an existing exclusive producer - self-deadlock issue on Pulsar 2.8.0
Posted by GitBox <gi...@apache.org>.
acjohnson closed issue #11654:
URL: https://github.com/apache/pulsar/issues/11654
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org