You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/03/01 19:49:42 UTC
[GitHub] [pulsar-client-node] alijmlzd opened a new issue #78: Error: Failed
to send message: AlreadyClosed
alijmlzd opened a new issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78
Hi there,
Why this example with 1 second timeout works fine:
```js
const client = new Pulsar.Client({
serviceUrl: 'pulsar://localhost:6650',
})
const producer = await client.createProducer({
topic: 'persistent://public/default/my-topic',
})
setTimeout(async() => {
producer.send({
data: Buffer.from('My Message'),
})
}, 1000)
```
But this example with a few more seconds timeout occurs an error?
```js
const client = new Pulsar.Client({
serviceUrl: 'pulsar://localhost:6650',
})
const producer = await client.createProducer({
topic: 'persistent://public/default/my-topic',
})
setTimeout(async() => {
producer.send({
data: Buffer.from('My Message'),
})
}, 9000)
```
Full error message:
```
You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection:
Error: Failed to send message: AlreadyClosed
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] yosiat commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
yosiat commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-601193248
@alexyans yeah this is an example, but in real system, I want to use the same producer and publish messages every now and then. we have a system that publish a message every one minute, in this case I will need to create new producer every-time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alijmlzd commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alijmlzd commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-595282893
@alexyans No, but I will try that and share my result.
By the way, what version you recommend to start?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] yosiat commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
yosiat commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-603524312
@sijie sounds like a good approach, I tried my approach of creating napi ref, but couldn't find a way to implement it.
I will implement your approach and create a pull request for that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] sijie commented on issue #78: Error: Failed to
send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-603522717
@alexyans @yosiat I think we can let Producer object keep a reference to the client. So if your application keeps references to Producer, it should keep the reference to the client as well. So the client instance will not be destroyed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alexyans commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alexyans commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-595671568
That would be hard to tell without looking at the code. We know the client uses a connection pool for the original client connection, but it's not clear to me whether the producers/consumers create separate connections or piggyback on the one client connection. I'm not exactly sure what happens when you create a producer, it could be cheaper than it looks.
But yeah, I share your frustration. Hope it gets resolved soon.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alijmlzd edited a comment on issue #78: Error:
Failed to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alijmlzd edited a comment on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-594750069
Is there any kind of activity in there to solve this issue?
To be more specific, I run pulsar inside Docker like this [as described here](https://pulsar.apache.org/docs/v2.0.1-incubating/getting-started/docker/):
```bash
$ docker run -it \
-p 6650:6650 \
-p 8080:8080 \
-v $PWD/data:/pulsar/data \
apachepulsar/pulsar:2.0.0-rc1-incubating \
bin/pulsar standalone
```
And here is the pulsar server logs:
```bash
18:33:03.492 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - New connection from /172.17.0.1:55506
18:33:03.607 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506][persistent://public/default/my-topic] Creating producer. producerId=0
18:33:03.722 [Thread-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506] persistent://public/default/my-topic configured with schema false
18:33:03.728 [Thread-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506] Created new producer: Producer{topic=PersistentTopic{topic=persistent://public/default/my-topic}, client=/172.17.0.1:55506, producerName=standalone-8-2, producerId=0}
18:33:19.431 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - Closed connection from /172.17.0.1:55506
18:33:20.480 [pulsar-web-57-1] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [04/Mar/2020:18:33:20 +0000] "GET /admin/v2/persistent/public/functions/coordinate/stats HTTP/1.1" 200 882 "-" "Pulsar-Java-v2.5.0" 9
```
I see on log `18:33:19.431` there is a connection close after a few second from producer creation. so this is not clear enough for me that this is the client behavior or pulsar server.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] sijie commented on issue #78: Error: Failed to
send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-600894572
"AlreadyClosed" indicates that the client is already closed. You need to avoid client destruction.
See a producer example: http://pulsar.apache.org/docs/en/client-libraries-node/#producer-example
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alexyans commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alexyans commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-604658303
@yosiat thank you!!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] equanz commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
equanz commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-597964706
It seems that `setTimeout` callback function run after pulsar client was destructed.
I think if use `setTimeout` in this case, should to use Promise and wait for timeout.
Here is an example.
```javascript
const Pulsar = require("pulsar-client");
async function produceTest(timeout) {
const client = new Pulsar.Client({
serviceUrl: "pulsar://localhost:6650"
});
const producer = await client.createProducer({
topic: "persistent://public/default/my-topic"
});
await new Promise((resolve, reject) => {
setTimeout(() => {
producer.send({
data: Buffer.from("My Message")
});
resolve();
}, timeout);
});
await client.close();
}
produceTest(1000);
produceTest(9000);
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alexyans commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alexyans commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-595136257
@alijmlzd did you try rolling back to an older version? Any luck finding one?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] equanz commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
equanz commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-598552987
As above, it seems that client has already been destructed when executing `send` after timeout ms. Therefore, it seems that the connection was closed by not Broker but client.
Here is the destructor implementation in C++ client(Node.js client wraps C++ client)
* https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/lib/ClientImpl.cc#L128
- https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/lib/ClientImpl.cc#L541-L568
I think you need to avoid client destruction before producing.
(Therefore, in my example, `await` is put on `setTimeout`)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] eaba commented on issue #78: Error: Failed to
send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
eaba commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-602017688
There are two sides to this, cause I experienced it!
1 - If your producer is actually being closed, have you tried Ping-ing the server every, lets say, 30s. Also try handling Ping command coming from the server.
2 - It might be that you created a producer with one connection and sending messages via another connection that is not meant for that producer - you will get alreadyclosed exception even if the producer is alive!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alexyans commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alexyans commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-601179535
This is a playground example though. In a real system you can't necessarily control the rate of messages produced, as they correspond to real-time events. You could send mock keep-alive messages but it's ugly and a waste of resoures.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alijmlzd edited a comment on issue #78: Error:
Failed to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alijmlzd edited a comment on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-594750069
Is there any kind of activity here to solve this issue?
To be more specific, I run pulsar server inside Docker like this [as described here](https://pulsar.apache.org/docs/v2.0.1-incubating/getting-started/docker/):
```bash
$ docker run -it \
-p 6650:6650 \
-p 8080:8080 \
-v $PWD/data:/pulsar/data \
apachepulsar/pulsar:2.0.0-rc1-incubating \
bin/pulsar standalone
```
And here is the pulsar server logs:
```bash
18:33:03.492 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - New connection from /172.17.0.1:55506
18:33:03.607 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506][persistent://public/default/my-topic] Creating producer. producerId=0
18:33:03.722 [Thread-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506] persistent://public/default/my-topic configured with schema false
18:33:03.728 [Thread-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506] Created new producer: Producer{topic=PersistentTopic{topic=persistent://public/default/my-topic}, client=/172.17.0.1:55506, producerName=standalone-8-2, producerId=0}
18:33:19.431 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - Closed connection from /172.17.0.1:55506
18:33:20.480 [pulsar-web-57-1] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [04/Mar/2020:18:33:20 +0000] "GET /admin/v2/persistent/public/functions/coordinate/stats HTTP/1.1" 200 882 "-" "Pulsar-Java-v2.5.0" 9
```
I see on log `18:33:19.431` there is a connection close after a few second from producer creation. so this is not clear enough for me that this is the client behavior or pulsar server.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] massakam closed issue #78: Error: Failed to
send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
massakam closed issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] yosiat commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
yosiat commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-604705233
@alexyans happy to help :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alexyans commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alexyans commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-595133166
I have the same problem. Sending a message immediately after initializing the producer works fine, so it can't be a TLS handshake issue. The connection seems to timeout after a second or so.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alijmlzd commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alijmlzd commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-594750069
Is there any kind of activity is there to solve this issue?
To be more specific, I run pulsar inside Docker like this [as described here](https://pulsar.apache.org/docs/v2.0.1-incubating/getting-started/docker/):
```bash
$ docker run -it \
-p 6650:6650 \
-p 8080:8080 \
-v $PWD/data:/pulsar/data \
apachepulsar/pulsar:2.0.0-rc1-incubating \
bin/pulsar standalone
```
And here is the pulsar server logs:
```bash
18:33:03.492 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - New connection from /172.17.0.1:55506
18:33:03.607 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506][persistent://public/default/my-topic] Creating producer. producerId=0
18:33:03.722 [Thread-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506] persistent://public/default/my-topic configured with schema false
18:33:03.728 [Thread-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506] Created new producer: Producer{topic=PersistentTopic{topic=persistent://public/default/my-topic}, client=/172.17.0.1:55506, producerName=standalone-8-2, producerId=0}
18:33:19.431 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - Closed connection from /172.17.0.1:55506
18:33:20.480 [pulsar-web-57-1] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [04/Mar/2020:18:33:20 +0000] "GET /admin/v2/persistent/public/functions/coordinate/stats HTTP/1.1" 200 882 "-" "Pulsar-Java-v2.5.0" 9
```
I see on log `18:33:19.431` there is a connection close after a few second from producer creation. so this is not clear enough for me that this is the client behavior or pulsar server.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] yosiat edited a comment on issue #78: Error:
Failed to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
yosiat edited a comment on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-601193248
@alexyans yeah this is an example, but in real system, I want to use the same producer and publish messages every now and then. we have a system that publish a message every one minute, in this case I will need to create new producer every-time.
I think there is no ref maintaned to producer and this why nodejs decided to close the producer. I think we need to fix this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alexyans commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alexyans commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-598421709
@equanz your example handles the rejection, sure. But there shouldn't be a rejection in the first place. A producer closing after some timeout is unspecified behavior. It also drifts from the convention: a producer connection should remain open as long as you need it open. Otherwise there would be no need to explicitly close it as per the docs. If it closes itself no matter what you do, it is safe from leaks anyway.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alijmlzd edited a comment on issue #78: Error:
Failed to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alijmlzd edited a comment on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-594750069
Is there any kind of activity here to solve this issue?
To be more specific, I run pulsar inside Docker like this [as described here](https://pulsar.apache.org/docs/v2.0.1-incubating/getting-started/docker/):
```bash
$ docker run -it \
-p 6650:6650 \
-p 8080:8080 \
-v $PWD/data:/pulsar/data \
apachepulsar/pulsar:2.0.0-rc1-incubating \
bin/pulsar standalone
```
And here is the pulsar server logs:
```bash
18:33:03.492 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - New connection from /172.17.0.1:55506
18:33:03.607 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506][persistent://public/default/my-topic] Creating producer. producerId=0
18:33:03.722 [Thread-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506] persistent://public/default/my-topic configured with schema false
18:33:03.728 [Thread-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.17.0.1:55506] Created new producer: Producer{topic=PersistentTopic{topic=persistent://public/default/my-topic}, client=/172.17.0.1:55506, producerName=standalone-8-2, producerId=0}
18:33:19.431 [pulsar-io-50-4] INFO org.apache.pulsar.broker.service.ServerCnx - Closed connection from /172.17.0.1:55506
18:33:20.480 [pulsar-web-57-1] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [04/Mar/2020:18:33:20 +0000] "GET /admin/v2/persistent/public/functions/coordinate/stats HTTP/1.1" 200 882 "-" "Pulsar-Java-v2.5.0" 9
```
I see on log `18:33:19.431` there is a connection close after a few second from producer creation. so this is not clear enough for me that this is the client behavior or pulsar server.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] yosiat commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
yosiat commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-602585875
Hey!
I see you what you are writing here, but this is a bug in the client.
I don't need to implement any keepalives or ping-pong mechanism on my own.
Here is the go client, doing the same thing I did with my nodejs example (https://github.com/apache/pulsar-client-node/issues/78#issuecomment-601153727) - https://gist.github.com/yosiat/97460fb4dee41c5d0d0f6873a31c77d9
The go code is producing all messages, while the node client is dying after some time.
the problem in my opinion that my function is maintaining a reference to the producer but no to the client which cause NodeJS/V8 to garage collect the client since it's not being referenced.
Changing my code to, instead of `console.log(`Sent message: ${msg}`);` to `console.log(`Sent message: ${msg} - client ${client}`);` fixes the issue for me.
IMO, But this is wrong.
@eaba my example has only one client and one producer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] yosiat commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
yosiat commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-601153727
How I avoid client destruction?
In the following example, I create a producer and produce a message every 1 second.
```javascript
const Pulsar = require('pulsar-client');
(async () => {
// Create a client
const client = new Pulsar.Client({
serviceUrl: 'pulsar://localhost:6650'
});
// Create a producer
const producer = await client.createProducer({
topic: 'persistent://public/default/my-topic',
sendTimeoutMs: 30000,
batchingEnabled: true,
});
let i =0;
async function produceMessage() {
i = i + 1;
const msg = `my-message-${i}`;
const ret = await producer.send({
data: Buffer.from(msg),
});
console.log(`Sent message: ${msg}`);
console.log(ret);
setTimeout(produceMessage, 1000);
}
produceMessage();
})();
```
Output:
```
2020-03-19 14:33:49.010 INFO ConnectionPool:85 | Created connection for pulsar://localhost:6650
2020-03-19 14:33:49.011 INFO ClientConnection:330 | [127.0.0.1:64239 -> 127.0.0.1:6650] Connected to broker
2020-03-19 14:33:49.111 INFO BatchMessageContainer:43 | { BatchContainer [size = 0] [batchSizeInBytes_ = 0] [maxAllowedMessageBatchSizeInBytes_ = 131072] [maxAllowedNumMessagesInBatch_ = 1000] [topicName = persistent://public/default/my-topic] [producerName_ = ] [batchSizeInBytes_ = 0] [numberOfBatchesSent = 0] [averageBatchSize = 0]} BatchMessageContainer constructed
2020-03-19 14:33:49.111 INFO HandlerBase:53 | [persistent://public/default/my-topic, ] Getting connection from pool
2020-03-19 14:33:49.155 INFO ProducerImpl:151 | [persistent://public/default/my-topic, ] Created producer on broker [127.0.0.1:64239 -> 127.0.0.1:6650]
Sent message: my-message-1
null
Sent message: my-message-2
null
Sent message: my-message-3
null
Sent message: my-message-4
null
Sent message: my-message-5
null
Sent message: my-message-6
null
Sent message: my-message-7
null
Sent message: my-message-8
null
2020-03-19 14:33:57.128 INFO ClientConnection:1349 | [127.0.0.1:64239 -> 127.0.0.1:6650] Connection closed
(node:37259) UnhandledPromiseRejectionWarning: Error: Failed to send message: AlreadyClosed
(node:37259) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:37259) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
2020-03-19 14:33:57.312 INFO ProducerImpl:472 | Producer - [persistent://public/default/my-topic, standalone-1-71] , [batchMessageContainer = { BatchContainer [size = 0] [batchSizeInBytes_ = 0] [maxAllowedMessageBatchSizeInBytes_ = 131072] [maxAllowedNumMessagesInBatch_ = 1000] [topicName = persistent://public/default/my-topic] [producerName_ = standalone-1-71] [batchSizeInBytes_ = 0] [numberOfBatchesSent = 8] [averageBatchSize = 1]}]
2020-03-19 14:33:57.312 INFO BatchMessageContainer:170 | [numberOfBatchesSent = 8] [averageBatchSize = 1]
2020-03-19 14:33:57.312 INFO ClientConnection:235 | [127.0.0.1:64239 -> 127.0.0.1:6650] Destroyed connection
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alexyans commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alexyans commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-595336522
I tried 2.4 of the client as it was the one recommended by Kafkaesque, but the problem was still there for me. Didn't bother going further back as this is unspecified behavior and most likely caused by the pulsar server, and I have no control of what version Kafkaesque uses. It appears that the server explicitly closes the connection for some reason.
For my use case, I worked around the problem by always creating the producer right before i emit a message. Flushing and closing after use should prevent leaks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alexyans commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alexyans commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-602643477
> Changing my code to, instead of `console.log(`Sent message: ${msg}`);` to `console.log(`Sent message: ${msg} - client ${client}`);` fixes the issue for me.
okay if that's true, i think you're right about the root cause. @sijie how do you want to proceed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] yosiat commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
yosiat commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-603868358
@sijie tried out creating a reference to a client from the producer but it didn't help (tried with InstanceAccesor) and it created a weird dependency on the code which I didn't feel comfortable doing.
But I found out how to create a reference for a client and un-reference it when the client closes the connection.
This PR - https://github.com/apache/pulsar-client-node/pull/85 fixed the code I wrote above :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar-client-node] alijmlzd commented on issue #78: Error: Failed
to send message: AlreadyClosed
Posted by GitBox <gi...@apache.org>.
alijmlzd commented on issue #78: Error: Failed to send message: AlreadyClosed
URL: https://github.com/apache/pulsar-client-node/issues/78#issuecomment-595434563
> For my use case, I worked around the problem by always creating the producer right before I emit a message. Flushing and closing after use should prevent leaks.
Oh, as a quick fix its a good idea. but what if I have about more than 1000 send messages per second? Is it ok to create a producer, send and then flush and close the producer each time?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services