You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/10/22 09:45:34 UTC
[GitHub] [pulsar] futeng opened a new issue #8339: [C Client] Server resource exhaustion due to C client memory leak
futeng opened a new issue #8339:
URL: https://github.com/apache/pulsar/issues/8339
**Describe the bug**
Testing with *valgrind* revealed a memory leak in the pulsar c client, as shown below.
And due to a memory leak problem, this can lead to:
1. The program restarts regularly
2. Host resources can be exhausted and cause a crash
**To Reproduce**
Server versions：
- JDK: 1.8.0_192
- Apache Pulsar: 2.5.0
- Zookeeper：3.6.1
- Client ：
```
$ rpm -qa | grep apache
apache-pulsar-client-debuginfo-2.6.0-1.x86_64
apache-commons-lang-2.6-15.el7.noarch
apache-pulsar-client-2.6.0-1.x86_64
apache-pulsar-client-devel-2.6.0-1.x86_64
apache-commons-io-2.4-12.el7.noarch
apache-commons-logging-1.1.2-7.el7.noarch
```
**C code (core logic)**
Please see [pulsar_tx_thread.c.main_logic.zip](https://github.com/apache/pulsar/files/5421766/pulsar_tx_thread.c.main_logic.zip)
**The command valgrind runs**
```shell
valgrind --log-file=/tmp/valgrind.log`date +%Y%m%d%H%M%S` --tool=memcheck --leak-check=full --show-leak-kinds=all --suppressions=valgrind_supp.txt -v ~/iprobe2sdtp/fv2sdtp/build/udp2sdtp -c 1fffffffff -n 1 -m 16384 --file-prefix=fv_xdr -- --config-file /zctt/dmp/etc/fv2sdtp.conf
```
**The entire file where VALGRIND prints the results**
Please see [valgrind.log20201021172208.zip](https://github.com/apache/pulsar/files/5421785/valgrind.log20201021172208.zip)
**Expected behavior**
Please help us solve this together. Hopefully fix in the next release.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on issue #8339: [C Client] Server resource exhaustion due to C client memory leak
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on issue #8339:
URL: https://github.com/apache/pulsar/issues/8339#issuecomment-714579552
First there is a flaw of C client's design, the callback needs to manual free the `pulsar_message_id_t`:
```c
static void asnyc_send_call_back(pulsar_result result, pulsar_message_id_t *msgId, void *ctx)
{
msg_ctx *pctx = (msg_ctx *)ctx;
(void)msgId; /* msgId points to a block of dynamic allocated memory, so the memory leaks here */
/* ... */
pulsar_message_free(pctx->msg);
free(pctx);
}
```
For the backward compatibility, it cannot be changed. You need to call `pulsar_message_id_free` in the callback function, see https://github.com/apache/pulsar/blob/e31d18c00e4967779f84f55b489468ccc0b83a40/pulsar-client-cpp/include/pulsar/c/message_id.h#L54
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] futeng closed issue #8339: [C Client] Server resource exhaustion due to C client memory leak
Posted by GitBox <gi...@apache.org>.
futeng closed issue #8339:
URL: https://github.com/apache/pulsar/issues/8339
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #8339: [C Client] Server resource exhaustion due to C client memory leak
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #8339:
URL: https://github.com/apache/pulsar/issues/8339#issuecomment-714690486
@futeng Can you check @BewareMyPower 's comment? You need to call `pulsar_message_id_free` to free the message-id. There is a known issue in cgo client (if you are using cgo client). The issue is tracked at apache/pulsar#8311
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] futeng commented on issue #8339: [C Client] Server resource exhaustion due to C client memory leak
Posted by GitBox <gi...@apache.org>.
futeng commented on issue #8339:
URL: https://github.com/apache/pulsar/issues/8339#issuecomment-716332362
The problem has not recurred for 4 days, by adjusting according to your suggestion. Thank you so much for your response and time. @BewareMyPower @sijie 😊👍.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org