You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/08/12 03:44:45 UTC
[GitHub] [rocketmq] zhxjdwh opened a new issue #3252: the rocketmq_client.log file spent to mush cache/buffer
zhxjdwh opened a new issue #3252:
URL: https://github.com/apache/rocketmq/issues/3252
**BUG REPORT**
the rocketmq_client.log file spent to mush cache/buffer !!!!!!!!!
These log files really consume too much cache/buffer. The jvm memory usage of my container is only 500M, but the log buffer/cache of Rocketmq consumes 2G of memory.
这几个日志文件实在消耗了太多cache/buffer了，我容器的jvm内存使用才500M，但是rocketmq的日志buffer/cache就消耗了2G的内存
```
[root@node135 f914151fc9ae5ce5f477a8d0d69ed9b0ebbb2ea45be3aec0f0acbde859d3e3b1]# /usr/local/bin/vmtouch ./diff/root/logs/rocketmqlogs/**rocketmq_client.log**
Files: 1
Directories: 0
**Resident Pages: 198301/198301 774M/774M 100%**
Elapsed: 0.03113 seconds
[root@node135 f914151fc9ae5ce5f477a8d0d69ed9b0ebbb2ea45be3aec0f0acbde859d3e3b1]# /usr/local/bin/vmtouch ./diff/root/logs/rocketmqlogs/**rocketmq_client.log.1**
Files: 1
Directories: 0
**Resident Pages: 262145/262145 1G/1G 100%**
Elapsed: 0.042067 seconds
[root@node135 f914151fc9ae5ce5f477a8d0d69ed9b0ebbb2ea45be3aec0f0acbde859d3e3b1]# /usr/local/bin/vmtouch **./diff/root/logs/rocketmqlogs/rocketmq_client.log.2**
Files: 1
Directories: 0
**Resident Pages: 120207/262145 469M/1G 45.9%**
Elapsed: 0.020819 seconds
```
**FEATURE REQUEST**
Reduce the buffer/cache memory usage of log files
减少log文件的buffer/cache内存占用
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] zhxjdwh removed a comment on issue #3252: the rocketmq_client.log file spent to mush cache/buffer
Posted by GitBox <gi...@apache.org>.
zhxjdwh removed a comment on issue #3252:
URL: https://github.com/apache/rocketmq/issues/3252#issuecomment-897402857
@ni-ze
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] zhxjdwh commented on issue #3252: the rocketmq_client.log file spent too much cache/buffer memory
Posted by GitBox <gi...@apache.org>.
zhxjdwh commented on issue #3252:
URL: https://github.com/apache/rocketmq/issues/3252#issuecomment-898128055
@ni-ze
I have 400 jvm processes running in k8s, and each pod that uses Rocketmq consumes 2-6 times more memory than other pods (k8s counts buffer/cache into the used memory), and excessive memory consumption leads to frequent pods Killed by k8s.
I can only adjust logLevel to ERROR, so that it consumes less memory
我有400个jvm进程运行在k8s,每个有用到rocketmq的pod都会比其他pod多消耗2-6倍的内存(k8s把buffer/cache算到已用内存里面),内存消耗过多导致pod频繁被k8s杀死。
我现在只能把logLevel调整为ERROR，这样消耗内存就会少一点
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] zhxjdwh commented on issue #3252: the rocketmq_client.log file spent to mush cache/buffer
Posted by GitBox <gi...@apache.org>.
zhxjdwh commented on issue #3252:
URL: https://github.com/apache/rocketmq/issues/3252#issuecomment-897402857
@ni-ze
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] ni-ze commented on issue #3252: the rocketmq_client.log file spent too much cache/buffer memory
Posted by GitBox <gi...@apache.org>.
ni-ze commented on issue #3252:
URL: https://github.com/apache/rocketmq/issues/3252#issuecomment-897586883
here cache/buffer has nothing to do with your jvm, it manage by os use it to to accelerate read/write. It would not affect rocketmq client work. Just ignore it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] ni-ze commented on issue #3252: the rocketmq_client.log file spent to mush cache/buffer
Posted by GitBox <gi...@apache.org>.
ni-ze commented on issue #3252:
URL: https://github.com/apache/rocketmq/issues/3252#issuecomment-897371476
can you give a way to repeat this? or just it will appear anyway?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] zhaohai1299002788 commented on issue #3252: the rocketmq_client.log file spent too much cache/buffer memory
Posted by GitBox <gi...@apache.org>.
zhaohai1299002788 commented on issue #3252:
URL: https://github.com/apache/rocketmq/issues/3252#issuecomment-897436932
If it is a log file, you can write a timer to delete the old log file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] ni-ze removed a comment on issue #3252: the rocketmq_client.log file spent too much cache/buffer memory
Posted by GitBox <gi...@apache.org>.
ni-ze removed a comment on issue #3252:
URL: https://github.com/apache/rocketmq/issues/3252#issuecomment-897371476
can you give a way to repeat this? or just it will appear anyway?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] ni-ze edited a comment on issue #3252: the rocketmq_client.log file spent too much cache/buffer memory
Posted by GitBox <gi...@apache.org>.
ni-ze edited a comment on issue #3252:
URL: https://github.com/apache/rocketmq/issues/3252#issuecomment-897586883
here cache/buffer has nothing to do with your jvm, it manage by os, use it to accelerate read/write. It would not affect rocketmq client work. Just ignore it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org