You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "sindhushreem (via GitHub)" <gi...@apache.org> on 2023/01/27 04:31:08 UTC
[GitHub] [pulsar] sindhushreem opened a new issue, #19334: bookkeeper is running out of direct memory
sindhushreem opened a new issue, #19334:
URL: https://github.com/apache/pulsar/issues/19334
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
2.8.2
### Minimal reproduce step
bookeeper runnning for some days and ran out of direct memory .
### What did you expect to see?
Bookkeeper should be running without any error .
### What did you see instead?
INFO org.apache.bookkeeper.bookie.JournalChannel - Opening journal data/bookkeeper/journal/current/181400c6508.txn
2022-11-14T22:01:22,123 [main] ERROR org.apache.bookkeeper.common.component.AbstractLifecycleComponent - Failed to start Component: bookie-server
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:695) ~[?:1.8.0_332]
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) ~[?:1.8.0_332]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) ~[?:1.8.0_332]
at [sun.nio.ch](http://sun.nio.ch/).Util.getTemporaryDirectBuffer(Util.java:247) ~[?:1.8.0_332]
at [sun.nio.ch](http://sun.nio.ch/).IOUtil.read(IOUtil.java:195) ~[?:1.8.0_332]
at [sun.nio.ch](http://sun.nio.ch/).FileChannelImpl.read(FileChannelImpl.java:159) ~[?:1.8.0_332]
at org.apache.bookkeeper.bookie.JournalChannel.read(JournalChannel.java:261) ~[org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.bookie.Journal.fullRead(Journal.java:1194) ~[org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.bookie.Journal.scanJournal(Journal.java:833) ~[org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.bookie.Bookie.replay(Bookie.java:995) ~[org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.bookie.Bookie.readJournal(Bookie.java:961) ~[org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.bookie.Bookie.start(Bookie.java:1015) ~[org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.proto.BookieServer.start(BookieServer.java:156) ~[org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.server.service.BookieService.doStart(BookieService.java:68) ~[org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:83) ~[org.apache.bookkeeper-bookkeeper-common-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.common.component.LifecycleComponentStack.lambda$start$4(LifecycleComponentStack.java:144) ~[org.apache.bookkeeper-bookkeeper-common-4.14.3.jar:4.14.3]
at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:406) [com.google.guava-guava-30.1-jre.jar:?]
at org.apache.bookkeeper.common.component.LifecycleComponentStack.start(LifecycleComponentStack.java:144) [org.apache.bookkeeper-bookkeeper-common-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.common.component.ComponentStarter.startComponent(ComponentStarter.java:85) [org.apache.bookkeeper-bookkeeper-common-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.server.Main.doMain(Main.java:234) [org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3]
at org.apache.bookkeeper.server.Main.main(Main.java:208) [org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3]
2022-11-14T22:01:22,129 [main] ERROR org.apache.bookkeeper.common.component.AbstractLifecycleComponent - Calling uncaughtExceptionHandler
2022-11-14T22:01:22,129 [main] ERROR org.apache.bookkeeper.common.component.ComponentStarter - Triggered exceptionHandler of Component: bookie-server because of Exception in Thread: Thread[main,5,main]
java.lang.OutOfMemoryError: Direct buffer memory
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sindhushreem commented on issue #19334: bookkeeper is running out of direct memory
Posted by "sindhushreem (via GitHub)" <gi...@apache.org>.
sindhushreem commented on issue #19334:
URL: https://github.com/apache/pulsar/issues/19334#issuecomment-1447573245
> Hi @sindhushreem, thank you for raising this issue. Could you share your direct memory configuration and the heap dump if possible?
Hi Hang ,
I have shared the direct memory configuration and heap dump with @codelipenghui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bookkeeper is running out of direct memory [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #19334: bookkeeper is running out of direct memory
URL: https://github.com/apache/pulsar/issues/19334
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #19334: bookkeeper is running out of direct memory
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #19334:
URL: https://github.com/apache/pulsar/issues/19334#issuecomment-1447453947
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bookkeeper is running out of direct memory [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #19334:
URL: https://github.com/apache/pulsar/issues/19334#issuecomment-6386663035
Thanks @sindhushreem for the report. This was reported on Pulsar 2.8, which is no longer supported; the supported releases are 4.0.x, 4.2.x and 5.0.x. Much of the code involved has changed since, and the problem hasn't been confirmed on a current release, so I'm closing it as part of cleaning up old reports. If you still see it on a supported release, please open a new issue with the version and steps to reproduce, and link this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #19334: bookkeeper is running out of direct memory
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #19334:
URL: https://github.com/apache/pulsar/issues/19334#issuecomment-1492786715
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hangc0276 commented on issue #19334: bookkeeper is running out of direct memory
Posted by "hangc0276 (via GitHub)" <gi...@apache.org>.
hangc0276 commented on issue #19334:
URL: https://github.com/apache/pulsar/issues/19334#issuecomment-1407319897
Hi @sindhushreem, thank you for raising this issue. Could you share your direct memory configuration and the heap dump if possible?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org