You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bookkeeper.apache.org by GitBox <gi...@apache.org> on 2022/01/13 06:57:12 UTC
[GitHub] [bookkeeper] Shoothzj opened a new pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Shoothzj opened a new pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989
### Motivation
Our `DirectMemoryUtils` has huge limit, it can't work well with other jvm. The Netty `PlatformDependent.maxDirectMemory();` is more generic.
### Changes
Use `PlatformDependent.maxDirectMemory();` instead of `DirectMemoryUtils`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] eolivelli commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1040496345
What about sending a PR to Netty in order to expose such value?
Netty project is very open to contributions, I sent some PRs in the past.
They also have a very frequent release cycle.
I would prefer to not copy paste, because in Netty they are maintaining that code and adapt it to new Java versions, and we will keep this version forever
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1013140800
rerun failure checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1040255189
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1016163989
>
@dlg99 Hi, We have a bookkeeper server, which is conifugred as "-Xmx6G -XDirectMemory4G". And we are using huawei jdk11. `DirectMemoryUtils` can't reed the `sun.misc.VM. maxDirectMemory`, fall back to the heap memory, which is 6G beyond the real `-XDirectMemory`.
Instead of enhace `DirectMemoryUtils`, we can use `PlatformDependent.maxDirectMemory()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1048628431
@eolivelli @dlg99 https://github.com/netty/netty/pull/12118 is merged. I will reopen this PR when we update to netty next version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1037676747
@dlg99 You are right. set `io.netty.maxDirectMemory` greater than zero might not work.
So we can enhance the `DirectMemoryUtils` with netty's code(They are Apache LICENSE too)
PTAL, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] dlg99 commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
dlg99 commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1017175199
Just switching to PlatformDependent.maxDirectMemory() feels like a misuse of it plus it might not work for some people if they e.g. set `io.netty.maxDirectMemory` greater than zero.
I'd try something like a configurable strategy for the DirectMemoryUtils to specify how to read the max memory (via Oracle JD specific methods, via Huawei's, via config parameter, etc)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj removed a comment on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj removed a comment on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1013140800
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1014437010
@eolivelli @dlg99 @merlimat @pkumar-singh PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1086821503
@eolivelli @dlg99 @merlimat @pkumar-singh PTAL again. **Netty** has exposed a util method for user to get maxDirectoryMemory. See https://github.com/netty/netty/pull/12118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj removed a comment on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj removed a comment on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1014176943
rerun failure checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1040926377
@eolivelli Thanks for your advice, I will try
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj removed a comment on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj removed a comment on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1014176943
rerun failure checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1011862574
rerun failure checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1037676747
@dlg99 You are right. set `io.netty.maxDirectMemory` greater than zero might not work.
So we can enhance the `DirectMemoryUtils` with netty's code(They are Apache LICENSE too)
PTAL, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1040414499
> Did you copy paste the code from Netty?
Yes, I did it because that these are more genernal, and netty is under apache license.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] eolivelli commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1040496345
What about sending a PR to Netty in order to expose such value?
Netty project is very open to contributions, I sent some PRs in the past.
They also have a very frequent release cycle.
I would prefer to not copy paste, because in Netty they are maintaining that code and adapt it to new Java versions, and we will keep this version forever
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1040255189
rerun failure checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Shoothzj commented on pull request #2989: Use netty maxDirectMemory instead of DirectMemoryUtils
Posted by GitBox <gi...@apache.org>.
Shoothzj commented on pull request #2989:
URL: https://github.com/apache/bookkeeper/pull/2989#issuecomment-1014176943
rerun failure checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org