You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shenyu.apache.org by GitBox <gi...@apache.org> on 2022/12/16 15:48:06 UTC
[GitHub] [shenyu] khuangz opened a new issue, #4274: [Question] proxy for file upload run a while will throw OutOfDirectMemoryError
khuangz opened a new issue, #4274:
URL: https://github.com/apache/shenyu/issues/4274
### Question
any suggestion to prevent this issue？
here is the error stack
`Caused by: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 4194304 byte(s) of direct memory (used: 2143289351, max: 2147483648)
at io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:806)
at io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:735)
at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:649)
at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:624)
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:203)
at io.netty.buffer.PoolArena.tcacheAllocateNormal(PoolArena.java:187)
at io.netty.buffer.PoolArena.allocate(PoolArena.java:136)
at io.netty.buffer.PoolArena.allocate(PoolArena.java:126)
at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:396)
at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)
at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
at io.netty.channel.unix.PreferredDirectByteBufAllocator.ioBuffer(PreferredDirectByteBufAllocator.java:53)
at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:120)
at io.netty.channel.epoll.EpollRecvByteAllocatorHandle.allocate(EpollRecvByteAllocatorHandle.java:75)
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:785)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:487)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:385)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:750)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shenyu.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shenyu] Zhoutzzz commented on issue #4274: [Question] proxy for file upload run a while will throw OutOfDirectMemoryError
Posted by GitBox <gi...@apache.org>.
Zhoutzzz commented on issue #4274:
URL: https://github.com/apache/shenyu/issues/4274#issuecomment-1356239312
You need to provide more information about this issus, like:
how big is this file.
how big is the jvm heap.
how much does you to used plugin.
This information needs to be related to the environment when you run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shenyu.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shenyu] yu199195 closed issue #4274: [Question] proxy for file upload run a while will throw OutOfDirectMemoryError
Posted by "yu199195 (via GitHub)" <gi...@apache.org>.
yu199195 closed issue #4274: [Question] proxy for file upload run a while will throw OutOfDirectMemoryError
URL: https://github.com/apache/shenyu/issues/4274
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shenyu.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shenyu] iwangjie commented on issue #4274: [Question] proxy for file upload run a while will throw OutOfDirectMemoryError
Posted by "iwangjie (via GitHub)" <gi...@apache.org>.
iwangjie commented on issue #4274:
URL: https://github.com/apache/shenyu/issues/4274#issuecomment-1454545651
This error is caused by running out of direct memory when trying to allocate memory for uploading a file through a proxy. The Java Virtual Machine (JVM) uses direct memory for ByteBuffer objects used by Netty, a popular networking library used for building high-performance network applications. Direct memory is memory outside of the Java heap and is allocated using the operating system's memory management system.
The error message indicates that the JVM has used up almost all of its available direct memory, leaving only 4 MB available, which is not enough to allocate the memory needed for the file upload. The maximum amount of direct memory available to the JVM is 2 GB (2147483648 bytes).
To fix this error, you can try increasing the maximum amount of direct memory available to the JVM by setting the -XX:MaxDirectMemorySize JVM option. For example, you can set this option to 4 GB by adding the following to your JVM arguments:
``` shell
-XX:MaxDirectMemorySize=4g
```
This will increase the maximum amount of direct memory available to the JVM to 4 GB, which should provide enough memory for the file upload.
Alternatively, you can try reducing the amount of direct memory used by your application by optimizing your code or reducing the size of the files being uploaded. You can also try using a different networking library that uses less direct memory or that allows you to configure the amount of direct memory used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shenyu.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shenyu] khuangz commented on issue #4274: [Question] proxy for file upload run a while will throw OutOfDirectMemoryError
Posted by GitBox <gi...@apache.org>.
khuangz commented on issue #4274:
URL: https://github.com/apache/shenyu/issues/4274#issuecomment-1356677935
>
below is the jvm setting, we use apache benchmarking to test upload file with file size from 1K to 10M
`java -server -Xmx4g -Xms4g -Xmn1g -Xss512k -XX:MaxDirectMemorySize=2g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/heapdump.hprof -XX:LargePageSizeInBytes=128m -Dshenyu.extPlugin.scheduleTime=3600 -Dplugin-ext=./ext-lib -Dspring.jmx.enabled=true -Djava.rmi.server.hostname=*** -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=8099 -Dcom.sun.management.jmxremote.rmi.port=8099 -Dcom.sun.management.jmxremote.access.file=/root/gateway-soap-test/gateway-bootstrap/jmxremote.access -Dcom.sun.management.jmxremote.password.file=/root/gateway-soap-test/gateway-bootstrap/jmxremote.password -Dcom.sun.management.jmxremote.ssl=false -Dspring.config.location=/root/gateway-soap-test/gateway-bootstrap/bootstrap.yml -Dshenyu.work.threads=1024 -jar /root/gateway-soap-test/gateway-bootstrap/tpass-gateway-soap.jar`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shenyu.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org