You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@uniffle.apache.org by GitBox <gi...@apache.org> on 2022/09/20 09:05:03 UTC

[GitHub] [incubator-uniffle] wfxxh opened a new issue, #233: [Problem] The shuffle server memory not release

wfxxh opened a new issue, #233:
URL: https://github.com/apache/incubator-uniffle/issues/233

   uniffle version: 0.6.0, I deploy on k8s,but the shuffle server  memory not release ,even my spark application is stop it does not release too
   my server config is below:
   > rss.rpc.server.port 19999
   rss.jetty.http.port 19998
   rss.rpc.executor.size 2000
   rss.storage.type MEMORY_LOCALFILE_HDFS
   rss.coordinator.quorum 10.100.41.162:19999
   rss.server.disk.capacity 50g
   rss.storage.basePath /home/data
   rss.server.flush.thread.alive 1
   rss.server.flush.threadPool.size 10
   rss.server.buffer.capacity 4g
   rss.server.read.buffer.capacity 2g
   rss.server.heartbeat.timeout 60000
   rss.server.heartbeat.interval 10000
   rss.rpc.message.max.size 1073741824
   rss.server.preAllocation.expired 120000
   rss.server.commit.timeout 600000
   rss.server.app.expired.withoutHeartbeat 120000
   rss.server.flush.cold.storage.threshold.size 128m
   
   ![image](https://user-images.githubusercontent.com/22764286/191216608-f04adaca-81fd-4a73-919d-0ea36f0657bc.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] wfxxh commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

wfxxh commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253317315

   I have reduced it to 8G，but the pod restart too


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] wfxxh commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

wfxxh commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253125456

   But when the memory is full, the shuffle server pod restart,this case my spark application faild


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] jerqi commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

jerqi commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253127980

   Why do the shuffle server restart? There should be some information in the logs or stdout.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] jerqi commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

jerqi commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253466766

   > I have reduced it to 8G，but the pod restart too
   
   Does the server restart because of the same reason? You give the pod 32G memory, XMX_SIZE is 8G, don't it?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] wfxxh commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

wfxxh commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253254155

   XMX_SIZE ? it is 30G now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] jerqi commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

jerqi commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253173625

   > It is restart by k8s, reason is memory is too high.I think if memory release ,it will not be appear
   
   Maybe we should give more memory to the pod.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] wfxxh commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

wfxxh commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253238540

   It is 32G now,I can not give more 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] jerqi commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

jerqi commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253122848

   JVM can occupy the memory although they don't process any data.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] jerqi commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

jerqi commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253245355

   You can adjust the parameter of memory in the `bin/rss-env.sh` and conf/server.conf.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] wfxxh commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

wfxxh commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253134208

   It is restart by k8s, reason is memory is too high.I think if memory release ,it will not be appear


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] wfxxh commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

wfxxh commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253238279

   It is 32G now,I can not give more 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-uniffle] jerqi commented on issue #233: [Problem] The shuffle server memory not release

Posted by GitBox <gi...@apache.org>.

jerqi commented on issue #233:
URL: https://github.com/apache/incubator-uniffle/issues/233#issuecomment-1253309375

   > XMX_SIZE ? it is 30G now.
   
   Could you reduce the value?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org