You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@linkis.apache.org by "sjgllgh (via GitHub)" <gi...@apache.org> on 2023/09/26 03:21:50 UTC

[GitHub] [linkis] sjgllgh opened a new pull request, #4917: [BUG] when submitting tasks in batches, em resources may exceed the threshold

sjgllgh opened a new pull request, #4917:
URL: https://github.com/apache/linkis/pull/4917

   ### What is the purpose of the change
   
   Fix bug: when submitting tasks in batches, em resources may exceed the threshold.
   
   ### Related issues/PRs
   
   Related issues: #4916 
   
   
   ### Brief change log
   
   - Modify the DefaultResourceManager.java to fix the bug.
   
   
   ### Checklist
   
   - [x] I have read the [Contributing Guidelines on pull requests](https://github.com/facebook/docusaurus/blob/main/CONTRIBUTING.md#pull-requests).
   - [x] I have explained the need for this PR and the problem it solves
   - [x] I have explained the changes or the new features added to this PR
   - [x] I have added tests corresponding to this change
   - [x] I have updated the documentation to reflect this change
   - [x] I have verified that this change is backward compatible (If not, please discuss on the [Linkis mailing list](https://linkis.apache.org/community/how-to-subscribe) first)
   - [x] **If this is a code change**: I have written unit tests to fully verify the new behavior.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org


[GitHub] [linkis] sjgllgh closed pull request #4917: [BUG] when submitting tasks in batches, em resources may exceed the threshold

Posted by "sjgllgh (via GitHub)" <gi...@apache.org>.
sjgllgh closed pull request #4917: [BUG] when submitting tasks in batches, em resources may exceed the threshold
URL: https://github.com/apache/linkis/pull/4917


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org


[GitHub] [linkis] peacewong commented on pull request #4917: [BUG] when submitting tasks in batches, em resources may exceed the threshold

Posted by "peacewong (via GitHub)" <gi...@apache.org>.
peacewong commented on PR #4917:
URL: https://github.com/apache/linkis/pull/4917#issuecomment-1735548482

   The logical judgment here is considered as follows:
   1. The ECM tag is not locked first for resource judgment. If the resource is judged, it will continue. Therefore, there is a certain probability of negative resources, but there will not be too many situations.
   2. The reason for this consideration is to reduce lock waiting, because here is a concurrent lock grab, so the lock granularity should be as small as possible to reduce concurrency pressure.
   3. If the usercreator lock is placed at a higher level, it will cause more lock waits.
   
   这里的逻辑判断是这样考虑的:
   1. ECM标签先不锁进行资源判断如果资源判断过就继续,所以存在一定概率出现负资源,但是不会出现太多情况
   2. 之所以这样考虑是为了减少锁的等待,因为这里是并发抢锁的,所以锁的粒度尽量小减少并发压力
   3. 如果把usercreator的锁放到更上一层,会导致更多的锁等待


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org


[GitHub] [linkis] sjgllgh commented on pull request #4917: [BUG] when submitting tasks in batches, em resources may exceed the threshold

Posted by "sjgllgh (via GitHub)" <gi...@apache.org>.
sjgllgh commented on PR #4917:
URL: https://github.com/apache/linkis/pull/4917#issuecomment-1736520234

   Thank you for your answer. Different scenarios may have different considerations. 
   
   > The logical judgment here is considered as follows:
   > 
   > 1. The ECM tag is not locked first for resource judgment. If the resource is judged, it will continue. Therefore, there is a certain probability of negative resources, but there will not be too many situations.
   > 2. The reason for this consideration is to reduce lock waiting, because here is a concurrent lock grab, so the lock granularity should be as small as possible to reduce concurrency pressure.
   > 3. If the usercreator lock is placed at a higher level, it will cause more lock waits.
   > 
   > 这里的逻辑判断是这样考虑的:
   > 
   > 1. ECM标签先不锁进行资源判断如果资源判断过就继续,所以存在一定概率出现负资源,但是不会出现太多情况
   > 2. 之所以这样考虑是为了减少锁的等待,因为这里是并发抢锁的,所以锁的粒度尽量小减少并发压力
   > 3. 如果把usercreator的锁放到更上一层,会导致更多的锁等待
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org