You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/07/05 13:47:49 UTC

[GitHub] [beam] steveniemitz opened a new issue, #22161: [Bug]: Significant scalability bottlenecks at high throughput due to coarse-grained locking

steveniemitz opened a new issue, #22161:
URL: https://github.com/apache/beam/issues/22161

   ### What happened?
   
   There are several spots in the critical (streaming) bundle execution path that serialize execution of all threads.  Importantly are:
   - `ExecutionStateSampler.add/removeTracker`
   - `ProxyInvocationHandler.as`
   - `DoFnSignatures.getSignature`
   - `ByteBuddyDoFnInvokerFactory.getByteBuddyInvokerConstructor`
   
   A JFR recording of a typical streaming load shows this:
   ![image](https://user-images.githubusercontent.com/1882981/177342493-008e0c79-bbe1-4b6f-80d0-8b8969d228b8.png)
   
   These functions should be optimized to not require a coarse-grained lock when executing.
   
   ### Issue Priority
   
   Priority: 2
   
   ### Issue Component
   
   Component: runner-core


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] steveniemitz commented on issue #22161: [Bug]: Significant scalability bottlenecks at high throughput due to coarse-grained locking

Posted by GitBox <gi...@apache.org>.
steveniemitz commented on issue #22161:
URL: https://github.com/apache/beam/issues/22161#issuecomment-1175083592

   .take-issue
   .remove-labels 'awaiting-triage'


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] github-actions[bot] closed issue #22161: [Bug]: Significant scalability bottlenecks at high throughput due to coarse-grained locking

Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #22161: [Bug]: Significant scalability bottlenecks at high throughput due to coarse-grained locking
URL: https://github.com/apache/beam/issues/22161


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] steveniemitz commented on issue #22161: [Bug]: Significant scalability bottlenecks at high throughput due to coarse-grained locking

Posted by GitBox <gi...@apache.org>.
steveniemitz commented on issue #22161:
URL: https://github.com/apache/beam/issues/22161#issuecomment-1185063820

   .close-issue


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] steveniemitz commented on issue #22161: [Bug]: Significant scalability bottlenecks at high throughput due to coarse-grained locking

Posted by GitBox <gi...@apache.org>.
steveniemitz commented on issue #22161:
URL: https://github.com/apache/beam/issues/22161#issuecomment-1175083838

   .take-issue


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on issue #22161: [Bug]: Significant scalability bottlenecks at high throughput due to coarse-grained locking

Posted by GitBox <gi...@apache.org>.
lukecwik commented on issue #22161:
URL: https://github.com/apache/beam/issues/22161#issuecomment-1184953007

   I think the above two PRs handled this, should we close this now?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org