[GitHub] [samza] shekhars-li opened a new pull request, #1644: Fix perfomance bug

[GitBox <gi...@apache.org>]

shekhars-li opened a new pull request, #1644:
URL: https://github.com/apache/samza/pull/1644

   Bug
   - We introduced blob store backed for state backup and restore in [this commit](https://github.com/apache/samza/commit/7cc4eaa96fff244f6dce9c18af804917db7c3b2b).
   - `StateBackendFactory` implementations like for Kafka and BlobStore create systemAdmins every time `getBackupFactory` is called from `SamzaContainer`. This leads to creation of duplicate kafka admin threads. 
   - This has impact on performance of large samza jobs as we have thousands of duplicate threads spawned.
   
   Fix
   - Pass `systemAdminsMap` to `getBackendFactory` method. This initializes the SystemAdmins with the system admin map rather than creating system admins every time the method is called. 
   
   Test
   - Tested with local jobs to verify the kafka admin threads are not duplicated. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [samza] dxichen merged pull request #1644: Fix performance bug in kafka/blobstore commit lifecycle.

[GitBox <gi...@apache.org>]

dxichen merged PR #1644:
URL: https://github.com/apache/samza/pull/1644


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org