[GitHub] [incubator-pekko-connectors] jtjeferreira commented on pull request #226: kinesis: use stage materializer with IODispatcher instead of injected EC

["jtjeferreira (via GitHub)" <gi...@apache.org>]

jtjeferreira commented on PR #226:
URL: https://github.com/apache/incubator-pekko-connectors/pull/226#issuecomment-1686541161

   > a proper `executionContext` which is explicitly designed for these IO blocking operations
   
   I agree that using `IODispatcher` would be an improvement from what we have now. However, do I understand correctly that `IODispatcher` is a [ThreadPool with fixed 16 threads](https://github.com/apache/incubator-pekko/blob/580f12c29fb61b65758af9b0c31494af0af17175/actor/src/main/resources/reference.conf#L573-L581)? 
   
   Then if a user wants to consume from more than 16 kinesis streams (i.e creates more than 16 KinesisSchedulerSource), then the thread pool is fully utilized, creating a deadlock. That's why I opted for creating a thread just for running the kinesis Scheduler, as in the example [documentation](https://docs.aws.amazon.com/streams/latest/dev/kcl2-standard-consumer-java-example.html) from AWS.
   
   That said, if you are not comfortable with this solution I can implement what you are suggesting with `IODispatcher`
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org