[GitHub] [beam] kennknowles opened a new issue, #19260: Too many open files due grpc channel not being closed (direct runnner Pubsub source)

[GitBox <gi...@apache.org>]

kennknowles opened a new issue, #19260:
URL: https://github.com/apache/beam/issues/19260

   When '_read_from_pubsub' (runnners/direct/transform_evaluator.py:448) is being invoked, it will create a Pubsub SubscriberClient. This will create a grpc channel which is not being closed. When you run a pipeline with the direct runner which has a pubsub source in it, it will run eventually into an error: "too many open files". This is because these grpc channels are never being closed.
   
   The suggestion is to wrap the code after the creation of the client in a try/finally block and close the channel (exposed on sub_client.api.transport._channel) in the finally block.
   
    
   
   Some extra info about the issue: https://github.com/googleapis/google-cloud-python/issues/5523
   
   Imported from Jira [BEAM-6708](https://issues.apache.org/jira/browse/BEAM-6708). Original Jira may contain additional context.
   Reported by: Boenders.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org