[GitHub] [pulsar] zbentley commented on issue #11953: C++ client (via Python bindings) `abort`s and hangs in child processes and is generally not `fork`-safe

[GitBox <gi...@apache.org>]

zbentley commented on issue #11953:
URL: https://github.com/apache/pulsar/issues/11953#issuecomment-914572210


   As a workaround, an `atfork` handler can be registered that intentionally swallows the error that occurs on shutdown.
   
   An example demonstrating this is here (the weakref parts aren't necessary for the demo, but they do prevent client instances from being kept away from GC by the atfork callback):
   ```python
   import os
   from functools import partial
   import weakref
   
   import pulsar
   
   
   def _shutdown_suppress_error(inst):
       inst = inst()
       if inst is not None:
           try:
               inst._client.shutdown()
           except RuntimeError as ex:
               if 'thread::detach failed: No such process' not in str(ex):
                   raise
   
   
   class ClientWrapper(pulsar.Client):
       def __init__(self, *args, **kwargs):
           super(ClientWrapper, self).__init__(*args, **kwargs)
           os.register_at_fork(after_in_child=partial(_shutdown_suppress_error, weakref.ref(self)))
   
   
   def main():
       cl = ClientWrapper(
           'pulsar://127.0.0.1:6650',
           operation_timeout_seconds=1,
           connection_timeout_ms=1000,
       )
       prod = cl.create_producer('foobar', send_timeout_millis=1000)
       prod.send(b'1')
       pid = os.fork()
       if pid == 0:  # Child
           # Shutdown is idempotent; since the first attempt's error was swallowed in the atfork handler, this one
           # succeeds.
           cl._client.shutdown()
           print("Child exiting")
       else:  # Parent
           prod.send(b'2')
           os.wait()
           prod.send(b'3')
   
   
   if __name__ == '__main__':
       main()
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org