[GitHub] [skywalking-python] tom-pytel opened a new pull request #157: exponential reconnection backoff

[GitBox <gi...@apache.org>]

tom-pytel opened a new pull request #157:
URL: https://github.com/apache/skywalking-python/pull/157


   Added generic hackish incrementing reconnect timeout on consecutive errors to avoid constant reconnect attempts from grpc if OAP server is down as mentioned in #138. This is a little ugly because each thread has its own timeout so all threads will attempt to reconnect individually after the timeout (no concept of a connection), but is not so bad with current maximum timeout of 1 min to have a few threads try to reconnect for now.
   
   Also this will handle any other unexpected exception which may happen during send for any reason, will prevent them from erroring constantly and eating 100% cpu and will enforce a delay between attempts if there is some problem.
   
   Also changed how timeout start is handled in report generator functions. Previously there was a very small but non-zero chance that if for some reason it took longer than 1 second to get from:
   ```py
   start = time()
   ```
   to:
   ```py
   timeout = config.QUEUE_TIMEOUT - int(time() - start)
   ```
   in the first pass through the loop then a send may not have been attempted at all. Now it will always send the first time through the loop.
   
   <!-- Uncomment the following checklist WHEN AND ONLY WHEN you're adding a new plugin -->
   <!--
   - [ ] Add a test case for the new plugin
   - [ ] Add a component id in [the main repo](https://github.com/apache/skywalking/blob/master/oap-server/server-bootstrap/src/main/resources/component-libraries.yml#L415)
   - [ ] Add a logo in [the UI repo](https://github.com/apache/skywalking-rocketbot-ui/tree/master/src/views/components/topology/assets)
   - [ ] Rebuild the `requirements.txt` by running `tools/env/build_requirements_(linux|windows).sh`
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking-python] kezhenxu94 merged pull request #157: exponential reconnection backoff

[GitBox <gi...@apache.org>]

kezhenxu94 merged pull request #157:
URL: https://github.com/apache/skywalking-python/pull/157


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org