[GitHub] [skywalking-python] tom-pytel commented on pull request #149: Enable HTTP Kafka log reporting

[GitBox <gi...@apache.org>]

tom-pytel commented on pull request #149:
URL: https://github.com/apache/skywalking-python/pull/149#issuecomment-896221383


   > But, I'm not sure if the `oap/v3/logs` endpoint is for such usage(It seems designed for fluent-bit batch reporting?). Reporting logs one by one through HTTP may not be ideal in terms of performance. I'm not sure whether the Java agent only implements gRPC reporter intentionally out of this reason.
   
   Not sure what the INTENDED usage of that endpoint was, but the same individual send inefficiency applies to how the spans are sent currently to `/v3/segment` instead of batching to `/v3/segments`, this should be optimized at some point. The hit is not that bad though since the http protocol uses `requests.Session` which should do persistent connections as per https://docs.python-requests.org/en/master/user/advanced/:
   
   "The Session object allows you to persist certain parameters across requests. It also persists cookies across all requests made from the Session instance, **and will use urllib3’s connection pooling. So if you’re making several requests to the same host, the underlying TCP connection will be reused**, which can result in a significant performance increase (see HTTP persistent connection)."
   
   In any case, if it works but may not yet be optional it is a step forward.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org