You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2020/12/01 09:11:38 UTC

[GitHub] [skywalking] wu-sheng opened a new issue #5930: Log collecting, analysis, and query

wu-sheng opened a new issue #5930:
URL: https://github.com/apache/skywalking/issues/5930


   Hi, as we are establishing the [Satellite](https://github.com/apache/skywalking-satellite), log collecting and analysis should be built on the OAP server too.
   
   I have submitted the [data collecting service](https://github.com/apache/skywalking-data-collect-protocol/pull/35), once it is approved, we could begin to build a new receiver(`log-receiver`) to provide backend features.
   In the log analysis, I propose to add the following features
   1. gRPC and HTTP format should be provided by following the proto definition.
   1. `service` in the reported data should support auto-filling if it is null. If the no previous service name, the log should be ignored.
   1. `timestamp` could be reported by the client-side or use the analysis component to read from the text.
   1.  Provide different analyzers for different format logs. Every analyzer implementation could have its own expression definition to read the data from the logs, such as for instance name, tags, traceid
   
   Query GraphQL definition will be submitted later, and ask UI side to confirm.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #5930: Log collecting, analysis, and query

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #5930:
URL: https://github.com/apache/skywalking/issues/5930#issuecomment-737683058


   Query protocol has been polished through https://github.com/apache/skywalking-query-protocol/pull/44.
   
   Take a look at this, https://github.com/apache/skywalking-query-protocol/blob/master/log.graphqls
   
   @arugal will lead this feature at the backend, @Fine0830 will follow up at the UI side after the backend is ready. This would be the core feature of 8.4.0. 
   
   FYI @dmsolr, if you have time, we could consider looking deeper into the OpenTelemetry collector log report format(gRPC), this would be a very useful integration and as an alternative with our Satellite.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] EvanLjp edited a comment on issue #5930: Log collecting, analysis, and query

Posted by GitBox <gi...@apache.org>.
EvanLjp edited a comment on issue #5930:
URL: https://github.com/apache/skywalking/issues/5930#issuecomment-736335750


   Do we need to plan for split storage? Log is larger. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] EvanLjp commented on issue #5930: Log collecting, analysis, and query

Posted by GitBox <gi...@apache.org>.
EvanLjp commented on issue #5930:
URL: https://github.com/apache/skywalking/issues/5930#issuecomment-736335750


   Do we need to plan for split storage? Logs have to be large. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #5930: Log collecting, analysis, and query

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #5930:
URL: https://github.com/apache/skywalking/issues/5930#issuecomment-736339427


   That is another topic, I have no plan to add it for now.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng closed issue #5930: Log collecting, analysis, and query

Posted by GitBox <gi...@apache.org>.
wu-sheng closed issue #5930:
URL: https://github.com/apache/skywalking/issues/5930


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org