You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2021/11/18 12:43:46 UTC

[GitHub] [skywalking] wu-sheng opened a new issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

wu-sheng opened a new issue #8145:
URL: https://github.com/apache/skywalking/issues/8145


   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Description
   
   I noticed in the showcase(also demo env), we always face `oap::` services, which mean no actual service name.
   I finally got a chance to confirm with @kezhenxu94 why. It turns out `oap`'s self-observability and `vm`'s monitoring.
   
   We need a mechanism to identify the source of a SampleFamily to avoid this case. I would like to propose these kind of mechanism.
   
   We add a field precondition field into all MAL related configuration files, including `oc-receiver`, `envoy-metrics-receiver`, `meter-receiver` etc. This field should be a closure returning true/false. When `false` is returned, this SampleFamily set isn't pushed into rules of all including expressions, but simply skip.
   
   @hanahmily Could you confirm this mechanism makes sense?
   
   ### Use case
   
   This feature could avoid empty service issue, and generally it increase the performance of MAL engine, because less calculation are applied by the input metrics set.
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] kezhenxu94 closed issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
kezhenxu94 closed issue #8145:
URL: https://github.com/apache/skywalking/issues/8145


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wankai123 edited a comment on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wankai123 edited a comment on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972857255


   We can get otel `job_name` from `node.getServiceInfo()`, then could add a tag like `job_name=skywalking` to all metrics from skywalking.
   I think we can use this to do the filter.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng edited a comment on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wu-sheng edited a comment on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972903345


   @kezhenxu94 @hanahmily @wankai123 Another point we should consider is, once the names of entity(such as service name for service, service/instance names for instance) are empty, we should skip this metric before forward it to streaming.
   
   This is not for fixing this case, using OTEL to add a job label seems more reliable and official, but we need to filter some illegal or unexpected data.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972837854


   FYI @wankai123 due to you asked about this in afternoon.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972903345


   @kezhenxu94 @hanahmily @wankai123 Another point we should consider is, once the names of entity(such as service name for service, service/instance names for instance) are empty, we should skip this metric before forward it to streaming.
   
   This is not for fixing this case, @wankai123 's job label seems more reliable and official, but we need to filter some illegal or unexpected data.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wankai123 commented on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wankai123 commented on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972912182


   one more thing...  
   the filter rule should consider one MAL config need multiple metrics source(otel job), like `k8s_service` require both `cAdvisor ` and `kube-state-metrics` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972927931


   > the filter rule should consider one MAL config need multiple metrics source(otel job), like k8s_service require both cAdvisor and kube-state-metrics
   
   That is why I proposed `precondition` as closure, then you could do `either` check, rather than struggling about using `at least having` or `includes`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972887564


   For making understanding easier, @hanahmily you could consider like this, MetricA from VM monitoring uses the same name SkyWalking OAP self-monitoring is using. Changing any side of these two doesn't make sense.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wankai123 commented on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wankai123 commented on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972857255


   We can get otel `job_name` from `node.getServiceInfo()`, then could add a tag like `job_name=skywalking` to all metrics.
   I think we can use this to do the filter.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972885242


   No, they are not redundant. 2 kinds of agent/source(s) are generating the metrics with same name.
   
   **oap's self-observability and vm's monitoring have the same metric name**. More importantly, this could happen again in other monitoring cases.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972866081


   Even we use a tag filter, we need the closure setup to filter irrelative metrics out.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] hanahmily commented on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
hanahmily commented on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972883284


   MAL tends to generate the metrics that are mandatory. If the metrics are invalid, related MALs are redundant, having to be removed from the configuration.
   
   What leads to such issues? Can we avoid them from the source? If their service tags are absent, could we have a chance to provide them? If not, I prefer to drop them.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng edited a comment on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wu-sheng edited a comment on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972903345


   @kezhenxu94 @hanahmily @wankai123 Another point we should consider is, once the names of entity(such as service name for service, service/instance names for instance) are empty, we should skip this metric before forward it to streaming.
   
   This is not for fixing this case, using OTEL to add a job label seems more reliable and official, but we need to filter some illegal or unexpected data. For OC receiver, we also could add `job_name` at the receiver side as label as well.
   
   ![image](https://user-images.githubusercontent.com/5441976/142436231-34a3b5bf-2ae8-4d85-afa5-c626f5560cb3.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wankai123 commented on issue #8145: [Feature] [MAL Core] Add a match pattern mechanism for the MAL core

Posted by GitBox <gi...@apache.org>.
wankai123 commented on issue #8145:
URL: https://github.com/apache/skywalking/issues/8145#issuecomment-972869568


   yes, the metrics should filter before into MAL rules


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org