You are viewing a plain text version of this content. The canonical link for it is here.

Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2021/01/02 01:53:43 UTC

[GitHub] [skywalking] wu-sheng opened a new issue #6118: Adopt Zabbix protocol

wu-sheng opened a new issue #6118:
URL: https://github.com/apache/skywalking/issues/6118


   Zabbix is a traditional and widely used metrics monitoring mechanism. We should consider to adopt its network protocol, and convert to our metrics system. 
   All @apache/skywalking-committers , this requirement will need more discussion on tech details. If any of you feels interested in this, please reply.
   
   NOTE, Zabbix is GPL licensed, but because we don't copy any codes, just implement its protocol in our own codes, and optional feature only. it is not an issue.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] mrproliu commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

mrproliu commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753486437

I have installed a simple Zabbix agent and server demo in the local machine, and using Wireshark to monitor the network base on `Active Checks Mode`. It will help to debug and watch data locally. I uploaded this [pcapng file](https://github.com/apache/skywalking/files/5760438/dump.pcapng.zip), you can view the specific data by downloading the `Wireshark` software.

The metrics key in Zabbix follow this format: `keyName[params]`, for more specifications, please refer to https://www.zabbix.com/documentation/current/manual/config/items/item/key

Let's take the key of `system.cpu.load[all,avg15]` as an example, and divide it into two parts: `Key Name` and `Parameter`:

#### Key Name
In the above example, it is `system.cpu.load`.
The metrics key in Zabbix is not particularly compatible with our MAL. When MAL parses, it will consider `system` as a metric and cannot find the `cpu` property. Zabbix does not provide a similar formatting key function, such as `system.cpu.load -> system_cpu_load`.

So I thought of two ways to solve it:
1. Users manually replace the "." with the "_" symbol when inputting expressions. In the above example, the expression of MAL must be manually replaced with `system_cpu_load`. When the backend receives the data and prepares to convert the MAL, it will also change the `.` -> `_` of the key in the metrics.
2. The user does not need to change the name of the metrics, it is solved by extending the function of MAL. When MAL cannot find `SampleFamily`, a new `SampleFamilyFinder` is provided to solve it. In `SampleFamilyFinder`, it can be solved by implementing the `propertyMissing` method.

#### Parameter
In the above example, it is `all,avg15`

We can convert it into the form of multiple tag values, and identify the index as a tag key. In the above data, it will become `{0: all, 1: avg15}`.
At this time, you can filter by `.tagEqual('0','all').tagEqual('1','avg15')` in MAL.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

wu-sheng commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753443808


   I see. Let's see how the metrics really look like, whether to be compatible with our MAL, then, we could set up the tech stack for TCP connection this time.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

wu-sheng commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753423041


   Doesn't Zabbix agent use push mode? The idea is replacing the Zabbix server, rather than reading data from it.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] mrproliu commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

mrproliu commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753425235


   > Doesn't Zabbix agent use push mode? The idea is replacing the Zabbix server, rather than reading data from it.
   
   Do you mean using [Sender](https://www.zabbix.com/documentation/current/manual/concepts/sender)? It seems like sending to the Zabbix proxy. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] mrproliu commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

mrproliu commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753578393


   > I think we don't have to add prefix like `zabbix:` forcely, we have the group concept on UI, which could be declared through the configuration. Check the oc-receiver rules.
   
   Of course, add a new configuration item named expSuffix, and add the tags service and service instance to each metric.
   
   > 
   > And many metrics of Zabbix should follow this #6108, which is also focusing on the VM metrics.
   
   Sure, I will following this issue, then I could add metrics into VM.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng closed issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

wu-sheng closed issue #6118:
URL: https://github.com/apache/skywalking/issues/6118


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

wu-sheng commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753487751


   For MAL details, @hanahmily what do you think? Personally, `.` -> `_` as an auto mechanism is fine.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] mrproliu commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

mrproliu commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753578324


   > I think we don't have to add prefix like `zabbix:` forcely, we have the group concept on UI, which could be declared through the configuration. Check the oc-receiver rules.
   Of course, add a new configuration item named `expSuffix`, and add the tags `service` and `service instance` to each metric.
   > 
   > And many metrics of Zabbix should follow this #6108, which is also focusing on the VM metrics.
   Sure, I will following this issue, then I could add metrics into `VM`.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] mrproliu commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

mrproliu commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753561680

I thought about the format of the configuration file. The configuration can also be placed in the same directory and can be distinguished by files based on different metrics.

```yaml
entities: # All entities support the metrics
- service: test_service # Combine all hosts, adding prefix "zabbix:"
instance-pattern: .+ # All hosts will support keys, adding prefix "zabbix:"
metrics:
- name: system_cpu_util # Save to meter system name, adding prefix "meter_"
keys: # Define need receive data keys from agent, all key update limit is `1m`, support number value only
- system.cpu.util[,system]
- system.cpu.util[,user]
- system.cpu.util[,iowait]
- system.cpu.util[,idle]
exp: system.cpu.util # MAL expression
```

When the agent asks which metrics keys need to be monitored, it will obtain all `metrics.keys` according to `entities.instance-pattern`. Also:
1. keys that are repeatedly declared will be deduplicated before sending.
2. If `entities.instance-pattern` is declared in multiple configuration files, then `metrics.keys` among them will tell the agent to query.

When pushing data content on the agent side, the data will also be aggregated into the `entities.service` service according to the specified `host`. The data content is generated based on all `metrics.exp` and `metrics.keys`.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

wu-sheng commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753571607


   I think we don't have to add prefix like `zabbix:` forcely, we have the group concept on UI, which could be declared through the configuration. Check the oc-receiver rules.
   
   And many metrics of Zabbix should follow this #6108, which is also focusing on the VM metrics.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] mrproliu edited a comment on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

mrproliu edited a comment on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753561680

I thought about the format of the configuration file. The configuration can also be placed in the same directory and can be distinguished by files based on different metrics.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] mrproliu removed a comment on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

mrproliu removed a comment on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753578324


   > I think we don't have to add prefix like `zabbix:` forcely, we have the group concept on UI, which could be declared through the configuration. Check the oc-receiver rules.
   Of course, add a new configuration item named `expSuffix`, and add the tags `service` and `service instance` to each metric.
   > 
   > And many metrics of Zabbix should follow this #6108, which is also focusing on the VM metrics.
   Sure, I will following this issue, then I could add metrics into `VM`.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] mrproliu commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

mrproliu commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753422754


   Using `zabbix:` prefix to build their service or service instance name. separate with agent side service and service instance name.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] mrproliu commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

mrproliu commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753420809

I have started reading their API document a few days, we need to using [Host](https://www.zabbix.com/documentation/current/manual/api/reference/host/get), [Host Group](https://www.zabbix.com/documentation/current/manual/api/reference/hostgroup/get), and [Item](https://www.zabbix.com/documentation/current/manual/api/reference/item/get).
Follow the API document, I think we need a new `fetcher` to process all the Zabbix hosts and metrics data.

At the `fetcher` processing, we need to follow these steps.

1. Creating a new schedule task to query their API, similar to Prometheus fetcher.
1. User needs to provide their Zabbix user account at the configuration, so we could access their API after [login](https://www.zabbix.com/documentation/current/manual/api/reference/user/login).
2. User needs to provide metrics conversion mapping, such as `MAL`. We need to use item API to get all these host metrics data and convert them to the meter system.
3. Using host group and host API to get The necessary data, to build the service or service instance data.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

wu-sheng commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753425810


   Take a look at https://www.zabbix.com/documentation/current/manual/introduction/overview
   
   SkyWalking OAP should be a replacement of Zabbix server to support Zabbix proxy and agents. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] mrproliu commented on issue #6118: Adopt Zabbix protocol

Posted by GitBox <gi...@apache.org>.

mrproliu commented on issue #6118:
URL: https://github.com/apache/skywalking/issues/6118#issuecomment-753427307


   > Take a look at https://www.zabbix.com/documentation/current/manual/introduction/overview
   > 
   > SkyWalking OAP should be a replacement of Zabbix server to support Zabbix proxy and agents.
   
   It seems like [Active Checks Mode](https://www.zabbix.com/documentation/current/manual/appendix/items/activepassive#active_checks), agent side query items need to be monitored and collecting metrics items sending to proxy or server.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org