You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/08/24 21:51:00 UTC
[jira] [Commented] (METRON-1132) Enhance Profiler Debug Functions
[ https://issues.apache.org/jira/browse/METRON-1132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140772#comment-16140772 ]
ASF GitHub Bot commented on METRON-1132:
----------------------------------------
GitHub user nickwallen opened a pull request:
https://github.com/apache/metron/pull/716
METRON-1132 Enhance Profiler Debug Functions
Enhanced the `PROFILER_APPLY` function so that multiple messages can be applied at once. This makes it much simpler to test with larger, likely more diverse, data sets.
This also enhanced the `StandAloneProfiler` to output the number of profiles defined, the number of messages applied, and details on how those messages were applied. This provides a greater understanding of the Profiler's internals while debugging profiles and also improves the unit tests.
### Testing
Along with the additional unit tests that were added, this was tested manually in the REPL. To test, follow along.
Create a Profiler configuration.
```
[Stellar]>>> conf := SHELL_EDIT()
[Stellar]>>> conf
{
"profiles": [
{
"profile": "hello-world",
"onlyif": "exists(ip_src_addr)",
"foreach": "ip_src_addr",
"init": { "count": "0" },
"update": { "count": "count + 1" },
"result": "count"
},
{
"profile": "hello-world",
"onlyif": "exists(ip_src_addr)",
"foreach": "ip_src_addr",
"init": { "count": "0" },
"update": { "count": "count + 1" },
"result": "count"
}
]
}
```
Create some sample message data.
```
[Stellar]>>> msgs := SHELL_EDIT()
[Stellar]>>> msgs
[
{
"ip_src_addr": "10.0.0.1",
"protocol": "HTTPS",
"length": "10",
"bytes_in": "234"
},
{
"ip_src_addr": "10.0.0.1",
"protocol": "HTTPS",
"length": "10",
"bytes_in": "234"
},
{
"ip_src_addr": "10.0.0.1",
"protocol": "HTTPS",
"length": "10",
"bytes_in": "234"
},
{
"ip_src_addr": "10.0.0.1",
"protocol": "HTTPS",
"length": "10",
"bytes_in": "234"
}
]
```
Initialize the Profiler.
```
[Stellar]>>> p := PROFILER_INIT(conf)
[Stellar]>>> p
Profiler{1 profile(s), 0 messages(s), 0 route(s)}
```
Apply 3 messages at a time. Notice how the number of messages and routes increase each time.
```
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 8 messages(s), 16 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 12 messages(s), 24 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 16 messages(s), 32 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 20 messages(s), 40 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 24 messages(s), 48 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 28 messages(s), 56 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 32 messages(s), 64 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 36 messages(s), 72 route(s)}
[Stellar]>>> PROFILER_APPLY(msgs, p)
Profiler{2 profile(s), 40 messages(s), 80 route(s)}
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/metron METRON-1132
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/716.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #716
----
commit cedf8ec802413fb797cb7d98fe7eb8679cc9b841
Author: Nick Allen <ni...@nickallen.org>
Date: 2017-08-24T21:38:42Z
METRON-1132 Enhance Profiler Debug Functions
commit a3b715b07a0d6b1deab16c5499c219ef42afc14c
Author: Nick Allen <ni...@nickallen.org>
Date: 2017-08-24T21:48:46Z
Forgot to initialize a counter
----
> Enhance Profiler Debug Functions
> --------------------------------
>
> Key: METRON-1132
> URL: https://issues.apache.org/jira/browse/METRON-1132
> Project: Metron
> Issue Type: Bug
> Affects Versions: 0.4.0
> Reporter: Nick Allen
> Assignee: Nick Allen
> Fix For: Next + 1
>
>
> Enhance the `PROFILER_APPLY` function so that multiple messages can be applied at once. This makes it much simpler to test with larger, likely more diverse, data sets.
> Enhance the `StandAloneProfiler` to output the number of profiles defined, the number of messages applied, and details on how those messages were applied.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)