You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/08/29 14:55:00 UTC
[jira] [Commented] (METRON-1138) Improve Error Message with Bad
Profile Expression
[ https://issues.apache.org/jira/browse/METRON-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145423#comment-16145423 ]
ASF GitHub Bot commented on METRON-1138:
----------------------------------------
GitHub user nickwallen opened a pull request:
https://github.com/apache/metron/pull/721
METRON-1138 Improve Error Message with Bad Profile Expression
I have found that it can be very natural for a new Profiler user to attempt to reference a field defined in a telemetry message, in a Profile's `groupBy` expression. Unfortunately, this does not work as the 'groupBy' expression runs during a flush and has no single telemetry message available in scope.
For example, this seems like a Profile that should work.
```
{
"profile": "calender-effects",
"onlyif": "exists(ip_src_addr) and exists(timestamp)",
"foreach": "ip_src_addr",
"init": { "count": 0 },
"update": { "count": "count + 1" },
"result": "count",
"groupBy": ["DAY_OF_WEEK(timestamp)"]
}
```
Currently, when this expression fails the error message looks-like this, which is not exactly enlightening.
```
org.apache.metron.stellar.dsl.ParseException: Bad 'groupBy' expression: Unexpected type:
expected=Object,
actual=null,
expression=DAY_OF_WEEK(timestamp),
profile=calender-effects,
entity=10.0.0.1
```
With this change, additional information is provided.
* The variables that are in-scope is logged in the error message.
* The exact expression that failed is logged. Previously, for the 'init' and 'update' expressions it wasn't clear which expression failed, if the user defined multiple.
This should make it much easier to debug for a user. The same error looks-like the following with the code change.
```
ERROR DefaultProfileBuilder:189 - ...
org.apache.metron.stellar.dsl.ParseException: Bad 'groupBy' expression:
error='Unexpected type: expected=Object, actual=null, expression=DAY_OF_WEEK(timestamp)', expr='DAY_OF_WEEK(timestamp)',
profile='calendar-effects',
entity='10.0.0.1',
variables-available='[duration, result, profile, start, x, end, entity]'
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/metron METRON-1138
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/721.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #721
----
commit b54db11a28737ca51d5a479ab8b4118a15d3abf2
Author: Nick Allen <ni...@nickallen.org>
Date: 2017-08-29T14:38:23Z
METRON-1138 Improve Error Message with Bad Profile Expression
----
> Improve Error Message with Bad Profile Expression
> -------------------------------------------------
>
> Key: METRON-1138
> URL: https://issues.apache.org/jira/browse/METRON-1138
> Project: Metron
> Issue Type: Improvement
> Affects Versions: 0.4.0
> Reporter: Nick Allen
> Assignee: Nick Allen
> Fix For: 0.4.1
>
>
> I have found that it can be very natural for a new Profiler user to attempt to reference a field defined in a telemetry message, in a Profile's `groupBy` expression. Unfortunately, this does not work as the 'groupBy' expression runs during a flush and has no single telemetry message available in scope.
> For example, this seems like a Profile that should work.
> {code}
> {
> "profile": "calender-effects",
> "onlyif": "exists(ip_src_addr) and exists(timestamp)",
> "foreach": "ip_src_addr",
> "init": { "count": 0 },
> "update": { "count": "count + 1" },
> "result": "count",
> "groupBy": ["DAY_OF_WEEK(timestamp)"]
> }
> {code}
> Currently, when this expression fails the error message looks-like this, which is not exactly enlightening.
> {code}
> org.apache.metron.stellar.dsl.ParseException: Bad 'groupBy' expression: Unexpected type: expected=Object, actual=null, expression=DAY_OF_WEEK(timestamp), profile=calender-effects, entity=10.0.0.1 at org.apache.metron.profiler.ProfileBuilder.execute(ProfileBuilder.java:243) at org.apache.metron.profiler.ProfileBuilder.flush(ProfileBuilder.java:164) at org.apache.metron.profiler.bolt.ProfileBuilderBolt.lambda$handleTick$2(ProfileBuilderBolt.java:183) at java.util.concurrent.ConcurrentMap.forEach(ConcurrentMap.java:114) at org.apache.metron.profiler.bolt.ProfileBuilderBolt.handleTick(ProfileBuilderBolt.java:179) at org.apache.metron.profiler.bolt.ProfileBuilderBolt.execute(ProfileBuilderBolt.java:150) at org.apache.storm.daemon.executor$fn__6573$tuple_action_fn__6575.invoke(executor.clj:734) at org.apache.storm.daemon.executor$mk_task_receiver$fn__6494.invoke(executor.clj:469) at org.apache.storm.disruptor$clojure_handler$reify__6007.onEvent(disruptor.clj:40) at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451) at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430) at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) at org.apache.storm.daemon.executor$fn__6573$fn__6586$fn__6639.invoke(executor.clj:853) at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.IllegalArgumentException: Unexpected type: expected=Object, actual=null, expression=DAY_OF_WEEK(timestamp) at org.apache.metron.stellar.common.DefaultStellarStatefulExecutor.execute(DefaultStellarStatefulExecutor.java:128) at org.apache.metron.profiler.ProfileBuilder.lambda$execute$6(ProfileBuilder.java:239) at java.util.ArrayList.forEach(ArrayList.java:1249) at org.apache.metron.profiler.ProfileBuilder.execute(ProfileBuilder.java:239) ... 15 mo
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)