[jira] [Commented] (METRON-668) Remove the "tickUpdate" profile config and make the "init" phase not reset variables

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/METRON-668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830685#comment-15830685 ] 

ASF GitHub Bot commented on METRON-668:
---------------------------------------

GitHub user cestella opened a pull request:

    https://github.com/apache/incubator-metron/pull/420

    METRON-668: Remove the "tickUpdate" profile config and make the "init" phase not reset variables

    Please see description at [METRON-668](https://issues.apache.org/jira/browse/METRON-668) for a full description.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/cestella/incubator-metron METRON-668

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-metron/pull/420.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #420
    
----
commit 7cb5c60462448b6c35a9d1def58489903a649834
Author: cstella <ce...@gmail.com>
Date:   2017-01-19T21:58:27Z

    METRON-668: Remove the 'tickUpdate' profile config and make the 'init' phase not reset variables

----


> Remove the "tickUpdate" profile config and make the "init" phase not reset variables
> ------------------------------------------------------------------------------------
>
>                 Key: METRON-668
>                 URL: https://issues.apache.org/jira/browse/METRON-668
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Casey Stella
>            Assignee: Casey Stella
>
> Originally during work on the MAD outlier work, I conceived of a need for a new callback in the profile configuration, "tickUpdate" that ran at the tick and had the variables from the tick that just completed available to it.  This was done so that I could merge the state accumulated in the tick with a lookback window of state for MAD.  The problem is that the "init" phase happens after this and blows away the changes done in "tickUpdate", so it never worked like intended.
> It occurs to me that what we really want is not to have two separate config phases, but only one, "init" and to not reset the variables on the tick for the profile.  You can, of course, choose to update them by overwriting them in the "init" phase *or* you can choose to use them as part of your init.
> For context, this would make the example for MAD:
> {code:javascript}
> {
>   "profiles": [
>     {
>       "profile": "sketchy_mad",
>       "foreach": "'global'",
>       "onlyif": "true",
>       "init" : {
>         "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'))"
>                },
>       "tickUpdate": {
>         "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'), s)"
>                 },
>       "update": {
>         "s": "OUTLIER_MAD_ADD(s, value)"
>                 },
>       "result": "s"
>     }
>   ]
> }
> {code}
> is functionally equivalent to
> {code:javascript}
> {
>   "profiles": [
>     {
>       "profile": "sketchy_mad",
>       "foreach": "'global'",
>       "onlyif": "true",
>       "init" : {
>         "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'))"
>                },
>       "update": {
>         "s": "OUTLIER_MAD_ADD(s, value)"
>                 },
>       "result": "s"
>     }
>   ]
> }
> {code}
> This resets the MAD state to the last 5 minute window.  If we did NOT reset the state and keep accumulating state (provided we did not clear the variables on init, we could do the following:
> {code:javascript}
> {
>   "profiles": [
>     {
>       "profile": "sketchy_mad",
>       "foreach": "'global'",
>       "onlyif": "true",
>       "init" : {
>         "s": "if exists(s) then s else OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'))"
>                },
>       "update": {
>         "s": "OUTLIER_MAD_ADD(s, value)"
>                 },
>       "result": "s"
>     }
>   ]
> }
> {code}
> s would get initialized sensibly and then always accumulate as long as the topology continued (rather than having a fixed lookback).
> In short, making init to not reset the variables shouldn't cause any harm and should provide another set of use-cases for the profiler.  Also, tickUpdate has no function whatsoever and should be removed because it gets overwritten by init directly after being called.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)