[jira] [Commented] (METRON-690) Create a DSL-based timestamp lookup for profiler to enable sparse windows

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/METRON-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860566#comment-15860566 ] 

ASF GitHub Bot commented on METRON-690:
---------------------------------------

GitHub user cestella opened a pull request:

    https://github.com/apache/incubator-metron/pull/450

    METRON-690: Create a DSL-based timestamp lookup for profiler to enable sparse windows

    Creating a small DSL to allow specifying profiles from windows of time that may:
    * repeat non-contiguously (e.g. the same hour every day for a week)
    * have inclusions and exclusions (e.g. the same hour every day for a week excluding holidays and weekends)
    
    This also provides a `PROFILE_WINDOW` Stellar function which accepts this DSL as a string and can return the set of profiler periods based on the times selected and suitable for passing to `PROFILE_GET`.  To be clear, you can compose the two functions.
    
     i.e. `PROFILE_GET('profile', 'entity', PROFILE_WINDOW('1 hour window every 24 hours starting from 14 days ago including the current day of the week excluding weekends, holidays:us'))` would retrieve all the profile measurements written for the profile "profile" and entity "entity" for the last hour on the same weekday excluding weekends and US holidays across the last 14 days
    
    For a complete description with examples, see [here](https://github.com/cestella/incubator-metron/tree/METRON-690-squashed/metron-analytics/metron-profiler-client#profile_window).
    
    Acceptance testing plan will be submitted as a follow-on comment.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/cestella/incubator-metron METRON-690-squashed

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-metron/pull/450.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #450
    
----
commit 04474a61c8f8cf85a491c6a693897a5cb0a6107f
Author: cstella <ce...@gmail.com>
Date:   2017-02-08T19:14:37Z

    Create a DSL-based timestamp lookup for profiler to enable sparse windows

commit eb729a821a7b8ee80a1b033ad884633a4d174138
Author: cstella <ce...@gmail.com>
Date:   2017-02-09T22:46:57Z

    Merge branch 'master' into METRON-690-squashed

commit 51381e70d66e01b10787779fcc10e69e39f05a83
Author: cstella <ce...@gmail.com>
Date:   2017-02-10T00:38:52Z

    Removed jodatime dependency as unnecessary

commit 877ba3df06a8cfb1e1a2f6101d8db9932d664d21
Author: cstella <ce...@gmail.com>
Date:   2017-02-10T01:27:52Z

    Updating docs and dependencies.

commit 8c36be979407f4bdfced1c49c351ba02b95a71d6
Author: cstella <ce...@gmail.com>
Date:   2017-02-10T02:18:04Z

    Finished readme

----


> Create a DSL-based timestamp lookup for profiler to enable sparse windows
> -------------------------------------------------------------------------
>
>                 Key: METRON-690
>                 URL: https://issues.apache.org/jira/browse/METRON-690
>             Project: Metron
>          Issue Type: New Feature
>            Reporter: Casey Stella
>
> I propose that we support the following features:
> * A starting point that is not current time
> * Sparse bins (i.e. the last hour for every tuesday for the last month)
> * The ability to skip events (e.g. weekends, holidays)
> This would result in a new function with the following arguments:
> from - The lookback starting point (default to now)
> fromUnits - The units for the lookback starting point
> to - The ending point for the lookback window (default to from + binSize)
> toUnits - The units for the lookback ending point
> including - A list of conditions which we would skip.
> weekend
> holiday
> sunday through saturday
> excluding - A list of conditions which we would skip.
> weekend
> holiday
> sunday through saturday
> binSize - The size of the lookback bin
> binUnits - The units of the lookback bin
> Given the number of arguments and their complexity and the fact that many, many are optional, 
> PROFILE_LOOKBACK accept a string backed by a DSL to express these criteria
> Base Case: A lookback of 1 hour ago
> PROFILE_LOOKBACK( '1 hour bins from now')
> Example 1: The same time window every tuesday for the last month starting one hour ago
> Just to make this as clear as possible, if this is run at 3PM on Monday January 23rd, 2017, it would include the following bins:
> January 17th, 2PM - 3PM
> January 10th, 2PM - 3PM
> January 3rd, 2PM - 3PM
> December 27th, 2PM - 3PM
> PROFILE_LOOKBACK( '1 hour bins from 1 hour to 1 month including tuesdays')
> Example 2: The same time window every sunday for the last month starting one hour ago skipping holidays
> Just to make this as clear as possible, if this is run at 3PM on Monday January 22rd, 2017, it would include the following bins:
> January 16th, 2PM - 3PM
> January 9th, 2PM - 3PM
> January 2rd, 2PM - 3PM
> NOT December 25th
> PROFILE_LOOKBACK( '1 hour bins from 1 hour to 1 month including tuesdays excluding holidays')



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)