You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@metron.apache.org by ce...@apache.org on 2017/02/24 20:53:26 UTC
[3/3] incubator-metron git commit: METRON-690: Create a DSL-based
timestamp lookup for profiler to enable sparse windows closes
apache/incubator-metron#450
METRON-690: Create a DSL-based timestamp lookup for profiler to enable sparse windows closes apache/incubator-metron#450
Project: http://git-wip-us.apache.org/repos/asf/incubator-metron/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-metron/commit/84d34719
Tree: http://git-wip-us.apache.org/repos/asf/incubator-metron/tree/84d34719
Diff: http://git-wip-us.apache.org/repos/asf/incubator-metron/diff/84d34719
Branch: refs/heads/master
Commit: 84d347195a817e70fb792b4b240151b7654b1a9c
Parents: 7abd7e8
Author: cstella <ce...@gmail.com>
Authored: Fri Feb 24 15:53:05 2017 -0500
Committer: cstella <ce...@gmail.com>
Committed: Fri Feb 24 15:53:05 2017 -0500
----------------------------------------------------------------------
dependencies_with_url.csv | 3 +-
.../metron-profiler-client/README.md | 287 ++++-
metron-analytics/metron-profiler-client/pom.xml | 27 +
.../profiler/client/window/generated/Window.g4 | 136 +++
.../src/main/java/Window.tokens | 17 +
.../src/main/java/WindowLexer.tokens | 17 +
.../client/stellar/IntervalPredicate.java | 106 ++
.../profiler/client/stellar/WindowLookback.java | 107 ++
.../metron/profiler/client/window/Window.java | 159 +++
.../profiler/client/window/WindowProcessor.java | 422 +++++++
.../window/generated/WindowBaseListener.java | 276 +++++
.../client/window/generated/WindowLexer.java | 477 ++++++++
.../client/window/generated/WindowListener.java | 231 ++++
.../client/window/generated/WindowParser.java | 1096 ++++++++++++++++++
.../predicates/DateSpecifierPredicate.java | 82 ++
.../client/window/predicates/DayPredicates.java | 113 ++
.../window/predicates/HolidaysPredicate.java | 87 ++
.../client/stellar/IntervalPredicateTest.java | 89 ++
.../client/stellar/WindowLookbackTest.java | 207 ++++
.../client/window/WindowProcessorTest.java | 314 +++++
.../apache/metron/profiler/ProfilePeriod.java | 12 +-
metron-platform/metron-common/README.md | 9 +
metron-platform/metron-common/pom.xml | 5 +-
.../apache/metron/common/dsl/GrammarUtils.java | 133 +++
.../org/apache/metron/common/dsl/Token.java | 2 +-
pom.xml | 1 +
26 files changed, 4361 insertions(+), 54 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/84d34719/dependencies_with_url.csv
----------------------------------------------------------------------
diff --git a/dependencies_with_url.csv b/dependencies_with_url.csv
index 3640b14..fded057 100644
--- a/dependencies_with_url.csv
+++ b/dependencies_with_url.csv
@@ -221,4 +221,5 @@ org.yaml:snakeyaml:jar:1.15:compile,Apache License Version 2.0,http://www.snakey
ring-cors:ring-cors:jar:0.1.5:compile,Eclipse Public License 1.0,https://github.com/r0man/ring-cors
xerces:xercesImpl:jar:2.9.1:compile,ASLv2,http://xerces.apache.org/xerces2-j
xml-apis:xml-apis:jar:1.3.04:compile,ASLv2,http://xml.apache.org/commons/components/external/
-
+de.jollyday:jollyday:jar:0.5.2:compile,ASLv2,http://jollyday.sourceforge.net/license.html
+org.threeten:threeten-extra:jar:1.0:compile,BSD,http://www.threeten.org/threeten-extra/license.html
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/84d34719/metron-analytics/metron-profiler-client/README.md
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler-client/README.md b/metron-analytics/metron-profiler-client/README.md
index 94da919..656bb25 100644
--- a/metron-analytics/metron-profiler-client/README.md
+++ b/metron-analytics/metron-profiler-client/README.md
@@ -4,27 +4,9 @@ This project provides a client API for accessing the profiles generated by the [
## Stellar Client API
-The following are usage examples that show how the Stellar API can be used to read profiles generated by the [Metron Profiler](../metron-profiler). This API would be used in conjunction with other Stellar functions like [`MAAS_MODEL_APPLY`](../../metron-platform/metron-common#maas_model_apply) to perform model scoring on streaming data.
-
-These examples assume a profile has been defined called 'snort-alerts' that tracks the number of Snort alerts associated with an IP address over time. The profile definition might look similar to the following.
-
-```
-{
- "profiles": [
- {
- "profile": "snort-alerts",
- "foreach": "ip_src_addr",
- "onlyif": "source.type == 'snort'",
- "update": { "s": "STATS_ADD(s, 1)" },
- "result": "STATS_MEAN(s)"
- }
- ]
-}
-```
+### `PROFILE_GET`
-During model scoring the entity being scored, in this case a particular IP address, will be known. The following examples shows how this profile data might be retrieved.
-
-The Stellar client consists of the `PROFILE_GET` command, which takes the following arguments:
+The `PROFILE_GET` command allows you to select all of the profile measurements written. This command takes the following arguments:
```
REQUIRED:
profile - The name of the profile
@@ -41,20 +23,7 @@ There is an older calling format where `groups_list` is specified as a sequence
The `periods` field is (likely) the output of another Stellar function which defines the times to include.
-`PROFILE_FIXED`: The profiler periods associated with a fixed lookback starting from now. These are ProfilePeriod objects.
-```
-REQUIRED:
- durationAgo - How long ago should values be retrieved from?
- units - The units of 'durationAgo'.
-OPTIONAL:
- config_overrides - Optional - Map (in curly braces) of name:value pairs, each overriding the global config parameter
- of the same name. Default is the empty Map, meaning no overrides.
-
-e.g. To retrieve all the profiles for the last 5 hours. PROFILE_GET('profile', 'entity', PROFILE_FIXED(5, 'HOURS'))
-```
-
-
-### Groups_list argument
+#### Groups_list argument
The `groups_list` argument in the client must exactly correspond to the [`groupBy`](../metron-profiler#groupby) configuration in the profile definition. If `groupBy` was not used in the profile, `groups_list` must be empty in the client. If `groupBy` was used in the profile, then the client `groups_list` is <b>not</b> optional; it must be the same length as the `groupBy` list, and specify exactly one selected group value for each `groupBy` criterion, in the same order. For example:
```
If in Profile, the groupBy criteria are: [ \u201cDAY_OF_WEEK()\u201d, \u201cURL_TO_PORT()\u201d ]
@@ -62,7 +31,7 @@ Then in PROFILE_GET, an allowed groups value would be: [ \u201c3\u201d, \u201c8080\u201d ]
which will select only records from Tuesdays with port number 8080.
```
-### Configuration and the config_overrides argument
+#### Configuration and the config_overrides argument
By default, the Profiler creates profiles with a period duration of 15 minutes. This means that data is accumulated, summarized and flushed every 15 minutes.
The Client API must also have knowledge of this duration to correctly retrieve the profile data. If the Client is expecting 15 minute periods, it will not be
@@ -91,37 +60,269 @@ want to change the global Client configuration so as not to disrupt the work of
| profiler.client.salt.divisor | The salt divisor used to store profile data. | Optional | 1000 |
| hbase.provider.impl | The name of the HBaseTableProvider implementation class. | Optional | |
+
+### Profile Selectors
+
+You will notice that the third argument for `PROFILE_GET` is a list of `ProfilePeriod` objects. This list is expected to
+be produced by another Stellar function. There are a couple options available.
+
+#### `PROFILE_FIXED`
+
+The profiler periods associated with a fixed lookback starting from now. These are ProfilePeriod objects.
+```
+REQUIRED:
+ durationAgo - How long ago should values be retrieved from?
+ units - The units of 'durationAgo'.
+OPTIONAL:
+ config_overrides - Optional - Map (in curly braces) of name:value pairs, each overriding the global config parameter
+ of the same name. Default is the empty Map, meaning no overrides.
+
+e.g. To retrieve all the profiles for the last 5 hours. PROFILE_GET('profile', 'entity', PROFILE_FIXED(5, 'HOURS'))
+```
+
+Note that the `config_overrides` parameter operates exactly as the `config_overrides` argument in `PROFILE_GET`.
+The only available parameters for override are:
+* `profiler.client.period.duration`
+* `profiler.client.period.duration.units`
+
+#### `PROFILE_WINDOW`
+
+`PROFILE_WINDOW` is intended to provide a finer-level of control over selecting windows for profiles:
+* Specify windows relative to the data timestamp (see the optional `now` parameter below)
+* Specify non-contiguous windows to better handle seasonal data (e.g. the last hour for every day for the last month)
+* Specify profile output excluding holidays
+* Specify only profile output on a specific day of the week
+
+It does this by a domain specific language mimicking natural language that defines the windows excluded.
+
+```
+REQUIRED:
+ windowSelector - The statement specifying the window to select.
+ now - Optional - The timestamp to use for now.
+OPTIONAL:
+ config_overrides - Optional - Map (in curly braces) of name:value pairs, each overriding the global config parameter
+ of the same name. Default is the empty Map, meaning no overrides.
+
+e.g. To retrieve all the measurements written for 'profile' and 'entity' for the last hour
+on the same weekday excluding weekends and US holidays across the last 14 days:
+PROFILE_GET('profile', 'entity', PROFILE_WINDOW('1 hour window every 24 hours starting from 14 days ago including the current day of the week excluding weekends, holidays:us'))
+```
+
+Note that the `config_overrides` parameter operates exactly as the `config_overrides` argument in `PROFILE_GET`.
+The only available parameters for override are:
+* `profiler.client.period.duration`
+* `profiler.client.period.duration.units`
+
+##### The Profile Selector Language
+
+The domain specific language can be broken into a series of clauses, some optional
+* <a href="#Temporal_Window_Width"><span style="color:blue">Total Temporal Duration</span></a> - The total range of time in which windows may be specified
+* <a href="#InclusionExclusion_specifiers"><span style="color:red">Temporal Window Width</span></a> - How large each temporal window
+* <a href="#Skip_distance"><span style="color:green">Skip distance</span></a> (optional)- How far to skip between when one window starts and when the next begins
+* <a href="#InclusionExclusion_specifiers"><span style="color:purple">Inclusion/Exclusion specifiers</span></a> (optional) - The set of specifiers to further filter the window
+
+One *must* specify either a total temporal duration or a temporal window width.
+The remaining clauses are optional.
+During the course of the following discussion, we will color code the clauses in the examples and link them
+to the relevant section for more detail.
+
+From a high level, the language fits the following three forms, which are composed of the clauses above:
+
+* <a href="#Temporal_Window_Width"><span style="color:red">time_interval WINDOW?</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">(INCLUDING specifier_list)? (EXCLUDING specifier_list)?</span></a>
+* <a href="#Temporal_Window_Width"><span style="color:red">time_interval WINDOW?</span></a> <a href="#Skip_distance"><span style="color:green">EVERY time_interval</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">FROM time_interval (TO time_interval)?</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">(INCLUDING specifier_list)? (EXCLUDING specifier_list)?</span></a>
+* <a href="#Total_Temporal_Duration"><span style="color:blue">FROM time_interval (TO time_interval)?</span></a>
+
+
+###### <span style="color:blue">Total Temporal Duration</span>
+
+Total temporal duration is specified by a phrase: `FROM time_interval AGO TO time_interval AGO`
+This indicates the beginning and ending of a time interval. This is an inclusive duration.
+* `FROM` - Can be the words "from" or "starting from"
+* `time_interval` - A time amount followed by a unit (e.g. 1 hour). Fractional amounts are not supported. The unit may be "minute", "day", "hour" with any pluralization.
+* `TO` - Can be the words "until" or "to"
+* `AGO` - Optionally the word "ago"
+
+The `TO time_interval AGO` portion is optional. If unspecified then it is expected that the time interval ends now.
+
+Due to the vagaries of the english language, the from and the to portions, if both specified, are interchangeable
+with regard to which one specifies the start and which specifies the end.
+
+In other words "<a href="#Total_Temporal_Duration"><span style="color:blue">starting from 1 hour ago to 30 minutes ago</span></a>" and
+"<a href="#Total_Temporal_Duration"><span style="color:blue">starting from 30 minutes ago to 1 hour ago</span></a>" specify the same
+temporal duration.
+
+**Examples**
+
+* A duration starting 1 hour ago and ending now
+ * <a href="#Total_Temporal_Duration"><span style="color:blue">from 1 hour ago</span></a>
+ * <a href="#Total_Temporal_Duration"><span style="color:blue">from 1 hour</span></a>
+ * <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 1 hour ago</span></a>
+ * <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 1 hour</span></a>
+* A duration starting 1 hour ago and ending 30 minutes ago:
+ * <a href="#Total_Temporal_Duration"><span style="color:blue">from 1 hour ago until 30 minutes ago</span></a>
+ * <a href="#Total_Temporal_Duration"><span style="color:blue">from 30 minutes ago until 1 hour ago</span></a>
+ * <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 1 hour ago to 30 minutes ago</span></a>
+ * <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 1 hour to 30 minutes</span></a>
+
+###### <span style="color:red">Temporal Window Width</span>
+
+Temporal window width is the specification of a window.
+A window is may either repeat within total temporal duration or may fill the total temporal duration. This is an inclusive window.
+A window is specified by the phrase: `time_interval WINDOW`
+* `time_interval` - A time amount followed by a unit (e.g. 1 hour). Fractional amounts are not supported. The unit may be "minute", "day", "hour" with any pluralization.
+* `WINDOW` - Optionally the word "window"
+
+**Examples**
+
+* A fixed window starting 2 hours ago and going until now
+ * <a href="#Temporal_Window_Width"><span style="color:red">2 hour</span></a>
+ * <a href="#Temporal_Window_Width"><span style="color:red">2 hours</span></a>
+ * <a href="#Temporal_Window_Width"><span style="color:red">2 hours window</span></a>
+* A repeating 30 minute window starting 2 hours ago and repeating every hour until now.
+This would result in 2 30-minute wide windows: 2 hours ago and 1 hour ago
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 2 hours ago</span></a>
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minute windows</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 2 hours ago</span></a>
+* A repeating 30 minute window starting 2 hours ago and repeating every hour until 30 minutes ago.
+This would result in 2 30-minute wide windows: 2 hours ago and 1 hour ago
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 2 hours ago until 30 minutes ago</span></a>
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minutes window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 2 hours ago to 30 minutes ago</span></a>
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minutes window</span></a> <a href="#Skip_distance"><span style="color:green">for every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 30 minutes ago to 2 hours ago</span></a>
+
+###### <span style="color:green">Skip distance</span>
+
+Skip distance is the amount of time between temporal window beginnings that the next window starts.
+It is, in effect, the window period.
+
+It is specified by the phrase `EVERY time_interval`
+* `time_interval` - A time amount followed by a unit (e.g. 1 hour). Fractional amounts are not supported. The unit may be "minute", "day", "hour" with any pluralization.
+* `EVERY` - The word/phrase "every" or "for every"
+
+**Examples**
+
+* A repeating 30 minute window starting 2 hours ago and repeating every hour until now.
+This would result in 2 30-minute wide windows: 2 hours ago and 1 hour ago
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 2 hours ago </span></a>
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minutes window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 2 hours ago </span></a>
+* A repeating 30 minute window starting 2 hours ago and repeating every hour until 30 minutes ago.
+This would result in 2 30-minute wide windows: 2 hours ago and 1 hour ago
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">starting from 2 hours ago until 30 minutes ago</span></a>
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minutes window</span></a> <a href="#Skip_distance"><span style="color:green">every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 2 hours ago to 30 minutes ago</span></a>
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minutes window</span></a> <a href="#Skip_distance"><span style="color:green">for every 1 hour</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 30 minutes ago to 2 hours ago</span></a>
+
+###### <span style="color:purple">Inclusion/Exclusion specifiers</span>
+Inclusion and Exclusion specifiers operate as filters on the set of windows.
+They operate on the window beginning timestamp.
+
+For inclusion specifiers, windows who are passed by _any_ of the set of inclusion specifiers are included.
+inclusion specifiers. Similarly, windows who are passed by _any_ of the set of exclusion specifiers are excluded.
+Exclusion specifiers trump inclusion specifiers.
+
+Specifiers follow one of the following formats depending on if it is an inclusion or exclusion specifier:
+* `INCLUSION specifier, specifier, ...`
+ * `INCLUSION` can be "include", "includes" or "including"
+* `EXCLUSION specifier, specifier, ...`
+ * `EXCLUSION` can be "exclude", "excludes" or "excluding"
+
+
+The specifiers are a set of fixed specifiers available as part of the language:
+* Fixed day of week-based specifiers - includes or excludes if the window is on the specified day of the week
+ * "monday" or "mondays"
+ * "tuesday" or "tuesdays"
+ * "wednesday" or "wednesdays"
+ * "thursday" or "thursdays"
+ * "friday" or "fridays"
+ * "saturday" or "saturdays"
+ * "sunday" or "sundays"
+ * "weekday" or "weekdays"
+ * "weekend" or ""weekends"
+* Relative day of week-based specifiers - includes or excludes based on the day of week relative to now
+ * "current day of the week"
+ * "current day of week"
+ * "this day of the week"
+ * "this day of week"
+* Specified date - includes or excludes based on the specified date
+ * "date" - Takes up to 2 arguments
+ * The day in `yyyy/MM/dd` format if no second argument is provided
+ * Optionally the format to specify the first argument in
+ * Example: `date:2017/12/25` would include or exclude December 25, 2017
+ * Example: `date:20171225:yyyyMMdd` would include or exclude December 25, 2017
+* Holidays - includes or excludes based on if the window starts during a holiday
+ * "holiday" or "holidays"
+ * Arguments form the jollyday hierarchy of holidays. e.g. "us:nyc" would be holidays for New York City, USA
+ * If none is specified, it will choose based on locale.
+ * Countries supported are those supported in [jollyday](https://github.com/svendiedrichsen/jollyday/tree/master/src/main/resources/holidays)
+ * Example: `holiday:us:nyc` would be the holidays of New York City, USA
+ * Example: `holiday:hu` would be the holidays of Hungary
+
+**Examples**
+
+Assume these are executed at noon.
+* A 1 hour window for the past 8 'current day of the week'
+ * <a href="#Temporal_Window_Width"><span style="color:red">1 hour window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 56 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">including this day of the week</span></a>
+* A 1 hour window for the past 8 tuesdays
+ * <a href="#Temporal_Window_Width"><span style="color:red">1 hour window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 56 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">including tuesdays</span></a>
+* A 30 minute window every tuesday at noon starting 14 days ago until now
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 14 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">including tuesdays</span></a>
+* A 30 minute window every day except holidays and weekends at noon starting 14 days ago until now
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 14 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">excluding holidays:us, weekends</span></a>
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 14 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">including weekdays excluding holidays:us, weekends</span></a>
+* A 30 minute window at noon every day from 7 days ago including saturdays and excluding weekends.
+Because exclusions trump inclusions, the following will never yield any windows
+ * <a href="#Temporal_Window_Width"><span style="color:red">30 minute window</span></a> <a href="#Skip_distance"><span style="color:green">every 24 hours</span></a> <a href="#Total_Temporal_Duration"><span style="color:blue">from 7 days ago</span></a> <a href="#InclusionExclusion_specifiers"><span style="color:purple">including saturdays excluding weekends</span></a>
+
### Errors
-The most common result of incorrect PROFILE_GET arguments or Client configuration parameters is an empty result set, rather than an error. The Client cannot effectively validate the arguments, because the Profiler configuration parameters may be changed and the profile itself does not store them. The person doing the querying must carry forward the knowledge of the Profiler configuration parameters from the time of profile creation, and use corresponding PROFILE_GET arguments and Client configuration parameters when querying the data.
+The most common result of incorrect `PROFILE_GET` arguments or Client configuration parameters is an empty result set, rather than an error.
+The Client cannot effectively validate the arguments, because the Profiler configuration parameters may be changed and the profile itself does not store them.
+The person doing the querying must carry forward the knowledge of the Profiler configuration parameters from the time of profile creation, and use corresponding `PROFILE_GET` arguments and Client configuration parameters when querying the data.
### Examples
+
+The following are usage examples that show how the Stellar API can be used to read profiles generated by the [Metron Profiler](../metron-profiler). This API would be used in conjunction with other Stellar functions like [`MAAS_MODEL_APPLY`](../../metron-platform/metron-common#maas_model_apply) to perform model scoring on streaming data.
+
+These examples assume a profile has been defined called 'snort-alerts' that tracks the number of Snort alerts associated with an IP address over time. The profile definition might look similar to the following.
+
+```
+{
+ "profiles": [
+ {
+ "profile": "snort-alerts",
+ "foreach": "ip_src_addr",
+ "onlyif": "source.type == 'snort'",
+ "update": { "s": "STATS_ADD(s, 1)" },
+ "result": "STATS_MEAN(s)"
+ }
+ ]
+}
+```
+
+During model scoring the entity being scored, in this case a particular IP address, will be known. The following examples shows how this profile data might be retrieved.
Retrieve all values of 'snort-alerts' from '10.0.0.1' over the past 4 hours.
```
-PROFILE_GET('snort-alerts', '10.0.0.1', 4, 'HOURS')
+PROFILE_GET('snort-alerts', '10.0.0.1', PROFILE_FIXED(4, 'HOURS'))
```
Retrieve all values of 'snort-alerts' from '10.0.0.1' over the past 2 days.
```
-PROFILE_GET('snort-alerts', '10.0.0.1', 2, 'DAYS')
+PROFILE_GET('snort-alerts', '10.0.0.1', PROFILE_FIXED(2, 'DAYS'))
```
If the profile had been defined to group the data by weekday versus weekend, then the following example would apply:
-Retrieve all values of 'snort-alerts' from '10.0.0.1' that occurred on 'weekdays' over the past month.
+Retrieve all values of 'snort-alerts' from '10.0.0.1' that occurred on 'weekdays' over the past 30 days.
```
-PROFILE_GET('snort-alerts', '10.0.0.1', 1, 'MONTHS', ['weekdays'] )
+PROFILE_GET('snort-alerts', '10.0.0.1', PROFILE_FIXED(30, 'DAYS'), ['weekdays'] )
```
The client may need to use a configuration different from the current Client configuration settings. For example, perhaps you are on a cluster shared with other analysts, and need to access a profile that was constructed 2 months ago using different period duration, while they are accessing more recent profiles constructed with the currently configured period duration. For this situation, you may use the `config_overrides` argument:
Retrieve all values of 'snort-alerts' from '10.0.0.1' over the past 2 days, with no `groupBy`, and overriding the usual global client configuration parameters for window duration.
```
-PROFILE_GET('profile1', 'entity1', 2, 'DAYS', [], {'profiler.client.period.duration' : '2', 'profiler.client.period.duration.units' : 'MINUTES'})
+PROFILE_GET('profile1', 'entity1', PROFILE_FIXED(2, 'DAYS', {'profiler.client.period.duration' : '2', 'profiler.client.period.duration.units' : 'MINUTES'}), [])
```
-Retrieve all values of 'snort-alerts' from '10.0.0.1' that occurred on 'weekdays' over the past month, overriding the usual global client configuration parameters for window duration.
+Retrieve all values of 'snort-alerts' from '10.0.0.1' that occurred on 'weekdays' over the past 30 days, overriding the usual global client configuration parameters for window duration.
```
-PROFILE_GET('profile1', 'entity1', 1, 'MONTHS', ['weekdays'], {'profiler.client.period.duration' : '2', 'profiler.client.period.duration.units' : 'MINUTES'})
+PROFILE_GET('profile1', 'entity1', PROFILE_FIXED(30, 'DAYS', {'profiler.client.period.duration' : '2', 'profiler.client.period.duration.units' : 'MINUTES'}), ['weekdays'] )
```
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/84d34719/metron-analytics/metron-profiler-client/pom.xml
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler-client/pom.xml b/metron-analytics/metron-profiler-client/pom.xml
index 77eb078..9b08be9 100644
--- a/metron-analytics/metron-profiler-client/pom.xml
+++ b/metron-analytics/metron-profiler-client/pom.xml
@@ -59,6 +59,11 @@
<scope>provided</scope>
</dependency>
<dependency>
+ <groupId>de.jollyday</groupId>
+ <artifactId>jollyday</artifactId>
+ <version>0.5.2</version>
+ </dependency>
+ <dependency>
<groupId>org.apache.metron</groupId>
<artifactId>metron-profiler-common</artifactId>
<version>${project.parent.version}</version>
@@ -236,6 +241,12 @@
</exclusion>
</exclusions>
</dependency>
+ <dependency>
+ <groupId>org.antlr</groupId>
+ <artifactId>antlr4-runtime</artifactId>
+ <version>${global_antlr_version}</version>
+ </dependency>
+
</dependencies>
<reporting>
<plugins>
@@ -266,6 +277,22 @@
<build>
<plugins>
<plugin>
+ <groupId>org.antlr</groupId>
+ <artifactId>antlr4-maven-plugin</artifactId>
+ <version>${global_antlr_version}</version>
+ <configuration>
+ <outputDirectory>${basedir}/src/main/java</outputDirectory>
+ </configuration>
+ <executions>
+ <execution>
+ <goals>
+ <goal>antlr4</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+
+ <plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>${global_shade_version}</version>
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/84d34719/metron-analytics/metron-profiler-client/src/main/antlr4/org/apache/metron/profiler/client/window/generated/Window.g4
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler-client/src/main/antlr4/org/apache/metron/profiler/client/window/generated/Window.g4 b/metron-analytics/metron-profiler-client/src/main/antlr4/org/apache/metron/profiler/client/window/generated/Window.g4
new file mode 100644
index 0000000..0090971
--- /dev/null
+++ b/metron-analytics/metron-profiler-client/src/main/antlr4/org/apache/metron/profiler/client/window/generated/Window.g4
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+grammar Window;
+
+@header {
+//CHECKSTYLE:OFF
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+}
+COMMA : ',';
+COLON : ':';
+
+WINDOW : 'window' | 'windows';
+
+INCLUDE : 'include' | 'INCLUDE' | 'includes' | 'INCLUDES' | 'including' | 'INCLUDING';
+EXCLUDE : 'exclude' | 'EXCLUDE' | 'excludes' | 'EXCLUDES' | 'excluding' | 'EXCLUDING';
+
+FROM : 'FROM' | 'from' | 'STARTING FROM' | 'starting from';
+EVERY : 'EVERY' | 'every' | 'FOR EVERY' | 'for every';
+TO : 'TO' | 'to' | 'until' | 'UNTIL';
+AGO : 'AGO' | 'ago';
+
+NUMBER : FIRST_DIGIT DIGIT*;
+IDENTIFIER : [:][a-zA-Z0-9][a-zA-Z0-9_\.\-/]*;
+
+DAY_SPECIFIER : MONDAY | TUESDAY | WEDNESDAY | THURSDAY
+ | FRIDAY | SATURDAY | SUNDAY
+ | CURRENT_DAY_OF_WEEK
+ | WEEKEND | WEEKDAY | HOLIDAYS
+ | DATE
+ ;
+
+TIME_UNIT : SECOND_UNIT | MINUTE_UNIT | HOUR_UNIT | DAY_UNIT ;
+
+WS : [ \r\t\u000C\n]+ -> skip;
+
+fragment SECOND_UNIT : 'SECOND' | 'second' | 'seconds' | 'SECONDS' | 'second(s)' | 'SECOND(S)';
+fragment MINUTE_UNIT : 'MINUTE' | 'minute' | 'minutes' | 'MINUTES' | 'minute(s)' | 'MINUTE(S)';
+fragment HOUR_UNIT : 'HOUR' | 'hour' | 'hours' | 'HOURS' | 'hour(s)' | 'HOUR(S)';
+fragment DAY_UNIT : 'DAY' | 'day' | 'days' | 'DAYS' | 'day(s)' | 'DAY(S)';
+fragment MONDAY : 'MONDAY' | 'monday' | 'MONDAYS' | 'mondays';
+fragment TUESDAY : 'TUESDAY' | 'tuesday' | 'TUESDAYS' | 'tuesdays';
+fragment WEDNESDAY : 'WEDNESDAY' | 'wednesday' | 'WEDNESDAYS' | 'wednesdays';
+fragment THURSDAY : 'THURSDAY' | 'thursday' | 'THURSDAYS' | 'thursdays';
+fragment FRIDAY : 'FRIDAY' | 'friday' | 'FRIDAYS' | 'fridays';
+fragment SATURDAY: 'SATURDAY' | 'saturday' | 'SATURDAYS' | 'saturdays';
+fragment SUNDAY : 'SUNDAY' | 'sunday' | 'SUNDAYS' | 'sundays';
+fragment CURRENT_DAY_OF_WEEK: 'this day of week' | 'THIS DAY OF WEEK' | 'this day of the week' | 'THIS DAY OF THE WEEK'
+ | 'current day of week' | 'CURRENT DAY OF WEEK'
+ | 'current day of the week' | 'CURRENT DAY OF THE WEEK';
+fragment WEEKEND : 'weekend' | 'WEEKEND' | 'weekends' | 'WEEKENDS';
+fragment WEEKDAY: 'weekday' | 'WEEKDAY' | 'weekdays' | 'WEEKDAYS';
+fragment HOLIDAYS: 'holiday' | 'HOLIDAY' | 'holidays' | 'HOLIDAYS';
+fragment DATE: 'date' | 'DATE';
+
+fragment DIGIT : '0'..'9';
+fragment FIRST_DIGIT : '1'..'9';
+
+window : window_expression EOF;
+
+window_expression : window_width including_specifier? excluding_specifier? #NonRepeatingWindow
+ | window_width skip_distance duration including_specifier? excluding_specifier? #RepeatingWindow
+ | duration #DenseWindow
+ ;
+
+excluding_specifier : EXCLUDE specifier_list
+ ;
+including_specifier : INCLUDE specifier_list
+ ;
+
+specifier : day_specifier
+ | day_specifier specifier_arg_list
+ ;
+
+specifier_arg_list : identifier
+ | identifier specifier_arg_list
+ ;
+
+day_specifier : DAY_SPECIFIER ;
+
+identifier : NUMBER | IDENTIFIER
+ ;
+
+specifier_list : specifier
+ | specifier_list COMMA specifier
+ ;
+
+duration : FROM time_interval AGO? TO time_interval AGO? #FromToDuration
+ | FROM time_interval AGO? #FromDuration
+ ;
+
+skip_distance : EVERY time_interval #SkipDistance
+ ;
+
+window_width : time_interval WINDOW? #WindowWidth
+ ;
+
+time_interval : time_amount time_unit #TimeInterval
+ ;
+
+time_amount : NUMBER #TimeAmount
+ ;
+
+time_unit : TIME_UNIT #TimeUnit
+ ;
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/84d34719/metron-analytics/metron-profiler-client/src/main/java/Window.tokens
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler-client/src/main/java/Window.tokens b/metron-analytics/metron-profiler-client/src/main/java/Window.tokens
new file mode 100644
index 0000000..b3390d2
--- /dev/null
+++ b/metron-analytics/metron-profiler-client/src/main/java/Window.tokens
@@ -0,0 +1,17 @@
+COMMA=1
+COLON=2
+WINDOW=3
+INCLUDE=4
+EXCLUDE=5
+NOW=6
+FROM=7
+EVERY=8
+TO=9
+AGO=10
+NUMBER=11
+IDENTIFIER=12
+DAY_SPECIFIER=13
+TIME_UNIT=14
+WS=15
+','=1
+':'=2
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/84d34719/metron-analytics/metron-profiler-client/src/main/java/WindowLexer.tokens
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler-client/src/main/java/WindowLexer.tokens b/metron-analytics/metron-profiler-client/src/main/java/WindowLexer.tokens
new file mode 100644
index 0000000..b3390d2
--- /dev/null
+++ b/metron-analytics/metron-profiler-client/src/main/java/WindowLexer.tokens
@@ -0,0 +1,17 @@
+COMMA=1
+COLON=2
+WINDOW=3
+INCLUDE=4
+EXCLUDE=5
+NOW=6
+FROM=7
+EVERY=8
+TO=9
+AGO=10
+NUMBER=11
+IDENTIFIER=12
+DAY_SPECIFIER=13
+TIME_UNIT=14
+WS=15
+','=1
+':'=2
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/84d34719/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/IntervalPredicate.java
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/IntervalPredicate.java b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/IntervalPredicate.java
new file mode 100644
index 0000000..f2f10d4
--- /dev/null
+++ b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/IntervalPredicate.java
@@ -0,0 +1,106 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.metron.profiler.client.stellar;
+
+import org.apache.commons.lang3.Range;
+
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+import java.util.Optional;
+import java.util.function.Function;
+import java.util.function.Predicate;
+
+/**
+ * A predicate applied to a type T which can be converted into a long which indicates whether it exists
+ * within a set of inclusive ranges of longs. Generally these ranges may be thought of as timestamps.
+ * In this interpretation, it will let you quickly indicate whether a given timestamp is within a set of timestamp
+ * ranges.
+ *
+ * @param <T>
+ */
+public class IntervalPredicate<T> implements Predicate<T> {
+ private final List<Range<Long>> intervals;
+ private final Function<T, Long> timestampTransformer;
+
+ /**
+ * In the situation where we want longs directly.
+ */
+ public static final class Identity extends IntervalPredicate<Long> {
+
+ public Identity(List<Range<Long>> intervals) {
+ super(x -> x, intervals, Long.class);
+ }
+ }
+
+ /**
+ * Construct an interval predicate given a set of intervals and a function to convert T's to timestamps.
+ * Please please please understand that intervals MUST be sorted.
+ *
+ * @param timestampTransformer The function to convert T's to timestamps.
+ * @param intervals A sorted list of timestamp intervals.
+ * @param clazz
+ */
+ public IntervalPredicate(Function<T, Long> timestampTransformer, List<Range<Long>> intervals, Class<T> clazz) {
+ this.intervals = intervals;
+ this.timestampTransformer = timestampTransformer;
+ }
+
+ private boolean containsInclusive(Range<Long> interval, long ts) {
+ return interval.contains(ts) || interval.getMaximum() == ts;
+ }
+
+
+ /**
+ * A helpful interval comparator that looks sorts the intervals according to left-side.
+ */
+ public static final Comparator<Range<Long>> INTERVAL_COMPARATOR = (o1, o2) -> {
+ if(o1.getMinimum() == o2.getMinimum() && o1.getMaximum() == o2.getMaximum()) {
+ return 0;
+ }
+ else {
+ int ret = Long.compare(o1.getMinimum(), o2.getMinimum());
+ if(ret == 0) {
+ return Long.compare(o1.getMaximum(), o2.getMaximum());
+ }
+ else {
+ return ret;
+ }
+ }
+ };
+
+ /**
+ * Determine if x is in the set of intervals in O(log*n) time.
+ * @param x
+ * @return true if in the set of intervals and false otherwise.
+ */
+ @Override
+ public boolean test(T x) {
+ long ts = timestampTransformer.apply(x);
+ int pos = Collections.binarySearch(intervals, Range.is(ts), INTERVAL_COMPARATOR);
+ if(pos < 0) {
+ pos = -pos - 1;
+ }
+
+ Optional<Range<Long>> right = pos >= 0 && pos < intervals.size()?Optional.of(intervals.get(pos)):Optional.empty();
+ Optional<Range<Long>> left = pos - 1 >= 0 && pos - 1 < intervals.size()?Optional.of(intervals.get(pos - 1)):Optional.empty();
+ return (right.isPresent()?containsInclusive(right.get(),ts):false) || (left.isPresent()?containsInclusive(left.get(),ts):false);
+ }
+}
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/84d34719/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/WindowLookback.java
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/WindowLookback.java b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/WindowLookback.java
new file mode 100644
index 0000000..c7390fa
--- /dev/null
+++ b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/stellar/WindowLookback.java
@@ -0,0 +1,107 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.metron.profiler.client.stellar;
+
+import com.google.common.cache.Cache;
+import com.google.common.cache.CacheBuilder;
+import org.apache.metron.common.dsl.Context;
+import org.apache.metron.common.dsl.ParseException;
+import org.apache.metron.common.dsl.Stellar;
+import org.apache.metron.common.dsl.StellarFunction;
+import org.apache.metron.common.utils.ConversionUtils;
+import org.apache.metron.profiler.ProfilePeriod;
+import org.apache.metron.profiler.client.window.Window;
+import org.apache.metron.profiler.client.window.WindowProcessor;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.TimeUnit;
+
+@Stellar(
+ namespace="PROFILE",
+ name="WINDOW",
+ description="The profiler periods associated with a window selector statement from an optional reference timestamp.",
+ params={
+ "windowSelector - The statement specifying the window to select.",
+ "now - Optional - The timestamp to use for now.",
+ "config_overrides - Optional - Map (in curly braces) of name:value pairs, each overriding the global config parameter " +
+ "of the same name. Default is the empty Map, meaning no overrides."
+ },
+ returns="The selected profile measurement periods. These are ProfilePeriod objects."
+)
+public class WindowLookback implements StellarFunction {
+
+ private Cache<String, Window> windowCache;
+
+ @Override
+ public Object apply(List<Object> args, Context context) throws ParseException {
+ Optional<Map> configOverridesMap = Optional.empty();
+ long now = System.currentTimeMillis();
+ String windowSelector = Util.getArg(0, String.class, args);
+ if(args.size() > 1) {
+ Optional<Object> arg2 = Optional.ofNullable(args.get(1));
+ Optional<Object> mapArg = args.size() > 2?Optional.ofNullable(args.get(2)):Optional.empty();
+ if(!mapArg.isPresent() && arg2.isPresent() && arg2.get() instanceof Map) {
+ mapArg = arg2;
+ }
+
+ if(arg2.isPresent() && arg2.get() instanceof Number) {
+ now = ConversionUtils.convert(arg2.get(), Long.class);
+ }
+
+ if(mapArg.isPresent()) {
+ Map rawMap = ConversionUtils.convert(mapArg.get(), Map.class);
+ configOverridesMap = rawMap == null || rawMap.isEmpty() ? Optional.empty() : Optional.of(rawMap);
+ }
+
+ }
+ Map<String, Object> effectiveConfigs = Util.getEffectiveConfig(context, configOverridesMap.orElse(null));
+ Long tickDuration = ProfilerConfig.PROFILER_PERIOD.get(effectiveConfigs, Long.class);
+ TimeUnit tickUnit = TimeUnit.valueOf(ProfilerConfig.PROFILER_PERIOD_UNITS.get(effectiveConfigs, String.class));
+ Window w = null;
+ try {
+ w = windowCache.get(windowSelector, () -> WindowProcessor.process(windowSelector));
+ } catch (ExecutionException e) {
+ throw new IllegalStateException("Unable to process " + windowSelector + ": " + e.getMessage(), e);
+ }
+ long end = w.getEndMillis(now);
+ long start = w.getStartMillis(now);
+ IntervalPredicate<ProfilePeriod> intervalSelector = new IntervalPredicate<>(period -> period.getStartTimeMillis()
+ , w.toIntervals(now)
+ , ProfilePeriod.class
+ );
+ return ProfilePeriod.visitPeriods(start, end, tickDuration, tickUnit, Optional.of(intervalSelector), period -> period);
+ }
+
+ @Override
+ public void initialize(Context context) {
+ windowCache = CacheBuilder.newBuilder()
+ .maximumSize(200)
+ .expireAfterAccess(10, TimeUnit.MINUTES)
+ .build();
+ }
+
+ @Override
+ public boolean isInitialized() {
+ return windowCache != null;
+ }
+}
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/84d34719/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/Window.java
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/Window.java b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/Window.java
new file mode 100644
index 0000000..8f697aa
--- /dev/null
+++ b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/Window.java
@@ -0,0 +1,159 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.metron.profiler.client.window;
+
+import com.google.common.collect.Iterables;
+import org.apache.commons.lang3.Range;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+import java.util.function.Function;
+import java.util.function.Predicate;
+import java.util.stream.Stream;
+
+/**
+ * A window is intended to compute the set of window intervals across time based on a reference time.
+ * The intervals are specified using a Window selector statement, which is a quasi-natural language grammar.
+ * Windows are intended to compute the set of intervals relative to a timestamp.
+ */
+public class Window {
+ private Function<Long, Long> startMillis ;
+ private Function<Long, Long> endMillis;
+ private List<Function<Long, Predicate<Long>>> includes = new ArrayList<>();
+ private List<Function<Long, Predicate<Long>>> excludes = new ArrayList<>();
+ private Optional<Long> binWidth = Optional.empty();
+ private Optional<Long> skipDistance = Optional.empty();
+
+ /**
+ * Return the start of the interval relative to the timestamp passed.
+ * @param now
+ * @return
+ */
+ public long getStartMillis(long now) {
+ return startMillis.apply(now);
+ }
+
+ void setStartMillis(Function<Long, Long> startMillis) {
+ this.startMillis = startMillis;
+ }
+
+ /**
+ * Return the end of the interval relative to the timestamp passed.
+ * @param now
+ * @return
+ */
+ public Long getEndMillis(long now) {
+ return endMillis.apply(now);
+ }
+
+ void setEndMillis(Function<Long, Long> endMillis) {
+ this.endMillis = endMillis;
+ }
+
+ /**
+ * Get the set of inclusion predicates. If any of these are true as applied to the window interval start time,
+ * then a field is included unless it's explicitly excluded.
+ * @param now
+ * @return
+ */
+ public Iterable<Predicate<Long>> getIncludes(long now) {
+ return Iterables.transform(includes, f -> f.apply(now));
+ }
+
+ void setIncludes(List<Function<Long, Predicate<Long>>> includes) {
+ this.includes = includes;
+ }
+
+ /**
+ * Get the set of exclusion predicates. If any of these exclusion predicates are true as applied to the window
+ * interval start time, then the interval is excluded. NOTE: Exclusions trump inclusions.
+ * @param now
+ * @return
+ */
+ public Iterable<Predicate<Long>> getExcludes(long now){
+ return Iterables.transform(excludes, f -> f.apply(now));
+ }
+
+ void setExcludes(List<Function<Long, Predicate<Long>>> excludes) {
+ this.excludes = excludes;
+ }
+
+ /**
+ * The bin width. This is fixed regardless of relative time.
+ * @return
+ */
+ public Optional<Long> getBinWidth() {
+ return binWidth;
+ }
+
+ void setBinWidth(long binWidth) {
+ this.binWidth = Optional.of(binWidth);
+ }
+
+ /**
+ * The skip distance. How long between interval windows that one must go.
+ * @return
+ */
+ public Optional<Long> getSkipDistance() {
+ return skipDistance;
+ }
+
+ void setSkipDistance(long skipDistance) {
+ this.skipDistance = Optional.of(skipDistance);
+ }
+
+ /**
+ * Compute the set of sorted (oldest to newest) window intervals relative to the passed timestamp
+ * given inclusion and exclusion predicates.
+ *
+ * @param now
+ * @return
+ */
+ public List<Range<Long>> toIntervals(long now) {
+ List<Range<Long>> intervals = new ArrayList<>();
+ long startMillis = getStartMillis(now);
+ long endMillis = getEndMillis(now);
+ Iterable<Predicate<Long>> includes = getIncludes(now);
+ Iterable<Predicate<Long>> excludes = getExcludes(now);
+ //if we don't have a skip distance, then we just skip past everything to make the window dense
+ long skipDistance = getSkipDistance().orElse(Long.MAX_VALUE);
+ //if we don't have a window width, then we want the window to be completely dense.
+ Optional<Long> binWidthOpt = getBinWidth();
+ long binWidth = binWidthOpt.isPresent()?binWidthOpt.get():endMillis-startMillis;
+
+ for(long left = startMillis;left >= 0 && left + binWidth <= endMillis;left += skipDistance) {
+ Range<Long> interval = Range.between(left, left + binWidth);
+ boolean include = includes.iterator().hasNext()?false:true;
+ for(Predicate<Long> inclusionPredicate : includes) {
+ include |= inclusionPredicate.test(left);
+ }
+ if(include) {
+ for(Predicate<Long> exclusionPredicate : excludes) {
+ include &= !exclusionPredicate.test(left);
+ }
+ }
+ if(include) {
+ intervals.add(interval);
+ }
+ }
+ return intervals;
+ }
+}
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/84d34719/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java
new file mode 100644
index 0000000..16a7513
--- /dev/null
+++ b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/WindowProcessor.java
@@ -0,0 +1,422 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.metron.profiler.client.window;
+
+import org.antlr.v4.runtime.*;
+import org.antlr.v4.runtime.tree.ParseTree;
+import org.apache.metron.common.dsl.ErrorListener;
+import org.apache.metron.common.dsl.GrammarUtils;
+import org.apache.metron.common.dsl.ParseException;
+import org.apache.metron.common.dsl.Token;
+import org.apache.metron.common.utils.ConversionUtils;
+import org.apache.metron.profiler.client.window.generated.WindowBaseListener;
+import org.apache.metron.profiler.client.window.generated.WindowLexer;
+import org.apache.metron.profiler.client.window.generated.WindowParser;
+import org.apache.metron.profiler.client.window.predicates.DayPredicates;
+
+import java.util.*;
+import java.util.concurrent.TimeUnit;
+import java.util.function.Function;
+import java.util.function.Predicate;
+
+import static org.apache.commons.lang3.StringUtils.isEmpty;
+
+/**
+ * The WindowProcessor instance provides the parser callbacks for the Window selector language. This constructs
+ * a Window object to be used to compute sparse window intervals across time.
+ */
+public class WindowProcessor extends WindowBaseListener {
+ private Throwable throwable;
+ private Deque<Token<?>> stack;
+ private static final Token<Object> LIST_MARKER = new Token<>(null, Object.class);
+ private static final Token<Object> SPECIFIER_MARKER = new Token<>(null, Object.class);
+ private Window window;
+
+ public WindowProcessor() {
+ this.stack = new ArrayDeque<>();
+ this.window = new Window();
+ }
+
+ /**
+ * Retrieve the window constructed from the window selector statement.
+ * @return
+ */
+ public Window getWindow() {
+ return window;
+ }
+
+ private void enterList() {
+ stack.push(LIST_MARKER);
+ }
+
+ private List<Function<Long, Predicate<Long>>> getPredicates() {
+ LinkedList<Function<Long, Predicate<Long>>> predicates = new LinkedList<>();
+ while (true) {
+ Token<?> token = stack.pop();
+ if (token == LIST_MARKER) {
+ break;
+ } else {
+ predicates.addFirst((Function<Long, Predicate<Long>>) token.getValue());
+ }
+ }
+ return predicates;
+ }
+
+ /**
+ * If we see an identifier, an argument for an inclusion/exclusion predicate, then we want to just push it onto the
+ * stack without its ':'.
+ * @param ctx
+ */
+ @Override
+ public void exitIdentifier(WindowParser.IdentifierContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ stack.push(new Token<>(ctx.getText().substring(1), String.class));
+ }
+
+ /**
+ * When we enter a specifier then we want to push onto the stack the specifier marker so we know when
+ * the specifier parameters end.
+ * @param ctx
+ */
+ @Override
+ public void enterSpecifier(WindowParser.SpecifierContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ stack.push(SPECIFIER_MARKER);
+ }
+
+ /**
+ * Read the specifier params off the stack in FIFO order until we get to the specifier marker. Now we can
+ * construct the specifier, which is a Function which constructs a Selector Predicate based on the args
+ * passed to the selector e.g. holidays:us:nyc would have 2 args us and nyc.
+ *
+ * @param ctx
+ */
+ @Override
+ public void exitSpecifier(WindowParser.SpecifierContext ctx) {
+ LinkedList<String> args = new LinkedList<>();
+
+ while (true) {
+ Token<?> token = stack.pop();
+ if (token == SPECIFIER_MARKER) {
+ break;
+ } else {
+ args.addFirst((String) token.getValue());
+ }
+ }
+ String specifier = args.removeFirst();
+ List<String> arg = args.size() > 0?args:new ArrayList<>();
+ Function<Long, Predicate<Long>> predicate = null;
+ try {
+ if (specifier.equals("THIS DAY OF THE WEEK") || specifier.equals("THIS DAY OF WEEK")) {
+ predicate = now -> DayPredicates.dayOfWeekPredicate(DayPredicates.getDayOfWeek(now));
+ } else {
+ final Predicate<Long> dayOfWeekPredicate = DayPredicates.create(specifier, arg);
+ predicate = now -> dayOfWeekPredicate;
+ }
+ stack.push(new Token<>(predicate, Function.class));
+ }
+ catch(Throwable t) {
+ throwable = t;
+ }
+ }
+
+ /**
+ * Normalize the day specifier e.g. tuesdays -> tuesday and push onto the stack.
+ * @param ctx
+ */
+ @Override
+ public void exitDay_specifier(WindowParser.Day_specifierContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ String specifier = ctx.getText().toUpperCase();
+ if(specifier.length() == 0 && ctx.exception != null){
+ IllegalStateException ise = new IllegalStateException("Invalid day specifier: " + ctx.getStart().getText(), ctx.exception);
+ throwable = ise;
+ throw ise;
+ }
+ if(specifier.endsWith("S")) {
+ specifier = specifier.substring(0, specifier.length() - 1);
+ }
+ stack.push(new Token<>(specifier, String.class));
+ }
+
+ /**
+ * When we're beginning an exclusion specifier list, then we push the list token so we
+ * know when we're done processing
+ * @param ctx
+ */
+ @Override
+ public void enterExcluding_specifier(WindowParser.Excluding_specifierContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ enterList();
+ }
+
+ /**
+ * And when we're done with the exclusions specifier, then we set the exclusions
+ * to the predicates we've put on the stack.
+ * @param ctx
+ */
+ @Override
+ public void exitExcluding_specifier(WindowParser.Excluding_specifierContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ window.setExcludes(getPredicates());
+ }
+
+ /**
+ * When we're beginning an inclusion specifier list, then we push the list token so we
+ * know when we're done processing
+ * @param ctx
+ */
+ @Override
+ public void enterIncluding_specifier(WindowParser.Including_specifierContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ enterList();
+ }
+
+ /**
+ * And when we're done with the inclusions specifier, then we set the exclusions
+ * to the predicates we've put on the stack.
+ * @param ctx
+ */
+ @Override
+ public void exitIncluding_specifier(WindowParser.Including_specifierContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ window.setIncludes(getPredicates());
+ }
+
+ private void setFromTo(long from, long to) {
+ window.setEndMillis(now -> now - Math.min(to, from));
+ window.setStartMillis(now -> now - Math.max(from, to));
+ }
+
+ /**
+ * If we have a total time interval that we've specified, then we want to set the interval.
+ * NOTE: the interval will be set based on the smallest to largest being the start and end time respectively.
+ * Thus 'from 1 hour ago to 1 day ago' and 'from 1 day ago to 1 hour ago' are equivalent.
+ * @param ctx
+ */
+ @Override
+ public void exitFromToDuration(org.apache.metron.profiler.client.window.generated.WindowParser.FromToDurationContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ Token<?> toInterval = stack.pop();
+ Token<?> fromInterval = stack.pop();
+ Long to = (Long)toInterval.getValue();
+ Long from = (Long)fromInterval.getValue();
+ setFromTo(from, to);
+ }
+
+ /**
+ * When we've done specifying a from, then we want to set it.
+ * @param ctx
+ */
+ @Override
+ public void exitFromDuration(org.apache.metron.profiler.client.window.generated.WindowParser.FromDurationContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ Token<?> timeInterval = stack.pop();
+ Long from = (Long)timeInterval.getValue();
+ setFromTo(from, 0);
+ }
+
+ /**
+ * We've set a skip distance.
+ * @param ctx
+ */
+ @Override
+ public void exitSkipDistance(org.apache.metron.profiler.client.window.generated.WindowParser.SkipDistanceContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ Token<?> timeInterval = stack.pop();
+ Long width = (Long)timeInterval.getValue();
+ window.setSkipDistance(width);
+ }
+
+ /**
+ * We've set a window width.
+ * @param ctx
+ */
+ @Override
+ public void exitWindowWidth(org.apache.metron.profiler.client.window.generated.WindowParser.WindowWidthContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ Token<?> timeInterval = stack.pop();
+ Long width = (Long)timeInterval.getValue();
+ window.setBinWidth(width);
+ window.setStartMillis(now -> now - width);
+ window.setEndMillis(now -> now);
+ }
+
+ /**
+ * We've set a time interval, which is a value along with a unit.
+ * @param ctx
+ */
+ @Override
+ public void exitTimeInterval(org.apache.metron.profiler.client.window.generated.WindowParser.TimeIntervalContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ Token<?> timeUnit = stack.pop();
+ Token<?> timeDuration = stack.pop();
+ long duration = ConversionUtils.convert(timeDuration.getValue(), Long.class);
+ TimeUnit unit = (TimeUnit) timeUnit.getValue();
+ stack.push(new Token<>(unit.toMillis(duration), Long.class));
+ }
+
+ /**
+ * We've set a time amount, which is integral.
+ * @param ctx
+ */
+ @Override
+ public void exitTimeAmount(org.apache.metron.profiler.client.window.generated.WindowParser.TimeAmountContext ctx) {
+ if(checkForException(ctx)) {
+ return;
+ }
+ if(ctx.getText().length() == 0) {
+ throwable = new IllegalStateException("Unable to process empty string.");
+ return;
+ }
+ long duration = Long.parseLong(ctx.getText());
+ stack.push(new Token<>(duration, Long.class));
+ }
+
+ /**
+ * We've set a time unit. We support the timeunits provided by java.util.concurrent.TimeUnit
+ * @param ctx
+ */
+ @Override
+ public void exitTimeUnit(org.apache.metron.profiler.client.window.generated.WindowParser.TimeUnitContext ctx) {
+ checkForException(ctx);
+ switch(normalizeTimeUnit(ctx.getText())) {
+ case "DAY":
+ stack.push(new Token<>(TimeUnit.DAYS, TimeUnit.class));
+ break;
+ case "HOUR":
+ stack.push(new Token<>(TimeUnit.HOURS, TimeUnit.class));
+ break;
+ case "MINUTE":
+ stack.push(new Token<>(TimeUnit.MINUTES, TimeUnit.class));
+ break;
+ case "SECOND":
+ stack.push(new Token<>(TimeUnit.SECONDS, TimeUnit.class));
+ break;
+ default:
+ throw new IllegalStateException("Unsupported time unit: " + ctx.getText()
+ + ". Supported units are limited to: day, hour, minute, second "
+ + "with any pluralization or capitalization.");
+ }
+ }
+
+ private boolean checkForException(ParserRuleContext ctx) {
+ if(throwable != null) {
+ return true;
+ }
+ else if(ctx.exception != null) {
+ return true;
+ }
+ return false;
+ }
+
+ private static String normalizeTimeUnit(String s) {
+ String ret = s.toUpperCase().replaceAll("[^A-Z]", "");
+ if(ret.endsWith("S")) {
+ return ret.substring(0, ret.length() - 1);
+ }
+ return ret;
+ }
+
+ private static TokenStream createTokenStream(String statement) {
+ if (statement == null || isEmpty(statement.trim())) {
+ return null;
+ }
+ statement = statement.trim();
+ ANTLRInputStream input = new ANTLRInputStream(statement);
+ WindowLexer lexer = new WindowLexer(input);
+ lexer.removeErrorListeners();
+ lexer.addErrorListener(new ErrorListener());
+ TokenStream tokens = new CommonTokenStream(lexer);
+ return tokens;
+ }
+
+ private static WindowParser createParser(TokenStream tokens, Optional<WindowProcessor> windowProcessor) {
+ WindowParser parser = new WindowParser(tokens);
+ if(windowProcessor.isPresent()) {
+ parser.addParseListener(windowProcessor.get());
+ }
+ parser.removeErrorListeners();
+ parser.addErrorListener(new ErrorListener());
+ return parser;
+ }
+
+ /**
+ * Create a reusable Window object (parameterized by time) from a statement specifying the window intervals
+ * conforming to the Window grammar.
+ *
+ * @param statement
+ * @return
+ * @throws ParseException
+ */
+ public static Window process(String statement) throws ParseException {
+ TokenStream tokens = createTokenStream(statement);
+ if(tokens == null) {
+ return null;
+ }
+ WindowProcessor treeBuilder = new WindowProcessor();
+ WindowParser parser = createParser(tokens, Optional.of(treeBuilder));
+ parser.window();
+ if(treeBuilder.throwable != null) {
+ throw new ParseException(treeBuilder.throwable.getMessage(), treeBuilder.throwable);
+ }
+ return treeBuilder.getWindow();
+ }
+
+ /**
+ * Create a textual representation of the syntax tree. This is useful for those intrepid souls
+ * who wish to extend the window selector language. God speed.
+ * @param statement
+ * @return A string representation of the syntax tree.
+ */
+ public static String syntaxTree(String statement) {
+ TokenStream tokens = createTokenStream(statement);
+ if(tokens == null) {
+ return null;
+ }
+ WindowParser parser = createParser(tokens, Optional.empty());
+ ParseTree tree = parser.window();
+ return GrammarUtils.toSyntaxTree(tree) ;
+ }
+}
http://git-wip-us.apache.org/repos/asf/incubator-metron/blob/84d34719/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/generated/WindowBaseListener.java
----------------------------------------------------------------------
diff --git a/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/generated/WindowBaseListener.java b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/generated/WindowBaseListener.java
new file mode 100644
index 0000000..163cf1f
--- /dev/null
+++ b/metron-analytics/metron-profiler-client/src/main/java/org/apache/metron/profiler/client/window/generated/WindowBaseListener.java
@@ -0,0 +1,276 @@
+// Generated from org/apache/metron/profiler/client/window/generated/Window.g4 by ANTLR 4.5
+package org.apache.metron.profiler.client.window.generated;
+
+//CHECKSTYLE:OFF
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+import org.antlr.v4.runtime.ParserRuleContext;
+import org.antlr.v4.runtime.misc.NotNull;
+import org.antlr.v4.runtime.tree.ErrorNode;
+import org.antlr.v4.runtime.tree.TerminalNode;
+
+/**
+ * This class provides an empty implementation of {@link WindowListener},
+ * which can be extended to create a listener which only needs to handle a subset
+ * of the available methods.
+ */
+public class WindowBaseListener implements WindowListener {
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterWindow(WindowParser.WindowContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitWindow(WindowParser.WindowContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterNonRepeatingWindow(WindowParser.NonRepeatingWindowContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitNonRepeatingWindow(WindowParser.NonRepeatingWindowContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterRepeatingWindow(WindowParser.RepeatingWindowContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitRepeatingWindow(WindowParser.RepeatingWindowContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterDenseWindow(WindowParser.DenseWindowContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitDenseWindow(WindowParser.DenseWindowContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterExcluding_specifier(WindowParser.Excluding_specifierContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitExcluding_specifier(WindowParser.Excluding_specifierContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterIncluding_specifier(WindowParser.Including_specifierContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitIncluding_specifier(WindowParser.Including_specifierContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterSpecifier(WindowParser.SpecifierContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitSpecifier(WindowParser.SpecifierContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterSpecifier_arg_list(WindowParser.Specifier_arg_listContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitSpecifier_arg_list(WindowParser.Specifier_arg_listContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterDay_specifier(WindowParser.Day_specifierContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitDay_specifier(WindowParser.Day_specifierContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterIdentifier(WindowParser.IdentifierContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitIdentifier(WindowParser.IdentifierContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterSpecifier_list(WindowParser.Specifier_listContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitSpecifier_list(WindowParser.Specifier_listContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterFromToDuration(WindowParser.FromToDurationContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitFromToDuration(WindowParser.FromToDurationContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterFromDuration(WindowParser.FromDurationContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitFromDuration(WindowParser.FromDurationContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterSkipDistance(WindowParser.SkipDistanceContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitSkipDistance(WindowParser.SkipDistanceContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterWindowWidth(WindowParser.WindowWidthContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitWindowWidth(WindowParser.WindowWidthContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterTimeInterval(WindowParser.TimeIntervalContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitTimeInterval(WindowParser.TimeIntervalContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterTimeAmount(WindowParser.TimeAmountContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitTimeAmount(WindowParser.TimeAmountContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterTimeUnit(WindowParser.TimeUnitContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitTimeUnit(WindowParser.TimeUnitContext ctx) { }
+
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void enterEveryRule(ParserRuleContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void exitEveryRule(ParserRuleContext ctx) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void visitTerminal(TerminalNode node) { }
+ /**
+ * {@inheritDoc}
+ *
+ * <p>The default implementation does nothing.</p>
+ */
+ @Override public void visitErrorNode(ErrorNode node) { }
+}
\ No newline at end of file