You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by nickwallen <gi...@git.apache.org> on 2016/08/25 13:50:08 UTC
[GitHub] incubator-metron pull request #231: METRON-394 Create Stellar Date Functions...
GitHub user nickwallen opened a pull request:
https://github.com/apache/incubator-metron/pull/231
METRON-394 Create Stellar Date Functions to Use with Profile 'Group By'
### [METRON-394](https://issues.apache.org/jira/browse/METRON-394)
[METRON-392](https://issues.apache.org/jira/browse/METRON-392) and #230 Allows a user to optionally define a custom set of 'groupBy' expressions that controls how Profile data is persisted. This is intended to allow for contiguous scans when training on subsets of the data.
A common use case would be grouping the data by some calendar period, like day of week. This would allow a contiguous scan to access all profile data, for example, on Mondays only. The Stellar expression DAY_OF_WEEK(start) would achieve this.
The following functions have been added for this purpose.
* WEEK_OF_MONTH
* WEEK_OF_YEAR
* DAY_OF_WEEK
* DAY_OF_MONTH
* DAY_OF_YEAR
* MONTH
* YEAR
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/incubator-metron METRON-394
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/231.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #231
----
commit 12873b151eafaa05c712efb42261fc3ca5e5bf43
Author: Nick Allen <ni...@nickallen.org>
Date: 2016-08-25T13:44:37Z
METRON-394 Create Stellar Date Functions to Use with Profile 'Group By'
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #231: METRON-394 Create Stellar Date Functions...
Posted by cestella <gi...@git.apache.org>.
Github user cestella commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/231#discussion_r76414651
--- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/DateFunctions.java ---
@@ -114,17 +122,176 @@ public Object apply(List<Object> objects) {
}
if(dateObj != null && formatObj != null) {
try {
- return getEpochTime(dateObj.toString()
- , formatObj.toString()
- , tzObj == null?Optional.empty():Optional.of(tzObj.toString())
- );
- } catch (ExecutionException e) {
- return null;
- } catch (ParseException e) {
+ Optional<String> tz = (tzObj == null) ? Optional.empty() : Optional.of(tzObj.toString());
+ return getEpochTime(dateObj.toString(), formatObj.toString(), tz);
+
+ } catch (ExecutionException | ParseException e) {
return null;
}
}
return null;
}
}
+
+ /**
+ * Stellar Function: DAY_OF_WEEK
+ *
+ * The numbered day within the week. The first day of the week, Sunday, has a value of 1.
+ */
+ public static class DayOfWeek extends BaseStellarFunction {
+ @Override
+ public Object apply(List<Object> args) {
+
+ // expect epoch milliseconds
+ Long epochMillis = ConversionUtils.convert(args.get(0), Long.class);
--- End diff --
can you do an args.get(0) null check before this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron issue #231: METRON-394 Create Stellar Date Functions to Use...
Posted by cestella <gi...@git.apache.org>.
Github user cestella commented on the issue:
https://github.com/apache/incubator-metron/pull/231
Looks good, +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #231: METRON-394 Create Stellar Date Functions...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/231#discussion_r76715633
--- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/DateFunctions.java ---
@@ -114,17 +122,176 @@ public Object apply(List<Object> objects) {
}
if(dateObj != null && formatObj != null) {
try {
- return getEpochTime(dateObj.toString()
- , formatObj.toString()
- , tzObj == null?Optional.empty():Optional.of(tzObj.toString())
- );
- } catch (ExecutionException e) {
- return null;
- } catch (ParseException e) {
+ Optional<String> tz = (tzObj == null) ? Optional.empty() : Optional.of(tzObj.toString());
+ return getEpochTime(dateObj.toString(), formatObj.toString(), tz);
+
+ } catch (ExecutionException | ParseException e) {
return null;
}
}
return null;
}
}
+
+ /**
+ * Stellar Function: DAY_OF_WEEK
+ *
+ * The numbered day within the week. The first day of the week, Sunday, has a value of 1.
+ */
+ public static class DayOfWeek extends BaseStellarFunction {
+ @Override
+ public Object apply(List<Object> args) {
+
+ // expect epoch milliseconds
+ Long epochMillis = ConversionUtils.convert(args.get(0), Long.class);
--- End diff --
No worries. Thanks for the review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #231: METRON-394 Create Stellar Date Functions...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-metron/pull/231
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-metron pull request #231: METRON-394 Create Stellar Date Functions...
Posted by cestella <gi...@git.apache.org>.
Github user cestella commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/231#discussion_r76414698
--- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/dsl/functions/DateFunctions.java ---
@@ -114,17 +122,176 @@ public Object apply(List<Object> objects) {
}
if(dateObj != null && formatObj != null) {
try {
- return getEpochTime(dateObj.toString()
- , formatObj.toString()
- , tzObj == null?Optional.empty():Optional.of(tzObj.toString())
- );
- } catch (ExecutionException e) {
- return null;
- } catch (ParseException e) {
+ Optional<String> tz = (tzObj == null) ? Optional.empty() : Optional.of(tzObj.toString());
+ return getEpochTime(dateObj.toString(), formatObj.toString(), tz);
+
+ } catch (ExecutionException | ParseException e) {
return null;
}
}
return null;
}
}
+
+ /**
+ * Stellar Function: DAY_OF_WEEK
+ *
+ * The numbered day within the week. The first day of the week, Sunday, has a value of 1.
+ */
+ public static class DayOfWeek extends BaseStellarFunction {
+ @Override
+ public Object apply(List<Object> args) {
+
+ // expect epoch milliseconds
+ Long epochMillis = ConversionUtils.convert(args.get(0), Long.class);
+ if(epochMillis == null) {
+ return null;
+ }
+
+ // create a calendar
+ Calendar calendar = Calendar.getInstance();
+ calendar.setTimeInMillis(epochMillis);
+
+ return calendar.get(Calendar.DAY_OF_WEEK);
+ }
+ }
+
+ /**
+ * Stellar Function: DAY_OF_MONTH
+ *
+ * The day within the month. The first day within the month has a value of 1.
+ */
+ public static class DayOfMonth extends BaseStellarFunction {
+ @Override
+ public Object apply(List<Object> args) {
+
+ // expect epoch milliseconds
+ Long epochMillis = ConversionUtils.convert(args.get(0), Long.class);
--- End diff --
args.get(0) null check, please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---