You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/05/27 13:56:12 UTC
[jira] [Commented] (METRON-189) Add the ability to do global
validations on messages passing through the parser.
[ https://issues.apache.org/jira/browse/METRON-189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15304086#comment-15304086 ]
ASF GitHub Bot commented on METRON-189:
---------------------------------------
GitHub user cestella opened a pull request:
https://github.com/apache/incubator-metron/pull/138
METRON-189: Add the ability to do global validations on messages passing through the parser.
Allow the user to specify field level or message level validations to ensure messages coming from the parser are valid. For instance, allow the ability ensure that a field is an IPv4 address.
If a field is invalid, send to a separate stream from the parser bolt. Follow-on work should be done to send this stream to the index for after-the-fact inspection.
I added the following validation functions:
* `MQL` : Execute a Query Language statement. Expects the query string in the `condition` field of the config.
* `IP` : Validates that the input fields are an IP addres. By default, if no configuration is set, it assumes `IPV4`, but you can specify the type by passing in the config by passing in `type` with either `IPV6` or `IPV4`.
* `DOMAIN` : Validates that the fields are all domains.
* `EMAIL` : Validates that the fields are all email addresses
* `URL` : Validates that the fields are all URLs
* `DATE` : Validates that the fields are a date. Expects `format` in the config.
* `INTEGER` : Validates that the fields are an integer. String representation of an integer is allowed.
* `REGEX_MATCH` : Validates that the fields match a regex. Expects `pattern` in the config.
* `NOT_EMPTY` : Validates that the fields exist and are not empty (after trimming.)
Because of the nice overlap, I also added these functions to the query language, so query language rules can take advantage of `IS_IP(field1)` for instance.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cestella/incubator-metron validation
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/138.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #138
----
commit 258ae1cde4cdb5092edeb00000dc6a50f0af2c1c
Author: cstella <ce...@gmail.com>
Date: 2016-05-27T02:46:35Z
Added validation framework.
commit d2ae7b3a8a96787523755ce0107c64909e2729bd
Author: cstella <ce...@gmail.com>
Date: 2016-05-27T03:39:52Z
Updating validators to work with unit tests.
commit ccc42b3d7390e98ade326c5651719d9b6d4533b8
Author: cstella <ce...@gmail.com>
Date: 2016-05-27T13:50:30Z
Updating readme.
----
> Add the ability to do global validations on messages passing through the parser.
> --------------------------------------------------------------------------------
>
> Key: METRON-189
> URL: https://issues.apache.org/jira/browse/METRON-189
> Project: Metron
> Issue Type: New Feature
> Reporter: Casey Stella
> Assignee: Casey Stella
>
> Allow the user to specify field level or message level validations to ensure messages coming from the parser are valid. For instance, allow the ability ensure that a field is an IPv4 address.
> If a field is invalid, send to a separate stream from the parser bolt. Follow-on work should be done to send this stream to the index for after-the-fact inspection.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)