[jira] [Commented] (METRON-189) Add the ability to do global validations on messages passing through the parser.

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/METRON-189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311360#comment-15311360 ] 

ASF GitHub Bot commented on METRON-189:
---------------------------------------

Github user merrimanr commented on a diff in the pull request:

    https://github.com/apache/incubator-metron/pull/138#discussion_r65461745
  
    --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java ---
    @@ -140,18 +142,25 @@ public void execute(Tuple tuple) {
         try {
           boolean ackTuple = true;
           if(sensorParserConfig != null) {
    +        List<FieldValidator> fieldValidations = getConfigurations().getFieldValidations();
             List<JSONObject> messages = parser.parse(originalMessage);
             for (JSONObject message : messages) {
               if (parser.validate(message)) {
    -            if (filter != null && filter.emitTuple(message)) {
    -              ackTuple = !isBulk;
    -              message.put(Constants.SENSOR_TYPE, getSensorType());
    -              for (FieldTransformer handler : sensorParserConfig.getFieldTransformations()) {
    -                if (handler != null) {
    -                  handler.transformAndUpdate(message, sensorParserConfig.getParserConfig());
    +            if(!isGloballyValid(message, fieldValidations)) {
    +              message.put(Constants.SENSOR_TYPE, getSensorType()+ ".invalid");
    +              collector.emit(Constants.INVALID_STREAM, new Values(message));
    +            }
    +            else if (filter != null && filter.emitTuple(message)) {
    +              if (filter != null && filter.emitTuple(message)) {
    --- End diff --
    
    Isn't this a duplicate of the previous line?


> Add the ability to do global validations on messages passing through the parser.
> --------------------------------------------------------------------------------
>
>                 Key: METRON-189
>                 URL: https://issues.apache.org/jira/browse/METRON-189
>             Project: Metron
>          Issue Type: New Feature
>            Reporter: Casey Stella
>            Assignee: Casey Stella
>
> Allow the user to specify field level or message level validations to ensure messages coming from the parser are valid.  For instance, allow the ability ensure that a field is an IPv4 address.
> If a field is invalid, send to a separate stream from the parser bolt.  Follow-on work should be done to send this stream to the index for after-the-fact inspection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)