You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Andrew Sammut <as...@kixeye.com> on 2014/03/28 06:55:59 UTC

morphline if conditions contains

Hi all,

I'm attempting to place a conditional statement in morphline (in order to
only process the syslog records that contain JSON data. However, I can
never get a match:

Here is the code snippet that I've written.

      { setValues { message_string : "test message" } }
      # only perform conversion on certain records
      {
        if {
          conditions : [
            { contains { message_string : "test" } }
          ]
          then : [
            { logDebug { format : "matched" } }
          ]
          else : [
            { logDebug { format : "not matched" } }
          ]
        }
      }
      { setValues { message_string : [] } }

And here is the information from the log file:

28 Mar 2014 05:48:19,779 DEBUG [pool-8-thread-1]
(com.cloudera.cdk.morphline.stdlib.ContainsBuilder$Contains.doProcess:86)
 - Contains command failed because it could not find any of [test] in
values: [test message] for command: {
    # /etc/flume-ng/conf/morphline.conf: 22
    "message_string" : "test"
}
28 Mar 2014 05:48:19,782 DEBUG [pool-8-thread-1]
(com.cloudera.cdk.morphline.base.AbstractCommand.process:161)  - Command
failed!
28 Mar 2014 05:48:19,786 DEBUG [pool-8-thread-1]
(com.cloudera.cdk.morphline.stdlib.LogDebugBuilder$LogDebug.log:63)  - not
matched

I know the syntax is correct, as the tcp syslog source would not start if
it was busted. Can anyone suggest why it's not matching?

Regards,
Andrew S

-- 
Andrew Sammut
Lead Reliability Engineer (AUS)

Mobile: +61 (0)435 072 784
Google Voice: +1 (415) 548 1927
Email: asammut@kixeye.com
Skype: kix.asammut

Re: morphline if conditions contains

Posted by Wolfgang Hoschek <wh...@cloudera.com>.
The “contains” command tests whether X is one of the elements in list Y, not a substring of some other string. You can use a mini script with the “java" command for that.

Wolfgang.

On Mar 27, 2014, at 10:55 PM, Andrew Sammut <as...@kixeye.com> wrote:

> 
> Hi all,
> 
> I'm attempting to place a conditional statement in morphline (in order to only process the syslog records that contain JSON data. However, I can never get a match:
> 
> Here is the code snippet that I've written.
> 
>       { setValues { message_string : "test message" } }
>       # only perform conversion on certain records
>       {
>         if {
>           conditions : [
>             { contains { message_string : "test" } }
>           ]
>           then : [
>             { logDebug { format : "matched" } }
>           ]
>           else : [
>             { logDebug { format : "not matched" } }
>           ]
>         }
>       }
>       { setValues { message_string : [] } }
> 
> And here is the information from the log file:
> 
> 28 Mar 2014 05:48:19,779 DEBUG [pool-8-thread-1] (com.cloudera.cdk.morphline.stdlib.ContainsBuilder$Contains.doProcess:86)  - Contains command failed because it could not find any of [test] in values: [test message] for command: {
>     # /etc/flume-ng/conf/morphline.conf: 22
>     "message_string" : "test"
> }
> 28 Mar 2014 05:48:19,782 DEBUG [pool-8-thread-1] (com.cloudera.cdk.morphline.base.AbstractCommand.process:161)  - Command failed!
> 28 Mar 2014 05:48:19,786 DEBUG [pool-8-thread-1] (com.cloudera.cdk.morphline.stdlib.LogDebugBuilder$LogDebug.log:63)  - not matched
> 
> I know the syntax is correct, as the tcp syslog source would not start if it was busted. Can anyone suggest why it's not matching?
> 
> Regards,
> Andrew S
> 
> -- 
> Andrew Sammut
> Lead Reliability Engineer (AUS)
> 
> Mobile: +61 (0)435 072 784
> Google Voice: +1 (415) 548 1927
> Email: asammut@kixeye.com
> Skype: kix.asammut