[jira] [Commented] (FLUME-2961) Make TaildirSource work with multiline

["Endre Major (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16685300#comment-16685300 ] 

Endre Major commented on FLUME-2961:
------------------------------------

I would like to create a PR from this issue. Could someone help me and explain the following parameter:
"multilineMatched: Whether to match the pattern. If 'false', a message not matching the pattern will be combined with the previous or the next line."
I tried understanding the working of this parameter from the tests.
If we use multiline mode why do we get events without the multiline pattern?
It seems to me that when we use multilineMatched=true, we get the line that matches the pattern and an the previous or the next non matching line, depending on the parameters. Is that correct? Is that the intention.

Please help.

> Make TaildirSource work with multiline
> --------------------------------------
>
>                 Key: FLUME-2961
>                 URL: https://issues.apache.org/jira/browse/FLUME-2961
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: 1.7.0
>            Reporter: tinawenqiao
>            Assignee: Endre Major
>            Priority: Major
>             Fix For: 1.9.0
>
>         Attachments: FLUME-2961_1.patch, FLUME-2961_2.patch, FLUME-2961_4.patch, FLUME-2961_5.patch
>
>
> TaidirSource defaults to LINE, this has issue when multiline log events like stack traces and have request/responses. Following part is Java traceback logs. We expect to have log line start regex Key to aggregate all the log lines till the next regex key is found.
> 2016-07-16 14:59:43,956 ERROR lifecycleSupervisor-1-7 LifecycleSupervisor.run - Unable to start EventDrivenSourceRunner: { source:cn.yottabyte.flume.source.http.HTTPSource{name:sourceHttp,state:IDLE} } - Exception follows.
> java.lang.IllegalStateException: Running HTTP Server found in source: sourceHttp before I started one. Will not attempt to start.
>     at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
>     at cn.yottabyte.flume.source.http.HTTPSource.startHttpSourceServer(HTTPSource.java:170)
>     at cn.yottabyte.flume.source.http.HTTPSource.start(HTTPSource.java:166)
>     at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
>     at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>     at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>     at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@flume.apache.org
For additional commands, e-mail: issues-help@flume.apache.org