You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flume.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/08/06 15:25:00 UTC
[jira] [Commented] (FLUME-3050) add counters for error conditions
and expose to monitor URL
[ https://issues.apache.org/jira/browse/FLUME-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16570353#comment-16570353 ]
ASF GitHub Bot commented on FLUME-3050:
---------------------------------------
GitHub user majorendre opened a pull request:
https://github.com/apache/flume/pull/222
FLUME-3050 add counters for error conditions and expose to monitor URL
Concept: an error is when an Exception is thrown or an ERROR level log is written during event processing.
In case of an error at least 1 error counter is increased at least once. (Preferably 1 counter once).
Errors during event processing are counted. Initialization errors are not handled here.
3 types of errors are differentiated.
-Channel read/write errors from the channel when the channel throws a ChannelException.
-Event read/write errors. E.g: A source cannot read an event due to
-Generic errors - e.g.: TaildirSource cannot write position file.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/majorendre/flume FLUME-3050
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flume/pull/222.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #222
----
commit c82d23011aa5dcc47df997f47792e8ececf88303
Author: emajor <em...@...>
Date: 2018-07-20T15:38:34Z
FLUME-3050 WIP
commit 8245d210f186fef06a3f7d996116f7c02e66f552
Author: emajor <em...@...>
Date: 2018-07-24T12:43:24Z
WIP
commit 83ae524a37acfcdd2442128fc19b26cdf30f1b45
Author: emajor <em...@...>
Date: 2018-07-30T10:03:39Z
WIP tests
commit eecd494a6b0c7e2000398429520014a143f8ea30
Author: emajor <em...@...>
Date: 2018-07-30T11:50:30Z
clean up 1
commit b4c9afabd4621d5f68a403644c75bc2c3f211be4
Author: emajor <em...@...>
Date: 2018-07-30T16:12:24Z
clean up 2
commit cc1d88abc31c5ae81cc16842d5d14418e5176b8b
Author: emajor <em...@...>
Date: 2018-07-30T16:17:59Z
clean up 3
commit 37594abeb2fbd2d695d3585d0351d7295810b5c4
Author: emajor <em...@...>
Date: 2018-07-31T14:57:39Z
WIP adding further tests
commit bc6e4fc18ecfabd0e2a8c9f7911573ee50ce60e7
Author: emajor <em...@...>
Date: 2018-08-01T16:40:31Z
further tests
commit d200eda3195f84b89580aabd5bdac19a9c8c0f8e
Author: emajor <em...@...>
Date: 2018-08-06T09:45:47Z
morphline error counter added
commit dd851dda8d3d95c1a37563a9012e153c79a17b37
Author: emajor <em...@...>
Date: 2018-08-06T13:51:15Z
cleanup and test fix
commit 63dff5781adeaab7d8aea74a45e0e9b33e2be06b
Author: emajor <em...@...>
Date: 2018-08-06T15:23:11Z
Adding error counters to ScribeSource
----
> add counters for error conditions and expose to monitor URL
> -----------------------------------------------------------
>
> Key: FLUME-3050
> URL: https://issues.apache.org/jira/browse/FLUME-3050
> Project: Flume
> Issue Type: Improvement
> Components: Channel, Shell, Sinks+Sources
> Affects Versions: 1.7.0
> Reporter: Yuval Lifshitz
> Assignee: Endre Major
> Priority: Major
> Labels: features
>
> currently error counters are not present when getting stats. for example:
> {code}
> > curl http://my-flume-host:44444/metrics
> {"SINK.k1":{"ConnectionCreatedCount":"1","ConnectionClosedCount":"0","Type":"SINK","BatchCompleteCount":"0","BatchEmptyCount":"4","EventDrainAttemptCount":"10","StartTime":"1485348138992","EventDrainSuccessCount":"10","BatchUnderflowCount":"1","StopTime":"0","ConnectionFailedCount":"0"},"CHANNEL.c1":{"ChannelCapacity":"1000000","ChannelFillPercentage":"0.0","Type":"CHANNEL","ChannelSize":"0","EventTakeSuccessCount":"10","EventTakeAttemptCount":"15","StartTime":"1485348138990","EventPutAttemptCount":"10","EventPutSuccessCount":"10","StopTime":"0"},"SOURCE.r1":{"EventReceivedCount":"10","AppendBatchAcceptedCount":"0","Type":"SOURCE","AppendReceivedCount":"0","EventAcceptedCount":"10","StartTime":"1485348138993","AppendAcceptedCount":"0","OpenConnectionCount":"0","AppendBatchReceivedCount":"0","StopTime":"0"}}
> {code}
> return only "good" stats for source, channel and sink.
> to get error you need to look into the log file. this makes it hard to integrate flume into automatic monitoring systems, NMS etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@flume.apache.org
For additional commands, e-mail: issues-help@flume.apache.org