You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flume.apache.org by jo...@apache.org on 2015/07/24 09:03:36 UTC
flume git commit: FLUME-2660. Add documentation for EventValidator
Repository: flume
Updated Branches:
refs/heads/trunk 4d7124700 -> baca45ae4
FLUME-2660. Add documentation for EventValidator
(Ashish Paliwal via Johny Rufus)
Project: http://git-wip-us.apache.org/repos/asf/flume/repo
Commit: http://git-wip-us.apache.org/repos/asf/flume/commit/baca45ae
Tree: http://git-wip-us.apache.org/repos/asf/flume/tree/baca45ae
Diff: http://git-wip-us.apache.org/repos/asf/flume/diff/baca45ae
Branch: refs/heads/trunk
Commit: baca45ae43756491ea71707635b151db5f928b2c
Parents: 4d71247
Author: Johny Rufus <jo...@apache.org>
Authored: Thu Jul 23 23:58:12 2015 -0700
Committer: Johny Rufus <jo...@apache.org>
Committed: Thu Jul 23 23:58:12 2015 -0700
----------------------------------------------------------------------
flume-ng-doc/sphinx/FlumeUserGuide.rst | 61 ++++++++++++++++++++++++++++-
1 file changed, 60 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flume/blob/baca45ae/flume-ng-doc/sphinx/FlumeUserGuide.rst
----------------------------------------------------------------------
diff --git a/flume-ng-doc/sphinx/FlumeUserGuide.rst b/flume-ng-doc/sphinx/FlumeUserGuide.rst
index 617e730..e959fa6 100644
--- a/flume-ng-doc/sphinx/FlumeUserGuide.rst
+++ b/flume-ng-doc/sphinx/FlumeUserGuide.rst
@@ -3748,7 +3748,7 @@ The tools can be run as follows::
$bin/flume-ng tool --conf ./conf FCINTEGRITYTOOL -l ./datadir
-where datadir the comma separated list of data directory to ve verified.
+where datadir is the comma separated list of data directory to be verified.
Following are the options available
@@ -3759,7 +3759,66 @@ h/help Displays help
**l/dataDirs** Comma-separated list of data directories which the tool must verify
======================= ====================================================================
+Event Validator Tool
+--------------------
+Event validator tool can be used to validate the File Channel Event's in application specific way.
+The tool applies the user provider validation login on each event and drop the event which do not
+confirm to the logic.
+
+The tools can be run as follows::
+
+ $bin/flume-ng tool --conf ./conf FCINTEGRITYTOOL -l ./datadir -e org.apache.flume.MyEventValidator -DmaxSize 2000
+
+where datadir is the comma separated list of data directory to be verified.
+
+Following are the options available
+
+======================= ====================================================================
+Option Name Description
+======================= ====================================================================
+h/help Displays help
+**l/dataDirs** Comma-separated list of data directories which the tool must verify
+e/eventValidator Fully Qualified Name of Event Validator Implementation. The jar must
+ be on Flume classpath
+======================= ====================================================================
+
+The Event validator implementation must implement EventValidator interface. It's recommended
+not to throw any exception from the implementation as they are treated as invalid events.
+Additional parameters can be passed to EventValitor implementation via -D options.
+
+Let's see an example of simple size based Event Validator, which shall reject event's larger
+than maximum size specified.
+
+.. code-block:: java
+ public static class MyEventValidator implements EventValidator {
+
+ private int value = 0;
+
+ private MyEventValidator(int val) {
+ value = val;
+ }
+
+ @Override
+ public boolean validateEvent(Event event) {
+ return event.getBody() <= value;
+ }
+
+ public static class Builder implements EventValidator.Builder {
+
+ private int sizeValidator = 0;
+
+ @Override
+ public EventValidator build() {
+ return new DummyEventVerifier(sizeValidator);
+ }
+
+ @Override
+ public void configure(Context context) {
+ binaryValidator = context.getInteger("maxSize");
+ }
+ }
+ }
Topology Design Considerations