RE: Possible to use Regex Interceptor in Flume 1.2

[Paul Chavez <pc...@verticalsearchworks.com>]

Hello,

Finally got around to testing this...

I have built and repackaged a JAR with the following classes:

/usr/lib/flume-ng/lib$ jar tf flume-ng-RegexCaptureInterceptor-1.3.jar
META-INF/
META-INF/MANIFEST.MF
META-INF/DEPENDENCIES
META-INF/LICENSE
META-INF/maven/
META-INF/maven/org.apache.flume/
META-INF/maven/org.apache.flume/flume-ng-core/
META-INF/maven/org.apache.flume/flume-ng-core/pom.properties
META-INF/maven/org.apache.flume/flume-ng-core/pom.xml
META-INF/NOTICE
org/
org/apache/
org/apache/flume/
org/apache/flume/interceptor/
org/apache/flume/interceptor/RegexExtractorInterceptor$1.class
org/apache/flume/interceptor/RegexExtractorInterceptor$Builder.class
org/apache/flume/interceptor/RegexExtractorInterceptor$NameAndSerializer.class
org/apache/flume/interceptor/RegexExtractorInterceptor.class
org/apache/flume/interceptor/RegexExtractorInterceptorMillisSerializer.class
org/apache/flume/interceptor/RegexExtractorInterceptorPassThroughSerializer.class
org/apache/flume/interceptor/RegexExtractorInterceptorSerializer.class

However when trying to start an agent with this setup I get the following errors:

4:55:16.866 PM	ERROR	org.apache.flume.channel.ChannelProcessor	
Could not instantiate Builder. Exception follows.
java.lang.InstantiationException: org.apache.flume.interceptor.RegexExtractorInterceptor
	at java.lang.Class.newInstance0(Class.java:340)
	at java.lang.Class.newInstance(Class.java:308)
	at org.apache.flume.interceptor.InterceptorBuilderFactory.newInstance(InterceptorBuilderFactory.java:48)
	at org.apache.flume.channel.ChannelProcessor.configureInterceptors(ChannelProcessor.java:109)
	at org.apache.flume.channel.ChannelProcessor.configure(ChannelProcessor.java:80)
	at org.apache.flume.conf.Configurables.configure(Configurables.java:41)

4:55:16.867 PM	ERROR	org.apache.flume.conf.properties.PropertiesFileConfigurationProvider	
Failed to load configuration data. Exception follows.
org.apache.flume.FlumeException: Interceptor.Builder not constructable.
	at org.apache.flume.channel.ChannelProcessor.configureInterceptors(ChannelProcessor.java:117)
	at org.apache.flume.channel.ChannelProcessor.configure(ChannelProcessor.java:80)
	at org.apache.flume.conf.Configurables.configure(Configurables.java:41)



Any insight as to what's going wrong is appreciated.

Thank you,
Paul Chavez
-----Original Message-----
From: Brock Noland [mailto:brock@cloudera.com] 
Sent: Monday, December 10, 2012 8:55 AM
To: user@flume.apache.org
Subject: Re: Possible to use Regex Interceptor in Flume 1.2

Hi,

If you built flume 1.3.0 and took the RegexExtractorInterceptor class files, built your own jar with just these class files, and placed that jar in Flumes classpath you'd be able to specify the fully qualified class name of the RegexExtractorInterceptor.

Too bad we did't make RegexExtractorInterceptor it's own module as it would be easier.

Brock

On Mon, Dec 10, 2012 at 10:36 AM, Paul Chavez <pc...@verticalsearchworks.com> wrote:
> I would like to use the new regex interceptor to pull timestamp values out of my event data. However, I do not manage the Hadoop installation at my work and we are using the CDH distributions which currently have Flume 1.2 packaged in them.
>
> Is there a straightforward way to use just the regex interceptor with Flume 1.2 as packaged by CDH 4.x?
>
> thank you,
> Paul Chavez



--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/