Camus add third party decoder jar, doesn't work

[Yang <te...@gmail.com>]

we tried to create out own decoder, we put it in a separate jar, and ran
Camus with the hadoop -libjars arg:


hadoop jar camus-etl-kafka-jar-with-dependencies
com.linkedIn.camus.CamusJob -libjars mydecoder-jar-with-dependencies.jar -P
camus.properties

then it gave me the following error

Caused by: org.springframework.beans.factory.BeanDefinitionStoreException:
Unexpected exception parsing XML document from class path resource
[yycontext.xml]; nested exception is
org.springframework.beans.FatalBeanException: Class
[org.springframework.context.config.ContextNamespaceHandler] for namespace [
http://www.springframework.org/schema/context] does not implement the
[org.springframework.beans.factory.xml.NamespaceHandler] interface
at
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:412)
at
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:334)
at
org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:302)
at
org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:174)
at
org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:209)
at
org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:180)
at
org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:243)
at
org.springframework.context.support.AbstractXmlApplicationContext.loadBeanDefinitions(AbstractXmlApplicationContext.java:127)
at
org.springframework.context.support.AbstractXmlApplicationContext.loadBeanDefinitions(AbstractXmlApplicationContext.java:93)
at
org.springframework.context.support.AbstractRefreshableApplicationContext.refreshBeanFactory(AbstractRefreshableApplicationContext.java:130)
at
org.springframework.context.support.AbstractApplicationContext.obtainFreshBeanFactory(AbstractApplicationContext.java:537)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:451)
at
org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
at
org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)
at avro.MyDecoder.init(MyDecoder.java:31)
at
com.linkedin.camus.etl.kafka.coders.MessageDecoderFactory.createMessageDecoder(MessageDecoderFactory.java:25)
... 21 more
Caused by: org.springframework.beans.FatalBeanException: Class
[org.springframework.context.config.ContextNamespaceHandler] for namespace [
http://www.springframework.org/schema/context] does not implement the
[org.springframework.beans.factory.xml.NamespaceHandler] interface
at
org.springframework.beans.factory.xml.DefaultNamespaceHandlerResolver.resolve(DefaultNamespaceHandlerResolver.java:126)
at
org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.parseCustomElement(BeanDefinitionParserDelegate.java:1433)
at
org.springframework.beans.factory.xml.BeanDefinitionParserDelegate.parseCustomElement(BeanDefinitionParserDelegate.java:1428)
at
org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.parseBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:195)
at
org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.doRegisterBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:139)
at
org.springframework.beans.factory.xml.DefaultBeanDefinitionDocumentReader.registerBeanDefinitions(DefaultBeanDefinitionDocumentReader.java:108)


(my decoder code uses spring somewhere)



then I manually combined the 2 jars together, and ran
hadoop jar combined_jar.jar com.linkedIn.camus.CamusJob -P camus.properties


then this time it works.




-libjars is a pretty standard feature of hadoop, I thought it would
essentially do the same as my manual combining of the jars, why do I get an
exception here? clearly the camus code itself does not contain references
to Spring.

thanks
Yang