[jira] [Created] (FLUME-2377) Don't automatically include Hadoop and HBase classpaths into Flume classpath

["Joshua Hyde (JIRA)" <ji...@apache.org>]

Joshua Hyde created FLUME-2377:
----------------------------------

             Summary: Don't automatically include Hadoop and HBase classpaths into Flume classpath
                 Key: FLUME-2377
                 URL: https://issues.apache.org/jira/browse/FLUME-2377
             Project: Flume
          Issue Type: Improvement
          Components: Shell
            Reporter: Joshua Hyde


HBase and Hadoop, for their own reasons, have Jersey 1.x libraries in their classpath. A custom source we wrote uses Jersey 2.x. For a number of reasons, the two versions of Jersey do not co-habitate on the same classpath nicely. As a result, we get stacktraces like the following if, for example, a developer has Hadoop or HBase installed on their system:

{noformat}
java.lang.NullPointerException
    at com.sun.jersey.core.provider.jaxb.AbstractJAXBProvider.setConfiguration(AbstractJAXBProvider.java:107)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.glassfish.hk2.utilities.reflection.ReflectionHelper.invoke(ReflectionHelper.java:1017)
    at org.jvnet.hk2.internal.ClazzCreator.methodMe(ClazzCreator.java:374)
    at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:427)
    at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)
    at org.jvnet.hk2.internal.PerLookupContext.findOrCreate(PerLookupContext.java:69)
    at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2488)
    at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:98)
    at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:87)
    at org.glassfish.jersey.internal.inject.ProviderToService.apply(ProviderToService.java:58)
    at org.glassfish.jersey.internal.inject.ProviderToService.apply(ProviderToService.java:54)
    at jersey.repackaged.com.google.common.collect.Iterators$8.transform(Iterators.java:860)
    at jersey.repackaged.com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
    at java.util.AbstractCollection.addAll(AbstractCollection.java:333)
    at java.util.LinkedHashSet.<init>(LinkedHashSet.java:169)
    at jersey.repackaged.com.google.common.collect.Sets.newLinkedHashSet(Sets.java:292)
    at org.glassfish.jersey.internal.inject.Providers.getClasses(Providers.java:364)
    at org.glassfish.jersey.internal.inject.Providers.getProviders(Providers.java:186)
    at org.glassfish.jersey.message.internal.MessageBodyFactory.<init>(MessageBodyFactory.java:304)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.glassfish.hk2.utilities.reflection.ReflectionHelper.makeMe(ReflectionHelper.java:1105)
    at org.jvnet.hk2.internal.ClazzCreator.createMe(ClazzCreator.java:291)
    at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:421)
    at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)
    at org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:114)
    at org.jvnet.hk2.internal.SingletonContext$1.compute(SingletonContext.java:102)
    at org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture$1.call(Cache.java:97)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at org.glassfish.hk2.utilities.cache.Cache$OriginThreadAwareFuture.run(Cache.java:154)
    at org.glassfish.hk2.utilities.cache.Cache.compute(Cache.java:199)
    at org.jvnet.hk2.internal.SingletonContext.findOrCreate(SingletonContext.java:153)
    at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2488)
    at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:629)
    at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:614)
    at org.glassfish.jersey.server.model.ComponentModelValidator.<init>(ComponentModelValidator.java:97)
    at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:457)
    at org.glassfish.jersey.server.ApplicationHandler.access$500(ApplicationHandler.java:162)
    at org.glassfish.jersey.server.ApplicationHandler$3.run(ApplicationHandler.java:322)
    at org.glassfish.jersey.internal.Errors$2.call(Errors.java:289)
    at org.glassfish.jersey.internal.Errors$2.call(Errors.java:286)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
    at org.glassfish.jersey.internal.Errors.processWithException(Errors.java:286)
    at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:319)
    at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:272)
    at org.glassfish.jersey.server.ContainerFactory.createContainer(ContainerFactory.java:79)
    at org.glassfish.jersey.jetty.JettyHttpContainerFactory.createServer(JettyHttpContainerFactory.java:139)
    at com.me.CustomSource.start(CustomSource.java:186)
    at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
    at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:724)
{noformat}

The fact that {{flume-ng}} includes the Hadoop and HBase classpath automatically, even if we have no need for it, is a bit surprising and, if you don't know that the shell script is doing that, it can take a little bit of digging to figure it out.

At the very least, I'd like the ability to tell {{flume-ng}}, "No, don't include the Hadoop classpath," and, "No, don't include the HBase classpath,", though if someone's willing to throw passivity to the wind, I'd be glad to just not have those external classpaths added to Flume's classpath to begin with.



--
This message was sent by Atlassian JIRA
(v6.2#6252)