You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kudu.apache.org by "Changyao Ye (JIRA)" <ji...@apache.org> on 2017/12/15 07:38:00 UTC
[jira] [Updated] (KUDU-2241) NoSuchMethodError happened when run
flume agent using kudu flume sink
[ https://issues.apache.org/jira/browse/KUDU-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Changyao Ye updated KUDU-2241:
------------------------------
Description:
I've installed kudu and flume components from cloudera manager, and when I start flume agent the following error happened.
---------------------------------------------------------------------------------------------------------------------------------
17/12/11 10:46:29 ERROR node.PollingPropertiesFileConfigurationProvider: Unhandled error
java.lang.NoSuchMethodError: org.apache.flume.Context.getSubProperties(Ljava/lang/String;)Lorg/apache/kudu/shaded/com/google/common/collect/ImmutableMap;
at org.apache.kudu.flume.sink.KuduSink.configure(KuduSink.java:206)
at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:411)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:141)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
---------------------------------------------------------------------------------------------------------------------------------
Version infomation:
- flume: 1.6.0-cdh5.13.0
- kudu: 1.5.0-cdh5.13.0
I checked the classpath of flume agent and found that maybe there is version conflict of guava jar between flume and kudu flume sink(kudu-flume-sink-1.5.0-cdh5.13.0.jar) which includes shaded guava jar.
Maybe this problem related to the changes made [here|https://github.com/apache/kudu/commit/5a258508f8d560f630512c237711a65cd137c6b3].
To make flume agent run properly, I excluded related class(ImmutableMap etc) from shade settings in pom.xml and found that worked.
{panel:title=My title}
<relocation>
<pattern>com.google.common</pattern>
<shadedPattern>org.apache.kudu.shaded.com.google.common</shadedPattern>
<excludes>
<exclude>com.google.common.collect.ImmutableMap*</exclude>
<exclude>com.google.common.collect.ImmutableEnumMap*</exclude>
</excludes>
</relocation>
{panel}
was:
I've installed kudu and flume components from cloudera manager, and when I start flume agent the following error happened.
```
17/12/11 10:46:29 ERROR node.PollingPropertiesFileConfigurationProvider: Unhandled error
java.lang.NoSuchMethodError: org.apache.flume.Context.getSubProperties(Ljava/lang/String;)Lorg/apache/kudu/shaded/com/google/common/collect/ImmutableMap;
at org.apache.kudu.flume.sink.KuduSink.configure(KuduSink.java:206)
at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:411)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:141)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
```
Version infomation:
- flume: 1.6.0-cdh5.13.0
- kudu: 1.5.0-cdh5.13.0
I checked the classpath of flume agent and found that maybe there is version conflict of guava jar between flume and kudu flume sink(kudu-flume-sink-1.5.0-cdh5.13.0.jar) which includes shaded guava jar.
Maybe this problem related to the changes made [here|https://github.com/apache/kudu/commit/5a258508f8d560f630512c237711a65cd137c6b3].
To make flume agent run properly, I excluded related class(ImmutableMap etc) from shaded settings in pom.xml and found that worked.
> NoSuchMethodError happened when run flume agent using kudu flume sink
> ---------------------------------------------------------------------
>
> Key: KUDU-2241
> URL: https://issues.apache.org/jira/browse/KUDU-2241
> Project: Kudu
> Issue Type: Bug
> Components: flume-sink
> Affects Versions: 1.5.0
> Reporter: Changyao Ye
>
> I've installed kudu and flume components from cloudera manager, and when I start flume agent the following error happened.
> ---------------------------------------------------------------------------------------------------------------------------------
> 17/12/11 10:46:29 ERROR node.PollingPropertiesFileConfigurationProvider: Unhandled error
> java.lang.NoSuchMethodError: org.apache.flume.Context.getSubProperties(Ljava/lang/String;)Lorg/apache/kudu/shaded/com/google/common/collect/ImmutableMap;
> at org.apache.kudu.flume.sink.KuduSink.configure(KuduSink.java:206)
> at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
> at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:411)
> at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
> at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:141)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> ---------------------------------------------------------------------------------------------------------------------------------
> Version infomation:
> - flume: 1.6.0-cdh5.13.0
> - kudu: 1.5.0-cdh5.13.0
> I checked the classpath of flume agent and found that maybe there is version conflict of guava jar between flume and kudu flume sink(kudu-flume-sink-1.5.0-cdh5.13.0.jar) which includes shaded guava jar.
> Maybe this problem related to the changes made [here|https://github.com/apache/kudu/commit/5a258508f8d560f630512c237711a65cd137c6b3].
> To make flume agent run properly, I excluded related class(ImmutableMap etc) from shade settings in pom.xml and found that worked.
> {panel:title=My title}
> <relocation>
> <pattern>com.google.common</pattern>
> <shadedPattern>org.apache.kudu.shaded.com.google.common</shadedPattern>
> <excludes>
> <exclude>com.google.common.collect.ImmutableMap*</exclude>
> <exclude>com.google.common.collect.ImmutableEnumMap*</exclude>
> </excludes>
> </relocation>
> {panel}
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)