[jira] [Updated] (STORM-967) Unable to package storm-kafka & storm-hdfs connectors in a topology

["Rick Kellogg (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/STORM-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rick Kellogg updated STORM-967:
-------------------------------
    Component/s: storm-kafka
                 storm-hdfs

> Unable to package storm-kafka & storm-hdfs connectors in a topology
> -------------------------------------------------------------------
>
>                 Key: STORM-967
>                 URL: https://issues.apache.org/jira/browse/STORM-967
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-hdfs, storm-kafka
>            Reporter: Sriharsha Chintalapani
>            Assignee: Priyank Shah
>            Priority: Critical
>
> Based on my testing storm-kafka & storm-hdfs connector along with it dependencies are unable to be packaged together .
> If we tried to package it using maven assembly plugin storm-hdfs errors out saying
> "java.lang.RuntimeException: Error preparing HdfsBolt: No FileSystem for scheme: hdfs"
> To avoid we need to use maven shade plugin as documented in the README here
> https://github.com/apache/storm/tree/master/external/storm-hdfs
> Now when we use maven shade plugin storm-kafka throws an initial error about
> {code}
>    java.lang.NoClassDefFoundError: Could not initialize class com.yammer.metrics.Metrics
> at kafka.metrics.KafkaMetricsGroup$class.newTimer(KafkaMetricsGroup.scala:52) at kafka.consumer.FetchRequestAndResponseMetrics.newTimer(FetchRequestAndResponseStats.scala:25)
> at kafka.consumer.FetchRequestAndResponseMetrics.<init>(FetchRequestAndResponseStats.scala:26)
> at kafka.consumer.FetchRequestAndResponseStats.<init>(FetchRequestAndResponseStats.scala:37)
> {code}
> After we explicitly added metrics dependency to the pom file 
> KafkaSpout now errors out with 
> IllegalStateException KafkaMetricsGroup.newTimer  
> KafkaSpout works fine if we use assembly plugin.  Due to the behavior of these two connectors one works assembly and another works with shade it looks like we won't be able to use them together.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)