[jira] [Created] (HUDI-2404) JMX Reporter broken using spark package

["Sarah Witt (Jira)" <ji...@apache.org>]

Sarah Witt created HUDI-2404:
--------------------------------

             Summary: JMX Reporter broken using spark package
                 Key: HUDI-2404
                 URL: https://issues.apache.org/jira/browse/HUDI-2404
             Project: Apache Hudi
          Issue Type: Bug
    Affects Versions: 0.9.0
            Reporter: Sarah Witt
             Fix For: 0.10.0


I am trying to enable JMX monitoring when using Hudi with Spark.

I followed the Spark Quickstart tutorial [https://hudi.apache.org/docs/quick-start-guide] and when I added the options to enable JMX monitoring in the insert data stage [https://hudi.apache.org/docs/quick-start-guide#insert-data], I get the following error: `java.lang.NoClassDefFoundError: org/apache/hudi/com/codahale/metrics/jmx/JmxReporter`

 

code used:

df.write.format("hudi").

  options(getQuickstartWriteConfigs).

  option("hoodie.metrics.on", "true").

  option("hoodie.metrics.reporter.type", "JMX").

  option("hoodie.metrics.reporter.port", "9889").

  option(PRECOMBINE_FIELD_OPT_KEY, "ts").

  option(RECORDKEY_FIELD_OPT_KEY, "uuid").

  option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").

  option(TABLE_NAME, tableName).

  mode(Append).

  save(basePath)

 

 

metrics-jmx is a new package (not included in metrics-core: https://metrics.dropwizard.io/4.2.0/about/release-notes.html), so I believe it should be included in the pom.xml here: https://github.com/apache/hudi/blob/master/packaging/hudi-spark-bundle/pom.xml#L92-L93. When I updated my local pom.xml to include metrics-jmx and repeated the setup I was able to successfully enable JMX. The issue seems to happen for flink as well. I can submit a PR if that is easier, too. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)