You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Sa Xiao <sa...@gmail.com> on 2016/02/24 02:24:10 UTC

metrics not reported by spark-cassandra-connector

Hi there,

I am trying to enable the metrics collection by spark-cassandra-connector,
following the instruction here:
https://github.com/datastax/spark-cassandra-connector/blob/master/doc/11_metrics.md

However, I was not able to see any metrics reported. I'm using
spark-cassandra-connector_2.10:1.5.0, and spark 1.5.1. I am trying to send
the metrics to statsD. My metrics.properties is as the following:

*.sink.statsd.class=org.apache.spark.metrics.sink.StatsDSink

*.sink.statsd.host=localhost

*.sink.statsd.port=18125

executor.source.cassandra-connector.class=org.apache.spark.metrics.CassandraConnectorSource

driver.source.cassandra-connector.class=org.apache.spark.metrics.CassandraConnectorSource

I'm able to see other metrics, e.g. DAGScheduler, but not any from the
CassandraConnectorSource. E.g. I tried to search "write-byte-meter", but
didn't find it. I didn't see the metrics on the spark UI either. I didn't
find any relevant error or info in the log that indicates the
CassandraConnectorSource is actually registered by the spark metrics
system. Any pointers would be very much appreciated!

Thanks,

Sa