using Codahale counters in source

[Steve Loughran <st...@hortonworks.com>]

Quick question: is it OK to use Codahale Metric classes (e.g. Counter) in source as generic thread-safe counters, with the option of hooking them to a Codahale metrics registry if there is one in the spark context?

The Counter class does extend LongAdder, which is by Doug Lea and promises to be a better performing long counter when update contention is low —such as when precisely one thread is doing the updates.

I've done that in other projects, and it works relatively well, in that you don't have to add extra code for metrics, you just have to make sure you implement counters that are relevant, and, when registring them, given them a useful name.

-Steve