You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Sa Li <sa...@gmail.com> on 2015/02/05 00:32:38 UTC
Re: metric-kafka problems
Hello, Joe
Continue this thread, I got following monitoring tools on my DEV,
1. graphite + statsD
2. kafka-web-console
3. JMX + jconsole
4. kafkaOffsetMonitor
5. Kafka Manager (yahoo just open source it)
They all work fine locally on my dev, but I am thinking to get it work out
of box cause I just do not want to install them on my production server,
how possible can I install all of them in a VM, remotely connecting the
production? It seems I can't find config to allow me to do this, or no such
out of box feature?
thanks
AL
On Fri, Jan 9, 2015 at 6:32 PM, Sa Li <sa...@gmail.com> wrote:
> Thank you very much, Joe, I will try all of them, keep here posted.
> On Jan 9, 2015 5:55 PM, "Joe Stein" <jo...@stealth.ly> wrote:
>
>> Hi, https://github.com/stealthly/metrics-kafka is a project to be used as
>> an example of how to use Kafka as a central point to send all of your
>> metrics for your entire infrastructure. The consumers integrate so as to
>> abstract the load and coupling of services so systems can just send their
>> stats to Kafka and then you can do whatever you want with them from there
>> (often multiple things). We also build a Yammer Metrics Reporter (which is
>> what Kafka uses to send its Metrics) for Kafka itself so brokers can send
>> their stats into a Kafka topic and used downstream (typically another
>> cluster). The issue you reported was caused by changes by github and I
>> just pushed fixes for them so things are working again.
>>
>> If you are not looking for that type of solution and want to just see and
>> chart broker metrics then I would suggest taking a look at
>> https://github.com/airbnb/kafka-statsd-metrics2 and you point it to
>> https://github.com/kamon-io/docker-grafana-graphite. I find this a very
>> quick out the box way to see what is going on with a broker when no stats
>> reporter is already in place. If you want a Kafka metrics reporter for
>> just
>> graphite check out https://github.com/damienclaveau/kafka-graphite for
>> just
>> ganglie https://github.com/criteo/kafka-ganglia for just Riemann
>> https://github.com/TheLadders/KafkaRiemannMetricsReporter and/or you also
>> can use a service like SPM
>> https://apps.sematext.com/spm-reports/mainPage.do?selectedApplication=4293
>> or DataDog https://www.datadoghq.com/
>>
>> Hope this help, thanks!
>>
>> /*******************************************
>> Joe Stein
>> Founder, Principal Consultant
>> Big Data Open Source Security LLC
>> http://www.stealth.ly
>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> ********************************************/
>>
>>
>> On Fri, Jan 9, 2015 at 7:51 PM, Sa Li <sa...@gmail.com> wrote:
>>
>> > Hello, all
>> >
>> > I like to use the tool metrics-kafka which seems to be attractive to
>> report
>> > kafka metric and use graphite to graph metrics, however I am having
>> trouble
>> > to make it work.
>> >
>> > In https://github.com/stealthly/metrics-kafka, it says:
>> >
>> > In the main metrics-kafka folder
>> >
>> > 1) sudo ./bootstrap.sh 2) ./gradlew test 3) sudo ./shutdown.sh
>> > When I run ./bootstrap, see this is what I got
>> > root@DO-mq-dev:/home/stuser/jmx/metrics-kafka# ././bootstrap.sh
>> > /dev/stdin: line 1: syntax error near unexpected token `newline'
>> > /dev/stdin: line 1: `<!DOCTYPE html>'
>> > /dev/stdin: line 1: syntax error near unexpected token `newline'
>> > /dev/stdin: line 1: `<!DOCTYPE html>'
>> > e348a98a5afb8b89b94fce51b125e8a2045d9834268ec64c3e38cb7b165ef642
>> > 2015/01/09 16:49:21 Error response from daemon: Could not find entity
>> for
>> > broker1
>> >
>> > And this is how I vagrant up:
>> > root@DO-mq-dev:/home/stuser/jmx/metrics-kafka# vagrant up
>> > /usr/share/vagrant/plugins/provisioners/docker/plugin.rb:13:in
>> > `require_relative':
>> > /usr/share/vagrant/plugins/provisioners/docker/config.rb:23: syntax
>> error,
>> > unexpected tPOW (SyntaxError)
>> > def run(name, **options)
>> > ^
>> > /usr/share/vagrant/plugins/provisioners/docker/config.rb:43: syntax
>> error,
>> > unexpected keyword_end, expecting $end
>> > from
>> /usr/share/vagrant/plugins/provisioners/docker/plugin.rb:13:in
>> > `block in <class:Plugin>'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/registry.rb:27:in `call'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/registry.rb:27:in `get'
>> > from
>> > /usr/share/vagrant/plugins/kernel_v2/config/vm_provisioner.rb:34:in
>> > `initialize'
>> > from /usr/share/vagrant/plugins/kernel_v2/config/vm.rb:223:in
>> `new'
>> > from /usr/share/vagrant/plugins/kernel_v2/config/vm.rb:223:in
>> > `provision'
>> > from /home/stuser/jmx/metrics-kafka/Vagrantfile:29:in `block (2
>> > levels) in <top (required)>'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/config/v2/loader.rb:37:in
>> > `call'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/config/v2/loader.rb:37:in
>> > `load'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/config/loader.rb:104:in
>> > `block (2 levels) in load'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/config/loader.rb:98:in
>> > `each'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/config/loader.rb:98:in
>> > `block in load'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/config/loader.rb:95:in
>> > `each'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/config/loader.rb:95:in
>> > `load'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:335:in
>> > `machine'
>> > from
>> /usr/lib/ruby/vendor_ruby/vagrant/plugin/v2/command.rb:142:in
>> > `block in with_target_vms'
>> > from
>> /usr/lib/ruby/vendor_ruby/vagrant/plugin/v2/command.rb:175:in
>> > `call'
>> > from
>> /usr/lib/ruby/vendor_ruby/vagrant/plugin/v2/command.rb:175:in
>> > `block in with_target_vms'
>> > from
>> /usr/lib/ruby/vendor_ruby/vagrant/plugin/v2/command.rb:174:in
>> > `map'
>> > from
>> /usr/lib/ruby/vendor_ruby/vagrant/plugin/v2/command.rb:174:in
>> > `with_target_vms'
>> > from /usr/share/vagrant/plugins/commands/up/command.rb:56:in
>> `block
>> > in execute'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:210:in
>> `block
>> > (2 levels) in batch'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:208:in
>> `tap'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:208:in
>> `block
>> > in batch'
>> > from <internal:prelude>:10:in `synchronize'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:207:in
>> > `batch'
>> > from /usr/share/vagrant/plugins/commands/up/command.rb:55:in
>> > `execute'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/cli.rb:38:in `execute'
>> > from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:484:in
>> `cli'
>> > from /usr/bin/vagrant:127:in `<main>'
>> >
>> > Any idea to make it work?
>> >
>> > thanks
>> >
>> >
>> > --
>> >
>> > Alec Li
>> >
>>
>
--
Alec Li