You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by bikash sharma <sh...@gmail.com> on 2011/04/01 22:09:21 UTC
Chukwa setup issues
Hi,
I am trying to setup Chukwa for a 16-node Hadoop cluster.
I followed the admin guide -
http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html#Agents
However, I ran two the following issues:
1. What should be the collector port that needs to be specified in
conf/collectors file
2. Am unable to see the collector running via web browser
Am I missing something?
Thanks in advance.
-bikash
p.s. - after i run collector, nothing happens
% bin/chukwa collector
2011-04-01 16:07:16.410::INFO: Logging to STDERR via
org.mortbay.log.StdErrLog
2011-04-01 16:07:16.523::INFO: jetty-6.1.11
2011-04-01 16:07:17.707::INFO: Started SelectChannelConnector@0.0.0.0:9999
started Chukwa http collector on port 9999
Re: Chukwa setup issues
Posted by Bernd Fondermann <bf...@brainlounge.de>.
[to: chukwa-user, cc: common-dev]
Chukwa has its own mailing lists now. Your question is better addressed
there.
Bernd
On 01.04.11 22:09, bikash sharma wrote:
> Hi,
> I am trying to setup Chukwa for a 16-node Hadoop cluster.
> I followed the admin guide -
> http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html#Agents
> However, I ran two the following issues:
> 1. What should be the collector port that needs to be specified in
> conf/collectors file
> 2. Am unable to see the collector running via web browser
>
> Am I missing something?
>
> Thanks in advance.
>
> -bikash
>
> p.s. - after i run collector, nothing happens
> % bin/chukwa collector
> 2011-04-01 16:07:16.410::INFO: Logging to STDERR via
> org.mortbay.log.StdErrLog
> 2011-04-01 16:07:16.523::INFO: jetty-6.1.11
> 2011-04-01 16:07:17.707::INFO: Started SelectChannelConnector@0.0.0.0:9999
> started Chukwa http collector on port 9999
>
Re: Chukwa setup issues
Posted by Bill Graham <bi...@gmail.com>.
Moving over to chukwa list and letting common-dev fall off via bcc.
On Fri, Apr 1, 2011 at 2:37 PM, Bill Graham <bi...@gmail.com> wrote:
> Unfortunately conf/collectors is used in two different ways in Chukwa,
> each with a different syntax. This should really be fixed.
>
> 1. The script that starts the collectors looks at it for a list of
> hostnames (no ports) to start collectors on. To start it just on one
> host, set it to localhost.
> 2. The agent looks at that file for the list of collectors to attempt
> to communicate with. In that case the format is a list of HTTP urls
> with ports of the collectors.
>
> Can you telnet to port 9999? It looks like it's listening, but
> nothing's being sent. Is there anything in logs/collector.log?
>
> On Fri, Apr 1, 2011 at 1:09 PM, bikash sharma <sh...@gmail.com> wrote:
>> Hi,
>> I am trying to setup Chukwa for a 16-node Hadoop cluster.
>> I followed the admin guide -
>> http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html#Agents
>> However, I ran two the following issues:
>> 1. What should be the collector port that needs to be specified in
>> conf/collectors file
>> 2. Am unable to see the collector running via web browser
>>
>> Am I missing something?
>>
>> Thanks in advance.
>>
>> -bikash
>>
>> p.s. - after i run collector, nothing happens
>> % bin/chukwa collector
>> 2011-04-01 16:07:16.410::INFO: Logging to STDERR via
>> org.mortbay.log.StdErrLog
>> 2011-04-01 16:07:16.523::INFO: jetty-6.1.11
>> 2011-04-01 16:07:17.707::INFO: Started SelectChannelConnector@0.0.0.0:9999
>> started Chukwa http collector on port 9999
>>
>
Re: Chukwa setup issues
Posted by Eric Yang <er...@gmail.com>.
Hi,
For starting hicc, the command is:
chukwa hicc
Are you using 0.4? If so, I would recommend to try the trunk version instead.
The setup procedure for trunk is located here:
http://wiki.apache.org/hadoop/Chukwa_Quick_Start
Trunk is more stable than 0.4, but it use hbase instead of mysql.
regards,
Eric
On Fri, Apr 1, 2011 at 10:50 PM, Bill Graham <bi...@gmail.com> wrote:
> I'm forwarding this email to chukwa-user, where you'll have a better
> chance of getting an answer than on the hadoop common lists. (I'm not
> that familiar with hicc.)
>
> On Fri, Apr 1, 2011 at 5:19 PM, bikash sharma <sh...@gmail.com> wrote:
>> I was trying to install HICC in Chukwa, but hicc.sh does not exist in the
>> repository.
>> Any idea?
>> -bikash
>>
>> On Fri, Apr 1, 2011 at 5:57 PM, bikash sharma <sh...@gmail.com>
>> wrote:
>>>
>>> Thanks Bill.
>>> I am able to connect via web now, actually had put wrong http port in
>>> config file.
>>> One following question - if i run a mapreduce program say terasort, how
>>> can we link chukwa to collect job metrics via web.
>>>
>>> On Fri, Apr 1, 2011 at 5:37 PM, Bill Graham <bi...@gmail.com> wrote:
>>>>
>>>> Unfortunately conf/collectors is used in two different ways in Chukwa,
>>>> each with a different syntax. This should really be fixed.
>>>>
>>>> 1. The script that starts the collectors looks at it for a list of
>>>> hostnames (no ports) to start collectors on. To start it just on one
>>>> host, set it to localhost.
>>>> 2. The agent looks at that file for the list of collectors to attempt
>>>> to communicate with. In that case the format is a list of HTTP urls
>>>> with ports of the collectors.
>>>>
>>>> Can you telnet to port 9999? It looks like it's listening, but
>>>> nothing's being sent. Is there anything in logs/collector.log?
>>>>
>>>> On Fri, Apr 1, 2011 at 1:09 PM, bikash sharma <sh...@gmail.com>
>>>> wrote:
>>>> > Hi,
>>>> > I am trying to setup Chukwa for a 16-node Hadoop cluster.
>>>> > I followed the admin guide -
>>>> > http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html#Agents
>>>> > However, I ran two the following issues:
>>>> > 1. What should be the collector port that needs to be specified in
>>>> > conf/collectors file
>>>> > 2. Am unable to see the collector running via web browser
>>>> >
>>>> > Am I missing something?
>>>> >
>>>> > Thanks in advance.
>>>> >
>>>> > -bikash
>>>> >
>>>> > p.s. - after i run collector, nothing happens
>>>> > % bin/chukwa collector
>>>> > 2011-04-01 16:07:16.410::INFO: Logging to STDERR via
>>>> > org.mortbay.log.StdErrLog
>>>> > 2011-04-01 16:07:16.523::INFO: jetty-6.1.11
>>>> > 2011-04-01 16:07:17.707::INFO: Started
>>>> > SelectChannelConnector@0.0.0.0:9999
>>>> > started Chukwa http collector on port 9999
>>>> >
>>>
>>
>>
>
Re: Chukwa setup issues
Posted by Bill Graham <bi...@gmail.com>.
I'm forwarding this email to chukwa-user, where you'll have a better
chance of getting an answer than on the hadoop common lists. (I'm not
that familiar with hicc.)
On Fri, Apr 1, 2011 at 5:19 PM, bikash sharma <sh...@gmail.com> wrote:
> I was trying to install HICC in Chukwa, but hicc.sh does not exist in the
> repository.
> Any idea?
> -bikash
>
> On Fri, Apr 1, 2011 at 5:57 PM, bikash sharma <sh...@gmail.com>
> wrote:
>>
>> Thanks Bill.
>> I am able to connect via web now, actually had put wrong http port in
>> config file.
>> One following question - if i run a mapreduce program say terasort, how
>> can we link chukwa to collect job metrics via web.
>>
>> On Fri, Apr 1, 2011 at 5:37 PM, Bill Graham <bi...@gmail.com> wrote:
>>>
>>> Unfortunately conf/collectors is used in two different ways in Chukwa,
>>> each with a different syntax. This should really be fixed.
>>>
>>> 1. The script that starts the collectors looks at it for a list of
>>> hostnames (no ports) to start collectors on. To start it just on one
>>> host, set it to localhost.
>>> 2. The agent looks at that file for the list of collectors to attempt
>>> to communicate with. In that case the format is a list of HTTP urls
>>> with ports of the collectors.
>>>
>>> Can you telnet to port 9999? It looks like it's listening, but
>>> nothing's being sent. Is there anything in logs/collector.log?
>>>
>>> On Fri, Apr 1, 2011 at 1:09 PM, bikash sharma <sh...@gmail.com>
>>> wrote:
>>> > Hi,
>>> > I am trying to setup Chukwa for a 16-node Hadoop cluster.
>>> > I followed the admin guide -
>>> > http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html#Agents
>>> > However, I ran two the following issues:
>>> > 1. What should be the collector port that needs to be specified in
>>> > conf/collectors file
>>> > 2. Am unable to see the collector running via web browser
>>> >
>>> > Am I missing something?
>>> >
>>> > Thanks in advance.
>>> >
>>> > -bikash
>>> >
>>> > p.s. - after i run collector, nothing happens
>>> > % bin/chukwa collector
>>> > 2011-04-01 16:07:16.410::INFO: Logging to STDERR via
>>> > org.mortbay.log.StdErrLog
>>> > 2011-04-01 16:07:16.523::INFO: jetty-6.1.11
>>> > 2011-04-01 16:07:17.707::INFO: Started
>>> > SelectChannelConnector@0.0.0.0:9999
>>> > started Chukwa http collector on port 9999
>>> >
>>
>
>
Re: Chukwa setup issues
Posted by bikash sharma <sh...@gmail.com>.
I was trying to install HICC in Chukwa, but hicc.sh does not exist in the
repository.
Any idea?
-bikash
On Fri, Apr 1, 2011 at 5:57 PM, bikash sharma <sh...@gmail.com>wrote:
> Thanks Bill.
> I am able to connect via web now, actually had put wrong http port in
> config file.
> One following question - if i run a mapreduce program say terasort, how can
> we link chukwa to collect job metrics via web.
>
>
> On Fri, Apr 1, 2011 at 5:37 PM, Bill Graham <bi...@gmail.com> wrote:
>
>> Unfortunately conf/collectors is used in two different ways in Chukwa,
>> each with a different syntax. This should really be fixed.
>>
>> 1. The script that starts the collectors looks at it for a list of
>> hostnames (no ports) to start collectors on. To start it just on one
>> host, set it to localhost.
>> 2. The agent looks at that file for the list of collectors to attempt
>> to communicate with. In that case the format is a list of HTTP urls
>> with ports of the collectors.
>>
>> Can you telnet to port 9999? It looks like it's listening, but
>> nothing's being sent. Is there anything in logs/collector.log?
>>
>> On Fri, Apr 1, 2011 at 1:09 PM, bikash sharma <sh...@gmail.com>
>> wrote:
>> > Hi,
>> > I am trying to setup Chukwa for a 16-node Hadoop cluster.
>> > I followed the admin guide -
>> > http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html#Agents
>> > However, I ran two the following issues:
>> > 1. What should be the collector port that needs to be specified in
>> > conf/collectors file
>> > 2. Am unable to see the collector running via web browser
>> >
>> > Am I missing something?
>> >
>> > Thanks in advance.
>> >
>> > -bikash
>> >
>> > p.s. - after i run collector, nothing happens
>> > % bin/chukwa collector
>> > 2011-04-01 16:07:16.410::INFO: Logging to STDERR via
>> > org.mortbay.log.StdErrLog
>> > 2011-04-01 16:07:16.523::INFO: jetty-6.1.11
>> > 2011-04-01 16:07:17.707::INFO: Started
>> SelectChannelConnector@0.0.0.0:9999
>> > started Chukwa http collector on port 9999
>> >
>>
>
>
Re: Chukwa setup issues
Posted by bikash sharma <sh...@gmail.com>.
I was trying to install HICC in Chukwa, but hicc.sh does not exist in the
repository.
Any idea?
-bikash
On Fri, Apr 1, 2011 at 5:57 PM, bikash sharma <sh...@gmail.com>wrote:
> Thanks Bill.
> I am able to connect via web now, actually had put wrong http port in
> config file.
> One following question - if i run a mapreduce program say terasort, how can
> we link chukwa to collect job metrics via web.
>
>
> On Fri, Apr 1, 2011 at 5:37 PM, Bill Graham <bi...@gmail.com> wrote:
>
>> Unfortunately conf/collectors is used in two different ways in Chukwa,
>> each with a different syntax. This should really be fixed.
>>
>> 1. The script that starts the collectors looks at it for a list of
>> hostnames (no ports) to start collectors on. To start it just on one
>> host, set it to localhost.
>> 2. The agent looks at that file for the list of collectors to attempt
>> to communicate with. In that case the format is a list of HTTP urls
>> with ports of the collectors.
>>
>> Can you telnet to port 9999? It looks like it's listening, but
>> nothing's being sent. Is there anything in logs/collector.log?
>>
>> On Fri, Apr 1, 2011 at 1:09 PM, bikash sharma <sh...@gmail.com>
>> wrote:
>> > Hi,
>> > I am trying to setup Chukwa for a 16-node Hadoop cluster.
>> > I followed the admin guide -
>> > http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html#Agents
>> > However, I ran two the following issues:
>> > 1. What should be the collector port that needs to be specified in
>> > conf/collectors file
>> > 2. Am unable to see the collector running via web browser
>> >
>> > Am I missing something?
>> >
>> > Thanks in advance.
>> >
>> > -bikash
>> >
>> > p.s. - after i run collector, nothing happens
>> > % bin/chukwa collector
>> > 2011-04-01 16:07:16.410::INFO: Logging to STDERR via
>> > org.mortbay.log.StdErrLog
>> > 2011-04-01 16:07:16.523::INFO: jetty-6.1.11
>> > 2011-04-01 16:07:17.707::INFO: Started
>> SelectChannelConnector@0.0.0.0:9999
>> > started Chukwa http collector on port 9999
>> >
>>
>
>
Re: Chukwa setup issues
Posted by bikash sharma <sh...@gmail.com>.
Thanks Bill.
I am able to connect via web now, actually had put wrong http port in config
file.
One following question - if i run a mapreduce program say terasort, how can
we link chukwa to collect job metrics via web.
On Fri, Apr 1, 2011 at 5:37 PM, Bill Graham <bi...@gmail.com> wrote:
> Unfortunately conf/collectors is used in two different ways in Chukwa,
> each with a different syntax. This should really be fixed.
>
> 1. The script that starts the collectors looks at it for a list of
> hostnames (no ports) to start collectors on. To start it just on one
> host, set it to localhost.
> 2. The agent looks at that file for the list of collectors to attempt
> to communicate with. In that case the format is a list of HTTP urls
> with ports of the collectors.
>
> Can you telnet to port 9999? It looks like it's listening, but
> nothing's being sent. Is there anything in logs/collector.log?
>
> On Fri, Apr 1, 2011 at 1:09 PM, bikash sharma <sh...@gmail.com>
> wrote:
> > Hi,
> > I am trying to setup Chukwa for a 16-node Hadoop cluster.
> > I followed the admin guide -
> > http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html#Agents
> > However, I ran two the following issues:
> > 1. What should be the collector port that needs to be specified in
> > conf/collectors file
> > 2. Am unable to see the collector running via web browser
> >
> > Am I missing something?
> >
> > Thanks in advance.
> >
> > -bikash
> >
> > p.s. - after i run collector, nothing happens
> > % bin/chukwa collector
> > 2011-04-01 16:07:16.410::INFO: Logging to STDERR via
> > org.mortbay.log.StdErrLog
> > 2011-04-01 16:07:16.523::INFO: jetty-6.1.11
> > 2011-04-01 16:07:17.707::INFO: Started
> SelectChannelConnector@0.0.0.0:9999
> > started Chukwa http collector on port 9999
> >
>
Re: Chukwa setup issues
Posted by Bill Graham <bi...@gmail.com>.
Moving over to chukwa list and letting common-dev fall off via bcc.
On Fri, Apr 1, 2011 at 2:37 PM, Bill Graham <bi...@gmail.com> wrote:
> Unfortunately conf/collectors is used in two different ways in Chukwa,
> each with a different syntax. This should really be fixed.
>
> 1. The script that starts the collectors looks at it for a list of
> hostnames (no ports) to start collectors on. To start it just on one
> host, set it to localhost.
> 2. The agent looks at that file for the list of collectors to attempt
> to communicate with. In that case the format is a list of HTTP urls
> with ports of the collectors.
>
> Can you telnet to port 9999? It looks like it's listening, but
> nothing's being sent. Is there anything in logs/collector.log?
>
> On Fri, Apr 1, 2011 at 1:09 PM, bikash sharma <sh...@gmail.com> wrote:
>> Hi,
>> I am trying to setup Chukwa for a 16-node Hadoop cluster.
>> I followed the admin guide -
>> http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html#Agents
>> However, I ran two the following issues:
>> 1. What should be the collector port that needs to be specified in
>> conf/collectors file
>> 2. Am unable to see the collector running via web browser
>>
>> Am I missing something?
>>
>> Thanks in advance.
>>
>> -bikash
>>
>> p.s. - after i run collector, nothing happens
>> % bin/chukwa collector
>> 2011-04-01 16:07:16.410::INFO: Logging to STDERR via
>> org.mortbay.log.StdErrLog
>> 2011-04-01 16:07:16.523::INFO: jetty-6.1.11
>> 2011-04-01 16:07:17.707::INFO: Started SelectChannelConnector@0.0.0.0:9999
>> started Chukwa http collector on port 9999
>>
>
Re: Chukwa setup issues
Posted by bikash sharma <sh...@gmail.com>.
Thanks Bill.
I am able to connect via web now, actually had put wrong http port in config
file.
One following question - if i run a mapreduce program say terasort, how can
we link chukwa to collect job metrics via web.
On Fri, Apr 1, 2011 at 5:37 PM, Bill Graham <bi...@gmail.com> wrote:
> Unfortunately conf/collectors is used in two different ways in Chukwa,
> each with a different syntax. This should really be fixed.
>
> 1. The script that starts the collectors looks at it for a list of
> hostnames (no ports) to start collectors on. To start it just on one
> host, set it to localhost.
> 2. The agent looks at that file for the list of collectors to attempt
> to communicate with. In that case the format is a list of HTTP urls
> with ports of the collectors.
>
> Can you telnet to port 9999? It looks like it's listening, but
> nothing's being sent. Is there anything in logs/collector.log?
>
> On Fri, Apr 1, 2011 at 1:09 PM, bikash sharma <sh...@gmail.com>
> wrote:
> > Hi,
> > I am trying to setup Chukwa for a 16-node Hadoop cluster.
> > I followed the admin guide -
> > http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html#Agents
> > However, I ran two the following issues:
> > 1. What should be the collector port that needs to be specified in
> > conf/collectors file
> > 2. Am unable to see the collector running via web browser
> >
> > Am I missing something?
> >
> > Thanks in advance.
> >
> > -bikash
> >
> > p.s. - after i run collector, nothing happens
> > % bin/chukwa collector
> > 2011-04-01 16:07:16.410::INFO: Logging to STDERR via
> > org.mortbay.log.StdErrLog
> > 2011-04-01 16:07:16.523::INFO: jetty-6.1.11
> > 2011-04-01 16:07:17.707::INFO: Started
> SelectChannelConnector@0.0.0.0:9999
> > started Chukwa http collector on port 9999
> >
>
Re: Chukwa setup issues
Posted by Bill Graham <bi...@gmail.com>.
Unfortunately conf/collectors is used in two different ways in Chukwa,
each with a different syntax. This should really be fixed.
1. The script that starts the collectors looks at it for a list of
hostnames (no ports) to start collectors on. To start it just on one
host, set it to localhost.
2. The agent looks at that file for the list of collectors to attempt
to communicate with. In that case the format is a list of HTTP urls
with ports of the collectors.
Can you telnet to port 9999? It looks like it's listening, but
nothing's being sent. Is there anything in logs/collector.log?
On Fri, Apr 1, 2011 at 1:09 PM, bikash sharma <sh...@gmail.com> wrote:
> Hi,
> I am trying to setup Chukwa for a 16-node Hadoop cluster.
> I followed the admin guide -
> http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html#Agents
> However, I ran two the following issues:
> 1. What should be the collector port that needs to be specified in
> conf/collectors file
> 2. Am unable to see the collector running via web browser
>
> Am I missing something?
>
> Thanks in advance.
>
> -bikash
>
> p.s. - after i run collector, nothing happens
> % bin/chukwa collector
> 2011-04-01 16:07:16.410::INFO: Logging to STDERR via
> org.mortbay.log.StdErrLog
> 2011-04-01 16:07:16.523::INFO: jetty-6.1.11
> 2011-04-01 16:07:17.707::INFO: Started SelectChannelConnector@0.0.0.0:9999
> started Chukwa http collector on port 9999
>