You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@chukwa.apache.org by Ken White <ke...@mailcatch.com> on 2009/06/30 11:24:54 UTC
chukwa download
Hmmm, where can I download Chukwa? :)
Page http://wiki.apache.org/hadoop/Chukwa says:
-----
Download
Chukwa is part of the Hadoop distribution. You can view the source as part of
the Hadoop Apache SVN repository [WWW] here
-----
I have Hadoop 0.20.0 and chukwa is not in it. Also, the link to SVN is not
valid anymore, it should be:
http://svn.apache.org:80/viewvc/hadoop/chukwa/
Should I:
1) use hadoop 0.18.3 (stable)
2) download chukwa through SVN
What is the best way to get it?
Kind regards,
Ken
Re: chukwa download
Posted by Ariel Rabkin <as...@gmail.com>.
Your JAVA_HOME is pointing to a java 1.4 JDK; needs to be 1.6.
On Tue, Jun 30, 2009 at 3:15 AM, Ken White<ke...@mailcatch.com> wrote:
>
> Next obstacle... ;)
>
> $ svn co https://svn.apache.org/repos/asf/hadoop/chukwa/trunk chukwa
> $ cd chukwa
> $ ant
> ...
> at first ok, then lots of compiler errors like these:
> [javac] Compiling 94 source files to
> /var/repos/tmp/hadoop_svn/chukwa/build/classes
> [javac] ----------
> [javac] 1. ERROR in
> /var/repos/tmp/hadoop_svn/chukwa/src/java/org/apache/hadoop/chukwa/ChukwaArchiveKey.java
> (at line 61)
> [javac]
> java.util.Iterator<org.apache.hadoop.record.meta.FieldTypeInfo> _rio_itFilter
> = _rio_
> [javac] rtiFilter
> [javac] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [javac] Syntax error, parameterized types are only available if source
> level is1.5
> [javac] ----------
> [javac] 2. ERROR in
> /var/repos/tmp/hadoop_svn/chukwa/src/java/org/apache/hadoop/chukwa/ChukwaArchiveKey.java
> (at line 65)
> [javac] org.apache.hadoop.record.meta.FieldTypeInfo _rio_tInfoFilter =
> _rio_itFilter
> [javac]
> [javac] .next();
> [javac]
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> [javac] Type mismatch: cannot convert from Object to FieldTypeInfo
> [javac] ----------
> ...
> [javac] 1153 problems (1103 errors, 50 warnings)
>
> Doew Chukwa build depend on Hadoop? Should I be checking out the while tree or
> just setting classpath to hadoop libraries?
>
> Thanks for help,
>
> Ken
>
>
> On Tuesday 30 June 2009, Ken White wrote:
>> @Ari: sorry, didn't notice your post before asking... I will get SVN trunk
>> and start playing with it.
>>
>> Thanks! :)
>>
>> Ken
>>
>> On Tuesday 30 June 2009, Ken White wrote:
>> > Hmmm, where can I download Chukwa? :)
>> >
>> >
>> > Page http://wiki.apache.org/hadoop/Chukwa says:
>> > -----
>> > Download
>> > Chukwa is part of the Hadoop distribution. You can view the source as
>> > part of the Hadoop Apache SVN repository [WWW] here
>> > -----
>> > I have Hadoop 0.20.0 and chukwa is not in it. Also, the link to SVN is
>> > not valid anymore, it should be:
>> > http://svn.apache.org:80/viewvc/hadoop/chukwa/
>> >
>> >
>> > Should I:
>> > 1) use hadoop 0.18.3 (stable)
>> > 2) download chukwa through SVN
>> >
>> > What is the best way to get it?
>> >
>> > Kind regards,
>> >
>> > Ken
>
>
>
--
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department
Re: chukwa download
Posted by Ariel Rabkin <as...@gmail.com>.
Hrm.
That document is badly out of date. You should probably be looking at
http://hadoop.apache.org/chukwa/docs/current/admin.html#2.+Configure+the+Agent
instead.
On Wed, Jul 1, 2009 at 5:25 AM, Ken White<ke...@mailcatch.com> wrote:
>
>>Your JAVA_HOME is pointing to a java 1.4 JDK; needs to be 1.6.
> I am following this guide:
> http://wiki.apache.org/hadoop/Chukwa_Quick_Start
>
> Right now I am trying to run `bash bin/jettyCollector.sh` but I get this
> exception:
> java.lang.ClassNotFoundException:
> org.apache.hadoop.chukwa.datacollection.collector.CollectorStub
>
> I guess my classpath is incomplete, but I can't figure out what is missing (in
> which jar CollectorStub is). From jettyCollector.sh:
> $ java ... -classpath ${CHUKWA_CONF_DIR}:${HADOOP_CONF_DIR}:${CLASSPATH}:
> ${CHUKWA_CORE}:${COMMON}:${HADOOP_JAR} ...
You need to move the jars from build to the chukwa root directory.
> Also, there are some things that are not clear to me in conf/chukwa-collector-
> conf.xml:
>
> 1) writer.hdfs.filesystem - I want to output to HDFS. Does that mean I have to
> specify hdfs://... style path? In Quick Start is says "path to the namenode",
> but that is probably not enough?
Actually, that's no longer needed, so long as you define
HADOOP_CONF_DIR in chukwa-env.sh
> 2) chukwaCollector.outputDir - is this the subdir of previous setting or is
> this the local output directory that Collector writes to ebfore it commits to
> HDFS?
I believe this is the directory on HDFS. But I think the default
setting is fine.
> 3) chukwaCollector.rotateInterval - the default value is 5 minutes, is there
> any reason for it to be specified in milliseconds? Is this the time between
> commits to HDFS?
This upper-bounds the time between commits to HDFS; it may happen more
frequently at high load. The issue is that HDFS commits when a file
closes, or else at the end of a block. This specifies when to close
the file. Specified in milliseconds because we wanted to use millis
thoughought to avoid unit confusion.
--Ari
--
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department
Re: chukwa download
Posted by Ken White <ke...@mailcatch.com>.
>Your JAVA_HOME is pointing to a java 1.4 JDK; needs to be 1.6.
Thank you!!! :)
Indeed - JAVA_HOME was not set so it probably used some old Java engine I had
lying around.
I would appreciate some additional help in setting this up, I hope I'm not
bothering you too much.
I am following this guide:
http://wiki.apache.org/hadoop/Chukwa_Quick_Start
Right now I am trying to run `bash bin/jettyCollector.sh` but I get this
exception:
java.lang.ClassNotFoundException:
org.apache.hadoop.chukwa.datacollection.collector.CollectorStub
I guess my classpath is incomplete, but I can't figure out what is missing (in
which jar CollectorStub is). From jettyCollector.sh:
$ java ... -classpath ${CHUKWA_CONF_DIR}:${HADOOP_CONF_DIR}:${CLASSPATH}:
${CHUKWA_CORE}:${COMMON}:${HADOOP_JAR} ...
Also, there are some things that are not clear to me in conf/chukwa-collector-
conf.xml:
1) writer.hdfs.filesystem - I want to output to HDFS. Does that mean I have to
specify hdfs://... style path? In Quick Start is says "path to the namenode",
but that is probably not enough?
2) chukwaCollector.outputDir - is this the subdir of previous setting or is
this the local output directory that Collector writes to ebfore it commits to
HDFS?
3) chukwaCollector.rotateInterval - the default value is 5 minutes, is there
any reason for it to be specified in milliseconds? Is this the time between
commits to HDFS?
Thank you again for your help, I appreciate it.
Kind regards,
Ken
Re: chukwa download
Posted by Ken White <ke...@mailcatch.com>.
Next obstacle... ;)
$ svn co https://svn.apache.org/repos/asf/hadoop/chukwa/trunk chukwa
$ cd chukwa
$ ant
...
at first ok, then lots of compiler errors like these:
[javac] Compiling 94 source files to
/var/repos/tmp/hadoop_svn/chukwa/build/classes
[javac] ----------
[javac] 1. ERROR in
/var/repos/tmp/hadoop_svn/chukwa/src/java/org/apache/hadoop/chukwa/ChukwaArchiveKey.java
(at line 61)
[javac]
java.util.Iterator<org.apache.hadoop.record.meta.FieldTypeInfo> _rio_itFilter
= _rio_
[javac] rtiFilter
[javac] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[javac] Syntax error, parameterized types are only available if source
level is1.5
[javac] ----------
[javac] 2. ERROR in
/var/repos/tmp/hadoop_svn/chukwa/src/java/org/apache/hadoop/chukwa/ChukwaArchiveKey.java
(at line 65)
[javac] org.apache.hadoop.record.meta.FieldTypeInfo _rio_tInfoFilter =
_rio_itFilter
[javac]
[javac] .next();
[javac]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[javac] Type mismatch: cannot convert from Object to FieldTypeInfo
[javac] ----------
...
[javac] 1153 problems (1103 errors, 50 warnings)
Doew Chukwa build depend on Hadoop? Should I be checking out the while tree or
just setting classpath to hadoop libraries?
Thanks for help,
Ken
On Tuesday 30 June 2009, Ken White wrote:
> @Ari: sorry, didn't notice your post before asking... I will get SVN trunk
> and start playing with it.
>
> Thanks! :)
>
> Ken
>
> On Tuesday 30 June 2009, Ken White wrote:
> > Hmmm, where can I download Chukwa? :)
> >
> >
> > Page http://wiki.apache.org/hadoop/Chukwa says:
> > -----
> > Download
> > Chukwa is part of the Hadoop distribution. You can view the source as
> > part of the Hadoop Apache SVN repository [WWW] here
> > -----
> > I have Hadoop 0.20.0 and chukwa is not in it. Also, the link to SVN is
> > not valid anymore, it should be:
> > http://svn.apache.org:80/viewvc/hadoop/chukwa/
> >
> >
> > Should I:
> > 1) use hadoop 0.18.3 (stable)
> > 2) download chukwa through SVN
> >
> > What is the best way to get it?
> >
> > Kind regards,
> >
> > Ken
Re: chukwa download
Posted by Ken White <ke...@mailcatch.com>.
@Ari: sorry, didn't notice your post before asking... I will get SVN trunk and
start playing with it.
Thanks! :)
Ken
On Tuesday 30 June 2009, Ken White wrote:
> Hmmm, where can I download Chukwa? :)
>
>
> Page http://wiki.apache.org/hadoop/Chukwa says:
> -----
> Download
> Chukwa is part of the Hadoop distribution. You can view the source as part
> of the Hadoop Apache SVN repository [WWW] here
> -----
> I have Hadoop 0.20.0 and chukwa is not in it. Also, the link to SVN is not
> valid anymore, it should be:
> http://svn.apache.org:80/viewvc/hadoop/chukwa/
>
>
> Should I:
> 1) use hadoop 0.18.3 (stable)
> 2) download chukwa through SVN
>
> What is the best way to get it?
>
> Kind regards,
>
> Ken