You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Jonathan Hunter <Jo...@oberlin.edu> on 2007/01/22 06:56:58 UTC
Compiling PruneIndexTool trouble
Dear nutch-users,
I am trying to make some changes to the Nutch's PruneIndexTool, but
before I start making those changes I wanted to make sure that I am able
to compile the current PruneIndexTool from the command line.
I checked to make sure that the java compiler works in general by using
it to compile a simple hello world program.
I did this by calling the following command from my nutch directory:
$ javac helloworld.java
//compiles with no errors
$ java helloworld
hello world
$
This led me to believe that the compiler is working.
After I determined that the compiler was working correctly, I then tried
to compile the PruneIndexTool.java file by calling:
$ javac PruneIndexTool.java
However, this command resulted in approx. 100 errors, which I have
listed at the end of this e-mail.
I think that problem might be the CLASSPATH variable in my .bash_profile.
My CLASSPATH is currently set to:
CLASSPATH=/home/jon/nutch-0.8.1/jakarta-oro-2.0.8
/src/java:/home/jon/nutch-0.8.1/lucene-2.0.0/src/java:/home/jon/nutch-0.8.1/src/java:/home/jon/nutch-0.8.1/src/plugin/urlfilter-regex/src/java:.
Here is the list of errors I am currently receiving. Any help you can
provide is much appreciated.
thank you,
....
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/crawl/MapWritable.java:148:
cannot resolve symbol
symbol : class Writable
location: class org.apache.nutch.crawl.MapWritable
public Writable get(Writable key) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/crawl/MapWritable.java:148:
cannot resolve symbol
symbol : class Writable
location: class org.apache.nutch.crawl.MapWritable
public Writable get(Writable key) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/crawl/MapWritable.java:183:
cannot resolve symbol
symbol : class Writable
location: class org.apache.nutch.crawl.MapWritable
public Writable put(Writable key, Writable value) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/crawl/MapWritable.java:183:
cannot resolve symbol
symbol : class Writable
location: class org.apache.nutch.crawl.MapWritable
public Writable put(Writable key, Writable value) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/crawl/MapWritable.java:183:
cannot resolve symbol
symbol : class Writable
location: class org.apache.nutch.crawl.MapWritable
public Writable put(Writable key, Writable value) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/crawl/MapWritable.java:213:
cannot resolve symbol
symbol : class Writable
location: class org.apache.nutch.crawl.MapWritable
public Writable remove(Writable key) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/crawl/MapWritable.java:213:
cannot resolve symbol
symbol : class Writable
location: class org.apache.nutch.crawl.MapWritable
public Writable remove(Writable key) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/crawl/MapWritable.java:286:
cannot resolve symbol
symbol : class Writable
location: class org.apache.nutch.crawl.MapWritable
private KeyValueEntry findEntryByKey(final Writable key) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/protocol/Content.java:22:
package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/protocol/Content.java:23:
package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.*;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/protocol/Content.java:24:
package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.*;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/protocol/Content.java:31:
cannot resolve symbol
symbol : class CompressedWritable
location: class org.apache.nutch.protocol.Content
public final class Content extends CompressedWritable {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/metadata/Metadata.java:34:
package org.apache.commons.lang does not exist
import org.apache.commons.lang.StringUtils;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/metadata/Metadata.java:37:
package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.UTF8;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/metadata/Metadata.java:38:
package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.Writable;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/metadata/Metadata.java:55:
cannot resolve symbol
symbol : class Writable
location: class org.apache.nutch.metadata.Metadata
Writable {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/util/mime/MimeTypes.java:28:
package org.apache.commons.logging does not exist
import org.apache.commons.logging.Log;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/util/mime/MimeTypes.java:69:
cannot resolve symbol
symbol : class Log
location: class org.apache.nutch.util.mime.MimeTypes
private MimeTypes(String filepath, Log logger) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/util/mime/MimeTypes.java:98:
cannot resolve symbol
symbol : class Log
location: class org.apache.nutch.util.mime.MimeTypes
public static MimeTypes get(String filepath, Log logger) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/protocol/Content.java:49:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.protocol.Content
Metadata metadata, Configuration conf) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseImpl.java:20:
package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseImpl.java:21:
package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.Configuration;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseImpl.java:22:
package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.Configurable;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseImpl.java:28:
cannot resolve symbol
symbol : class Writable
location: class org.apache.nutch.parse.ParseImpl
public class ParseImpl implements Parse, Writable, Configurable {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseImpl.java:28:
cannot resolve symbol
symbol : class Configurable
location: class org.apache.nutch.parse.ParseImpl
public class ParseImpl implements Parse, Writable, Configurable {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseText.java:20:
package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseText.java:21:
package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.*;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseText.java:22:
package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.*;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseText.java:28:
cannot resolve symbol
symbol : class VersionedWritable
location: class org.apache.nutch.parse.ParseText
public final class ParseText extends VersionedWritable {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseData.java:22:
package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseData.java:23:
package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.*;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseData.java:24:
package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.Configuration;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseData.java:25:
package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.Configurable;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseData.java:34:
cannot resolve symbol
symbol : class VersionedWritable
location: class org.apache.nutch.parse.ParseData
public final class ParseData extends VersionedWritable implements
Configurable {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseData.java:34:
cannot resolve symbol
symbol : class Configurable
location: class org.apache.nutch.parse.ParseData
public final class ParseData extends VersionedWritable implements
Configurable {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseImpl.java:31:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.parse.ParseImpl
private Configuration conf;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseImpl.java:66:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.parse.ParseImpl
public static ParseImpl read(DataInput in, Configuration conf) throws
IOException {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseImpl.java:73:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.parse.ParseImpl
public void setConf(Configuration conf) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseImpl.java:78:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.parse.ParseImpl
public Configuration getConf() {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/Outlink.java:22:
package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/Outlink.java:24:
package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.Configuration;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/Outlink.java:27:
cannot resolve symbol
symbol : class Writable
location: class org.apache.nutch.parse.Outlink
public class Outlink implements Writable {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/Outlink.java:34:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.parse.Outlink
public Outlink(String toUrl, String anchor, Configuration conf) throws
MalformedURLException {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseStatus.java:12:
package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.VersionedWritable;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseStatus.java:13:
package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.WritableUtils;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseStatus.java:14:
package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.Configuration;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseStatus.java:22:
cannot resolve symbol
symbol : class VersionedWritable
location: class org.apache.nutch.parse.ParseStatus
public class ParseStatus extends VersionedWritable {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseStatus.java:168:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.parse.ParseStatus
public Parse getEmptyParse(Configuration conf) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseData.java:44:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.parse.ParseData
private Configuration conf;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseData.java:239:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.parse.ParseData
public void setConf(Configuration conf) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/parse/ParseData.java:243:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.parse.ParseData
public Configuration getConf() {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/fetcher/FetcherOutput.java:33:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.fetcher.FetcherOutput
private Configuration conf;
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/fetcher/FetcherOutput.java:83:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.fetcher.FetcherOutput
public void setConf(Configuration conf) {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/fetcher/FetcherOutput.java:87:
cannot resolve symbol
symbol : class Configuration
location: class org.apache.nutch.fetcher.FetcherOutput
public Configuration getConf() {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/crawl/CrawlDatum.java:240:
cannot resolve symbol
symbol : class WritableComparator
location: class org.apache.nutch.crawl.CrawlDatum.Comparator
public static class Comparator extends WritableComparator {
^
/home/jon/nutch-0.8.1/src/java/org/apache/nutch/net/UrlNormalizerFactory.java:20:
package org.apache.commons.logging does not exist
import org.apache.commons.logging.Log;
^
Re: Compiling PruneIndexTool trouble
Posted by Jonathan Hunter <Jo...@oberlin.edu>.
dear nutch-users
I am trying to make modifications to the PruneIndexTool.java file in my
nutch directory. When I compile this using ant:
$ export ANT_HOME=/usr/local/ant/apache-ant-1.6.5
$ export PATH=${PATH}:${ANT_HOME}/bin
$ ant
then call:
$ bin/nutch org.apache.nutch.tools.PruneIndexTool
I get the missing arguments error, which correctly calls the usage()
method. The problem lies in that when I try to make any type of
modification, bin/nutch uses the original version of the Tool, never the
modified Tool. I've made sure to edit the Tool in the src/java/.../tools
directory, which ant compiles from. I've put test print statements at
the beginning of the usage() method as well as at the start of the main
method, but these statements are not called when running the Tool.
my PATH and CLASSPATH are as followed:
PATH=$PATH:/usr/local/j2sdk1.3.2_08/bin/
CLASSPATH=/home/jon/nutch-0.8.1/jakarta-oro-2.0.8/src/java:/home/jon/nutch-0.8.1/lucene-2.0.0/src/java:/home/jon/nutch-0.8.1/src/java:/home/jon/nutch-0.8.1/src/plugin/urlfilter-regex/src/java:.
thank you,
~Jon
----- Original Message -----
From: Renaud Richardet <re...@oslutions.com>
Date: Tuesday, January 23, 2007 7:07 pm
Subject: Re: Compiling PruneIndexTool trouble
To: nutch-user@lucene.apache.org
> Hey Jon,
>
> Ant should have already compiled it for you.
> Try bin/nutch org.apache.nutch.tools.PruneIndexTool [...] and it
> should
> work.
>
> HTH,
> Renaud
>
>
> Jonathan Hunter wrote:
> > nutch-users,
> > I was able to use ant to build and compile my nutch directory. I was
> > wondering if ant is the correct tool to compile the
> PruneIndexTool.java> specifically. If so, what is the command I
> should call to compile?
> >
> > thank you,
> > ~Jon
> >
> > ----- Original Message -----
> > From: Sami Siren <ss...@gmail.com>
> > Date: Monday, January 22, 2007 10:10 am
> > Subject: Re: Compiling PruneIndexTool trouble
> > To: nutch-user@lucene.apache.org
> >
> >
> >> Jonathan Hunter wrote:
> >>
> >>> Dear nutch-users,
> >>>
> >>> I am trying to make some changes to the Nutch's PruneIndexTool,
> but>>> before I start making those changes I wanted to make sure
> that I
> >>>
> >> am able
> >>
> >>> to compile the current PruneIndexTool from the command line.
> >>>
> >>> I checked to make sure that the java compiler works in general
> by
> >>>
> >> using> it to compile a simple hello world program.
> >>
> >>> I did this by calling the following command from my nutch
> directory:>>>
> >>> $ javac helloworld.java
> >>> //compiles with no errors
> >>> $ java helloworld
> >>> hello world
> >>> $
> >>>
> >>>
> >> You should use "ant" command to compile nutch (including
> >> PruneIndexTool).
> >> $ ant
> >>
> >> --
> >> Sami Siren
> >>
> >>
> >
> >
>
>
> --
> renaud richardet +1 617 230 9112
> renaud <at> oslutions.com http://www.oslutions.com
>
>
Re: Compiling PruneIndexTool trouble
Posted by Renaud Richardet <re...@oslutions.com>.
Hey Jon,
Ant should have already compiled it for you.
Try bin/nutch org.apache.nutch.tools.PruneIndexTool [...] and it should
work.
HTH,
Renaud
Jonathan Hunter wrote:
> nutch-users,
> I was able to use ant to build and compile my nutch directory. I was
> wondering if ant is the correct tool to compile the PruneIndexTool.java
> specifically. If so, what is the command I should call to compile?
>
> thank you,
> ~Jon
>
> ----- Original Message -----
> From: Sami Siren <ss...@gmail.com>
> Date: Monday, January 22, 2007 10:10 am
> Subject: Re: Compiling PruneIndexTool trouble
> To: nutch-user@lucene.apache.org
>
>
>> Jonathan Hunter wrote:
>>
>>> Dear nutch-users,
>>>
>>> I am trying to make some changes to the Nutch's PruneIndexTool, but
>>> before I start making those changes I wanted to make sure that I
>>>
>> am able
>>
>>> to compile the current PruneIndexTool from the command line.
>>>
>>> I checked to make sure that the java compiler works in general by
>>>
>> using> it to compile a simple hello world program.
>>
>>> I did this by calling the following command from my nutch directory:
>>>
>>> $ javac helloworld.java
>>> //compiles with no errors
>>> $ java helloworld
>>> hello world
>>> $
>>>
>>>
>> You should use "ant" command to compile nutch (including
>> PruneIndexTool).
>> $ ant
>>
>> --
>> Sami Siren
>>
>>
>
>
--
renaud richardet +1 617 230 9112
renaud <at> oslutions.com http://www.oslutions.com
Re: Compiling PruneIndexTool trouble
Posted by Jonathan Hunter <Jo...@oberlin.edu>.
nutch-users,
I was able to use ant to build and compile my nutch directory. I was
wondering if ant is the correct tool to compile the PruneIndexTool.java
specifically. If so, what is the command I should call to compile?
thank you,
~Jon
----- Original Message -----
From: Sami Siren <ss...@gmail.com>
Date: Monday, January 22, 2007 10:10 am
Subject: Re: Compiling PruneIndexTool trouble
To: nutch-user@lucene.apache.org
> Jonathan Hunter wrote:
> > Dear nutch-users,
> >
> > I am trying to make some changes to the Nutch's PruneIndexTool, but
> > before I start making those changes I wanted to make sure that I
> am able
> > to compile the current PruneIndexTool from the command line.
> >
> > I checked to make sure that the java compiler works in general by
> using> it to compile a simple hello world program.
> >
> > I did this by calling the following command from my nutch directory:
> >
> > $ javac helloworld.java
> > //compiles with no errors
> > $ java helloworld
> > hello world
> > $
> >
>
>
> You should use "ant" command to compile nutch (including
> PruneIndexTool).
> $ ant
>
> --
> Sami Siren
>
Re: Compiling PruneIndexTool trouble
Posted by Sami Siren <ss...@gmail.com>.
Jonathan Hunter wrote:
> Dear nutch-users,
>
> I am trying to make some changes to the Nutch's PruneIndexTool, but
> before I start making those changes I wanted to make sure that I am able
> to compile the current PruneIndexTool from the command line.
>
> I checked to make sure that the java compiler works in general by using
> it to compile a simple hello world program.
>
> I did this by calling the following command from my nutch directory:
>
> $ javac helloworld.java
> //compiles with no errors
> $ java helloworld
> hello world
> $
>
You should use "ant" command to compile nutch (including PruneIndexTool).
$ ant
--
Sami Siren