You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by ahammad <ah...@gmail.com> on 2009/01/13 22:04:12 UTC
Question about writing plug ins
Hello,
I've been looking at the plugin writing tutorial on the Nutch wiki
(http://wiki.apache.org/nutch/WritingPluginExample-0%2e9).
First it tells you to start by "downloading the Nutch source code". The
thing that I downloaded is the release zip file of Nutch that pretty much
works out of the box (I didn't compile or package anything). Within the
Nutch folder, I have a folder called src. Is that what the tutorial is
referring to or was I supposed to download something else?
My next issue is where to save the files. The tutorial starts off well but
then becomes less and less descriptive. What I did was I created a folder
within src/plugin called "recommended" like they suggested, and then I
placed the following files from the code that they had:
1) build.xml and plugin.xml in C:\nutch-0.9\src\plugin\recommended
2) RecommendedIndexer.java, RecommendedParser.java, QueryFilter.java in
C:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\recommended
When I try to compile using the ant command in cygwin, I get the following:
----------------------------------------------------------------------
Buildfile: build.xml
init:
init-plugin:
deps-jar:
compile:
[echo] Compiling plugin: recommended
[javac] Compiling 3 source files to
c:\nutch-0.9\build\recommended\classes
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\QueryFilter.java:12: class RecommendedQueryFilter is public,
should
be declared in a file named RecommendedQueryFilter.java
[javac] public class RecommendedQueryFilter extends FieldQueryFilter {
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\QueryFilter.java:3: package org.apache.nutch.searcher does not
exist
[javac] import org.apache.nutch.searcher.FieldQueryFilter;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\QueryFilter.java:12: cannot find symbol
[javac] symbol: class FieldQueryFilter
[javac] public class RecommendedQueryFilter extends FieldQueryFilter {
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:12: package org.apache.nutch.util does
not e
xist
[javac] import org.apache.nutch.util.LogUtil;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:13: package org.apache.nutch.fetcher
does no
t exist
[javac] import org.apache.nutch.fetcher.FetcherOutput;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:14: package org.apache.nutch.indexer
does no
t exist
[javac] import org.apache.nutch.indexer.IndexingFilter;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:15: package org.apache.nutch.indexer
does no
t exist
[javac] import org.apache.nutch.indexer.IndexingException;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:16: cannot find symbol
[javac] symbol : class Parse
[javac] location: package org.apache.nutch.parse
[javac] import org.apache.nutch.parse.Parse;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:20: package org.apache.nutch.crawl does
not
exist
[javac] import org.apache.nutch.crawl.CrawlDatum;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:21: package org.apache.nutch.crawl does
not
exist
[javac] import org.apache.nutch.crawl.Inlinks;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:27: cannot find symbol
[javac] symbol: class IndexingFilter
[javac] public class RecommendedIndexer implements IndexingFilter {
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:36: cannot find symbol
[javac] symbol : class Parse
[javac] location: class
org.apache.nutch.parse.recommended.RecommendedIndexe
r
[javac] public Document filter(Document doc, Parse parse, Text url,
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:37: cannot find symbol
[javac] symbol : class CrawlDatum
[javac] location: class
org.apache.nutch.parse.recommended.RecommendedIndexe
r
[javac] CrawlDatum datum, Inlinks inlinks)
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:37: cannot find symbol
[javac] symbol : class Inlinks
[javac] location: class
org.apache.nutch.parse.recommended.RecommendedIndexe
r
[javac] CrawlDatum datum, Inlinks inlinks)
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedIndexer.java:38: cannot find symbol
[javac] symbol : class IndexingException
[javac] location: class
org.apache.nutch.parse.recommended.RecommendedIndexe
r
[javac] throws IndexingException {
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedParser.java:10: cannot find symbol
[javac] symbol : class HTMLMetaTags
[javac] location: package org.apache.nutch.parse
[javac] import org.apache.nutch.parse.HTMLMetaTags;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedParser.java:11: cannot find symbol
[javac] symbol : class Parse
[javac] location: package org.apache.nutch.parse
[javac] import org.apache.nutch.parse.Parse;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedParser.java:12: cannot find symbol
[javac] symbol : class HtmlParseFilter
[javac] location: package org.apache.nutch.parse
[javac] import org.apache.nutch.parse.HtmlParseFilter;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedParser.java:13: package org.apache.nutch.protocol
does no
t exist
[javac] import org.apache.nutch.protocol.Content;
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedParser.java:22: cannot find symbol
[javac] symbol: class HtmlParseFilter
[javac] public class RecommendedParser implements HtmlParseFilter {
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedParser.java:34: cannot find symbol
[javac] symbol : class Content
[javac] location: class
org.apache.nutch.parse.recommended.RecommendedParser
[javac] public Parse filter(Content content, Parse parse,
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedParser.java:34: cannot find symbol
[javac] symbol : class Parse
[javac] location: class
org.apache.nutch.parse.recommended.RecommendedParser
[javac] public Parse filter(Content content, Parse parse,
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedParser.java:35: cannot find symbol
[javac] symbol : class HTMLMetaTags
[javac] location: class
org.apache.nutch.parse.recommended.RecommendedParser
[javac] HTMLMetaTags metaTags, DocumentFragment doc) {
[javac] ^
[javac]
c:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\
recommended\RecommendedParser.java:34: cannot find symbol
[javac] symbol : class Parse
[javac] location: class
org.apache.nutch.parse.recommended.RecommendedParser
[javac] public Parse filter(Content content, Parse parse,
[javac] ^
[javac] 24 errors
BUILD FAILED
c:\nutch-0.9\src\plugin\build-plugin.xml:111: Compile failed; see the
compiler e
rror output for details.
Total time: 2 seconds
------------------------------------------------------------------------
It seems to me like my file placement is incorrect. All the other plugins
have only one .java file whereas I have 3. The problem is I'm not sure where
they're supposed to go.
As I said, I did not modify any of the code whatsoever. I want to get this
plugin running first, and then I'll proceed to code what I need (indexing
meta tags). If you can help me sort out these issues or point me to a more
descriptive tutorial I'll be extremely happy. Any help would be very much
appreciated.
Regards
--
View this message in context: http://www.nabble.com/Question-about-writing-plug-ins-tp21444458p21444458.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Question about writing plug ins
Posted by ahammad <ah...@gmail.com>.
Hello,
I tried deleting the build folder before, but never the plugins folder. It
worked after I got rid of the plugins folder. When I ran ant, the plugins
folder was never recreated.
It works fine now and the plugin is used. I'm just wondering what the
purpose of the plugins folder was.
Cheers
Ankur Garg-2 wrote:
>
> Ahmed,
>
> *"I also noticed that my recommended plugin was not in
> C:\nutch-0.9\plugins.
> Is there anything I need to add there?*"
>
> I think you should try by first deleting nutch-0.9/build/ and
> nutch-0.9/plugin/ folders and then doing ant from nutch-0.9 folder(u don't
> need to do ant seperately in recommended plugin )
> then your plugin should appear in nutch-0.9/plugin folder thus enabling
> the
> plugin to run while crawling.
>
>
> Ankur Garg
>
> On Fri, Jan 16, 2009 at 8:01 PM, ahammad <ah...@gmail.com> wrote:
>
>>
>> Hello,
>>
>> I tried that but it still didn't work. Note that I am working off of the
>> files that were extracted from the zip file. Here are the exact steps
>> that
>> I
>> took:
>>
>> 1) Ran "ant" in C:\nutch-0.9 to build nutch. A folder called "build" was
>> created in C:\nutch-0.9
>> 2) Ran "ant" in C:\nutch-0.9\src\plugin\recommended to build my plugin. A
>> folder called "recommended" appeared in C:\nutch-0.9\build
>> 3) Modified the build.xml file in C:\nutch-0.9\src\plugin to include <ant
>> dir="recommended" target="deploy"/>
>> 4) Modified nutch-default.xml like so (note the addition of "recommended"
>> at
>> the end of the string:
>>
>> <value>protocol-http|urlfilter-regex|parse-(text|html|js)|index-basic|query-(basic|site|url)|summary-basic|scoring-opic|urlnormalizer-(pass|regex|basic)|recommended</value>
>> 5) Rebuilt nutch by running ant again in C:\nutch-0.9
>>
>> Then I tried crawling. In hadoop.log, there was no reference to the newly
>> added plugin.
>>
>> I noticed that whenever you build nutch, a new nutch-0.9.job appears in
>> the
>> build folder. Is that at all relevant? Do I have to replace any of the
>> files
>> in C:\nutch-0.9 by the files in the build folder? Do I need to run ant
>> war
>> in order to generate the .jar and .war files, and then will I need to
>> replace the .jar and .war files in C:\nutch-0.9 by the ones that were
>> created in the build folder?
>>
>> I also noticed that my recommended plugin was not in
>> C:\nutch-0.9\plugins.
>> Is there anything I need to add there?
>>
>> I'm sure that what I am missing is a small step. I have a lot of things I
>> want to add in the future, and this is the only thing that is holding me
>> back.
>>
>> Thanks a lot for the help.
>>
>> Cheers
>>
>>
>>
>> Ankur Garg-2 wrote:
>> >
>> > Hi Ahmed,
>> >
>> > To be able to use the plugin after you have sucessfully compiled
>> > it,
>> > you it to do one entry in
>> > conf/nutch-default.xml file in the property "plugin.includes". After
>> doing
>> > this entry only, the plugin will be activated.
>> >
>> >
>> > hope it helps you
>> >
>> > Ankur Garg
>> >
>> > On Thu, Jan 15, 2009 at 10:45 PM, ahammad <ah...@gmail.com>
>> wrote:
>> >
>> >>
>> >> I was messing around with this and I got a step further.
>> >>
>> >> Background information:
>> >> I am trying to install the sample plugin from the Nutch Wiki. I
>> >> downloaded
>> >> the Nutch 0.9 release version (not sure if this is what I need to
>> >> download)
>> >> and I added the following files in the nutch-0.9\src\ directory:
>> >>
>> >> 1) build.xml and plugin.xml in C:\nutch-0.9\src\plugin\recommended
>> >> 2) RecommendedIndexer.java, RecommendedParser.java, QueryFilter.java
>> in
>> >>
>> >>
>> C:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\recommended
>> >>
>> >> note- all the code came from the Nutch Wiki
>> >>
>> >> I also changed the build.xml file in src\plugin to include the new
>> >> plugin:
>> >> <ant dir="recommended" target="deploy"/>
>> >>
>> >> What I tried:
>> >> I started off by attempting to compile the plugin by using ant. When I
>> >> did
>> >> this, I get errors about some packages not existing. I also get errors
>> >> about
>> >> the file QueryFilter being misnamed and saying that it needs to be
>> called
>> >> RecommendedQueryFilter. I made that change and renamed it like it was
>> >> suggested.
>> >>
>> >> I ended up going to the root directory of nutch and running ant from
>> >> there.
>> >> I believe this builds the whole program. A folder called "build" was
>> >> created
>> >> containing a bunch of stuff in there.
>> >>
>> >> After that was done, I went on to C:\nutch-0.9\src\plugin\recommended
>> and
>> >> ran ant again. This time, I did not get errors and everything worked
>> >> fine.
>> >>
>> >> The issue:
>> >> In the hadoop.log file I do not see any references made to the new
>> plugin
>> >> ("Recommended"). It goes on about added plugins/extensions, but there
>> is
>> >> no
>> >> mention of the one I added.
>> >>
>> >> Did I do everything correctly? What am I missing here? I am completely
>> >> lost
>> >> right now and I really need help with this.
>> >>
>> >> Thank you and regards.
>> >>
>> >>
>> >> --
>> >> View this message in context:
>> >>
>> http://www.nabble.com/Question-about-writing-plug-ins-tp21444458p21482275.html
>> >> Sent from the Nutch - User mailing list archive at Nabble.com.
>> >>
>> >>
>> >
>> >
>> > --
>> > Ankur Garg
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Question-about-writing-plug-ins-tp21444458p21500533.html
>> Sent from the Nutch - User mailing list archive at Nabble.com.
>>
>>
>
>
> --
> Ankur Garg
>
> CDAC Noida
> Anusandhan Bhawan
> Sec-62, Noida
> Mob. 9968274797
>
>
--
View this message in context: http://www.nabble.com/Question-about-writing-plug-ins-tp21444458p21508329.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Question about writing plug ins
Posted by Ankur Garg <ga...@gmail.com>.
Ahmed,
*"I also noticed that my recommended plugin was not in C:\nutch-0.9\plugins.
Is there anything I need to add there?*"
I think you should try by first deleting nutch-0.9/build/ and
nutch-0.9/plugin/ folders and then doing ant from nutch-0.9 folder(u don't
need to do ant seperately in recommended plugin )
then your plugin should appear in nutch-0.9/plugin folder thus enabling the
plugin to run while crawling.
Ankur Garg
On Fri, Jan 16, 2009 at 8:01 PM, ahammad <ah...@gmail.com> wrote:
>
> Hello,
>
> I tried that but it still didn't work. Note that I am working off of the
> files that were extracted from the zip file. Here are the exact steps that
> I
> took:
>
> 1) Ran "ant" in C:\nutch-0.9 to build nutch. A folder called "build" was
> created in C:\nutch-0.9
> 2) Ran "ant" in C:\nutch-0.9\src\plugin\recommended to build my plugin. A
> folder called "recommended" appeared in C:\nutch-0.9\build
> 3) Modified the build.xml file in C:\nutch-0.9\src\plugin to include <ant
> dir="recommended" target="deploy"/>
> 4) Modified nutch-default.xml like so (note the addition of "recommended"
> at
> the end of the string:
>
> <value>protocol-http|urlfilter-regex|parse-(text|html|js)|index-basic|query-(basic|site|url)|summary-basic|scoring-opic|urlnormalizer-(pass|regex|basic)|recommended</value>
> 5) Rebuilt nutch by running ant again in C:\nutch-0.9
>
> Then I tried crawling. In hadoop.log, there was no reference to the newly
> added plugin.
>
> I noticed that whenever you build nutch, a new nutch-0.9.job appears in the
> build folder. Is that at all relevant? Do I have to replace any of the
> files
> in C:\nutch-0.9 by the files in the build folder? Do I need to run ant war
> in order to generate the .jar and .war files, and then will I need to
> replace the .jar and .war files in C:\nutch-0.9 by the ones that were
> created in the build folder?
>
> I also noticed that my recommended plugin was not in C:\nutch-0.9\plugins.
> Is there anything I need to add there?
>
> I'm sure that what I am missing is a small step. I have a lot of things I
> want to add in the future, and this is the only thing that is holding me
> back.
>
> Thanks a lot for the help.
>
> Cheers
>
>
>
> Ankur Garg-2 wrote:
> >
> > Hi Ahmed,
> >
> > To be able to use the plugin after you have sucessfully compiled
> > it,
> > you it to do one entry in
> > conf/nutch-default.xml file in the property "plugin.includes". After
> doing
> > this entry only, the plugin will be activated.
> >
> >
> > hope it helps you
> >
> > Ankur Garg
> >
> > On Thu, Jan 15, 2009 at 10:45 PM, ahammad <ah...@gmail.com>
> wrote:
> >
> >>
> >> I was messing around with this and I got a step further.
> >>
> >> Background information:
> >> I am trying to install the sample plugin from the Nutch Wiki. I
> >> downloaded
> >> the Nutch 0.9 release version (not sure if this is what I need to
> >> download)
> >> and I added the following files in the nutch-0.9\src\ directory:
> >>
> >> 1) build.xml and plugin.xml in C:\nutch-0.9\src\plugin\recommended
> >> 2) RecommendedIndexer.java, RecommendedParser.java, QueryFilter.java in
> >>
> >>
> C:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\recommended
> >>
> >> note- all the code came from the Nutch Wiki
> >>
> >> I also changed the build.xml file in src\plugin to include the new
> >> plugin:
> >> <ant dir="recommended" target="deploy"/>
> >>
> >> What I tried:
> >> I started off by attempting to compile the plugin by using ant. When I
> >> did
> >> this, I get errors about some packages not existing. I also get errors
> >> about
> >> the file QueryFilter being misnamed and saying that it needs to be
> called
> >> RecommendedQueryFilter. I made that change and renamed it like it was
> >> suggested.
> >>
> >> I ended up going to the root directory of nutch and running ant from
> >> there.
> >> I believe this builds the whole program. A folder called "build" was
> >> created
> >> containing a bunch of stuff in there.
> >>
> >> After that was done, I went on to C:\nutch-0.9\src\plugin\recommended
> and
> >> ran ant again. This time, I did not get errors and everything worked
> >> fine.
> >>
> >> The issue:
> >> In the hadoop.log file I do not see any references made to the new
> plugin
> >> ("Recommended"). It goes on about added plugins/extensions, but there is
> >> no
> >> mention of the one I added.
> >>
> >> Did I do everything correctly? What am I missing here? I am completely
> >> lost
> >> right now and I really need help with this.
> >>
> >> Thank you and regards.
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Question-about-writing-plug-ins-tp21444458p21482275.html
> >> Sent from the Nutch - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > Ankur Garg
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Question-about-writing-plug-ins-tp21444458p21500533.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
>
--
Ankur Garg
CDAC Noida
Anusandhan Bhawan
Sec-62, Noida
Mob. 9968274797
Re: Question about writing plug ins
Posted by ahammad <ah...@gmail.com>.
Hello,
I tried that but it still didn't work. Note that I am working off of the
files that were extracted from the zip file. Here are the exact steps that I
took:
1) Ran "ant" in C:\nutch-0.9 to build nutch. A folder called "build" was
created in C:\nutch-0.9
2) Ran "ant" in C:\nutch-0.9\src\plugin\recommended to build my plugin. A
folder called "recommended" appeared in C:\nutch-0.9\build
3) Modified the build.xml file in C:\nutch-0.9\src\plugin to include <ant
dir="recommended" target="deploy"/>
4) Modified nutch-default.xml like so (note the addition of "recommended" at
the end of the string:
<value>protocol-http|urlfilter-regex|parse-(text|html|js)|index-basic|query-(basic|site|url)|summary-basic|scoring-opic|urlnormalizer-(pass|regex|basic)|recommended</value>
5) Rebuilt nutch by running ant again in C:\nutch-0.9
Then I tried crawling. In hadoop.log, there was no reference to the newly
added plugin.
I noticed that whenever you build nutch, a new nutch-0.9.job appears in the
build folder. Is that at all relevant? Do I have to replace any of the files
in C:\nutch-0.9 by the files in the build folder? Do I need to run ant war
in order to generate the .jar and .war files, and then will I need to
replace the .jar and .war files in C:\nutch-0.9 by the ones that were
created in the build folder?
I also noticed that my recommended plugin was not in C:\nutch-0.9\plugins.
Is there anything I need to add there?
I'm sure that what I am missing is a small step. I have a lot of things I
want to add in the future, and this is the only thing that is holding me
back.
Thanks a lot for the help.
Cheers
Ankur Garg-2 wrote:
>
> Hi Ahmed,
>
> To be able to use the plugin after you have sucessfully compiled
> it,
> you it to do one entry in
> conf/nutch-default.xml file in the property "plugin.includes". After doing
> this entry only, the plugin will be activated.
>
>
> hope it helps you
>
> Ankur Garg
>
> On Thu, Jan 15, 2009 at 10:45 PM, ahammad <ah...@gmail.com> wrote:
>
>>
>> I was messing around with this and I got a step further.
>>
>> Background information:
>> I am trying to install the sample plugin from the Nutch Wiki. I
>> downloaded
>> the Nutch 0.9 release version (not sure if this is what I need to
>> download)
>> and I added the following files in the nutch-0.9\src\ directory:
>>
>> 1) build.xml and plugin.xml in C:\nutch-0.9\src\plugin\recommended
>> 2) RecommendedIndexer.java, RecommendedParser.java, QueryFilter.java in
>>
>> C:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\recommended
>>
>> note- all the code came from the Nutch Wiki
>>
>> I also changed the build.xml file in src\plugin to include the new
>> plugin:
>> <ant dir="recommended" target="deploy"/>
>>
>> What I tried:
>> I started off by attempting to compile the plugin by using ant. When I
>> did
>> this, I get errors about some packages not existing. I also get errors
>> about
>> the file QueryFilter being misnamed and saying that it needs to be called
>> RecommendedQueryFilter. I made that change and renamed it like it was
>> suggested.
>>
>> I ended up going to the root directory of nutch and running ant from
>> there.
>> I believe this builds the whole program. A folder called "build" was
>> created
>> containing a bunch of stuff in there.
>>
>> After that was done, I went on to C:\nutch-0.9\src\plugin\recommended and
>> ran ant again. This time, I did not get errors and everything worked
>> fine.
>>
>> The issue:
>> In the hadoop.log file I do not see any references made to the new plugin
>> ("Recommended"). It goes on about added plugins/extensions, but there is
>> no
>> mention of the one I added.
>>
>> Did I do everything correctly? What am I missing here? I am completely
>> lost
>> right now and I really need help with this.
>>
>> Thank you and regards.
>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Question-about-writing-plug-ins-tp21444458p21482275.html
>> Sent from the Nutch - User mailing list archive at Nabble.com.
>>
>>
>
>
> --
> Ankur Garg
>
>
--
View this message in context: http://www.nabble.com/Question-about-writing-plug-ins-tp21444458p21500533.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: Question about writing plug ins
Posted by Ankur Garg <ga...@gmail.com>.
Hi Ahmed,
To be able to use the plugin after you have sucessfully compiled it,
you it to do one entry in
conf/nutch-default.xml file in the property "plugin.includes". After doing
this entry only, the plugin will be activated.
hope it helps you
Ankur Garg
On Thu, Jan 15, 2009 at 10:45 PM, ahammad <ah...@gmail.com> wrote:
>
> I was messing around with this and I got a step further.
>
> Background information:
> I am trying to install the sample plugin from the Nutch Wiki. I downloaded
> the Nutch 0.9 release version (not sure if this is what I need to download)
> and I added the following files in the nutch-0.9\src\ directory:
>
> 1) build.xml and plugin.xml in C:\nutch-0.9\src\plugin\recommended
> 2) RecommendedIndexer.java, RecommendedParser.java, QueryFilter.java in
>
> C:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\recommended
>
> note- all the code came from the Nutch Wiki
>
> I also changed the build.xml file in src\plugin to include the new plugin:
> <ant dir="recommended" target="deploy"/>
>
> What I tried:
> I started off by attempting to compile the plugin by using ant. When I did
> this, I get errors about some packages not existing. I also get errors
> about
> the file QueryFilter being misnamed and saying that it needs to be called
> RecommendedQueryFilter. I made that change and renamed it like it was
> suggested.
>
> I ended up going to the root directory of nutch and running ant from there.
> I believe this builds the whole program. A folder called "build" was
> created
> containing a bunch of stuff in there.
>
> After that was done, I went on to C:\nutch-0.9\src\plugin\recommended and
> ran ant again. This time, I did not get errors and everything worked fine.
>
> The issue:
> In the hadoop.log file I do not see any references made to the new plugin
> ("Recommended"). It goes on about added plugins/extensions, but there is no
> mention of the one I added.
>
> Did I do everything correctly? What am I missing here? I am completely lost
> right now and I really need help with this.
>
> Thank you and regards.
>
>
> --
> View this message in context:
> http://www.nabble.com/Question-about-writing-plug-ins-tp21444458p21482275.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
>
--
Ankur Garg
Re: Question about writing plug ins
Posted by ahammad <ah...@gmail.com>.
I was messing around with this and I got a step further.
Background information:
I am trying to install the sample plugin from the Nutch Wiki. I downloaded
the Nutch 0.9 release version (not sure if this is what I need to download)
and I added the following files in the nutch-0.9\src\ directory:
1) build.xml and plugin.xml in C:\nutch-0.9\src\plugin\recommended
2) RecommendedIndexer.java, RecommendedParser.java, QueryFilter.java in
C:\nutch-0.9\src\plugin\recommended\src\java\org\apache\nutch\parse\recommended
note- all the code came from the Nutch Wiki
I also changed the build.xml file in src\plugin to include the new plugin:
<ant dir="recommended" target="deploy"/>
What I tried:
I started off by attempting to compile the plugin by using ant. When I did
this, I get errors about some packages not existing. I also get errors about
the file QueryFilter being misnamed and saying that it needs to be called
RecommendedQueryFilter. I made that change and renamed it like it was
suggested.
I ended up going to the root directory of nutch and running ant from there.
I believe this builds the whole program. A folder called "build" was created
containing a bunch of stuff in there.
After that was done, I went on to C:\nutch-0.9\src\plugin\recommended and
ran ant again. This time, I did not get errors and everything worked fine.
The issue:
In the hadoop.log file I do not see any references made to the new plugin
("Recommended"). It goes on about added plugins/extensions, but there is no
mention of the one I added.
Did I do everything correctly? What am I missing here? I am completely lost
right now and I really need help with this.
Thank you and regards.
--
View this message in context: http://www.nabble.com/Question-about-writing-plug-ins-tp21444458p21482275.html
Sent from the Nutch - User mailing list archive at Nabble.com.