Question about adding tags or attributes to indexed info

[Gene Campbell <ge...@gmail.com>]

Hello

I'm new to Nutch, and this mailing list.  From what I've seen in my
short 3 hours of work with Nutch, I'm very impressed.  Excellent
work!!   I was able to get an index built and serve it up with Tomcat
in short order.  So, now that I've had that success, I'm on to
creating my site with Nutch.

I have a number of boolean type tags that I want to attach to each url
(from a domain) that I crawl and add to the index.   Of course the
various urls will get some of the tags and others will not.  This
breakdown of domain to tags is stored in a flat file at the moment.
To the user, the tags will be shown as checkboxes in a search UI, such
that, if you put a check in the box, you will get all urls that have
that tag associated, or set to true.

How do I do this Nutch?  I'm guessing I could use a plugin, but I'm
not sure which one.  Or, do I need to create my own plugin?  Are there
example anywhere of implementing something like this?

thanks!!

Re: Fwd: Question about adding tags or attributes to indexed info

[Sathyam Y <sa...@yahoo.com>]

The following documentation could be very helpful: http://wiki.apache.org/nutch/WritingPluginExample-0.9
   
  AFAIK, there is a limitation in nutch source FieldQueryFilter.java  where you have to create a separate plugin for each field to be indexed. One way I was able to workaround this is by enhancing FieldQueryFilter.java. It is fairly straightforward to modify FieldQueryFilter.java to support a list of fields, although that would require change to nutch base code.
   
  - Sathyam

Sathyam Y <sa...@yahoo.com> wrote:
  The easiest way to achieve this is by adding a custom query plugin. I added a plugin to parse META tags on HTML pages and add them to the index, which can be used during queries. 

- Enhance HtmlParseFilter to parse meta tags and store it in the Document
- Enhance IndexingFilter to add the tags to the index

- Sathyam

Gene Campbell wrote:
Hello

I'm new to Nutch.

I want each URL in the index to be optionally tagged with extra
information that a searcher with filter on.

How do I do this Nutch/Lucene? Some indexing plugin? Which one? Do
I need to create my own plugin? Are there examples anywhere?



---------------------------------
Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now.

       
---------------------------------
Be a better friend, newshound, and know-it-all with Yahoo! Mobile.  Try it now.

Re: Fwd: Question about adding tags or attributes to indexed info

[Sathyam Y <sa...@yahoo.com>]

The easiest way to achieve this is by adding a custom query plugin. I added a plugin to parse META tags on HTML pages and add them to the index, which can be used during queries. 
   
  - Enhance HtmlParseFilter to parse meta tags and store it in the Document
  - Enhance IndexingFilter to add the tags to the index
   
  - Sathyam
  
Gene Campbell <ge...@gmail.com> wrote:
  Hello

I'm new to Nutch.

I want each URL in the index to be optionally tagged with extra
information that a searcher with filter on.

How do I do this Nutch/Lucene? Some indexing plugin? Which one? Do
I need to create my own plugin? Are there examples anywhere?


       
---------------------------------
Be a better friend, newshound, and know-it-all with Yahoo! Mobile.  Try it now.

Fwd: Question about adding tags or attributes to indexed info

[Gene Campbell <ge...@gmail.com>]

Hello

 I'm new to Nutch.

 I want each URL in the index to be optionally tagged with extra
information that a searcher with filter on.

 How do I do this Nutch/Lucene?  Some indexing plugin?  Which one?  Do
I need to create my own plugin?  Are there examples anywhere?