You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Enrico Triolo <en...@gmail.com> on 2006/01/31 12:05:32 UTC
Problem with plugins
Hi all, I developed a couple of plugins to add and search a custom field.
The indexing plugin works fine (with Luke I can see the field is added to
the index). But when it comes to search no documents are returned...
Here the plugins code:
----------------- Indexing plugin ---------------------------
package org.apache.nutch.indexer.quiew;
import ...;
public class QuiewIndexingFilter implements IndexingFilter {
public Document filter(Document doc, Parse parse, FetcherOutput fo)
throws IndexingException {
String url = fo.getUrl().toString();
doc.add(new Field("quiewId", MD5Hash.digest(url).toString(), true, true,
false));
return doc;
}
}
---------------- Query plugin ----------------------
package org.apache.nutch.searcher.quiew;
import org.apache.nutch.searcher.FieldQueryFilter;
public class QuiewQueryFilter extends FieldQueryFilter {
public QuiewQueryFilter() {
super("quiewId");
}
}
And here's how I try to perform the query:
String query = "quiewId:a3d32ce0cae0da47677f30cc6182d421";
NutchBean bean = new NutchBean(new File(indexPath));
Query q = Query.parse(query);
Hits hits = bean.search(q, 1000);
long tot = hits.getTotal();
System.out.println("Query returned " + tot + " hits.");
for(int i = 0; i < tot; i++) {
Hit hit = hits.getHit(i);
HitDetails details = bean.getDetails(hit);
System.out.println("Resource title: " + details.getValue("title"));
System.out.println("Resource url: " + details.getValue("url"));
}
If I perform the same query with Luke, I get correct results, but only if I
use the KeywordAnalyzer...
What am I doing wrong?
Thanks,
Enrico
Re: Problem with plugins
Posted by Enrico Triolo <en...@gmail.com>.
Quite interesting (I think), if I use lucene api to perform the query I get
exactly what I'm expecting...
Searcher searcher = new IndexSearcher(indexDir);
Analyzer analyzer = new NutchDocumentAnalyzer();
Query query = QueryParser.parse("a3d32ce0cae0da47677f30cc6182d421",
"quiewId", analyzer);
System.out.println(query);
System.out.println("Searching for: " + query.toString("quiewId"));
Hits hits = searcher.search(query);
System.out.println(hits.length() + " total matching documents");
Do you have any suggestion?
Thanks,
Enrico
On 1/31/06, Enrico Triolo <en...@gmail.com> wrote:
>
> No... :-(
> What would it be supposed to do?
>
> Enrico
>
> On 1/31/06, Stefan Groschupf <sg...@media-style.com> wrote:
> >
> > What happens if you change your query string to:
> > String query = "quiewId:a3d32ce0cae0da47677f30cc6182d421 HTTP" ;
> > (adding space http)??
> > Does this return any hits?
> >
> > Stefan
> >
> > Am 31.01.2006 um 12:05 schrieb Enrico Triolo:
> >
> > > Hi all, I developed a couple of plugins to add and search a custom
> > > field.
> > > The indexing plugin works fine (with Luke I can see the field is
> > > added to
> > > the index). But when it comes to search no documents are returned...
> > >
> > > Here the plugins code:
> > >
> > > ----------------- Indexing plugin ---------------------------
> > >
> > > package org.apache.nutch.indexer.quiew;
> > >
> > > import ...;
> > >
> > > public class QuiewIndexingFilter implements IndexingFilter {
> > >
> > > public Document filter(Document doc, Parse parse, FetcherOutput fo)
> > > throws IndexingException {
> > >
> > > String url = fo.getUrl().toString();
> > > doc.add(new Field("quiewId", MD5Hash.digest(url).toString(),
> > > true, true,
> > > false));
> > >
> > > return doc;
> > > }
> > >
> > > }
> > >
> > > ---------------- Query plugin ----------------------
> > > package org.apache.nutch.searcher.quiew;
> > >
> > > import org.apache.nutch.searcher.FieldQueryFilter ;
> > >
> > > public class QuiewQueryFilter extends FieldQueryFilter {
> > >
> > > public QuiewQueryFilter() {
> > > super("quiewId");
> > > }
> > > }
> > >
> > > And here's how I try to perform the query:
> > >
> > > String query = "quiewId:a3d32ce0cae0da47677f30cc6182d421";
> > > NutchBean bean = new NutchBean(new File(indexPath));
> > > Query q = Query.parse(query);
> > > Hits hits = bean.search(q, 1000);
> > > long tot = hits.getTotal();
> > >
> > > System.out.println("Query returned " + tot + " hits.");
> > >
> > > for(int i = 0; i < tot; i++) {
> > >
> > > Hit hit = hits.getHit(i);
> > > HitDetails details = bean.getDetails(hit);
> > >
> > > System.out.println("Resource title: " + details.getValue("title"));
> > > System.out.println("Resource url: " + details.getValue("url"));
> > > }
> > >
> > > If I perform the same query with Luke, I get correct results, but
> > > only if I
> > > use the KeywordAnalyzer...
> > >
> > > What am I doing wrong?
> > >
> > > Thanks,
> > > Enrico
> >
> >
>
Re: Problem with plugins
Posted by Enrico Triolo <en...@gmail.com>.
No... :-(
What would it be supposed to do?
Enrico
On 1/31/06, Stefan Groschupf <sg...@media-style.com> wrote:
>
> What happens if you change your query string to:
> String query = "quiewId:a3d32ce0cae0da47677f30cc6182d421 HTTP" ;
> (adding space http)??
> Does this return any hits?
>
> Stefan
>
> Am 31.01.2006 um 12:05 schrieb Enrico Triolo:
>
> > Hi all, I developed a couple of plugins to add and search a custom
> > field.
> > The indexing plugin works fine (with Luke I can see the field is
> > added to
> > the index). But when it comes to search no documents are returned...
> >
> > Here the plugins code:
> >
> > ----------------- Indexing plugin ---------------------------
> >
> > package org.apache.nutch.indexer.quiew;
> >
> > import ...;
> >
> > public class QuiewIndexingFilter implements IndexingFilter {
> >
> > public Document filter(Document doc, Parse parse, FetcherOutput fo)
> > throws IndexingException {
> >
> > String url = fo.getUrl().toString();
> > doc.add(new Field("quiewId", MD5Hash.digest(url).toString(),
> > true, true,
> > false));
> >
> > return doc;
> > }
> >
> > }
> >
> > ---------------- Query plugin ----------------------
> > package org.apache.nutch.searcher.quiew;
> >
> > import org.apache.nutch.searcher.FieldQueryFilter;
> >
> > public class QuiewQueryFilter extends FieldQueryFilter {
> >
> > public QuiewQueryFilter() {
> > super("quiewId");
> > }
> > }
> >
> > And here's how I try to perform the query:
> >
> > String query = "quiewId:a3d32ce0cae0da47677f30cc6182d421";
> > NutchBean bean = new NutchBean(new File(indexPath));
> > Query q = Query.parse(query);
> > Hits hits = bean.search(q, 1000);
> > long tot = hits.getTotal();
> >
> > System.out.println("Query returned " + tot + " hits.");
> >
> > for(int i = 0; i < tot; i++) {
> >
> > Hit hit = hits.getHit(i);
> > HitDetails details = bean.getDetails(hit);
> >
> > System.out.println("Resource title: " + details.getValue("title"));
> > System.out.println("Resource url: " + details.getValue("url"));
> > }
> >
> > If I perform the same query with Luke, I get correct results, but
> > only if I
> > use the KeywordAnalyzer...
> >
> > What am I doing wrong?
> >
> > Thanks,
> > Enrico
>
>
Re: Problem with plugins
Posted by Stefan Groschupf <sg...@media-style.com>.
What happens if you change your query string to:
String query = "quiewId:a3d32ce0cae0da47677f30cc6182d421 HTTP" ;
(adding space http)??
Does this return any hits?
Stefan
Am 31.01.2006 um 12:05 schrieb Enrico Triolo:
> Hi all, I developed a couple of plugins to add and search a custom
> field.
> The indexing plugin works fine (with Luke I can see the field is
> added to
> the index). But when it comes to search no documents are returned...
>
> Here the plugins code:
>
> ----------------- Indexing plugin ---------------------------
>
> package org.apache.nutch.indexer.quiew;
>
> import ...;
>
> public class QuiewIndexingFilter implements IndexingFilter {
>
> public Document filter(Document doc, Parse parse, FetcherOutput fo)
> throws IndexingException {
>
> String url = fo.getUrl().toString();
> doc.add(new Field("quiewId", MD5Hash.digest(url).toString(),
> true, true,
> false));
>
> return doc;
> }
>
> }
>
> ---------------- Query plugin ----------------------
> package org.apache.nutch.searcher.quiew;
>
> import org.apache.nutch.searcher.FieldQueryFilter;
>
> public class QuiewQueryFilter extends FieldQueryFilter {
>
> public QuiewQueryFilter() {
> super("quiewId");
> }
> }
>
> And here's how I try to perform the query:
>
> String query = "quiewId:a3d32ce0cae0da47677f30cc6182d421";
> NutchBean bean = new NutchBean(new File(indexPath));
> Query q = Query.parse(query);
> Hits hits = bean.search(q, 1000);
> long tot = hits.getTotal();
>
> System.out.println("Query returned " + tot + " hits.");
>
> for(int i = 0; i < tot; i++) {
>
> Hit hit = hits.getHit(i);
> HitDetails details = bean.getDetails(hit);
>
> System.out.println("Resource title: " + details.getValue("title"));
> System.out.println("Resource url: " + details.getValue("url"));
> }
>
> If I perform the same query with Luke, I get correct results, but
> only if I
> use the KeywordAnalyzer...
>
> What am I doing wrong?
>
> Thanks,
> Enrico