You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by ramires <uy...@beriltech.com> on 2010/04/01 14:57:48 UTC
description and keywords
Is there any plugin for fetcher and indexer to get description and keywords
in web pages meta tags?
--
View this message in context: http://n3.nabble.com/description-and-keywords-tp690681p690681.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: description and keywords
Posted by Julien Nioche <li...@gmail.com>.
See https://issues.apache.org/jira/browse/NUTCH-783 for a utility which
allows you to check the fields generated by the IndexingFilters.
Using Luke is also a good way of making sure the fields/values in a Lucene
index
Are you using the SVN version? If so check that the indexing filter is
listed in indexingfilter.order
You can also try :* ./nutch org.apache.nutch.searcher.Query* to see how the
user query is converted into a Lucene one.
J.
--
DigitalPebble Ltd
http://www.digitalpebble.com
On 6 April 2010 08:19, ramires <uy...@beriltech.com> wrote:
>
> Hi
>
> I put these lines to nutch-site.xml then search a keywords but there is no
> result page. I think Indexer not index metatags.
>
> <property>
> <name>metatags.names</name>
> <value>description;keywords</value>
> </property>
>
> <property>
> <name>query.basic.description.boost</name>
> <value>5.0</value>
> </property>
>
> <property>
> <name>query.basic.keywords.boost</name>
> <value>5.0</value>
> </property>
>
> --
> View this message in context:
> http://n3.nabble.com/description-and-keywords-tp690681p699659.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
Re: description and keywords
Posted by ramires <uy...@beriltech.com>.
Hi
I put these lines to nutch-site.xml then search a keywords but there is no
result page. I think Indexer not index metatags.
<property>
<name>metatags.names</name>
<value>description;keywords</value>
</property>
<property>
<name>query.basic.description.boost</name>
<value>5.0</value>
</property>
<property>
<name>query.basic.keywords.boost</name>
<value>5.0</value>
</property>
--
View this message in context: http://n3.nabble.com/description-and-keywords-tp690681p699659.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: description and keywords
Posted by Julien Nioche <li...@gmail.com>.
Hi
The query-basic plugin is used to include these fields in the search e.g. in
nutch-site.xml
{code:xml}
<property>
<name>query.basic.description.boost</name>
<value>2.0</value>
</property>
<property>
<name>query.basic.keywords.boost</name>
<value>2.0</value>
</property>
{code}
The query filter included in the batch was a botched attempt which I forgot
to remove.
Let me know if it works
J.
--
DigitalPebble Ltd
http://www.digitalpebble.com
On 5 April 2010 13:44, ramires <uy...@beriltech.com> wrote:
>
> hi
> Metatag parser work great. When I dumped with readseg I saw
> metatag.keywords field and data.But I can't solve query part. I put
> nutch-site.xml this and deploy it. When I querying a keyword there is no
> search result. Is there anything wrong with this situation ?
>
> <property>
> <name>metatags.names</name>
> <value>description;keywords</value>
> </property>
>
>
> --
> View this message in context:
> http://n3.nabble.com/description-and-keywords-tp690681p697621.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
Re: description and keywords
Posted by ramires <uy...@beriltech.com>.
hi
Metatag parser work great. When I dumped with readseg I saw
metatag.keywords field and data.But I can't solve query part. I put
nutch-site.xml this and deploy it. When I querying a keyword there is no
search result. Is there anything wrong with this situation ?
<property>
<name>metatags.names</name>
<value>description;keywords</value>
</property>
--
View this message in context: http://n3.nabble.com/description-and-keywords-tp690681p697621.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: description and keywords
Posted by Julien Nioche <li...@gmail.com>.
FYI : I have fixed the issue with parse-tika. The plugin should work in the
same way regardless of the implementation used for parsing HTML
J.
On 2 April 2010 15:21, Julien Nioche <li...@gmail.com> wrote:
> It's available from https://issues.apache.org/jira/browse/NUTCH-809
>
> Please read the notes carefully as there is an issue with parse-tika (which
> hopefully will be solved soon).
>
> HTH
>
>
> Julien
> --
> DigitalPebble Ltd
> http://www.digitalpebble.com
>
> On 2 April 2010 14:43, ramires <uy...@beriltech.com> wrote:
>
>>
>> +1
>>
>> I'm very appriciated if you put it to JIRA.
>> --
>> View this message in context:
>> http://n3.nabble.com/description-and-keywords-tp690681p692880.html
>> Sent from the Nutch - User mailing list archive at Nabble.com.
>>
>
>
>
>
--
DigitalPebble Ltd
http://www.digitalpebble.com
Re: description and keywords
Posted by Julien Nioche <li...@gmail.com>.
It's available from https://issues.apache.org/jira/browse/NUTCH-809
Please read the notes carefully as there is an issue with parse-tika (which
hopefully will be solved soon).
HTH
Julien
--
DigitalPebble Ltd
http://www.digitalpebble.com
On 2 April 2010 14:43, ramires <uy...@beriltech.com> wrote:
>
> +1
>
> I'm very appriciated if you put it to JIRA.
> --
> View this message in context:
> http://n3.nabble.com/description-and-keywords-tp690681p692880.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
Re: description and keywords
Posted by ramires <uy...@beriltech.com>.
+1
I'm very appriciated if you put it to JIRA.
--
View this message in context: http://n3.nabble.com/description-and-keywords-tp690681p692880.html
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: description and keywords
Posted by MilleBii <mi...@gmail.com>.
I would use it if it comes 'for free'
2010/4/1, Julien Nioche <li...@gmail.com>:
> Description and keywords are not provided by 'index-more' - you'll need to
> write a custom plugin for this. I have written one which does exactly that
> and will probably put it in JIRA if there is interest for it. It is quite
> straightforward to do and it would be a nice way of learning about Nutch and
> the plugin system.
>
> Julien
>
>
> --
> DigitalPebble Ltd
> http://www.digitalpebble.com
>
> On 1 April 2010 14:49, <to...@gmx.de> wrote:
>
>> hi,
>>
>> may be the "index-more" and "query-more" plugin as part of values of
>> property "plugin.includes" in nutch-default.xml or nutch-site.xml? (see
>> http://wiki.apache.org/nutch/PluginCentral)
>>
>> regards marcel :)
>>
>> Am 01.04.2010 um 14:57 schrieb ramires:
>>
>> >
>> > Is there any plugin for fetcher and indexer to get description and
>> keywords
>> > in web pages meta tags?
>> > --
>> > View this message in context:
>> http://n3.nabble.com/description-and-keywords-tp690681p690681.html
>> > Sent from the Nutch - User mailing list archive at Nabble.com.
>>
>>
>
--
-MilleBii-
Re: description and keywords
Posted by Julien Nioche <li...@gmail.com>.
Description and keywords are not provided by 'index-more' - you'll need to
write a custom plugin for this. I have written one which does exactly that
and will probably put it in JIRA if there is interest for it. It is quite
straightforward to do and it would be a nice way of learning about Nutch and
the plugin system.
Julien
--
DigitalPebble Ltd
http://www.digitalpebble.com
On 1 April 2010 14:49, <to...@gmx.de> wrote:
> hi,
>
> may be the "index-more" and "query-more" plugin as part of values of
> property "plugin.includes" in nutch-default.xml or nutch-site.xml? (see
> http://wiki.apache.org/nutch/PluginCentral)
>
> regards marcel :)
>
> Am 01.04.2010 um 14:57 schrieb ramires:
>
> >
> > Is there any plugin for fetcher and indexer to get description and
> keywords
> > in web pages meta tags?
> > --
> > View this message in context:
> http://n3.nabble.com/description-and-keywords-tp690681p690681.html
> > Sent from the Nutch - User mailing list archive at Nabble.com.
>
>
Re: description and keywords
Posted by to...@gmx.de.
hi,
may be the "index-more" and "query-more" plugin as part of values of property "plugin.includes" in nutch-default.xml or nutch-site.xml? (see http://wiki.apache.org/nutch/PluginCentral)
regards marcel :)
Am 01.04.2010 um 14:57 schrieb ramires:
>
> Is there any plugin for fetcher and indexer to get description and keywords
> in web pages meta tags?
> --
> View this message in context: http://n3.nabble.com/description-and-keywords-tp690681p690681.html
> Sent from the Nutch - User mailing list archive at Nabble.com.