You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Siddharth Jonathan <jo...@gmail.com> on 2007/03/31 11:52:08 UTC
trouble adding fields to index
Hi,
I'm a newbie to nutch.
I am trying to add a new field (and make it searchable) in nutch.
I followed the wiki, http://wiki.apache.org/nutch/CreateNewFilter
but the field doesnt appear to be searchable.
Is there anything else I should be doing?
Is this the right wiki? It seemed a little outdated (for eg. the filter
method here has a different signature than the one in nutch 0.8.1)
I have also tried the wiki for writing a nutch plugin with similar results.
I think the field is added to the index but it is not searchable.
If anyone has sample code, that would be very helpful.
thanks
Jonathan
Re: trouble adding fields to index
Posted by Siddharth Jonathan <jo...@gmail.com>.
Hi Ratnesh,
It worked! Thanks!
I found that I had made a careless mistake in my plugin.xml file (in naming
the id of the plugin).
So I guess all that needs to be done is to carefully copy
the functionality of index-more and query-site and
adapt it to the fields that we need. So the wiki is correct although
v.v.brief on details.
Jonathan
On 3/31/07, Ratnesh,V2Solutions India <ra...@in.v2solutions.com>
wrote:
>
>
> I think , u better try to set the field in plugin.xml where it is declared
> as
> FIELD=DEFAULT, with your own field name u want to index like
> <parameter name=fields value="f1,f2,f3">
>
> and for compilation purpose, you can set the source path of
> build-plugin.xml
> of nutch/src/plugin. in the build.xml file.
>
> I hope that it will help u a bit.
>
> Ratnesh
>
>
>
> Siddharth Jonathan wrote:
> >
> > I see.
> > I added this to nutch-default. (I think adding it to nutch-site.xml also
> > might not be necessary. But I can give it a shot)
> > I also had to modify the necessary build files. But I guess if I messed
> up
> > here I would not even have gotten the necessary jar.
> >
> > I did not change parse-plugins.xml since I am not getting the value for
> my
> > field by parsing anything. I am currently
> > keeping it super simple and just adding a constant value for this field
> to
> > each document that I index.
> >
> > I'll wait for your reply. And keep hacking away to see if I can figure
> > something out.
> >
> > thanks!
> > Jonathan
> >
> > On 3/31/07, Ratnesh,V2Solutions India
> > <ra...@in.v2solutions.com>
> > wrote:
> >>
> >>
> >> Oh,
> >> Ya I was also struggling with the same given example in wikki, can u
> >> brief
> >> me what the problem r u getting??
> >> have u added this plugin to nutch-site.xml. and did u configure
> >> parse-plugins.xml for this .
> >>
> >> you can go through the citeseer meta data example which explains
> indexing
> >> better the problem with this is , it is in nutch-0.5.
> >>
> >> Or let me reach to the final solution then only I can explain better.
> >>
> >> Regards,
> >> Ratnesh
> >>
> >>
> >> Siddharth Jonathan wrote:
> >> >
> >> > Hi Ratnesh,
> >> > Thanks for your reply.
> >> > I am currently trying to index a "recommended" field
> in
> >> > addition to the usual fields.
> >> > To keep it simple, I am indexing a constant value for this Field as
> of
> >> > now,
> >> >
> >> > For eg. in my indexing filter plugin my filter method has this line,
> >> > doc.add(new Field("recommended","someConstantValue",Field.Store.YES,
> >> > Field.Index.UN_TOKENIZED));
> >> >
> >> > and my query filter plugin has the line super("recommended") in the
> >> right
> >> > place.
> >> >
> >> > My goal for now is to query like,
> >> > recommended:someConstantValue
> >> >
> >> > and have results returned.
> >> >
> >> > Its really surprising that not too many people appear
> to
> >> > have
> >> > this problem.
> >> > I was wondering if I was not understanding the wiki correctly.
> >> >
> >> > Jonathan
> >> >
> >> >
> >> > On 3/31/07, Ratnesh,V2Solutions India
> >> > <ra...@in.v2solutions.com>
> >> > wrote:
> >> >>
> >> >>
> >> >> HI,
> >> >> Siddharth I do have this kind of problem , which I am near to sort
> out
> >> >> may be in day or two I will be able to solve this problem.
> >> >>
> >> >> But can you please explain me which kind of application u r
> testing??
> >> >>
> >> >>
> >> >> Ratnesh
> >> >>
> >> >>
> >> >> Siddharth Jonathan wrote:
> >> >> >
> >> >> > Hi,
> >> >> > I'm a newbie to nutch.
> >> >> > I am trying to add a new field (and make it searchable) in nutch.
> >> >> > I followed the wiki, http://wiki.apache.org/nutch/CreateNewFilter
> >> >> > but the field doesnt appear to be searchable.
> >> >> >
> >> >> > Is there anything else I should be doing?
> >> >> > Is this the right wiki? It seemed a little outdated (for eg. the
> >> filter
> >> >> > method here has a different signature than the one in nutch 0.8.1)
> >> >> > I have also tried the wiki for writing a nutch plugin with similar
> >> >> > results.
> >> >> > I think the field is added to the index but it is not searchable.
> >> >> >
> >> >> > If anyone has sample code, that would be very helpful.
> >> >> >
> >> >> > thanks
> >> >> > Jonathan
> >> >> >
> >> >> >
> >> >>
> >> >> --
> >> >> View this message in context:
> >> >>
> >>
> http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9765953
> >> >> Sent from the Nutch - User mailing list archive at Nabble.com.
> >> >>
> >> >>
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9766064
> >> Sent from the Nutch - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9766175
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
>
Re: trouble adding fields to index
Posted by "Ratnesh,V2Solutions India" <ra...@in.v2solutions.com>.
I think , u better try to set the field in plugin.xml where it is declared as
FIELD=DEFAULT, with your own field name u want to index like
<parameter name=fields value="f1,f2,f3">
and for compilation purpose, you can set the source path of build-plugin.xml
of nutch/src/plugin. in the build.xml file.
I hope that it will help u a bit.
Ratnesh
Siddharth Jonathan wrote:
>
> I see.
> I added this to nutch-default. (I think adding it to nutch-site.xml also
> might not be necessary. But I can give it a shot)
> I also had to modify the necessary build files. But I guess if I messed up
> here I would not even have gotten the necessary jar.
>
> I did not change parse-plugins.xml since I am not getting the value for my
> field by parsing anything. I am currently
> keeping it super simple and just adding a constant value for this field to
> each document that I index.
>
> I'll wait for your reply. And keep hacking away to see if I can figure
> something out.
>
> thanks!
> Jonathan
>
> On 3/31/07, Ratnesh,V2Solutions India
> <ra...@in.v2solutions.com>
> wrote:
>>
>>
>> Oh,
>> Ya I was also struggling with the same given example in wikki, can u
>> brief
>> me what the problem r u getting??
>> have u added this plugin to nutch-site.xml. and did u configure
>> parse-plugins.xml for this .
>>
>> you can go through the citeseer meta data example which explains indexing
>> better the problem with this is , it is in nutch-0.5.
>>
>> Or let me reach to the final solution then only I can explain better.
>>
>> Regards,
>> Ratnesh
>>
>>
>> Siddharth Jonathan wrote:
>> >
>> > Hi Ratnesh,
>> > Thanks for your reply.
>> > I am currently trying to index a "recommended" field in
>> > addition to the usual fields.
>> > To keep it simple, I am indexing a constant value for this Field as of
>> > now,
>> >
>> > For eg. in my indexing filter plugin my filter method has this line,
>> > doc.add(new Field("recommended","someConstantValue",Field.Store.YES,
>> > Field.Index.UN_TOKENIZED));
>> >
>> > and my query filter plugin has the line super("recommended") in the
>> right
>> > place.
>> >
>> > My goal for now is to query like,
>> > recommended:someConstantValue
>> >
>> > and have results returned.
>> >
>> > Its really surprising that not too many people appear to
>> > have
>> > this problem.
>> > I was wondering if I was not understanding the wiki correctly.
>> >
>> > Jonathan
>> >
>> >
>> > On 3/31/07, Ratnesh,V2Solutions India
>> > <ra...@in.v2solutions.com>
>> > wrote:
>> >>
>> >>
>> >> HI,
>> >> Siddharth I do have this kind of problem , which I am near to sort out
>> >> may be in day or two I will be able to solve this problem.
>> >>
>> >> But can you please explain me which kind of application u r testing??
>> >>
>> >>
>> >> Ratnesh
>> >>
>> >>
>> >> Siddharth Jonathan wrote:
>> >> >
>> >> > Hi,
>> >> > I'm a newbie to nutch.
>> >> > I am trying to add a new field (and make it searchable) in nutch.
>> >> > I followed the wiki, http://wiki.apache.org/nutch/CreateNewFilter
>> >> > but the field doesnt appear to be searchable.
>> >> >
>> >> > Is there anything else I should be doing?
>> >> > Is this the right wiki? It seemed a little outdated (for eg. the
>> filter
>> >> > method here has a different signature than the one in nutch 0.8.1)
>> >> > I have also tried the wiki for writing a nutch plugin with similar
>> >> > results.
>> >> > I think the field is added to the index but it is not searchable.
>> >> >
>> >> > If anyone has sample code, that would be very helpful.
>> >> >
>> >> > thanks
>> >> > Jonathan
>> >> >
>> >> >
>> >>
>> >> --
>> >> View this message in context:
>> >>
>> http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9765953
>> >> Sent from the Nutch - User mailing list archive at Nabble.com.
>> >>
>> >>
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9766064
>> Sent from the Nutch - User mailing list archive at Nabble.com.
>>
>>
>
>
--
View this message in context: http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9766175
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: trouble adding fields to index
Posted by Siddharth Jonathan <jo...@gmail.com>.
I see.
I added this to nutch-default. (I think adding it to nutch-site.xml also
might not be necessary. But I can give it a shot)
I also had to modify the necessary build files. But I guess if I messed up
here I would not even have gotten the necessary jar.
I did not change parse-plugins.xml since I am not getting the value for my
field by parsing anything. I am currently
keeping it super simple and just adding a constant value for this field to
each document that I index.
I'll wait for your reply. And keep hacking away to see if I can figure
something out.
thanks!
Jonathan
On 3/31/07, Ratnesh,V2Solutions India <ra...@in.v2solutions.com>
wrote:
>
>
> Oh,
> Ya I was also struggling with the same given example in wikki, can u brief
> me what the problem r u getting??
> have u added this plugin to nutch-site.xml. and did u configure
> parse-plugins.xml for this .
>
> you can go through the citeseer meta data example which explains indexing
> better the problem with this is , it is in nutch-0.5.
>
> Or let me reach to the final solution then only I can explain better.
>
> Regards,
> Ratnesh
>
>
> Siddharth Jonathan wrote:
> >
> > Hi Ratnesh,
> > Thanks for your reply.
> > I am currently trying to index a "recommended" field in
> > addition to the usual fields.
> > To keep it simple, I am indexing a constant value for this Field as of
> > now,
> >
> > For eg. in my indexing filter plugin my filter method has this line,
> > doc.add(new Field("recommended","someConstantValue",Field.Store.YES,
> > Field.Index.UN_TOKENIZED));
> >
> > and my query filter plugin has the line super("recommended") in the
> right
> > place.
> >
> > My goal for now is to query like,
> > recommended:someConstantValue
> >
> > and have results returned.
> >
> > Its really surprising that not too many people appear to
> > have
> > this problem.
> > I was wondering if I was not understanding the wiki correctly.
> >
> > Jonathan
> >
> >
> > On 3/31/07, Ratnesh,V2Solutions India
> > <ra...@in.v2solutions.com>
> > wrote:
> >>
> >>
> >> HI,
> >> Siddharth I do have this kind of problem , which I am near to sort out
> >> may be in day or two I will be able to solve this problem.
> >>
> >> But can you please explain me which kind of application u r testing??
> >>
> >>
> >> Ratnesh
> >>
> >>
> >> Siddharth Jonathan wrote:
> >> >
> >> > Hi,
> >> > I'm a newbie to nutch.
> >> > I am trying to add a new field (and make it searchable) in nutch.
> >> > I followed the wiki, http://wiki.apache.org/nutch/CreateNewFilter
> >> > but the field doesnt appear to be searchable.
> >> >
> >> > Is there anything else I should be doing?
> >> > Is this the right wiki? It seemed a little outdated (for eg. the
> filter
> >> > method here has a different signature than the one in nutch 0.8.1)
> >> > I have also tried the wiki for writing a nutch plugin with similar
> >> > results.
> >> > I think the field is added to the index but it is not searchable.
> >> >
> >> > If anyone has sample code, that would be very helpful.
> >> >
> >> > thanks
> >> > Jonathan
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9765953
> >> Sent from the Nutch - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9766064
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
>
Re: trouble adding fields to index
Posted by "Ratnesh,V2Solutions India" <ra...@in.v2solutions.com>.
Oh,
Ya I was also struggling with the same given example in wikki, can u brief
me what the problem r u getting??
have u added this plugin to nutch-site.xml. and did u configure
parse-plugins.xml for this .
you can go through the citeseer meta data example which explains indexing
better the problem with this is , it is in nutch-0.5.
Or let me reach to the final solution then only I can explain better.
Regards,
Ratnesh
Siddharth Jonathan wrote:
>
> Hi Ratnesh,
> Thanks for your reply.
> I am currently trying to index a "recommended" field in
> addition to the usual fields.
> To keep it simple, I am indexing a constant value for this Field as of
> now,
>
> For eg. in my indexing filter plugin my filter method has this line,
> doc.add(new Field("recommended","someConstantValue",Field.Store.YES,
> Field.Index.UN_TOKENIZED));
>
> and my query filter plugin has the line super("recommended") in the right
> place.
>
> My goal for now is to query like,
> recommended:someConstantValue
>
> and have results returned.
>
> Its really surprising that not too many people appear to
> have
> this problem.
> I was wondering if I was not understanding the wiki correctly.
>
> Jonathan
>
>
> On 3/31/07, Ratnesh,V2Solutions India
> <ra...@in.v2solutions.com>
> wrote:
>>
>>
>> HI,
>> Siddharth I do have this kind of problem , which I am near to sort out
>> may be in day or two I will be able to solve this problem.
>>
>> But can you please explain me which kind of application u r testing??
>>
>>
>> Ratnesh
>>
>>
>> Siddharth Jonathan wrote:
>> >
>> > Hi,
>> > I'm a newbie to nutch.
>> > I am trying to add a new field (and make it searchable) in nutch.
>> > I followed the wiki, http://wiki.apache.org/nutch/CreateNewFilter
>> > but the field doesnt appear to be searchable.
>> >
>> > Is there anything else I should be doing?
>> > Is this the right wiki? It seemed a little outdated (for eg. the filter
>> > method here has a different signature than the one in nutch 0.8.1)
>> > I have also tried the wiki for writing a nutch plugin with similar
>> > results.
>> > I think the field is added to the index but it is not searchable.
>> >
>> > If anyone has sample code, that would be very helpful.
>> >
>> > thanks
>> > Jonathan
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9765953
>> Sent from the Nutch - User mailing list archive at Nabble.com.
>>
>>
>
>
--
View this message in context: http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9766064
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: trouble adding fields to index
Posted by Siddharth Jonathan <jo...@gmail.com>.
Hi Ratnesh,
Thanks for your reply.
I am currently trying to index a "recommended" field in
addition to the usual fields.
To keep it simple, I am indexing a constant value for this Field as of now,
For eg. in my indexing filter plugin my filter method has this line,
doc.add(new Field("recommended","someConstantValue",Field.Store.YES,
Field.Index.UN_TOKENIZED));
and my query filter plugin has the line super("recommended") in the right
place.
My goal for now is to query like,
recommended:someConstantValue
and have results returned.
Its really surprising that not too many people appear to have
this problem.
I was wondering if I was not understanding the wiki correctly.
Jonathan
On 3/31/07, Ratnesh,V2Solutions India <ra...@in.v2solutions.com>
wrote:
>
>
> HI,
> Siddharth I do have this kind of problem , which I am near to sort out
> may be in day or two I will be able to solve this problem.
>
> But can you please explain me which kind of application u r testing??
>
>
> Ratnesh
>
>
> Siddharth Jonathan wrote:
> >
> > Hi,
> > I'm a newbie to nutch.
> > I am trying to add a new field (and make it searchable) in nutch.
> > I followed the wiki, http://wiki.apache.org/nutch/CreateNewFilter
> > but the field doesnt appear to be searchable.
> >
> > Is there anything else I should be doing?
> > Is this the right wiki? It seemed a little outdated (for eg. the filter
> > method here has a different signature than the one in nutch 0.8.1)
> > I have also tried the wiki for writing a nutch plugin with similar
> > results.
> > I think the field is added to the index but it is not searchable.
> >
> > If anyone has sample code, that would be very helpful.
> >
> > thanks
> > Jonathan
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9765953
> Sent from the Nutch - User mailing list archive at Nabble.com.
>
>
Re: trouble adding fields to index
Posted by "Ratnesh,V2Solutions India" <ra...@in.v2solutions.com>.
HI,
Siddharth I do have this kind of problem , which I am near to sort out
may be in day or two I will be able to solve this problem.
But can you please explain me which kind of application u r testing??
Ratnesh
Siddharth Jonathan wrote:
>
> Hi,
> I'm a newbie to nutch.
> I am trying to add a new field (and make it searchable) in nutch.
> I followed the wiki, http://wiki.apache.org/nutch/CreateNewFilter
> but the field doesnt appear to be searchable.
>
> Is there anything else I should be doing?
> Is this the right wiki? It seemed a little outdated (for eg. the filter
> method here has a different signature than the one in nutch 0.8.1)
> I have also tried the wiki for writing a nutch plugin with similar
> results.
> I think the field is added to the index but it is not searchable.
>
> If anyone has sample code, that would be very helpful.
>
> thanks
> Jonathan
>
>
--
View this message in context: http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9765953
Sent from the Nutch - User mailing list archive at Nabble.com.
Re: trouble adding fields to index
Posted by "Ratnesh,V2Solutions India" <ra...@in.v2solutions.com>.
gr8,
i will update u sometime later about some more details.
Thnx
Siddharth Jonathan wrote:
>
> Hi,
> I'm a newbie to nutch.
> I am trying to add a new field (and make it searchable) in nutch.
> I followed the wiki, http://wiki.apache.org/nutch/CreateNewFilter
> but the field doesnt appear to be searchable.
>
> Is there anything else I should be doing?
> Is this the right wiki? It seemed a little outdated (for eg. the filter
> method here has a different signature than the one in nutch 0.8.1)
> I have also tried the wiki for writing a nutch plugin with similar
> results.
> I think the field is added to the index but it is not searchable.
>
> If anyone has sample code, that would be very helpful.
>
> thanks
> Jonathan
>
>
--
View this message in context: http://www.nabble.com/trouble-adding-fields-to-index-tf3496357.html#a9766509
Sent from the Nutch - User mailing list archive at Nabble.com.