Spelling

[Aled Jones <Al...@comtec-europe.co.uk>]

Hi

Does nutch do spelling suggestions?
If so, is it a plugin I should install or is there something I should
configure?

If not are there any plans to implement it?

Kind Regards
Aled

************************************************************************
This e-mail and any attachments are strictly confidential and intended solely for the addressee. They may contain information which is covered by legal, professional or other privilege. If you are not the intended addressee, you must not copy the e-mail or the attachments, or use them for any purpose or disclose their contents to any other person. To do so may be unlawful. If you have received this transmission in error, please notify us as soon as possible and delete the message and attachments from all places in your computer where they are stored. 

Although we have scanned this e-mail and any attachments for viruses, it is your responsibility to ensure that they are actually virus free.
 

Re: Spelling

[Byron Miller <by...@yahoo.com>]

Yes, i would love to see it committed so it is
maintained through the branches..

--- Jérôme Charron <je...@gmail.com> wrote:

> > I have it loaded on mozdex.com <http://mozdex.com>
> and it works fairly
> > well.
> 
> Thanks for your feedback Byron.
> So it is a good candidate for a commit... I note it.
> 
> Regards
> 
> Jérôme
> 
> --
> http://motrech.free.fr/
> http://www.frutch.org/
> 

Slow updatedb?

[Howie Wang <ho...@hotmail.com>]

Hi,

Recently my updatedb step has been taking a long time to
complete. There are about 7M links and 1M pages, and they
take up about 4.5GB of space. The update step seems to be
taking about 2 hours to complete.

It looked like it was limiting its memory usage to about 100mb
so I increased the io.sort.mb to 512mb.  After that updatedb
has used up to 250mb of memory, but the overall process
still takes about 2 hours.

Any tips on increasing performance? Is this about the right level
of performance for this DB size? Do I just need a faster disk?

I'm on Nutch 0.7 on Windows XP with 768MB ram, by the way.

Thanks!
Howie

Re: Spelling

[Jérôme Charron <je...@gmail.com>]

> I have it loaded on mozdex.com <http://mozdex.com> and it works fairly
> well.

Thanks for your feedback Byron.
So it is a good candidate for a commit... I note it.

Regards

Jérôme

--
http://motrech.free.fr/
http://www.frutch.org/

Re: Spelling

[Byron Miller <by...@yahoo.com>]

I have it loaded on mozdex.com and it works fairly
well.

Only thing i noticed is it seems to look for longer
versions of a matching phrase vs immediate common
mistakes.

For example "diat pill" (which is a very common query)
comes up as diatribe pill  instead of "diet pill" :)

BUT as my index grows perhaps these will trickle out.

-byron

--- Jérôme Charron <je...@gmail.com> wrote:

> > Does nutch do spelling suggestions?
> >
> If so, is it a plugin I should install or is there
> something I should
> > configure?
> 
> Take a look at this nutch issue :
> http://issues.apache.org/jira/browse/NUTCH-48
> I never tested this patch.
> If you try it, please give us feed back.
> 
> Regards,
> 
> Jérôme
> 
> --
> http://motrech.free.fr/
> http://www.frutch.org/
> 

Re: Spelling

[Jérôme Charron <je...@gmail.com>]

> Does nutch do spelling suggestions?
>
If so, is it a plugin I should install or is there something I should
> configure?

Take a look at this nutch issue :
http://issues.apache.org/jira/browse/NUTCH-48
I never tested this patch.
If you try it, please give us feed back.

Regards,

Jérôme

--
http://motrech.free.fr/
http://www.frutch.org/