RE: Slow Highlighting -> CopyField maxSize property

[Nicolas DESSAIGNE <ni...@arisem.com>]

Koji,

The patch is now available at https://issues.apache.org/jira/browse/SOLR-538

Tell me if it fits your needs.
Nicolas

-----Message d'origine-----
De : Koji Sekiguchi [mailto:koji@r.email.ne.jp]
Envoyé : vendredi 21 mars 2008 16:50
À : solr-user@lucene.apache.org
Objet : Re: Slow Highlighting -> CopyField maxSize property

Hello Nocolas,

This has been in the back of my mind for a time.
Can you make a patch for it? I'd like to use it.

Thank you,

Koji

nicolas.dessaigne@arisem.com wrote:
> Hi all,
>
>
>
> I would like to propose a new property on copy fields that limit the number
> of characters that are copied.
>
>
>
> The use case is the following: Among other documents, we index very big
> documents (several Mo of text) and want to be able to use highlighting.
> However, as soon as one or more big documents are included in the matches,
> the response time is awful. The maxAnalyzedChars is not enough as the full
> document is uploaded in memory before to do any processing and that alone
> can be very long.
>
>
>
> For this kind of situations, we propose to use a dedicated copy field for
> highlighting and to limit the number of characters that are copied. For
> exemple:
>
> <copyField source="text" dest="highlight" maxSize="30000" />
>
>
>
> This approach has also the advantage of limiting the index size for large
> documents (the original text field does not need to be stored and to have
> term vectors). However, the index is bigger for small documents...
>
>
>
> Of course, if the only terms that are matched by a query are after the
> limit, no highlight is possible.
>
>
>
> What do you think of this feature?
>
>
>
> Best regards,
>
> Nicolas
>
>

Re: Slow Highlighting -> CopyField maxSize property

[Koji Sekiguchi <ko...@r.email.ne.jp>]

Hello Nicolas,

Thank you for letting me know this.

Yes, your patch will solve my problem (highlighter performance w/ large 
doc).
BTW, I posted similar ticket to solve my another problem (hl.alternateField
w/ large field).

https://issues.apache.org/jira/browse/SOLR-516

Thank you again,

Koji

Nicolas DESSAIGNE wrote:
> Koji,
>
> The patch is now available at https://issues.apache.org/jira/browse/SOLR-538
>
> Tell me if it fits your needs.
> Nicolas
>
>