MultiLingual Search

["Sachit P. Menon" <sa...@mindtree.com>]

Hello folks,

 

My project requires having the same content (mostly) in multiple languages. 

 

Do I need to have different files for the same content in different
languages? This will need indexing of every file whenever a new content
appears?

Or can we have a mapping of the content with all other languages?

 

Suppose I have hundred documents on particular search criteria (say Solr) in
different languages, how can I get a Chinese result only when I enter Solr in
Chinese as search criteria? Transliteration in this case will be handled from
the front end.

 

Can anyone tell me the steps to implement multilingual search in Solr as I'm
very new to Solr?

 

 

Thanks,

Sachit



DISCLAIMER:
This message (including attachment if any) is confidential and may be privileged. If you have received this message by mistake please notify the sender by return e-mail and delete this message from your system. Any unauthorized use or dissemination of this message in whole or in part is strictly prohibited.
E-mail may contain viruses. Before opening attachments please check them for viruses and defects. While MindTree Limited (MindTree) has put in place checks to minimize the risks, MindTree will not be responsible for any viruses or defects or any forwarded attachments emanating either from within MindTree or outside.
Please note that e-mails are susceptible to change and MindTree shall not be liable for any improper, untimely or incomplete transmission.
MindTree reserves the right to monitor and review the content of all messages sent to or from MindTree e-mail address. Messages sent to or from this e-mail address may be stored on the MindTree e-mail system or else where.

RE: MultiLingual Search

["Sachit P. Menon" <sa...@mindtree.com>]

Hi All,

I would like to know if Indian regional languages (like Malayalam, Kannada,
Tamil, etc.) can also be indexed through Solr.

Thanks and Regards
Sachit P. Menon


DISCLAIMER:
This message (including attachment if any) is confidential and may be privileged. If you have received this message by mistake please notify the sender by return e-mail and delete this message from your system. Any unauthorized use or dissemination of this message in whole or in part is strictly prohibited.
E-mail may contain viruses. Before opening attachments please check them for viruses and defects. While MindTree Limited (MindTree) has put in place checks to minimize the risks, MindTree will not be responsible for any viruses or defects or any forwarded attachments emanating either from within MindTree or outside.
Please note that e-mails are susceptible to change and MindTree shall not be liable for any improper, untimely or incomplete transmission.
MindTree reserves the right to monitor and review the content of all messages sent to or from MindTree e-mail address. Messages sent to or from this e-mail address may be stored on the MindTree e-mail system or else where.

Re: MultiLingual Search

[Alexander Ramos Jardim <al...@gmail.com>]

 I would look toward some implementations:

1st: You could have one index for each language. Just record the preferred
language in session and use it to select the index you are searching in.
Pros: It is easy to add a new language, just create put another index
instance online.
Cons:It can become expensive and difficult to maintain as you need to
replicate the changes on all indexes.

2nd: You could have fields like english_title, chinese_title, and so on, so
you would use the language concatenated to the field name,getting all the
fields you want when you know the language.
Pros: It is easy to maintain. One index, just make a new field for each
language. Restart the solr server and you have all you need.
Cons: Each document will become large, so they will be slower to search
through.


2008/5/12 Sachit P. Menon <sa...@mindtree.com>:

> Hello folks,
>
>
>
> My project requires having the same content (mostly) in multiple
> languages.
>
>
>
> Do I need to have different files for the same content in different
> languages? This will need indexing of every file whenever a new content
> appears?
>
> Or can we have a mapping of the content with all other languages?
>
>
>
> Suppose I have hundred documents on particular search criteria (say Solr)
> in
> different languages, how can I get a Chinese result only when I enter Solr
> in
> Chinese as search criteria? Transliteration in this case will be handled
> from
> the front end.
>
>
>
> Can anyone tell me the steps to implement multilingual search in Solr as
> I'm
> very new to Solr?
>
>
>
>
>
> Thanks,
>
> Sachit
>
>
>
> DISCLAIMER:
> This message (including attachment if any) is confidential and may be
> privileged. If you have received this message by mistake please notify the
> sender by return e-mail and delete this message from your system. Any
> unauthorized use or dissemination of this message in whole or in part is
> strictly prohibited.
> E-mail may contain viruses. Before opening attachments please check them
> for viruses and defects. While MindTree Limited (MindTree) has put in place
> checks to minimize the risks, MindTree will not be responsible for any
> viruses or defects or any forwarded attachments emanating either from within
> MindTree or outside.
> Please note that e-mails are susceptible to change and MindTree shall not
> be liable for any improper, untimely or incomplete transmission.
> MindTree reserves the right to monitor and review the content of all
> messages sent to or from MindTree e-mail address. Messages sent to or from
> this e-mail address may be stored on the MindTree e-mail system or else
> where.
>



-- 
Alexander Ramos Jardim

Re: MultiLingual Search

[Norberto Meijome <fr...@meijome.net>]

On Mon, 12 May 2008 16:16:28 +0530
"Sachit P. Menon" <sa...@mindtree.com> wrote:

> My project requires having the same content (mostly) in multiple languages. 

hi Sachit,
please search the archives of the list. this topic seems to come up twice a week or thereabouts :) 

You are of course encouraged to ask back when you have specific questions that haven't been answered already.

Everyone - time for MultiLingual FAQ @ Wiki ? 
B

_________________________
{Beto|Norberto|Numard} Meijome

"Lots of people who complained about us receiving the MBE received theirs for heroism in the war -- for killing people. We received ours for entertaining other people. I'd say we deserve ours more."
  John Lennon

I speak for myself, not my employer. Contents may be hot. Slippery when wet. Reading disclaimers makes you go blind. Writing them is worse. You have been Warned.