Problem with uploading Large synonym files in cloud mode

[Salmaan Rashid Syed <sa...@mroads.com>]

Hi Solr Users,

I have a very big synonym file (>5MB). I am unable to start Solr in cloud
mode as it throws an error message stating that the synonmys file is
too large. I figured out that the zookeeper doesn't take a file greater
than 1MB size.

I tried to break down my synonyms file to smaller chunks less than 1MB
each. But, I am not sure about how to include all the filenames into the
Solr schema.

Should it be seperated by commas like synonyms = "__1_synonyms.txt,
__2_synonyms.txt, __3synonyms.txt"

Or is there a better way of doing that? Will the bigger file when broken
down to smaller chunks will be uploaded to zookeeper as well.

Please help or please guide me to relevant documentation regarding this.

Thank you.

Regards.
Salmaan.

Re: Problem with uploading Large synonym files in cloud mode

[Jörn Franke <jo...@gmail.com>]

Aside that a 5 MB synonym file is rather strange (what is the use case for such a large synonym file?) and that it will have impact on index size and/or query time:

You can configure zookeeper server and the Solr client to allow larger files using the jute.maxbuffer option.

> Am 30.07.2019 um 13:09 schrieb Salmaan Rashid Syed <sa...@mroads.com>:
> 
> Hi Solr Users,
> 
> I have a very big synonym file (>5MB). I am unable to start Solr in cloud
> mode as it throws an error message stating that the synonmys file is
> too large. I figured out that the zookeeper doesn't take a file greater
> than 1MB size.
> 
> I tried to break down my synonyms file to smaller chunks less than 1MB
> each. But, I am not sure about how to include all the filenames into the
> Solr schema.
> 
> Should it be seperated by commas like synonyms = "__1_synonyms.txt,
> __2_synonyms.txt, __3synonyms.txt"
> 
> Or is there a better way of doing that? Will the bigger file when broken
> down to smaller chunks will be uploaded to zookeeper as well.
> 
> Please help or please guide me to relevant documentation regarding this.
> 
> Thank you.
> 
> Regards.
> Salmaan.