You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Pratik Patel <pr...@semandex.net> on 2019/06/10 22:32:28 UTC
How to increase maximum size of files allowed in configuration for MiniSolrCloudCluster
Hi,
I am trying to upload a configuration to "MiniSolrCloudCluster" in my unit
test. This configuration has some binary files for NLP related
functionality. Some of these binary files are bigger than 5 MB. If I try to
upload configuration with these files then it doesn't work. I can set up
the cluster fine if I remove all binary files bigger than 5 MB.
I have noticed the same issue when I try to restore a backup having
configuration files bigger than 5 MB.
Does jetty have some limit on the size of configuration files? Is there a
way to override this?
Thanks,
Pratik
Re: How to increase maximum size of files allowed in configuration
for MiniSolrCloudCluster
Posted by Pratik Patel <pr...@semandex.net>.
That was spot on. Thanks a lot for your help!
On Tue, Jun 11, 2019 at 2:14 AM Jörn Franke <jo...@gmail.com> wrote:
> It is probably a Zookeeper limit. You have to set jute.maxbuffer in the
> Java System properties of all (!) zookeeper Servers and clients to the same
> value (in your case it should be a little bit larger than your largest
> file).
> If possible you can try to avoid storing the NLP / ML models in Solr but
> provide them on a share or similar where all Solr nodes have access to.
>
> > Am 11.06.2019 um 00:32 schrieb Pratik Patel <pr...@semandex.net>:
> >
> > Hi,
> >
> > I am trying to upload a configuration to "MiniSolrCloudCluster" in my
> unit
> > test. This configuration has some binary files for NLP related
> > functionality. Some of these binary files are bigger than 5 MB. If I try
> to
> > upload configuration with these files then it doesn't work. I can set up
> > the cluster fine if I remove all binary files bigger than 5 MB.
> >
> > I have noticed the same issue when I try to restore a backup having
> > configuration files bigger than 5 MB.
> >
> > Does jetty have some limit on the size of configuration files? Is there a
> > way to override this?
> >
> > Thanks,
> > Pratik
>
Re: How to increase maximum size of files allowed in configuration for MiniSolrCloudCluster
Posted by Jörn Franke <jo...@gmail.com>.
It is probably a Zookeeper limit. You have to set jute.maxbuffer in the Java System properties of all (!) zookeeper Servers and clients to the same value (in your case it should be a little bit larger than your largest file).
If possible you can try to avoid storing the NLP / ML models in Solr but provide them on a share or similar where all Solr nodes have access to.
> Am 11.06.2019 um 00:32 schrieb Pratik Patel <pr...@semandex.net>:
>
> Hi,
>
> I am trying to upload a configuration to "MiniSolrCloudCluster" in my unit
> test. This configuration has some binary files for NLP related
> functionality. Some of these binary files are bigger than 5 MB. If I try to
> upload configuration with these files then it doesn't work. I can set up
> the cluster fine if I remove all binary files bigger than 5 MB.
>
> I have noticed the same issue when I try to restore a backup having
> configuration files bigger than 5 MB.
>
> Does jetty have some limit on the size of configuration files? Is there a
> way to override this?
>
> Thanks,
> Pratik