You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Venkat Paranjothi <vp...@us.ibm.com> on 2015/12/11 17:40:16 UTC
Unable to create lot of cores -- Failing after 100 cores
Hello all,
We need to create around 300 collections with replication factor 2. But
after creating 100, we couldn't create more and most of them are in RED
state in the solrcloud.
Is this issue related to zookeeper jute.maxBuffer issue? If so, how can
we increase the size of zookeeper maxbuffer and memory size.
Thanks
Venkat
Re: Unable to create lot of cores -- Failing after 100 cores
Posted by Venkat Paranjothi <vp...@us.ibm.com>.
Eric,
Sorry for the confusion.
Noticed both issues, some times collection was showing in RED in the
solrcloud.. if i leave this for a day, next day it shows GREEN. This is
not true for all the cases and not seeing the consistent behavior.
After restarting the zookeeper & tomcat services, I was able to create
the first collection successfully and second collection was showing RED
in the solrcloud.
At the end, the behavior is not consistent.
From: Erick Erickson <er...@gmail.com>
To: solr-user <so...@lucene.apache.org>
Date: 12/11/2015 03:48 PM
Subject: Re: Unable to create lot of cores -- Failing after 100 cores
You seem to have changed the problem statement from
being unable to create cores to it takes a long time for
them to show up.
What's the current problem you're having?
Best,
Erick
On Fri, Dec 11, 2015 at 12:45 PM, Venkat Paranjothi <vp...@us.ibm.com>
wrote:
> Thanks shalin..
>
> Sorry i forgot to mention the verison.. 4.6.1 + tomcat6 + zookeeper 3.4.5
>
> Thanks,
>
>
>
>
>
> [image: Inactive hide details for Shalin Shekhar Mangar ---12/11/2015
> 03:18:53 PM---Which version of Solr are you using? As Erick, said]Shalin
> Shekhar Mangar ---12/11/2015 03:18:53 PM---Which version of Solr are you
> using? As Erick, said, use the latest 5.3.1 release, it is much more b
>
> From: Shalin Shekhar Mangar <sh...@gmail.com>
> To: solr-user@lucene.apache.org
> Date: 12/11/2015 03:18 PM
> Subject: Re: Unable to create lot of cores -- Failing after 100 cores
> ------------------------------
>
>
>
> Which version of Solr are you using? As Erick, said, use the latest 5.3.1
> release, it is much more better in handling many collections.
>
> On Sat, Dec 12, 2015 at 1:39 AM, Venkat Paranjothi <vp...@us.ibm.com>
> wrote:
>
> > Thanks Eric,
> >
> > This is what we did to the zookeeper and solr settings.. Still, we are
> not
> > seeing the improvement in the collection creation.. it takes lot of
time
> to
> > see the collection on the Solrcloud.
> >
> > added the following line in zkServer.sh
> >
> > export JVMFLAGS="$JVMFLAGS -Xms256m -Xmx1g -Djute.maxBuffer=10485760"
> >
> > added the following into catalina.sh
> > JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/opt/apache_solr/installed/solr
> > -Dport=8080 -DhostContext=solr -Dsolr.Data.dir=/solr/dataidx
> > -Djute.maxBuffer=10485760 -DzkClientTimeout=200000
> > -DzkHost=x.x.x.x:2181,y.y.y.y:2181,z.z.z.z:2181
> > -Dcollection.configname=collection_configuration_v1"
> >
> >
> > *Here is my zoo.cfg settings*
> >
> > # The number of milliseconds of each tick
> > tickTime=3000
> >
> > # The number of ticks that the initial synchronization phase can take
> > initLimit=200
> >
> > maxClientCnxns=0
> >
> > # The number of ticks that can pass between
> > # sending a request and getting an acknowledgement
> > syncLimit=50
> >
> > # the directory where the snapshot is stored.
> > # Choose appropriately for your environment
> > dataDir=/opt/apache_solr/solrcloud/zookeeper_data
> >
> > # the port at which the clients will connect
> > clientPort=2181
> >
> > # the directory where transaction log is stored.
> > # this parameter provides dedicated log device for ZooKeeper
> > dataLogDir=/opt/apache_solr/solrcloud/zookeeper_log
> >
> > # ZooKeeper server and its port no.
> > # ZooKeeper ensemble should know about every other machine in the
> ensemble
> > # specify server id by creating 'myid' file in the dataDir
> > # use hostname instead of IP address for convenient maintenance
> > server.1=x.x.x.x:2888:3888
> > server.2=y.y.y.y:2888:3888
> > server.3=z.z.z.z:2888:3888
> >
> >
> >
> > Thanks,
> >
> > *Venkat Paranjothi*
> > Kenexa 2xB Software Engineering Team
> > ------------------------------
> > *Phone:* 1-978-899-2746
> > *E-mail:* *vparanjo@us.ibm.com* <vp...@us.ibm.com>
> >
> >
> >
> >
> > [image: Inactive hide details for Erick Erickson ---12/11/2015 01:16:57
> > PM---A quick Google search shows the following: "you must set -]Erick
> > Erickson ---12/11/2015 01:16:57 PM---A quick Google search shows the
> > following: "you must set -Djute.maxbuffer in zookeeper and solr..."
> >
> > From: Erick Erickson <er...@gmail.com>
> > To: solr-user <so...@lucene.apache.org>
> > Date: 12/11/2015 01:16 PM
> > Subject: Re: Unable to create lot of cores -- Failing after 100 cores
> > ------------------------------
> >
> >
> >
> > A quick Google search shows the following:
> >
> > "you must set -Djute.maxbuffer in zookeeper and solr..."
> >
> > What have you tried? What were the results?
> >
> > What version of Solr are you using? 5.x defaults to
> > an individual state.json file per collection rather than
> > one big one for all collections, that will also help.
> >
> > Best,
> > Erick
> >
> > On Fri, Dec 11, 2015 at 8:40 AM, Venkat Paranjothi
<vp...@us.ibm.com>
> > wrote:
> > > Hello all,
> > >
> > > We need to create around 300 collections with replication factor 2.
> But
> > > after creating 100, we couldn't create more and most of them are in
RED
> > > state in the solrcloud.
> > >
> > > Is this issue related to zookeeper jute.maxBuffer issue? If so, how
> can
> > > we increase the size of zookeeper maxbuffer and memory size.
> > >
> > > Thanks
> > > Venkat
> >
> >
> >
> >
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
>
>
Re: Unable to create lot of cores -- Failing after 100 cores
Posted by Erick Erickson <er...@gmail.com>.
You seem to have changed the problem statement from
being unable to create cores to it takes a long time for
them to show up.
What's the current problem you're having?
Best,
Erick
On Fri, Dec 11, 2015 at 12:45 PM, Venkat Paranjothi <vp...@us.ibm.com>
wrote:
> Thanks shalin..
>
> Sorry i forgot to mention the verison.. 4.6.1 + tomcat6 + zookeeper 3.4.5
>
> Thanks,
>
> *Venkat Paranjothi*
> Kenexa 2xB Software Engineering Team
> ------------------------------
> *Phone:* 1-978-899-2746
> *E-mail:* *vparanjo@us.ibm.com* <vp...@us.ibm.com>
>
>
>
>
> [image: Inactive hide details for Shalin Shekhar Mangar ---12/11/2015
> 03:18:53 PM---Which version of Solr are you using? As Erick, said]Shalin
> Shekhar Mangar ---12/11/2015 03:18:53 PM---Which version of Solr are you
> using? As Erick, said, use the latest 5.3.1 release, it is much more b
>
> From: Shalin Shekhar Mangar <sh...@gmail.com>
> To: solr-user@lucene.apache.org
> Date: 12/11/2015 03:18 PM
> Subject: Re: Unable to create lot of cores -- Failing after 100 cores
> ------------------------------
>
>
>
> Which version of Solr are you using? As Erick, said, use the latest 5.3.1
> release, it is much more better in handling many collections.
>
> On Sat, Dec 12, 2015 at 1:39 AM, Venkat Paranjothi <vp...@us.ibm.com>
> wrote:
>
> > Thanks Eric,
> >
> > This is what we did to the zookeeper and solr settings.. Still, we are
> not
> > seeing the improvement in the collection creation.. it takes lot of time
> to
> > see the collection on the Solrcloud.
> >
> > added the following line in zkServer.sh
> >
> > export JVMFLAGS="$JVMFLAGS -Xms256m -Xmx1g -Djute.maxBuffer=10485760"
> >
> > added the following into catalina.sh
> > JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/opt/apache_solr/installed/solr
> > -Dport=8080 -DhostContext=solr -Dsolr.Data.dir=/solr/dataidx
> > -Djute.maxBuffer=10485760 -DzkClientTimeout=200000
> > -DzkHost=x.x.x.x:2181,y.y.y.y:2181,z.z.z.z:2181
> > -Dcollection.configname=collection_configuration_v1"
> >
> >
> > *Here is my zoo.cfg settings*
> >
> > # The number of milliseconds of each tick
> > tickTime=3000
> >
> > # The number of ticks that the initial synchronization phase can take
> > initLimit=200
> >
> > maxClientCnxns=0
> >
> > # The number of ticks that can pass between
> > # sending a request and getting an acknowledgement
> > syncLimit=50
> >
> > # the directory where the snapshot is stored.
> > # Choose appropriately for your environment
> > dataDir=/opt/apache_solr/solrcloud/zookeeper_data
> >
> > # the port at which the clients will connect
> > clientPort=2181
> >
> > # the directory where transaction log is stored.
> > # this parameter provides dedicated log device for ZooKeeper
> > dataLogDir=/opt/apache_solr/solrcloud/zookeeper_log
> >
> > # ZooKeeper server and its port no.
> > # ZooKeeper ensemble should know about every other machine in the
> ensemble
> > # specify server id by creating 'myid' file in the dataDir
> > # use hostname instead of IP address for convenient maintenance
> > server.1=x.x.x.x:2888:3888
> > server.2=y.y.y.y:2888:3888
> > server.3=z.z.z.z:2888:3888
> >
> >
> >
> > Thanks,
> >
> > *Venkat Paranjothi*
> > Kenexa 2xB Software Engineering Team
> > ------------------------------
> > *Phone:* 1-978-899-2746
> > *E-mail:* *vparanjo@us.ibm.com* <vp...@us.ibm.com>
> >
> >
> >
> >
> > [image: Inactive hide details for Erick Erickson ---12/11/2015 01:16:57
> > PM---A quick Google search shows the following: "you must set -]Erick
> > Erickson ---12/11/2015 01:16:57 PM---A quick Google search shows the
> > following: "you must set -Djute.maxbuffer in zookeeper and solr..."
> >
> > From: Erick Erickson <er...@gmail.com>
> > To: solr-user <so...@lucene.apache.org>
> > Date: 12/11/2015 01:16 PM
> > Subject: Re: Unable to create lot of cores -- Failing after 100 cores
> > ------------------------------
> >
> >
> >
> > A quick Google search shows the following:
> >
> > "you must set -Djute.maxbuffer in zookeeper and solr..."
> >
> > What have you tried? What were the results?
> >
> > What version of Solr are you using? 5.x defaults to
> > an individual state.json file per collection rather than
> > one big one for all collections, that will also help.
> >
> > Best,
> > Erick
> >
> > On Fri, Dec 11, 2015 at 8:40 AM, Venkat Paranjothi <vp...@us.ibm.com>
> > wrote:
> > > Hello all,
> > >
> > > We need to create around 300 collections with replication factor 2.
> But
> > > after creating 100, we couldn't create more and most of them are in RED
> > > state in the solrcloud.
> > >
> > > Is this issue related to zookeeper jute.maxBuffer issue? If so, how
> can
> > > we increase the size of zookeeper maxbuffer and memory size.
> > >
> > > Thanks
> > > Venkat
> >
> >
> >
> >
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
>
>
Re: Unable to create lot of cores -- Failing after 100 cores
Posted by Venkat Paranjothi <vp...@us.ibm.com>.
Thanks shalin..
Sorry i forgot to mention the verison.. 4.6.1 + tomcat6 + zookeeper 3.4.5
Thanks,
Venkat Paranjothi
Kenexa 2xB Software Engineering Team
Phone: 1-978-899-2746
E-mail: vparanjo@us.ibm.com
From: Shalin Shekhar Mangar <sh...@gmail.com>
To: solr-user@lucene.apache.org
Date: 12/11/2015 03:18 PM
Subject: Re: Unable to create lot of cores -- Failing after 100 cores
Which version of Solr are you using? As Erick, said, use the latest 5.3.1
release, it is much more better in handling many collections.
On Sat, Dec 12, 2015 at 1:39 AM, Venkat Paranjothi <vp...@us.ibm.com>
wrote:
> Thanks Eric,
>
> This is what we did to the zookeeper and solr settings.. Still, we are
not
> seeing the improvement in the collection creation.. it takes lot of time
to
> see the collection on the Solrcloud.
>
> added the following line in zkServer.sh
>
> export JVMFLAGS="$JVMFLAGS -Xms256m -Xmx1g -Djute.maxBuffer=10485760"
>
> added the following into catalina.sh
> JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/opt/apache_solr/installed/solr
> -Dport=8080 -DhostContext=solr -Dsolr.Data.dir=/solr/dataidx
> -Djute.maxBuffer=10485760 -DzkClientTimeout=200000
> -DzkHost=x.x.x.x:2181,y.y.y.y:2181,z.z.z.z:2181
> -Dcollection.configname=collection_configuration_v1"
>
>
> *Here is my zoo.cfg settings*
>
> # The number of milliseconds of each tick
> tickTime=3000
>
> # The number of ticks that the initial synchronization phase can take
> initLimit=200
>
> maxClientCnxns=0
>
> # The number of ticks that can pass between
> # sending a request and getting an acknowledgement
> syncLimit=50
>
> # the directory where the snapshot is stored.
> # Choose appropriately for your environment
> dataDir=/opt/apache_solr/solrcloud/zookeeper_data
>
> # the port at which the clients will connect
> clientPort=2181
>
> # the directory where transaction log is stored.
> # this parameter provides dedicated log device for ZooKeeper
> dataLogDir=/opt/apache_solr/solrcloud/zookeeper_log
>
> # ZooKeeper server and its port no.
> # ZooKeeper ensemble should know about every other machine in the
ensemble
> # specify server id by creating 'myid' file in the dataDir
> # use hostname instead of IP address for convenient maintenance
> server.1=x.x.x.x:2888:3888
> server.2=y.y.y.y:2888:3888
> server.3=z.z.z.z:2888:3888
>
>
>
> Thanks,
>
> *Venkat Paranjothi*
> Kenexa 2xB Software Engineering Team
> ------------------------------
> *Phone:* 1-978-899-2746
> *E-mail:* *vparanjo@us.ibm.com* <vp...@us.ibm.com>
>
>
>
>
> [image: Inactive hide details for Erick Erickson ---12/11/2015 01:16:57
> PM---A quick Google search shows the following: "you must set -]Erick
> Erickson ---12/11/2015 01:16:57 PM---A quick Google search shows the
> following: "you must set -Djute.maxbuffer in zookeeper and solr..."
>
> From: Erick Erickson <er...@gmail.com>
> To: solr-user <so...@lucene.apache.org>
> Date: 12/11/2015 01:16 PM
> Subject: Re: Unable to create lot of cores -- Failing after 100 cores
> ------------------------------
>
>
>
> A quick Google search shows the following:
>
> "you must set -Djute.maxbuffer in zookeeper and solr..."
>
> What have you tried? What were the results?
>
> What version of Solr are you using? 5.x defaults to
> an individual state.json file per collection rather than
> one big one for all collections, that will also help.
>
> Best,
> Erick
>
> On Fri, Dec 11, 2015 at 8:40 AM, Venkat Paranjothi <vp...@us.ibm.com>
> wrote:
> > Hello all,
> >
> > We need to create around 300 collections with replication factor 2.
But
> > after creating 100, we couldn't create more and most of them are in RED
> > state in the solrcloud.
> >
> > Is this issue related to zookeeper jute.maxBuffer issue? If so, how
can
> > we increase the size of zookeeper maxbuffer and memory size.
> >
> > Thanks
> > Venkat
>
>
>
>
--
Regards,
Shalin Shekhar Mangar.
Re: Unable to create lot of cores -- Failing after 100 cores
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Which version of Solr are you using? As Erick, said, use the latest 5.3.1
release, it is much more better in handling many collections.
On Sat, Dec 12, 2015 at 1:39 AM, Venkat Paranjothi <vp...@us.ibm.com>
wrote:
> Thanks Eric,
>
> This is what we did to the zookeeper and solr settings.. Still, we are not
> seeing the improvement in the collection creation.. it takes lot of time to
> see the collection on the Solrcloud.
>
> added the following line in zkServer.sh
>
> export JVMFLAGS="$JVMFLAGS -Xms256m -Xmx1g -Djute.maxBuffer=10485760"
>
> added the following into catalina.sh
> JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/opt/apache_solr/installed/solr
> -Dport=8080 -DhostContext=solr -Dsolr.Data.dir=/solr/dataidx
> -Djute.maxBuffer=10485760 -DzkClientTimeout=200000
> -DzkHost=x.x.x.x:2181,y.y.y.y:2181,z.z.z.z:2181
> -Dcollection.configname=collection_configuration_v1"
>
>
> *Here is my zoo.cfg settings*
>
> # The number of milliseconds of each tick
> tickTime=3000
>
> # The number of ticks that the initial synchronization phase can take
> initLimit=200
>
> maxClientCnxns=0
>
> # The number of ticks that can pass between
> # sending a request and getting an acknowledgement
> syncLimit=50
>
> # the directory where the snapshot is stored.
> # Choose appropriately for your environment
> dataDir=/opt/apache_solr/solrcloud/zookeeper_data
>
> # the port at which the clients will connect
> clientPort=2181
>
> # the directory where transaction log is stored.
> # this parameter provides dedicated log device for ZooKeeper
> dataLogDir=/opt/apache_solr/solrcloud/zookeeper_log
>
> # ZooKeeper server and its port no.
> # ZooKeeper ensemble should know about every other machine in the ensemble
> # specify server id by creating 'myid' file in the dataDir
> # use hostname instead of IP address for convenient maintenance
> server.1=x.x.x.x:2888:3888
> server.2=y.y.y.y:2888:3888
> server.3=z.z.z.z:2888:3888
>
>
>
> Thanks,
>
> *Venkat Paranjothi*
> Kenexa 2xB Software Engineering Team
> ------------------------------
> *Phone:* 1-978-899-2746
> *E-mail:* *vparanjo@us.ibm.com* <vp...@us.ibm.com>
>
>
>
>
> [image: Inactive hide details for Erick Erickson ---12/11/2015 01:16:57
> PM---A quick Google search shows the following: "you must set -]Erick
> Erickson ---12/11/2015 01:16:57 PM---A quick Google search shows the
> following: "you must set -Djute.maxbuffer in zookeeper and solr..."
>
> From: Erick Erickson <er...@gmail.com>
> To: solr-user <so...@lucene.apache.org>
> Date: 12/11/2015 01:16 PM
> Subject: Re: Unable to create lot of cores -- Failing after 100 cores
> ------------------------------
>
>
>
> A quick Google search shows the following:
>
> "you must set -Djute.maxbuffer in zookeeper and solr..."
>
> What have you tried? What were the results?
>
> What version of Solr are you using? 5.x defaults to
> an individual state.json file per collection rather than
> one big one for all collections, that will also help.
>
> Best,
> Erick
>
> On Fri, Dec 11, 2015 at 8:40 AM, Venkat Paranjothi <vp...@us.ibm.com>
> wrote:
> > Hello all,
> >
> > We need to create around 300 collections with replication factor 2. But
> > after creating 100, we couldn't create more and most of them are in RED
> > state in the solrcloud.
> >
> > Is this issue related to zookeeper jute.maxBuffer issue? If so, how can
> > we increase the size of zookeeper maxbuffer and memory size.
> >
> > Thanks
> > Venkat
>
>
>
>
--
Regards,
Shalin Shekhar Mangar.
Re: Unable to create lot of cores -- Failing after 100 cores
Posted by Venkat Paranjothi <vp...@us.ibm.com>.
Thanks Eric,
This is what we did to the zookeeper and solr settings.. Still, we are
not seeing the improvement in the collection creation.. it takes lot of
time to see the collection on the Solrcloud.
added the following line in zkServer.sh
export JVMFLAGS="$JVMFLAGS -Xms256m -Xmx1g -Djute.maxBuffer=10485760"
added the following into catalina.sh
JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/opt/apache_solr/installed/solr
-Dport=8080 -DhostContext=solr -Dsolr.Data.dir=/solr/dataidx
-Djute.maxBuffer=10485760 -DzkClientTimeout=200000
-DzkHost=x.x.x.x:2181,y.y.y.y:2181,z.z.z.z:2181
-Dcollection.configname=collection_configuration_v1"
Here is my zoo.cfg settings
# The number of milliseconds of each tick
tickTime=3000
# The number of ticks that the initial synchronization phase can take
initLimit=200
maxClientCnxns=0
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=50
# the directory where the snapshot is stored.
# Choose appropriately for your environment
dataDir=/opt/apache_solr/solrcloud/zookeeper_data
# the port at which the clients will connect
clientPort=2181
# the directory where transaction log is stored.
# this parameter provides dedicated log device for ZooKeeper
dataLogDir=/opt/apache_solr/solrcloud/zookeeper_log
# ZooKeeper server and its port no.
# ZooKeeper ensemble should know about every other machine in the ensemble
# specify server id by creating 'myid' file in the dataDir
# use hostname instead of IP address for convenient maintenance
server.1=x.x.x.x:2888:3888
server.2=y.y.y.y:2888:3888
server.3=z.z.z.z:2888:3888
Thanks,
Venkat Paranjothi
Kenexa 2xB Software Engineering Team
Phone: 1-978-899-2746
E-mail: vparanjo@us.ibm.com
From: Erick Erickson <er...@gmail.com>
To: solr-user <so...@lucene.apache.org>
Date: 12/11/2015 01:16 PM
Subject: Re: Unable to create lot of cores -- Failing after 100 cores
A quick Google search shows the following:
"you must set -Djute.maxbuffer in zookeeper and solr..."
What have you tried? What were the results?
What version of Solr are you using? 5.x defaults to
an individual state.json file per collection rather than
one big one for all collections, that will also help.
Best,
Erick
On Fri, Dec 11, 2015 at 8:40 AM, Venkat Paranjothi <vp...@us.ibm.com>
wrote:
> Hello all,
>
> We need to create around 300 collections with replication factor 2. But
> after creating 100, we couldn't create more and most of them are in RED
> state in the solrcloud.
>
> Is this issue related to zookeeper jute.maxBuffer issue? If so, how can
> we increase the size of zookeeper maxbuffer and memory size.
>
> Thanks
> Venkat
Re: Unable to create lot of cores -- Failing after 100 cores
Posted by Erick Erickson <er...@gmail.com>.
A quick Google search shows the following:
"you must set -Djute.maxbuffer in zookeeper and solr..."
What have you tried? What were the results?
What version of Solr are you using? 5.x defaults to
an individual state.json file per collection rather than
one big one for all collections, that will also help.
Best,
Erick
On Fri, Dec 11, 2015 at 8:40 AM, Venkat Paranjothi <vp...@us.ibm.com> wrote:
> Hello all,
>
> We need to create around 300 collections with replication factor 2. But
> after creating 100, we couldn't create more and most of them are in RED
> state in the solrcloud.
>
> Is this issue related to zookeeper jute.maxBuffer issue? If so, how can
> we increase the size of zookeeper maxbuffer and memory size.
>
> Thanks
> Venkat