You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by kaustubh147 <ka...@gmail.com> on 2013/10/25 00:30:46 UTC
Problem with glassfish and zookeeper 3.4.5
Hi,
Glassfish 3.1.2.2
Solr 4.5
Zookeeper 3.4.5
We have set up a SolrCloud with 4 Solr nodes and 3 zookeeper instances. It
seems to be working fine from Solr admin page.
but when I am trying to connect it to web application using Solrj 4.5.
I am creating my Solr Cloud Server as suggested on the wiki page
LBHttpSolrServer lbHttpSolrServer = new LBHttpSolrServer(
SOLR_INSTANCE01,
SOLR_INSTANCE02,
SOLR_INSTANCE03,
SOLR_INSTANCE04);
solrServer = new CloudSolrServer(zk1:p1, zk2:p1, zk3:p1, lbHttpSolrServer);
solrServer.setDefaultCollection(collection);
It seems to be working fine for a while even though I am getting a WARNING
as below
-----
SASL configuration failed: javax.security.auth.login.LoginException: No JAAS
configuration section named 'Client' was found in specified JAAS
configuration file: 'XYZ_path/SolrCloud_04/config/login.conf'. Will continue
connection to Zookeeper server without SASL authentication, if Zookeeper
server allows it.
--------------
The application is deployed on a single node cluster on glassfish.
as soon as my application has made some queries to the Solr server it will
start throwing error in the solrServer.runQuery() method. The reason of the
error is not clear..
Application logs shows following error trace many times...
-----------------------------------------
[#|2013-10-24T14:07:53.750-0700|WARNING|glassfish3.1.2|org.apache.zookeeper.ClientCnxn|_ThreadID=1434;_ThreadName=Thread-2;|SASL
configuration failed: javax.security.auth.login.LoginException: No JAAS
configuration section named 'Client' was found in specified JAAS
configuration file: 'XYZ_PATH/config/login.conf'. Will continue connection
to Zookeeper server without SASL authentication, if Zookeeper server allows
it.|#]
[#|2013-10-24T14:07:53.750-0700|INFO|glassfish3.1.2|org.apache.zookeeper.ClientCnxn|_ThreadID=1434;_ThreadName=Thread-2;|Opening
socket connection to server server_name/IP3:2181|#]
[#|2013-10-24T14:07:53.750-0700|INFO|glassfish3.1.2|org.apache.solr.common.cloud.ConnectionManager|_ThreadID=1435;_ThreadName=Thread-2;|Watcher
org.apache.solr.common.cloud.ConnectionManager@187eaada
name:ZooKeeperConnection Watcher:IP1:2181,IP2:2181,IP3:2181 got event
WatchedEvent state:AuthFailed type:None path:null path:null type:None|#]
[#|2013-10-24T14:07:53.750-0700|INFO|glassfish3.1.2|org.apache.solr.common.cloud.ConnectionManager|_ThreadID=1435;_ThreadName=Thread-2;|Client->ZooKeeper
status change trigger but we are already closed|#]
[#|2013-10-24T14:07:53.751-0700|INFO|glassfish3.1.2|org.apache.zookeeper.ClientCnxn|_ThreadID=1434;_ThreadName=Thread-2;|Socket
connection established to server_name/IP3:2181, initiating session|#]
[#|2013-10-24T14:07:53.751-0700|INFO|glassfish3.1.2|org.apache.solr.common.cloud.ConnectionManager|_ThreadID=1420;_ThreadName=Thread-2;|Watcher
org.apache.solr.common.cloud.ConnectionManager@4ba50169
name:ZooKeeperConnection Watcher:IP1:2181,IP2:2181,IP3:2181 got event
WatchedEvent state:Disconnected type:None path:null path:null type:None|#]
[#|2013-10-24T14:07:53.751-0700|WARNING|glassfish3.1.2|org.apache.zookeeper.ClientCnxn|_ThreadID=1434;_ThreadName=Thread-2;|Session
0x0 for serverserver_name/IP3:2181, unexpected error, closing socket
connection and attempting reconnect
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198)
at sun.nio.ch.IOUtil.read(IOUtil.java:166)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:245)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68)
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:355)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
|#]
------------------------------------------
before this happen the zookeeper logs on all the 3 instances starts showing
following warning
2013-10-24 14:05:55,200 [myid:3] - WARN
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too
many connections from /IP_APPLICATION_SEVER - max is 200
it means that my application is making too many connections with the
zookeeper and it is exceeding the limit which is set to 200.
Is there a way I can control the number of connections my application is
making with the zookeeper.
The only component which is connecting to zookeeper in my application is
CloudSolrServer object.
As per my investigation SASL warning is related to a existing bug in
Zookeeper 3.4.5 and is being solved for Zookeeper 3.5 and it should not
cause this issue....
I need help and guidance..
Thanks,
Kaustubh
--
View this message in context: http://lucene.472066.n3.nabble.com/Problem-with-glassfish-and-zookeeper-3-4-5-tp4097635.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with glassfish and zookeeper 3.4.5
Posted by kaustubh147 <ka...@gmail.com>.
Thanks Shawn,
I found a bug in my code, it was creating too many CloudSolrServer objects.
Thanks
Kaustubh
--
View this message in context: http://lucene.472066.n3.nabble.com/Problem-with-glassfish-and-zookeeper-3-4-5-tp4097635p4098732.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Problem with glassfish and zookeeper 3.4.5
Posted by Shawn Heisey <so...@elyograg.org>.
On 10/24/2013 4:30 PM, kaustubh147 wrote:
> Glassfish 3.1.2.2
> Solr 4.5
> Zookeeper 3.4.5
>
> We have set up a SolrCloud with 4 Solr nodes and 3 zookeeper instances. It
> seems to be working fine from Solr admin page.
>
> but when I am trying to connect it to web application using Solrj 4.5.
> I am creating my Solr Cloud Server as suggested on the wiki page
>
> LBHttpSolrServer lbHttpSolrServer = new LBHttpSolrServer(
> SOLR_INSTANCE01,
> SOLR_INSTANCE02,
> SOLR_INSTANCE03,
> SOLR_INSTANCE04);
> solrServer = new CloudSolrServer(zk1:p1, zk2:p1, zk3:p1, lbHttpSolrServer);
> solrServer.setDefaultCollection(collection);
If this is what you are seeing as instructions for connecting from SolrJ
to SolrCloud, then something's really screwy. Can you give me the URL
that shows this, so I can see about getting it changed? The following
code example is how you should be doing that. For this example,
zookeeper is using the default port of 2181 and the zookeeper hosts are
zoo1, zoo2, and zoo3.
String zkHost = "zoo1:2181,zoo2:2181,zoo3:2181";
// If you are using a chroot, use something like this instead:
// String zkHost = "zoo1:2181,zoo2:2181,zoo3:2181/chroot";
CloudSolrServer server = new CloudSolrServer(zkHost);
server.setDefaultCollection("collection1");
> It seems to be working fine for a while even though I am getting a WARNING
> as below
> -----
> SASL configuration failed: javax.security.auth.login.LoginException: No JAAS
> configuration section named 'Client' was found in specified JAAS
> configuration file: 'XYZ_path/SolrCloud_04/config/login.conf'. Will continue
> connection to Zookeeper server without SASL authentication, if Zookeeper
> server allows it.
> --------------
Later you said this sounds like a bug you saw in ZK 3.4.5. It may be
that glassfish turns on some system-wide setting related to
authentication that zookeeper picks up on. I would tend to agree that
this probably is not related to the other problems mentioned below.
> before this happen the zookeeper logs on all the 3 instances starts showing
> following warning
>
> 2013-10-24 14:05:55,200 [myid:3] - WARN
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@193] - Too
> many connections from /IP_APPLICATION_SEVER - max is 200
>
> it means that my application is making too many connections with the
> zookeeper and it is exceeding the limit which is set to 200.
Are you creating one CloudSolrServer object (static would be OK) and
using it for all interaction with SolrCloud, or are you creating many
CloudSolrServer objects over the life of your application? Is there
more than one thread or instance of your application running, and each
one has its own CloudSolrServer object? It is strongly recommended that
you only create one object for your entire application and use it for
all queries, updates, etc. You can set the "collection" parameter on
each query or request object that you use, if you need to use more than one.
If you *DO* create many CloudSolrServer objects over the life of your
application and cannot immediately change your code so that it uses one
object, be sure to shutdown() each one when it is no longer required.
Depending on the exact nature of your application, you may also need to
increase the maximum number of connections allowed in your zookeeper config.
Thanks,
Shawn