You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Rajesh Khan <ra...@gmail.com> on 2013/07/26 20:45:31 UTC
Re: Problem connecting "No protocol received closing"
We are trying to run the Qpid C++ broker on Fedora 18, but when we increase
the number of connections to Qpid topics we get a tremendous number of
these messages in the log:
Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
Could not accept socket: Too many open files
(/builddir/build/BUILD/qpid-0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
Could not accept socket: Too many open files
(/builddir/build/BUILD/qpid-0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
Could not accept socket: Too many open files
(/builddir/build/BUILD/qpid-0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
Could not accept socket: Too many open files
(/builddir/build/BUILD/qpid-0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
Could not accept socket: Too many open files
(/builddir/build/BUILD/qpid-0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
Could not accept socket: Too many open files
(/builddir/build/BUILD/qpid-0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
Could not accept socket: Too many open files
(/builddir/build/BUILD/qpid-0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
Could not accept socket: Too many open files
(/builddir/build/BUILD/qpid-0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
Could not accept socket: Too many open files
(/builddir/build/BUILD/qpid-0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
When I say we get a tremendous number of these messages in the log, I mean
over ten thousand per second. In the following example, 11690 within a
single second:
root@jackson:/var/log> grep "Too many open files" | grep '18:09:36'
messages | wc -l
11690
We see these messages when there are only 1071 open file handles for the
qpidd daemon according to lsof.
In trying to address the issue, we have increased, system wide, the maximum
number of open file handles per user to 65535, or 65k:
root@jackson:~> ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 128143
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65535
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
To verify that the user running the qpid daemon is truly running with the
new increased open files limit:
qpidd@jackson:~> ulimit -n
65535
Here is line 214 in BSDSocket.cpp that is responsible for all the messages
in the log:
204 Socket* BSDSocket::accept() const
205 {
206 int afd = ::accept(fd, 0, 0);
207 if ( afd >= 0) {
208 BSDSocket* s = new BSDSocket(afd);
209 s->localname = localname;
210 return s;
211 }
212 else if (errno == EAGAIN)
213 return 0;
214 else throw QPID_POSIX_ERROR(errno);
215 }
It appears that increasing the maximum number of open file handles for the
qpid user has not solved the problem. It seems that others on the email
list have experienced this probem, but we haven't seen a solution. Any
ideas?
On Mon, Jun 17, 2013 at 9:07 AM, zhumin <zh...@git.com.cn> wrote:
> I also meet it when i use SSL C++ client to connect qpid server.
>
>
>
>
> --
> View this message in context:
> http://qpid.2158936.n2.nabble.com/Problem-connecting-No-protocol-received-closing-tp7594171p7594201.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
RE: Problem connecting "No protocol received closing"
Posted by Steve Huston <sh...@riverace.com>.
I'm certainly not sure about your case, but here are some points from our knowledge:
Make sure that /proc/sys/fs/file-max (total number of files that can be opened by all users) is large enough. To check, you can: cat /proc/sys/fs/file-max
If you need to increase it, modify /etc/sysctl.conf and add: fs.file-max = <some big number> and then execute sysctl -p. Most likely you will not have to make any changes here.
Edit /etc/security/limits.conf and add: <user name> hard nofile <the handle limit you want> (no larger than the __FD_SETSIZE value from above).
Login as user specified in previous step and execute ulimit -n 2048. At this point kernel will allow any process in current session to open 2048 handles. To check what are your limits you can always use ulimit -a. Since this setting will be valid only for this session, you can add ulimit -n 2048 to your .profile or .bash_profile if the permanent setting is required.
> -----Original Message-----
> From: Rajesh Khan [mailto:rajeshkhan808@gmail.com]
> Sent: Friday, July 26, 2013 2:46 PM
> To: users@qpid.apache.org
> Subject: Re: Problem connecting "No protocol received closing"
>
> We are trying to run the Qpid C++ broker on Fedora 18, but when we
> increase the number of connections to Qpid topics we get a tremendous
> number of these messages in the log:
>
> Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
> Could not accept socket: Too many open files
> (/builddir/build/BUILD/qpid-
> 0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
> Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
> Could not accept socket: Too many open files
> (/builddir/build/BUILD/qpid-
> 0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
> Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
> Could not accept socket: Too many open files
> (/builddir/build/BUILD/qpid-
> 0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
> Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
> Could not accept socket: Too many open files
> (/builddir/build/BUILD/qpid-
> 0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
> Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
> Could not accept socket: Too many open files
> (/builddir/build/BUILD/qpid-
> 0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
> Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
> Could not accept socket: Too many open files
> (/builddir/build/BUILD/qpid-
> 0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
> Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
> Could not accept socket: Too many open files
> (/builddir/build/BUILD/qpid-
> 0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
> Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
> Could not accept socket: Too many open files
> (/builddir/build/BUILD/qpid-
> 0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
> Jul 26 12:09:36 jackson qpidd[31823]: 2013-07-26 18:09:36 [System] error
> Could not accept socket: Too many open files
> (/builddir/build/BUILD/qpid-
> 0.22/cpp/src/qpid/sys/posix/BSDSocket.cpp:214)
>
> When I say we get a tremendous number of these messages in the log, I
> mean over ten thousand per second. In the following example, 11690 within
> a single second:
>
> root@jackson:/var/log> grep "Too many open files" | grep '18:09:36'
> messages | wc -l
> 11690
>
> We see these messages when there are only 1071 open file handles for the
> qpidd daemon according to lsof.
>
> In trying to address the issue, we have increased, system wide, the
> maximum number of open file handles per user to 65535, or 65k:
>
> root@jackson:~> ulimit -a
> core file size (blocks, -c) 0
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) unlimited
> pending signals (-i) 128143
> max locked memory (kbytes, -l) 64
> max memory size (kbytes, -m) unlimited
> open files (-n) 65535
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 8192
> cpu time (seconds, -t) unlimited
> max user processes (-u) 1024
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
>
> To verify that the user running the qpid daemon is truly running with the new
> increased open files limit:
>
> qpidd@jackson:~> ulimit -n
> 65535
>
> Here is line 214 in BSDSocket.cpp that is responsible for all the messages in
> the log:
>
> 204 Socket* BSDSocket::accept() const
> 205 {
> 206 int afd = ::accept(fd, 0, 0);
> 207 if ( afd >= 0) {
> 208 BSDSocket* s = new BSDSocket(afd);
> 209 s->localname = localname;
> 210 return s;
> 211 }
> 212 else if (errno == EAGAIN)
> 213 return 0;
> 214 else throw QPID_POSIX_ERROR(errno);
> 215 }
>
> It appears that increasing the maximum number of open file handles for the
> qpid user has not solved the problem. It seems that others on the email list
> have experienced this probem, but we haven't seen a solution. Any ideas?
>
>
>
> On Mon, Jun 17, 2013 at 9:07 AM, zhumin <zh...@git.com.cn> wrote:
>
> > I also meet it when i use SSL C++ client to connect qpid server.
> >
> >
> >
> >
> > --
> > View this message in context:
> > http://qpid.2158936.n2.nabble.com/Problem-connecting-No-protocol-
> recei
> > ved-closing-tp7594171p7594201.html
> > Sent from the Apache Qpid users mailing list archive at Nabble.com.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For
> > additional commands, e-mail: users-help@qpid.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org