You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Ul...@t-systems.com on 2014/07/25 12:37:22 UTC

[users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Hi,

we have apaches as reverse proxies for some tomcat and wso2-application servers.

We get this error message on a high-load-webserver:

[proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

And we have no clue, which resource is unavailable and what to do against it.

Apache-2.4.10 / apr-1.5.1

Ulimit of apache-user:
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 46666
max locked memory       (kbytes, -l) 655360
max memory size         (kbytes, -m) unlimited
open files                      (-n) 655360
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 655360
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

$ cat /proc/sys/kernel/threads-max
2065067

=>      So, I don't see any visible shortage of anything.

Any ideas how to find the reason for this error ?

Uli




AW: [users@httpd] AW: error reading response

Posted by Ul...@t-systems.com.
Thank you for your answer:

-          No quota

-          no issue with inodes

So I don't think, we have a "real" storage issue.

Uli


T-Systems International GmbH
Production / CSS / GCU Automotive & Finance
Ulrich Herbst
Fachbereich Automotive / Daimler AS Open Systems
Head of Application Services Central
Bannwaldallee 42, D-76185 Karlsruhe
+49 721 990-2342 (Tel.)
+49 151 16 20 73 13 (Mobil)
E-Mail: Ulrich.Herbst@t-systems.com<ma...@t-systems.com>
Internet: http://www.t-systems.de

Die gesetzlichen Pflichtangaben finden Sie unter:
http://www.t-systems.de/pflichtangaben

Von: Bremser, Kurt (AMOS Austria GmbH) [mailto:Kurt.Bremser@allianz.at]
Gesendet: Freitag, 25. Juli 2014 13:01
An: users@httpd.apache.org
Betreff: AW: [users@httpd] AW: error reading response

Could a quota be involved, or a native file size limit imposed by the file system?

Kurt Bremser
AMOS Austria
________________________________
Von: Ulrich.Herbst@t-systems.com<ma...@t-systems.com> [Ulrich.Herbst@t-systems.com]
Gesendet: Freitag, 25. Juli 2014 12:53
An: users@httpd.apache.org<ma...@httpd.apache.org>
Betreff: [users@httpd] AW: [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response [wd-vc]
Additionally we get sometimes this message:

[Fri Jul 25 12:52:18.203975 2014] [proxy_http:error] [pid 13256:tid 47012293342976] (28)No space left on device: [client 172.31.0.224:20507] AH01110: error reading response

And there is plenty of room in all filesystems. So maybe this is misleading and we have some issues with shared memory too small ? (just my wild guess)

Uli

Von: Herbst, Ulrich
Gesendet: Freitag, 25. Juli 2014 12:37
An: users@httpd.apache.org<ma...@httpd.apache.org>
Betreff: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Hi,

we have apaches as reverse proxies for some tomcat and wso2-application servers.

We get this error message on a high-load-webserver:

[proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

And we have no clue, which resource is unavailable and what to do against it.

Apache-2.4.10 / apr-1.5.1

Ulimit of apache-user:
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 46666
max locked memory       (kbytes, -l) 655360
max memory size         (kbytes, -m) unlimited
open files                      (-n) 655360
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 655360
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

$ cat /proc/sys/kernel/threads-max
2065067


 *   So, I don't see any visible shortage of anything.

Any ideas how to find the reason for this error ?

Uli




AMOS Austria GmbH
1130 Wien, Hietzinger Kai 101-105
FN 365014k, Handelsgericht Wien
UID: ATU 66614737

http://www.allianz.at

********************************************************
Dieses E-Mail und allfaellig daran angeschlossene Anhaenge
enthalten Informationen, die vertraulich und
ausschliesslich fuer den (die) bezeichneten Adressaten
bestimmt sind.
Wenn Sie nicht der genannte Adressat sind, darf dieses
E-Mail samt allfaelliger Anhaenge von Ihnen weder anderen
Personen zugaenglich gemacht noch in anderer Weise
verwertet werden.
Wenn Sie nicht der beabsichtigte Empfaenger sind, bitten
wir Sie, dieses E-Mail und saemtliche angeschlossene
Anhaenge zu loeschen.

Please note: This email and any files transmitted with it is
intended only for the named recipients and may contain
confidential and/or privileged information. If you are not the
intended recipient, please do not read, copy, use or disclose
the contents of this communication to others and notify the
sender immediately. Then please delete the email and any
copies of it. Thank you.
********************************************************

AW: [users@httpd] AW: error reading response

Posted by "Bremser, Kurt (AMOS Austria GmbH)" <Ku...@allianz.at>.
Could a quota be involved, or a native file size limit imposed by the file system?

Kurt Bremser
AMOS Austria
________________________________
Von: Ulrich.Herbst@t-systems.com [Ulrich.Herbst@t-systems.com]
Gesendet: Freitag, 25. Juli 2014 12:53
An: users@httpd.apache.org
Betreff: [users@httpd] AW: [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response [wd-vc]

Additionally we get sometimes this message:

[Fri Jul 25 12:52:18.203975 2014] [proxy_http:error] [pid 13256:tid 47012293342976] (28)No space left on device: [client 172.31.0.224:20507] AH01110: error reading response

And there is plenty of room in all filesystems. So maybe this is misleading and we have some issues with shared memory too small ? (just my wild guess)

Uli

Von: Herbst, Ulrich
Gesendet: Freitag, 25. Juli 2014 12:37
An: users@httpd.apache.org
Betreff: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Hi,

we have apaches as reverse proxies for some tomcat and wso2-application servers.

We get this error message on a high-load-webserver:

[proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

And we have no clue, which resource is unavailable and what to do against it.

Apache-2.4.10 / apr-1.5.1

Ulimit of apache-user:
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 46666
max locked memory       (kbytes, -l) 655360
max memory size         (kbytes, -m) unlimited
open files                      (-n) 655360
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 655360
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

$ cat /proc/sys/kernel/threads-max
2065067


 *   So, I don’t see any visible shortage of anything.

Any ideas how to find the reason for this error ?

Uli




AMOS Austria GmbH 
1130 Wien, Hietzinger Kai 101-105 
FN 365014k, Handelsgericht Wien 
UID: ATU 66614737 

http://www.allianz.at 

******************************************************** 
Dieses E-Mail und allfaellig daran angeschlossene Anhaenge 
enthalten Informationen, die vertraulich und 
ausschliesslich fuer den (die) bezeichneten Adressaten 
bestimmt sind. 
Wenn Sie nicht der genannte Adressat sind, darf dieses 
E-Mail samt allfaelliger Anhaenge von Ihnen weder anderen 
Personen zugaenglich gemacht noch in anderer Weise 
verwertet werden.
Wenn Sie nicht der beabsichtigte Empfaenger sind, bitten
wir Sie, dieses E-Mail und saemtliche angeschlossene
Anhaenge zu loeschen. 

Please note: This email and any files transmitted with it is 
intended only for the named recipients and may contain 
confidential and/or privileged information. If you are not the 
intended recipient, please do not read, copy, use or disclose 
the contents of this communication to others and notify the 
sender immediately. Then please delete the email and any 
copies of it. Thank you.
********************************************************

[users@httpd] AW: [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Posted by Ul...@t-systems.com.
Additionally we get sometimes this message:

[Fri Jul 25 12:52:18.203975 2014] [proxy_http:error] [pid 13256:tid 47012293342976] (28)No space left on device: [client 172.31.0.224:20507] AH01110: error reading response

And there is plenty of room in all filesystems. So maybe this is misleading and we have some issues with shared memory too small ? (just my wild guess)

Uli

Von: Herbst, Ulrich
Gesendet: Freitag, 25. Juli 2014 12:37
An: users@httpd.apache.org
Betreff: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Hi,

we have apaches as reverse proxies for some tomcat and wso2-application servers.

We get this error message on a high-load-webserver:

[proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

And we have no clue, which resource is unavailable and what to do against it.

Apache-2.4.10 / apr-1.5.1

Ulimit of apache-user:
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 46666
max locked memory       (kbytes, -l) 655360
max memory size         (kbytes, -m) unlimited
open files                      (-n) 655360
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 655360
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

$ cat /proc/sys/kernel/threads-max
2065067


 *   So, I don't see any visible shortage of anything.

Any ideas how to find the reason for this error ?

Uli




AW: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Posted by Ul...@t-systems.com.
Running out of shm?

=> Yes, possible. But: How to check this ?

Ulimit -l and ulimit -u show nothing interesting (that is: The usage (ulimit -u) is much less than the limits from ulimit -l)

Uli


T-Systems International GmbH
Production / CSS / GCU Automotive & Finance
Ulrich Herbst
Fachbereich Automotive / Daimler AS Open Systems
Head of Application Services Central
Bannwaldallee 42, D-76185 Karlsruhe
+49 721 990-2342 (Tel.)
+49 151 16 20 73 13 (Mobil)
E-Mail: Ulrich.Herbst@t-systems.com
Internet: http://www.t-systems.de

Die gesetzlichen Pflichtangaben finden Sie unter:
http://www.t-systems.de/pflichtangaben


-----Ursprüngliche Nachricht-----
Von: Tom Evans [mailto:tevans.uk@googlemail.com] 
Gesendet: Freitag, 25. Juli 2014 16:50
An: users@httpd.apache.org
Betreff: Re: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

On Fri, Jul 25, 2014 at 11:37 AM,  <Ul...@t-systems.com> wrote:
> Hi,
>
> we have apaches as reverse proxies for some tomcat and 
> wso2-application servers.
>
> We get this error message on a high-load-webserver:
>
> [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource 
> temporarily
> unavailable: [client 172.31.0.142:21815] AH01110: error reading 
> response
>
> And we have no clue, which resource is unavailable and what to do 
> against it.
>
> Apache-2.4.10 / apr-1.5.1
>
> Ulimit of apache-user:
> core file size          (blocks, -c) 0
> data seg size           (kbytes, -d) unlimited
> scheduling priority             (-e) 0
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 46666
> max locked memory       (kbytes, -l) 655360
> max memory size         (kbytes, -m) unlimited
> open files                      (-n) 655360
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) unlimited
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 655360
> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
>
> $ cat /proc/sys/kernel/threads-max
> 2065067
>
>
> So, I don’t see any visible shortage of anything.
>
>
> Any ideas how to find the reason for this error ?
>
> Uli
>

Running out of shm?

How are you proxying to tomcat (http/ajp/jk2)?

Cheers

Tom

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

AW: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Posted by Ul...@t-systems.com.
Or inotify perhaps:

http://unix.stackexchange.com/questions/65278/httpd-no-space-left-on-device

=> Not sure about this: We had never issues with tail...

But nevertheless: I changed these values and we just wait for new errors.

Uli

-----Ursprüngliche Nachricht-----
Von: Tom Evans [mailto:tevans.uk@googlemail.com] 
Gesendet: Freitag, 25. Juli 2014 16:51
An: users@httpd.apache.org
Betreff: Re: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

On Fri, Jul 25, 2014 at 3:49 PM, Tom Evans <te...@googlemail.com> wrote:
> On Fri, Jul 25, 2014 at 11:37 AM,  <Ul...@t-systems.com> wrote:
>> Hi,
>>
>> we have apaches as reverse proxies for some tomcat and 
>> wso2-application servers.
>>
>> We get this error message on a high-load-webserver:
>>
>> [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource 
>> temporarily
>> unavailable: [client 172.31.0.142:21815] AH01110: error reading 
>> response
>>
>> And we have no clue, which resource is unavailable and what to do 
>> against it.
>>
>> Apache-2.4.10 / apr-1.5.1
>>
>> Ulimit of apache-user:
>> core file size          (blocks, -c) 0
>> data seg size           (kbytes, -d) unlimited
>> scheduling priority             (-e) 0
>> file size               (blocks, -f) unlimited
>> pending signals                 (-i) 46666
>> max locked memory       (kbytes, -l) 655360
>> max memory size         (kbytes, -m) unlimited
>> open files                      (-n) 655360
>> pipe size            (512 bytes, -p) 8
>> POSIX message queues     (bytes, -q) 819200
>> real-time priority              (-r) 0
>> stack size              (kbytes, -s) unlimited
>> cpu time               (seconds, -t) unlimited
>> max user processes              (-u) 655360
>> virtual memory          (kbytes, -v) unlimited
>> file locks                      (-x) unlimited
>>
>> $ cat /proc/sys/kernel/threads-max
>> 2065067
>>
>>
>> So, I don’t see any visible shortage of anything.
>>
>>
>> Any ideas how to find the reason for this error ?
>>
>> Uli
>>
>
> Running out of shm?
>
> How are you proxying to tomcat (http/ajp/jk2)?
>

Or inotify perhaps:

http://unix.stackexchange.com/questions/65278/httpd-no-space-left-on-device

Cheers

Tom

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Posted by Tom Evans <te...@googlemail.com>.
On Fri, Jul 25, 2014 at 3:49 PM, Tom Evans <te...@googlemail.com> wrote:
> On Fri, Jul 25, 2014 at 11:37 AM,  <Ul...@t-systems.com> wrote:
>> Hi,
>>
>> we have apaches as reverse proxies for some tomcat and wso2-application
>> servers.
>>
>> We get this error message on a high-load-webserver:
>>
>> [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily
>> unavailable: [client 172.31.0.142:21815] AH01110: error reading response
>>
>> And we have no clue, which resource is unavailable and what to do against
>> it.
>>
>> Apache-2.4.10 / apr-1.5.1
>>
>> Ulimit of apache-user:
>> core file size          (blocks, -c) 0
>> data seg size           (kbytes, -d) unlimited
>> scheduling priority             (-e) 0
>> file size               (blocks, -f) unlimited
>> pending signals                 (-i) 46666
>> max locked memory       (kbytes, -l) 655360
>> max memory size         (kbytes, -m) unlimited
>> open files                      (-n) 655360
>> pipe size            (512 bytes, -p) 8
>> POSIX message queues     (bytes, -q) 819200
>> real-time priority              (-r) 0
>> stack size              (kbytes, -s) unlimited
>> cpu time               (seconds, -t) unlimited
>> max user processes              (-u) 655360
>> virtual memory          (kbytes, -v) unlimited
>> file locks                      (-x) unlimited
>>
>> $ cat /proc/sys/kernel/threads-max
>> 2065067
>>
>>
>> So, I don’t see any visible shortage of anything.
>>
>>
>> Any ideas how to find the reason for this error ?
>>
>> Uli
>>
>
> Running out of shm?
>
> How are you proxying to tomcat (http/ajp/jk2)?
>

Or inotify perhaps:

http://unix.stackexchange.com/questions/65278/httpd-no-space-left-on-device

Cheers

Tom

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Fwd: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Posted by Eric Covener <co...@gmail.com>.
I didn't consider this at all.  Should we remove the default of
LimitRequestBody if it is operating on a PROXYREQ_REVERSE?


---------- Forwarded message ----------
From: Jeff Trawick <tr...@gmail.com>
Date: Fri, Jul 25, 2014 at 11:10 AM
Subject: Re: [users@httpd] [proxy_http:error] [pid 13256:tid
47013272524544] (11)Resource temporarily unavailable: [client
172.31.0.142:21815] AH01110: error reading response
To: "users@httpd.apache.org" <us...@httpd.apache.org>


On Fri, Jul 25, 2014 at 10:49 AM, Tom Evans <te...@googlemail.com> wrote:
>
> On Fri, Jul 25, 2014 at 11:37 AM,  <Ul...@t-systems.com> wrote:
> > Hi,
> >
> > we have apaches as reverse proxies for some tomcat and wso2-application
> > servers.
> >
> > We get this error message on a high-load-webserver:
> >
> > [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily
> > unavailable: [client 172.31.0.142:21815] AH01110: error reading response
> >
> > And we have no clue, which resource is unavailable and what to do against
> > it.
> >
> > Apache-2.4.10 / apr-1.5.1
> >
> > Ulimit of apache-user:
> > core file size          (blocks, -c) 0
> > data seg size           (kbytes, -d) unlimited
> > scheduling priority             (-e) 0
> > file size               (blocks, -f) unlimited
> > pending signals                 (-i) 46666
> > max locked memory       (kbytes, -l) 655360
> > max memory size         (kbytes, -m) unlimited
> > open files                      (-n) 655360
> > pipe size            (512 bytes, -p) 8
> > POSIX message queues     (bytes, -q) 819200
> > real-time priority              (-r) 0
> > stack size              (kbytes, -s) unlimited
> > cpu time               (seconds, -t) unlimited
> > max user processes              (-u) 655360
> > virtual memory          (kbytes, -v) unlimited
> > file locks                      (-x) unlimited
> >
> > $ cat /proc/sys/kernel/threads-max
> > 2065067
> >
> >
> > So, I don’t see any visible shortage of anything.
> >
> >
> > Any ideas how to find the reason for this error ?
> >
> > Uli
> >
>
> Running out of shm?
>
> How are you proxying to tomcat (http/ajp/jk2)?


The message comes from mod_proxy_http, so "http" it is.

Looking at how to get the ENOSPC error in that particular context, I
see that several pieces of code can return ENOSPC based on exceeding
some configured limit, but most don't apply to the path where that
message is issued.

Is it possible that mod_deflate is configured, perhaps unexpectedly,
to decompress the response from the backend via an input filter?

See http://httpd.apache.org/docs/current/mod/mod_deflate.html#deflateinflatelimitrequestbody
 (new with 2.4.10)  That is part of a fix to vulnerability
CVE-2014-0118.


>
>
> Cheers
>
> Tom
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>



--
Born in Roswell... married an alien...
http://emptyhammock.com/
http://edjective.org/



-- 
Eric Covener
covener@gmail.com

Re: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Posted by Eric Covener <co...@gmail.com>.
Any word on this? Safe to assume it did not occur prior to 2.4.10?  Do
you just have setoutputfilter DEFLATE or something else?


On Sun, Jul 27, 2014 at 8:23 AM, Jeff Trawick <tr...@gmail.com> wrote:
> On Sat, Jul 26, 2014 at 10:58 AM, <Ul...@t-systems.com> wrote:
>>
>> Hi,
>>
>>
>>
>> Is it possible that mod_deflate is configured, perhaps unexpectedly, to
>> decompress the response from the backend via an input filter?
>>
>>
>>
>>
>>
>> ð  Yes, for sure. mod_deflate is enabled and we compress the
>> (uncompressed) response from the backend before sending to the clients.
>>
>>
>>
>> See
>> http://httpd.apache.org/docs/current/mod/mod_deflate.html#deflateinflatelimitrequestbody
>> (new with 2.4.10)
>>
>>
>>
>> ð  So, you think, that our compressed body is too large sometimes, and we
>> got “no space” because of this ?
>
>
> I think it is possible to get that error with proxy when a compressed
> response from a backend server is relatively large.
>
> Are you seeing any warning messages in the log from mod_deflate?  What is
> the log level that applies to mod_deflate?  (Or, what is your LogLevel
> directive if that doesn't make sense to you ;) )  You'd need to see log
> level warn to check this theory on mod_deflate.
>
> Another case is with the http layer returning ENOSPC when exceeding a
> similar limit when the body is not compressed.  The log level for http must
> log info messages.
>
> If your log level is higher than info (e.g., error), try something like
>
> LogLevel error http:info deflate:warn
>
> to see log messages that could pinpoint exactly where the error comes from.
>>
>>
>>
>> Uli
>>
>>
>>
>>
>>
>> Von: Jeff Trawick [mailto:trawick@gmail.com]
>> Gesendet: Freitag, 25. Juli 2014 17:10
>> An: users@httpd.apache.org
>> Betreff: Re: [users@httpd] [proxy_http:error] [pid 13256:tid
>> 47013272524544] (11)Resource temporarily unavailable: [client
>> 172.31.0.142:21815] AH01110: error reading response
>>
>>
>>
>> On Fri, Jul 25, 2014 at 10:49 AM, Tom Evans <te...@googlemail.com>
>> wrote:
>>
>> On Fri, Jul 25, 2014 at 11:37 AM,  <Ul...@t-systems.com> wrote:
>> > Hi,
>> >
>> > we have apaches as reverse proxies for some tomcat and wso2-application
>> > servers.
>> >
>> > We get this error message on a high-load-webserver:
>> >
>> > [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource
>> > temporarily
>> > unavailable: [client 172.31.0.142:21815] AH01110: error reading response
>> >
>> > And we have no clue, which resource is unavailable and what to do
>> > against
>> > it.
>> >
>> > Apache-2.4.10 / apr-1.5.1
>> >
>> > Ulimit of apache-user:
>> > core file size          (blocks, -c) 0
>> > data seg size           (kbytes, -d) unlimited
>> > scheduling priority             (-e) 0
>> > file size               (blocks, -f) unlimited
>> > pending signals                 (-i) 46666
>> > max locked memory       (kbytes, -l) 655360
>> > max memory size         (kbytes, -m) unlimited
>> > open files                      (-n) 655360
>> > pipe size            (512 bytes, -p) 8
>> > POSIX message queues     (bytes, -q) 819200
>> > real-time priority              (-r) 0
>> > stack size              (kbytes, -s) unlimited
>> > cpu time               (seconds, -t) unlimited
>> > max user processes              (-u) 655360
>> > virtual memory          (kbytes, -v) unlimited
>> > file locks                      (-x) unlimited
>> >
>> > $ cat /proc/sys/kernel/threads-max
>> > 2065067
>> >
>> >
>> > So, I don’t see any visible shortage of anything.
>> >
>> >
>> > Any ideas how to find the reason for this error ?
>> >
>> > Uli
>> >
>>
>> Running out of shm?
>>
>> How are you proxying to tomcat (http/ajp/jk2)?
>>
>>
>>
>> The message comes from mod_proxy_http, so "http" it is.
>>
>>
>>
>> Looking at how to get the ENOSPC error in that particular context, I see
>> that several pieces of code can return ENOSPC based on exceeding some
>> configured limit, but most don't apply to the path where that message is
>> issued.
>>
>>
>>
>> Is it possible that mod_deflate is configured, perhaps unexpectedly, to
>> decompress the response from the backend via an input filter?
>>
>>
>>
>> See
>> http://httpd.apache.org/docs/current/mod/mod_deflate.html#deflateinflatelimitrequestbody
>> (new with 2.4.10)  That is part of a fix to vulnerability CVE-2014-0118.
>>
>>
>>
>>
>>
>>
>> Cheers
>>
>> Tom
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>> For additional commands, e-mail: users-help@httpd.apache.org
>>
>>
>>
>>
>>
>> --
>>
>> Born in Roswell... married an alien...
>> http://emptyhammock.com/
>>
>> http://edjective.org/
>>
>>
>
>
>
>
> --
> Born in Roswell... married an alien...
> http://emptyhammock.com/
> http://edjective.org/
>



-- 
Eric Covener
covener@gmail.com

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Posted by Jeff Trawick <tr...@gmail.com>.
On Sat, Jul 26, 2014 at 10:58 AM, <Ul...@t-systems.com> wrote:

> Hi,
>
>
>
> Is it possible that mod_deflate is configured, perhaps unexpectedly, to
> decompress the response from the backend via an input filter?
>
>
>
>
>
> ð  Yes, for sure. mod_deflate is enabled and we compress the
> (uncompressed) response from the backend before sending to the clients.
>
>
>
> See
> http://httpd.apache.org/docs/current/mod/mod_deflate.html#deflateinflatelimitrequestbody
>  (new with 2.4.10)
>
>
>
> ð  So, you think, that our compressed body is too large sometimes, and we
> got “no space” because of this ?
>

I think it is possible to get that error with proxy when a compressed
response from a backend server is relatively large.

Are you seeing any warning messages in the log from mod_deflate?  What is
the log level that applies to mod_deflate?  (Or, what is your LogLevel
directive if that doesn't make sense to you ;) )  You'd need to see log
level warn to check this theory on mod_deflate.

Another case is with the http layer returning ENOSPC when exceeding a
similar limit when the body is not compressed.  The log level for http must
log info messages.

If your log level is higher than info (e.g., error), try something like

LogLevel error http:info deflate:warn

to see log messages that could pinpoint exactly where the error comes from.

>
>
> Uli
>
>
>
>
>
> *Von:* Jeff Trawick [mailto:trawick@gmail.com]
> *Gesendet:* Freitag, 25. Juli 2014 17:10
> *An:* users@httpd.apache.org
> *Betreff:* Re: [users@httpd] [proxy_http:error] [pid 13256:tid
> 47013272524544] (11)Resource temporarily unavailable: [client
> 172.31.0.142:21815] AH01110: error reading response
>
>
>
> On Fri, Jul 25, 2014 at 10:49 AM, Tom Evans <te...@googlemail.com>
> wrote:
>
> On Fri, Jul 25, 2014 at 11:37 AM,  <Ul...@t-systems.com> wrote:
> > Hi,
> >
> > we have apaches as reverse proxies for some tomcat and wso2-application
> > servers.
> >
> > We get this error message on a high-load-webserver:
> >
> > [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource
> temporarily
> > unavailable: [client 172.31.0.142:21815] AH01110: error reading response
> >
> > And we have no clue, which resource is unavailable and what to do against
> > it.
> >
> > Apache-2.4.10 / apr-1.5.1
> >
> > Ulimit of apache-user:
> > core file size          (blocks, -c) 0
> > data seg size           (kbytes, -d) unlimited
> > scheduling priority             (-e) 0
> > file size               (blocks, -f) unlimited
> > pending signals                 (-i) 46666
> > max locked memory       (kbytes, -l) 655360
> > max memory size         (kbytes, -m) unlimited
> > open files                      (-n) 655360
> > pipe size            (512 bytes, -p) 8
> > POSIX message queues     (bytes, -q) 819200
> > real-time priority              (-r) 0
> > stack size              (kbytes, -s) unlimited
> > cpu time               (seconds, -t) unlimited
> > max user processes              (-u) 655360
> > virtual memory          (kbytes, -v) unlimited
> > file locks                      (-x) unlimited
> >
> > $ cat /proc/sys/kernel/threads-max
> > 2065067
> >
> >
> > So, I don’t see any visible shortage of anything.
> >
> >
> > Any ideas how to find the reason for this error ?
> >
> > Uli
> >
>
> Running out of shm?
>
> How are you proxying to tomcat (http/ajp/jk2)?
>
>
>
> The message comes from mod_proxy_http, so "http" it is.
>
>
>
> Looking at how to get the ENOSPC error in that particular context, I see
> that several pieces of code can return ENOSPC based on exceeding some
> configured limit, but most don't apply to the path where that message is
> issued.
>
>
>
> Is it possible that mod_deflate is configured, perhaps unexpectedly, to
> decompress the response from the backend via an input filter?
>
>
>
> See
> http://httpd.apache.org/docs/current/mod/mod_deflate.html#deflateinflatelimitrequestbody
>  (new with 2.4.10)  That is part of a fix to vulnerability CVE-2014-0118.
>
>
>
>
>
>
> Cheers
>
> Tom
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>
>
>
> --
>
> Born in Roswell... married an alien...
> http://emptyhammock.com/
>
> http://edjective.org/
>
>
>



-- 
Born in Roswell... married an alien...
http://emptyhammock.com/
http://edjective.org/

AW: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Posted by Ul...@t-systems.com.
Hi,

Is it possible that mod_deflate is configured, perhaps unexpectedly, to decompress the response from the backend via an input filter?



ð  Yes, for sure. mod_deflate is enabled and we compress the (uncompressed) response from the backend before sending to the clients.

See http://httpd.apache.org/docs/current/mod/mod_deflate.html#deflateinflatelimitrequestbody  (new with 2.4.10)


ð  So, you think, that our compressed body is too large sometimes, and we got “no space” because of this ?

Uli


Von: Jeff Trawick [mailto:trawick@gmail.com]
Gesendet: Freitag, 25. Juli 2014 17:10
An: users@httpd.apache.org
Betreff: Re: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

On Fri, Jul 25, 2014 at 10:49 AM, Tom Evans <te...@googlemail.com>> wrote:
On Fri, Jul 25, 2014 at 11:37 AM,  <Ul...@t-systems.com>> wrote:
> Hi,
>
> we have apaches as reverse proxies for some tomcat and wso2-application
> servers.
>
> We get this error message on a high-load-webserver:
>
> [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily
> unavailable: [client 172.31.0.142:21815<http://172.31.0.142:21815>] AH01110: error reading response
>
> And we have no clue, which resource is unavailable and what to do against
> it.
>
> Apache-2.4.10 / apr-1.5.1
>
> Ulimit of apache-user:
> core file size          (blocks, -c) 0
> data seg size           (kbytes, -d) unlimited
> scheduling priority             (-e) 0
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 46666
> max locked memory       (kbytes, -l) 655360
> max memory size         (kbytes, -m) unlimited
> open files                      (-n) 655360
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) unlimited
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 655360
> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
>
> $ cat /proc/sys/kernel/threads-max
> 2065067
>
>
> So, I don’t see any visible shortage of anything.
>
>
> Any ideas how to find the reason for this error ?
>
> Uli
>
Running out of shm?

How are you proxying to tomcat (http/ajp/jk2)?

The message comes from mod_proxy_http, so "http" it is.

Looking at how to get the ENOSPC error in that particular context, I see that several pieces of code can return ENOSPC based on exceeding some configured limit, but most don't apply to the path where that message is issued.

Is it possible that mod_deflate is configured, perhaps unexpectedly, to decompress the response from the backend via an input filter?

See http://httpd.apache.org/docs/current/mod/mod_deflate.html#deflateinflatelimitrequestbody  (new with 2.4.10)  That is part of a fix to vulnerability CVE-2014-0118.



Cheers

Tom

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org<ma...@httpd.apache.org>
For additional commands, e-mail: users-help@httpd.apache.org<ma...@httpd.apache.org>



--
Born in Roswell... married an alien...
http://emptyhammock.com/
http://edjective.org/


Re: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Posted by Jeff Trawick <tr...@gmail.com>.
On Fri, Jul 25, 2014 at 10:49 AM, Tom Evans <te...@googlemail.com>
wrote:

> On Fri, Jul 25, 2014 at 11:37 AM,  <Ul...@t-systems.com> wrote:
> > Hi,
> >
> > we have apaches as reverse proxies for some tomcat and wso2-application
> > servers.
> >
> > We get this error message on a high-load-webserver:
> >
> > [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource
> temporarily
> > unavailable: [client 172.31.0.142:21815] AH01110: error reading response
> >
> > And we have no clue, which resource is unavailable and what to do against
> > it.
> >
> > Apache-2.4.10 / apr-1.5.1
> >
> > Ulimit of apache-user:
> > core file size          (blocks, -c) 0
> > data seg size           (kbytes, -d) unlimited
> > scheduling priority             (-e) 0
> > file size               (blocks, -f) unlimited
> > pending signals                 (-i) 46666
> > max locked memory       (kbytes, -l) 655360
> > max memory size         (kbytes, -m) unlimited
> > open files                      (-n) 655360
> > pipe size            (512 bytes, -p) 8
> > POSIX message queues     (bytes, -q) 819200
> > real-time priority              (-r) 0
> > stack size              (kbytes, -s) unlimited
> > cpu time               (seconds, -t) unlimited
> > max user processes              (-u) 655360
> > virtual memory          (kbytes, -v) unlimited
> > file locks                      (-x) unlimited
> >
> > $ cat /proc/sys/kernel/threads-max
> > 2065067
> >
> >
> > So, I don’t see any visible shortage of anything.
> >
> >
> > Any ideas how to find the reason for this error ?
> >
> > Uli
> >
>
> Running out of shm?
>
> How are you proxying to tomcat (http/ajp/jk2)?
>

The message comes from mod_proxy_http, so "http" it is.

Looking at how to get the ENOSPC error in that particular context, I see
that several pieces of code can return ENOSPC based on exceeding some
configured limit, but most don't apply to the path where that message is
issued.

Is it possible that mod_deflate is configured, perhaps unexpectedly, to
decompress the response from the backend via an input filter?

See
http://httpd.apache.org/docs/current/mod/mod_deflate.html#deflateinflatelimitrequestbody
 (new with 2.4.10)  That is part of a fix to vulnerability CVE-2014-0118.



>
> Cheers
>
> Tom
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>


-- 
Born in Roswell... married an alien...
http://emptyhammock.com/
http://edjective.org/

Re: [users@httpd] [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily unavailable: [client 172.31.0.142:21815] AH01110: error reading response

Posted by Tom Evans <te...@googlemail.com>.
On Fri, Jul 25, 2014 at 11:37 AM,  <Ul...@t-systems.com> wrote:
> Hi,
>
> we have apaches as reverse proxies for some tomcat and wso2-application
> servers.
>
> We get this error message on a high-load-webserver:
>
> [proxy_http:error] [pid 13256:tid 47013272524544] (11)Resource temporarily
> unavailable: [client 172.31.0.142:21815] AH01110: error reading response
>
> And we have no clue, which resource is unavailable and what to do against
> it.
>
> Apache-2.4.10 / apr-1.5.1
>
> Ulimit of apache-user:
> core file size          (blocks, -c) 0
> data seg size           (kbytes, -d) unlimited
> scheduling priority             (-e) 0
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 46666
> max locked memory       (kbytes, -l) 655360
> max memory size         (kbytes, -m) unlimited
> open files                      (-n) 655360
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) unlimited
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 655360
> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
>
> $ cat /proc/sys/kernel/threads-max
> 2065067
>
>
> So, I don’t see any visible shortage of anything.
>
>
> Any ideas how to find the reason for this error ?
>
> Uli
>

Running out of shm?

How are you proxying to tomcat (http/ajp/jk2)?

Cheers

Tom

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org