Broken pipe error on Push

[João Bernardo <jb...@gmail.com>]

Hi,

I have a Python application doing http PUSH on ATS 2.1.7 and, when the file
is big (tested with 105MB mp4 video), the script sometimes accuse "Errno 23:
Broken pipe" after sending some amount of data (something about 90 to 95MB).

Recently I changed the way data is sent to ATS from loading the full file
into memory to getting chunks of data and sending to ATS.

I never saw that happening when loading a file from the local harddrive, but
when the file is a open connection to a server, the "Broken pipe" occurs
very often.

Python snippet:

*socket.send(header)
for chunk in file_like_object:
    socket.send(chunk)  #Error after some time
socket.send(b'\r\n')*

so, every chunk (I'm using 4KB) is downloaded and sent to ATS. And the
action is repeated until the end of the file.

I guess the pauses between sending 4KB of data and downloading more data
from the web server is making ATS close the connection by thinking the
client was idle for too much time...

Any thoughts on what should I do?

Thanks,
João Bernardo

Re: Broken pipe error on Push

[sridhar basam <sr...@basam.org>]

On Mon, Aug 15, 2011 at 10:34 AM, João Bernardo <jb...@gmail.com> wrote:

> Hi,
>
> I still couldn't solve the problem and, if possible, would like to know the
> behavior of ATS on these conditions. Will it close the connection if it
> takes some time for the client to answer? How much time?
>
> My system always works with small files but keep showing this problem with
> big ones (sometimes)...
>
>
Are you on a slow connection? You could run tcpdump to see why the
connection is being torn down but i would guess you are hitting an
inactivity timer.

Look at your records.config file for these values and try adjusting after
you determine if one of these is biting you. Look at transaction_no and
transaction timers since this isn't related to keepalive.

 Sridhar

     ###################################
   # HTTP connection timeouts (secs) #
   ###################################
   # out: proxy -> origin server connection
   # in : ua -> proxy connection
CONFIG proxy.config.http.keep_alive_no_activity_timeout_in INT 115
CONFIG proxy.config.http.keep_alive_no_activity_timeout_out INT 120
CONFIG proxy.config.http.transaction_no_activity_timeout_in INT 30
CONFIG proxy.config.http.transaction_no_activity_timeout_out INT 30
CONFIG proxy.config.http.transaction_active_timeout_in INT 900
CONFIG proxy.config.http.transaction_active_timeout_out INT 0

Re: Broken pipe error on Push

[João Bernardo <jb...@gmail.com>]

Hi,

I still couldn't solve the problem and, if possible, would like to know the
behavior of ATS on these conditions. Will it close the connection if it
takes some time for the client to answer? How much time?

My system always works with small files but keep showing this problem with
big ones (sometimes)...

Thanks,
João Bernardo


2011/8/9 João Bernardo <jb...@gmail.com>

> Hi,
>
> I have a Python application doing http PUSH on ATS 2.1.7 and, when the file
> is big (tested with 105MB mp4 video), the script sometimes accuse "Errno 23:
> Broken pipe" after sending some amount of data (something about 90 to 95MB).
>
> Recently I changed the way data is sent to ATS from loading the full file
> into memory to getting chunks of data and sending to ATS.
>
> I never saw that happening when loading a file from the local harddrive,
> but when the file is a open connection to a server, the "Broken pipe" occurs
> very often.
>
> Python snippet:
>
> *socket.send(header)
> for chunk in file_like_object:
>     socket.send(chunk)  #Error after some time
> socket.send(b'\r\n')*
>
> so, every chunk (I'm using 4KB) is downloaded and sent to ATS. And the
> action is repeated until the end of the file.
>
> I guess the pauses between sending 4KB of data and downloading more data
> from the web server is making ATS close the connection by thinking the
> client was idle for too much time...
>
> Any thoughts on what should I do?
>
> Thanks,
> João Bernardo
>