You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Steve Smith <st...@isay.com.au> on 2000/11/17 04:42:43 UTC
Tempfile and send_fd()
Hi,
Could somebody tell me why the following testcase doesn't work?
use Apache ();
use Apache::File ();
my $r = Apache->request();
$r->content_type('text/plain');
$r->send_http_header();
my $f = Apache::File->tmpfile();
print $f "test\ntest\n";
$r->send_fd($f);
$f->close;
All I get is an empty document. My understanding is that the data
written to the tmpfile should be available immediately through the
filehandle even if it hasn't been flushed.
This is running under Registry, on Linux.
Thanks,
Steve
Re: Tempfile and send_fd()
Posted by barries <ba...@slaysys.com>.
On Fri, Nov 17, 2000 at 02:42:43PM +1100, Steve Smith wrote:
> Hi,
>
> All I get is an empty document. My understanding is that the data
> written to the tmpfile should be available immediately through the
> filehandle even if it hasn't been flushed.
I wouldn't bet on it flushing: mod_perl may not have written it out yet.
perl often doesn't flush writes to disk until close. Try adding a
select( (select( $fh ), $| = 1 )[0] ) ;
just after the open and see if that fixes things.
Also, Apache doesn't seek or rewind the FD (see below), so even if it's flushed,
your file position may no longer be at the beginning of the file, depending
on your C RTL. I seem to recall that some require you to seek() between
reads and writes to ensure synchronization.
That lack of a rewind can be nice when you're caching things: you can sysread
some header lines, then turn the rest over to Apache to send.
- Barrie
API_EXPORT(long) ap_send_fd_length(FILE *f, request_rec *r, long length)
{
char buf[IOBUFSIZE];
long total_bytes_sent = 0;
register int n, w, o, len;
if (length == 0)
return 0;
ap_soft_timeout("send body", r);
while (!r->connection->aborted) {
if ((length > 0) && (total_bytes_sent + IOBUFSIZE) > length)
len = length - total_bytes_sent;
else
len = IOBUFSIZE;
while ((n = fread(buf, sizeof(char), len, f)) < 1
&& ferror(f) && errno == EINTR && !r->connection->aborted)
continue;
Re: Tempfile and send_fd()
Posted by barries <ba...@slaysys.com>.
On Fri, Nov 17, 2000 at 12:32:33AM -0500, barries wrote:
> On Fri, Nov 17, 2000 at 03:51:35PM +1100, Steve Smith wrote:
> >
> > seek $f, 0, 0;
> ^^^^
>
> Had a look in Apache::File (below), and it sysopens, so you might want
> to sysseek(...) instead.
No, nevermind, don't: sysseek won't flush the buffer:
strace perl -we 'use Fcntl ; sysopen F, "foo", O_RDWR | O_CREAT or die $! ; print F "hi1\n" ; sysseek F, 0, 0 ; sleep 5 ; print F "hi2\n" ; close F'
open("foo", O_RDWR|O_CREAT|0x8000, 0666) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|0x8000)
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40018000
_llseek(0x3, 0, 0, 0xbffff66c, 0x1) = 0
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
_llseek(0x3, 0, 0, 0xbffff8f8, 0) = 0
time([974439311]) = 974439311
SYS_175(0, 0xbffff8c0, 0xbffff840, 0x8, 0) = 0
SYS_174(0x11, 0, 0xbffff644, 0x8, 0x11) = 0
SYS_175(0x2, 0xbffff840, 0, 0x8, 0x2) = 0
nanosleep(0xbffff7ac, 0xbffff7ac, 0x401791b4, 0xbffff7ac, 0xbffff8c0) = 0
time([974439316]) = 974439316
write(3, "hi1\nhi2\n", 8) = 8
close(3) = 0
Re: Tempfile and send_fd()
Posted by barries <ba...@slaysys.com>.
On Fri, Nov 17, 2000 at 03:51:03PM +0100, Roger Espel Llima wrote:
>
> Internally sysopen() does a plain libc
> open(), but then it follows with a fdopen() call to get a stdio handle
> anyway.
That explains the results I saw, thanks.
- Barrie
Re: Tempfile and send_fd()
Posted by Roger Espel Llima <es...@iagora.net>.
barries <ba...@slaysys.com> wrote:
> On Fri, Nov 17, 2000 at 03:51:35PM +1100, Steve Smith wrote:
> > seek $f, 0, 0;
>
> Had a look in Apache::File (below), and it sysopens, so you might want
> to sysseek(...) instead.
Just to clear up one thing: sysopen() doesn't actually belong to the
sys* family of functions (syswrite, sysread, sysseek). File handles
under perl are buffered by default, whether you open them with perl's
open() or with sysopen(). Internally sysopen() does a plain libc
open(), but then it follows with a fdopen() call to get a stdio handle
anyway.
Example (Linux 2.2):
$ strace -e trace=open,write,close,nanosleep perl -MFcntl -e 'sysopen(FH, "/tmp/bla", O_WRONLY|O_CREAT); print FH "hey"; sleep 1; print FH "hoy\n"; close FH;'
[ ... ]
open("/tmp/bla", O_WRONLY|O_CREAT|O_LARGEFILE, 0666) = 3
nanosleep({1, 0}, {1, 0}) = 0
write(3, "heyhoy\n", 7) = 7
close(3) = 0
So the rule of thumb is: use either sysread/syswrite/sysseek, or
read/<>/print/seek, but don't mix the two sets on the same filehandle.
But you can open your filehandles with open(), sysopen() or any of the
OO file packages, as you see fit; both sets of IO functions will work on
a filehandle no matter how it was opened.
Personally, I don't like the magic parsing in open(), so I tend to use
sysopen() always, in combination with the usual read/<>/print/seek.
--
Roger Espel Llima, espel@iagora.net
http://www.iagora.com/~espel/index.html
Re: Tempfile and send_fd()
Posted by barries <ba...@slaysys.com>.
On Fri, Nov 17, 2000 at 03:51:35PM +1100, Steve Smith wrote:
> "Steve" == Steve Smith <st...@isay.com.au> writes:
> > Hi, Could somebody tell me why the following testcase doesn't work?
>
> <snip>
>
> Nevermind, I got it from the archives eventually :
>
> seek $f, 0, 0;
^^^^
Had a look in Apache::File (below), and it sysopens, so you might want
to sysseek(...) instead.
That seek is flushing the filehandle for you though, here's without the
seek (on Linux, YMMV):
strace perl -we 'use Fcntl ; sysopen F, "foo", O_RDWR | O_CREAT or die $! ; print F "hi1\n" ; sleep 5 ; print F "hi2\n" ; close F'
[snip]
open("foo", O_RDWR|O_CREAT|0x8000, 0666) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|0x8000)
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) =
0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x400180
00
_llseek(0x3, 0, 0, 0xbffff67c, 0x1) = 0
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) =
0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
time([974438839]) = 974438839
SYS_175(0, 0xbffff8d0, 0xbffff850, 0x8, 0) = 0
SYS_174(0x11, 0, 0xbffff654, 0x8, 0x11) = 0
SYS_175(0x2, 0xbffff850, 0, 0x8, 0x2) = 0
nanosleep(0xbffff7bc, 0xbffff7bc, 0x401791b4, 0xbffff7bc, 0xbffff8d0) = 0
time([974438844]) = 974438844
write(3, "hi1\nhi2\n", 8) = 8
close(3) = 0
and here's with the seek():
strace perl -we 'use Fcntl ; sysopen F, "foo", O_RDWR | O_CREAT or die $! ; print F "hi1\n" ; seek F, 0, 0 ; sleep 5 ; print F "hi2\n" ; close F'
open("foo", O_RDWR|O_CREAT|0x8000, 0666) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|0x8000)
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40018000
_llseek(0x3, 0, 0, 0xbffff66c, 0x1) = 0
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
write(3, "hi1\n", 4) = 4
_llseek(0x3, 0, 0, 0xbffff788, 0) = 0
time([974438924]) = 974438924
SYS_175(0, 0xbffff8c0, 0xbffff840, 0x8, 0) = 0
SYS_174(0x11, 0, 0xbffff644, 0x8, 0x11) = 0
SYS_175(0x2, 0xbffff840, 0, 0x8, 0x2) = 0
nanosleep(0xbffff7ac, 0xbffff7ac, 0x401791b4, 0xbffff7ac, 0xbffff8c0) = 0
time([974438929]) = 974438929
write(3, "hi2\n", 4) = 4
close(3) = 0
Calling syswrite bypasses Perl's buffering, but doesn't address the seek
issue:
open("foo", O_RDWR|O_CREAT|0x8000, 0666) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|0x8000)
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40018000
_llseek(0x3, 0, 0, 0xbffff66c, 0x1) = 0
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
write(3, "hi1\n", 4) = 4
time([974439066]) = 974439066
SYS_175(0, 0xbffff8c0, 0xbffff840, 0x8, 0) = 0
SYS_174(0x11, 0, 0xbffff644, 0x8, 0x11) = 0
SYS_175(0x2, 0xbffff840, 0, 0x8, 0x2) = 0
nanosleep(0xbffff7ac, 0xbffff7ac, 0x401791b4, 0xbffff7ac, 0xbffff8c0) = 0
time([974439071]) = 974439071
write(3, "hi2\n", 4) = 4
close(3) = 0
- Barrie
sub tmpfile {
my $class = shift;
my $limit = 100;
my $r = Apache->request;
while($limit--) {
my $tmpfile = "$TMPDIR/${$}" . $TMPNAM++;
my $fh = $class->new;
sysopen($fh, $tmpfile, $Mode, $Perms);
$r->register_cleanup(sub { unlink $tmpfile }) if $r;
if($fh) {
return wantarray ? ($tmpfile,$fh) : $fh;
}
}
}
Re:Tempfile and send_fd()
Posted by Steve Smith <st...@isay.com.au>.
"Steve" == Steve Smith <st...@isay.com.au> writes:
> Hi, Could somebody tell me why the following testcase doesn't work?
<snip>
Nevermind, I got it from the archives eventually :
seek $f, 0, 0;
$r->send_fd($f);
Cheers,
Steve