You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Victor Star <gr...@victorstar.com> on 2006/09/24 04:36:01 UTC
[users@httpd] Corrupted source files
Hi,
I'm new to Apache. I've just installed Apache 2.2.3 on FreeBSD 6.0. Configured it and got it
running. But I'm having something that looks like quite a weird problem. I've done my research on
Google, read FAQ and docs but couldn't find anything even closely resembling my situation.
Description of the problem: when I'm requesting documents like HTML files or images with the browser
- it displays properly the very first time, and after this the original file on the server is
corrupted - filled with some binary garbage. And this garbage is what's displayed on every next
request. Actually this garbage changes from request to request. So what it looks like - that when
Apache grabs the file to send it to the browser, it writes some garbage into it. I've never seen
anything like that before.
Some additional info: when I request php files (served by php_mod) - everything is fine, nothing
gets corrupted.
I would greatly appreciate any hints on where to look for a fix.
Thanks!
--
Best regards,
Victor
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Victor Star <gr...@victorstar.com>.
Hi Joshua,
> Change your Listen directive to a port over 1024 and then strace after
> starting from a non-root account.
I've tried this one as well, finally made it work after fiddling with permissions.
Here is the log (looks the same as the previous one):
====- 8< -===================================================
42061 accept(3, {sa_family=AF_INET6, sin6_port=htons(1697), inet_pton(AF_INET6, "::ffff:10.10.10.10", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 16
42061 getsockname(16, {sa_family=AF_INET6, sin6_port=htons(1024), inet_pton(AF_INET6, "::ffff:10.10.10.60", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
42061 fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
42061 fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0
42061 break(0x817d000) = 0
42061 break(0x817f000) = 0
42061 read(16, "GET /index2.html HTTP/1.1\r\nUser-"..., 8000) = 585
42061 gettimeofday({1852139329, 1293957748}, NULL) = 0
42061 stat("/usr/local/www/apache22/data/index2.html", {st_mode=0, st_size=0, ...}) = 0
42061 getpid() = 42061 (ppid 42060)
42061 open("/usr/local/www/apache22/data/index2.html", O_RDONLY) = 17
42061 issetugid(0) = 0
42061 open("", O_RDONLY) = -1 ENOENT (No such file or directory)
42061 issetugid(0) = 0
42061 open("", O_RDONLY) = 18
42061 fstat(18, {st_mode=0, st_size=0, ...}) = 0
42061 read(18, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 7944) = 1267
42061 close(18) = 0
42061 break(0x8181000) = 0
42061 break(0x8183000) = 0
42061 syscall_393(0x11, 0x10, 0, 0, 0x202, 0xbfbfc660, 0xbfbfc658, 0) = 0
42061 read(16, 0x817d028, 8000) = -1 EAGAIN (Resource temporarily unavailable)
42061 write(8, "10.10.10.10 - - [24/Sep/2006:17:"..., 153) = 153
42061 close(17) = 0
42061 poll([{fd=0, events=0}], 1, 5000) = 1
42061 read(16, "GET /favicon.ico HTTP/1.1\r\nUser-"..., 8000) = 628
42061 gettimeofday({1852139329, 1293957748}, NULL) = 0
42061 stat("/usr/local/www/apache22/data/favicon.ico", 0xbfbfe8d0) = -1 ENOENT (No such file or directory)
42061 lstat("/usr", {st_mode=0, st_size=0, ...}) = 0
42061 lstat("/usr/local", {st_mode=0, st_size=0, ...}) = 0
42061 lstat("/usr/local/www", {st_mode=0, st_size=0, ...}) = 0
42061 lstat("/usr/local/www/apache22", {st_mode=0, st_size=0, ...}) = 0
42061 lstat("/usr/local/www/apache22/data", {st_mode=0, st_size=0, ...}) = 0
42061 lstat("/usr/local/www/apache22/data/favicon.ico", 0xbfbfe8d0) = -1 ENOENT (No such file or directory)
42061 getpid() = 42061 (ppid 42060)
42061 gettimeofday({135186120, 135186712}, NULL) = 0
42061 write(7, "8\317\16\10\0\0\0\0\0\0\0\0\0\0\0\0h\347\16\10\0\0\0\0"..., 161) = 161
42061 break(0x8185000) = 0
42061 writev(16, [{"P\372\16\10\220\372\16\10\320\372\16\10`\372\16\10\30\300"..., 135200072}, {NULL, 135201768}], 2) = 463
42061 read(16, 0x817b028, 8000) = -1 EAGAIN (Resource temporarily unavailable)
42061 write(8, "10.10.10.10 - - [24/Sep/2006:17:"..., 184) = 184
42061 poll([{fd=0, events=0}], 1, 5000) = 0
42061 gettimeofday({0, 0}, NULL) = 0
42061 shutdown(16, 1 /* send */) = 0
42061 poll([{fd=0, events=0}], 1, 2000) = 1
42061 read(16, "", 512) = 0
42061 close(16) = 0
42061 read(5, 0xbfbfeb53, 1) = -1 EAGAIN (Resource temporarily unavailable)
42061 accept(3,
====- 8< -===================================================
--
Best regards,
Victor
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Joshua Slive <jo...@slive.ca>.
On 9/24/06, Victor Star <gr...@victorstar.com> wrote:
> I've tried strace -u www httpd -X
> but the result is exactly the same as above.
> As far as I know Apache needs to start as root to begin listening http ports and switches to www
> user later for securinty reasons.
>
> And yes, I'm starting it with -X option. And server seems to work except strace is not able to trace
> anything...
>
Change your Listen directive to a port over 1024 and then strace after
starting from a non-root account.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re[3]: [users@httpd] Corrupted source files
Posted by Victor Star <gr...@victorstar.com>.
Hi Victor,
> I'm in the process of upgrading to 6.1 right now. Will see if it changes anything.
> If not I guess I'll post it in freebsd forum and we'll see if we can dig something out.
> Thanks again for your help!
I don't know what it was, but upgrade to 6.1 fixed it. Running no problems now!
Thanks for your help!
Best regards,
Victor
***
"Life is full of little surprises" -Pandora
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re[2]: [users@httpd] Corrupted source files
Posted by Victor Star <gr...@victorstar.com>.
Hi Joshua,
>> You're right. "EnableSendfile Off" makes the trick. :-/
>> Now what could it be with my FreeBSD installation to cause such a mess? Until now I didn't notice
>> any problems with any other applications.
>> I have FreeBSD 6.0 here. Not sure if 6.1 would make a difference...
> You've got me stumped. I've never heard of any such problem before.
> Lots of systems have flaky sendfile problems, but I've never heard of
> one that actually corrupts the disk. Perhaps a freebsd forum would
> have more ideas.
I'm in the process of upgrading to 6.1 right now. Will see if it changes anything.
If not I guess I'll post it in freebsd forum and we'll see if we can dig something out.
Thanks again for your help!
Best regards,
Victor
***
Nothing is fool-proof to a sufficiently talented fool.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Joshua Slive <jo...@slive.ca>.
On 9/25/06, Victor Star <gr...@victorstar.com> wrote:
> You're right. "EnableSendfile Off" makes the trick. :-/
> Now what could it be with my FreeBSD installation to cause such a mess? Until now I didn't notice
> any problems with any other applications.
> I have FreeBSD 6.0 here. Not sure if 6.1 would make a difference...
You've got me stumped. I've never heard of any such problem before.
Lots of systems have flaky sendfile problems, but I've never heard of
one that actually corrupts the disk. Perhaps a freebsd forum would
have more ideas.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Victor Star <gr...@victorstar.com>.
Hi Joshua,
> There are only two writing actions there: syscall_393 which is (I
> believe) sendfile and the log write. Neither of those should have any
> effect on your file.
> Just on a whim, I would try "EnableSendfile Off" in httpd.conf. But
> even if that works, the only explanation I would have would be
> something very screwed up with your freebsd install. (Note that 1.3
> doesn't use sendfile, which would explain why it doesn't have that
> problem.)
You're right. "EnableSendfile Off" makes the trick. :-/
Now what could it be with my FreeBSD installation to cause such a mess? Until now I didn't notice
any problems with any other applications.
I have FreeBSD 6.0 here. Not sure if 6.1 would make a difference...
BTW did I mention I really appreciate your help? Thanks a lot! People like you are priceless for
open-source projects!
--
Best regards,
Victor
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by "Philip M. Gollucci" <pg...@p6m7g8.com>.
Joshua Slive wrote:
> On 9/24/06, Victor Star <gr...@victorstar.com> wrote:
>> syscall_393(0x10, 0xf, 0, 0, 0x24f, 0xbfbfc660, 0xbfbfc658, 0) = 0
> There are only two writing actions there: syscall_393 which is (I
grep 393 /usr/include/sys/syscall.h
#define SYS_sendfile 393
> Just on a whim, I would try "EnableSendfile Off" in httpd.conf. But
> even if that works, the only explanation I would have would be
> something very screwed up with your freebsd install.
people.apache.org et al are AFAIK successfully using Sendfile on FBSD 6.1
Defaults to on FYI.
--
------------------------------------------------------------------------
Philip M. Gollucci (pgollucci@p6m7g8.com) 323.219.4708
Consultant / http://p6m7g8.net/Resume/resume.shtml
Senior Software Engineer - TicketMaster - http://ticketmaster.com
1024D/A79997FA F357 0FDD 2301 6296 690F 6A47 D55A 7172 A799 97F
When I call your name, Girl, it starts to flame
Burning in my heart, Tearing it all apart..
No matter how I try My love I cannot hide....
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Joshua Slive <jo...@slive.ca>.
On 9/24/06, Victor Star <gr...@victorstar.com> wrote:
> And completely manual request (file gets corrupted):
> ====- 8< -===================================================
> accept(4, {sa_family=AF_INET, sin_port=htons(2305), sin_addr=inet_addr("10.10.10.10")}, [16]) = 15
> getsockname(15, {sa_family=AF_INET, sin_port=htons(1024), sin_addr=inet_addr("10.10.10.60")}, [16]) = 0
> fcntl(15, F_GETFL) = 0x2 (flags O_RDWR)
> fcntl(15, F_SETFL, O_RDWR|O_NONBLOCK) = 0
> read(15, "GET /index2.html HTTP/1.1\r\nhost:"..., 8000) = 45
> gettimeofday({1919510048, 1818321509}, NULL) = 0
> stat("/usr/local/www/apache22/data/index2.html", {st_mode=0, st_size=0, ...}) = 0
> getpid() = 94563 (ppid 94562)
> open("/usr/local/www/apache22/data/index2.html", O_RDONLY) = 16
> syscall_393(0x10, 0xf, 0, 0, 0x24f, 0xbfbfc660, 0xbfbfc658, 0) = 0
> read(15, 0x817d028, 8000) = -1 EAGAIN (Resource temporarily unavailable)
> write(7, "10.10.10.10 - - [24/Sep/2006:18:"..., 89) = 89
> close(16) = 0
> poll([{fd=859451954, events=POLLERR|POLLHUP|POLLNVAL|0x2000, revents=POLLIN|POLLOUT|POLLERR|POLLNVAL|0x3000}], 1, 5000) = 0
> gettimeofday({1563439156, 1162289696}, NULL) = 0
> shutdown(15, 1 /* send */) = 0
> poll([{fd=1764696148, events=POLLPRI|POLLOUT|POLLRDNORM|POLLERR|POLLNVAL|0x6400, revents=POLLIN|POLLOUT|POLLRDNORM|POLLNVAL|0x7800}], 1, 2000) = 1
> read(15, "", 512) = 0
> close(15) = 0
> read(3, 0xbfbfeb53, 1) = -1 EAGAIN (Resource temporarily unavailable)
> accept(4,
There are only two writing actions there: syscall_393 which is (I
believe) sendfile and the log write. Neither of those should have any
effect on your file.
Just on a whim, I would try "EnableSendfile Off" in httpd.conf. But
even if that works, the only explanation I would have would be
something very screwed up with your freebsd install. (Note that 1.3
doesn't use sendfile, which would explain why it doesn't have that
problem.)
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Victor Star <gr...@victorstar.com>.
And completely manual request (file gets corrupted):
====- 8< -===================================================
accept(4, {sa_family=AF_INET, sin_port=htons(2305), sin_addr=inet_addr("10.10.10.10")}, [16]) = 15
getsockname(15, {sa_family=AF_INET, sin_port=htons(1024), sin_addr=inet_addr("10.10.10.60")}, [16]) = 0
fcntl(15, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(15, F_SETFL, O_RDWR|O_NONBLOCK) = 0
read(15, "GET /index2.html HTTP/1.1\r\nhost:"..., 8000) = 45
gettimeofday({1919510048, 1818321509}, NULL) = 0
stat("/usr/local/www/apache22/data/index2.html", {st_mode=0, st_size=0, ...}) = 0
getpid() = 94563 (ppid 94562)
open("/usr/local/www/apache22/data/index2.html", O_RDONLY) = 16
syscall_393(0x10, 0xf, 0, 0, 0x24f, 0xbfbfc660, 0xbfbfc658, 0) = 0
read(15, 0x817d028, 8000) = -1 EAGAIN (Resource temporarily unavailable)
write(7, "10.10.10.10 - - [24/Sep/2006:18:"..., 89) = 89
close(16) = 0
poll([{fd=859451954, events=POLLERR|POLLHUP|POLLNVAL|0x2000, revents=POLLIN|POLLOUT|POLLERR|POLLNVAL|0x3000}], 1, 5000) = 0
gettimeofday({1563439156, 1162289696}, NULL) = 0
shutdown(15, 1 /* send */) = 0
poll([{fd=1764696148, events=POLLPRI|POLLOUT|POLLRDNORM|POLLERR|POLLNVAL|0x6400, revents=POLLIN|POLLOUT|POLLRDNORM|POLLNVAL|0x7800}], 1, 2000) = 1
read(15, "", 512) = 0
close(15) = 0
read(3, 0xbfbfeb53, 1) = -1 EAGAIN (Resource temporarily unavailable)
accept(4,
====- 8< -===================================================
--
Best regards,
Victor
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Victor Star <gr...@victorstar.com>.
>>> write(7, "h\347\16\10\0\0\0\0\320\355\16\0108\362\16\10@\372\16\10"..., 156) = 156
>>> writev(16, [{NULL, 135209656},
>>> {"@*\26\10\0\0\0\0\0\0\0\0\0\0\0\0\2\0\0\0\1\0\0\0LoadMo"..., 135211888}], 2) = 463
>> Looks pretty ordinary except for the above two lines. Can you try to
>> figure out what the descriptor 7 is pointing to, and when it is
>> getting opened. 16 is obviously the network connection, but it seems
>> like there is garbage getting written to it.
> Sorry, but how do I find out? Is this the NODE column of lsof output?
> If it is, than here's what I see:
> ====- 8< -===================================================
> # lsof | grep " 7 "
> su 1005 root cwd VDIR 0,71 512 7 /root
> su 41683 root cwd VDIR 0,71 512 7 /root
> csh 41684 root cwd VDIR 0,71 512 7 /root
> su 57674 root cwd VDIR 0,71 512 7 /root
> # lsof | grep " 16 "
> lsof 47656 root txt VREG 0,71 21836 16 /lib/libkvm.so.3
> ====- 8< -===================================================
>> These are both associated with the favicon request. You might try a
>> manual request (not from a browser) to just the index2.html file to
>> see if it still gets corrupted. If not, it is probably your 404 error
>> handler that is mangling things.
Also, I've tried requesting index2.html from the other browser that doesn't ask for favicon.ico:
====- 8< -===================================================
accept(3, {sa_family=AF_INET6, sin6_port=htons(1936), inet_pton(AF_INET6, "::ffff:10.10.10.10", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 16
getsockname(16, {sa_family=AF_INET6, sin6_port=htons(1024), inet_pton(AF_INET6, "::ffff:10.10.10.60", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0
read(16, "GET /index2.html HTTP/1.1\r\nAccep"..., 8000) = 253
gettimeofday({706755188, 168634927}, NULL) = 0
stat("/usr/local/www/apache22/data/index2.html", {st_mode=0, st_size=0, ...}) = 0
getpid() = 61292 (ppid 61291)
open("/usr/local/www/apache22/data/index2.html", O_RDONLY) = 17
syscall_393(0x11, 0x10, 0, 0, 0x24f, 0xbfbfc660, 0xbfbfc658, 0) = 0
read(16, 0x8181028, 8000) = -1 EAGAIN (Resource temporarily unavailable)
write(8, "10.10.10.10 - - [24/Sep/2006:17:"..., 138) = 138
close(17) = 0
poll([{fd=893007925, events=POLLIN|POLLHUP|POLLNVAL|0x2000, revents=POLLIN|POLLOUT|POLLERR|POLLNVAL|0x3000}], 1, 5000) = 0
gettimeofday({1563439156, 1162289696}, NULL) = 0
shutdown(16, 1 /* send */) = 0
poll([{fd=1764696148, events=POLLPRI|POLLOUT|POLLRDNORM|POLLERR|POLLNVAL|0x6400, revents=POLLIN|POLLOUT|POLLRDNORM|POLLNVAL|0x7800}], 1, 2000) = 0
close(16) = 0
read(5, 0xbfbfeb53, 1) = -1 EAGAIN (Resource temporarily unavailable)
accept(3,
====- 8< -===================================================
File still gets corrupted. :-/
I've tried installing Apache 1.3 and it doesn't seem to have such a problem.
--
Best regards,
Victor
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Victor Star <gr...@victorstar.com>.
Hi Joshua,
> On 9/24/06, Victor Star <gr...@victorstar.com> wrote:
>> write(7, "h\347\16\10\0\0\0\0\320\355\16\0108\362\16\10@\372\16\10"..., 156) = 156
>> writev(16, [{NULL, 135209656}, {"@*\26\10\0\0\0\0\0\0\0\0\0\0\0\0\2\0\0\0\1\0\0\0LoadMo"..., 135211888}], 2) = 463
> Looks pretty ordinary except for the above two lines. Can you try to
> figure out what the descriptor 7 is pointing to, and when it is
> getting opened. 16 is obviously the network connection, but it seems
> like there is garbage getting written to it.
Sorry, but how do I find out? Is this the NODE column of lsof output?
If it is, than here's what I see:
====- 8< -===================================================
# lsof | grep " 7 "
su 1005 root cwd VDIR 0,71 512 7 /root
su 41683 root cwd VDIR 0,71 512 7 /root
csh 41684 root cwd VDIR 0,71 512 7 /root
su 57674 root cwd VDIR 0,71 512 7 /root
# lsof | grep " 16 "
lsof 47656 root txt VREG 0,71 21836 16 /lib/libkvm.so.3
====- 8< -===================================================
> These are both associated with the favicon request. You might try a
> manual request (not from a browser) to just the index2.html file to
> see if it still gets corrupted. If not, it is probably your 404 error
> handler that is mangling things.
> Joshua.
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
--
Best regards,
Victor
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Joshua Slive <jo...@slive.ca>.
On 9/24/06, Victor Star <gr...@victorstar.com> wrote:
> write(7, "h\347\16\10\0\0\0\0\320\355\16\0108\362\16\10@\372\16\10"..., 156) = 156
> writev(16, [{NULL, 135209656}, {"@*\26\10\0\0\0\0\0\0\0\0\0\0\0\0\2\0\0\0\1\0\0\0LoadMo"..., 135211888}], 2) = 463
Looks pretty ordinary except for the above two lines. Can you try to
figure out what the descriptor 7 is pointing to, and when it is
getting opened. 16 is obviously the network connection, but it seems
like there is garbage getting written to it.
These are both associated with the favicon request. You might try a
manual request (not from a browser) to just the index2.html file to
see if it still gets corrupted. If not, it is probably your 404 error
handler that is mangling things.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Victor Star <gr...@victorstar.com>.
Ok, I've been able to get some tracing info by running strace -u www -p <httpd PID>
(attaching to the running process). Here's what I got:
====- 8< -===================================================
getsockname(16, {sa_family=AF_INET6, sin6_port=htons(80), inet_pton(AF_INET6, "::ffff:10.10.10.60", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
fcntl(16, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(16, F_SETFL, O_RDWR|O_NONBLOCK) = 0
read(16, "GET /index2.html HTTP/1.1\r\nUser-"..., 8000) = 580
gettimeofday({1852139329, 1293957748}, NULL) = 0
stat("/usr/local/www/apache22/data/index2.html", {st_mode=S_IFCHR|S_ISUID|0160, st_rdev=makedev(49, 135201016), ...}) = 0
getpid() = 41930 (ppid 1)
open("/usr/local/www/apache22/data/index2.html", O_RDONLY) = 17
syscall_393(0x11, 0x10, 0, 0, 0x202, 0xbfbfc5b0, 0xbfbfc5a8, 0) = 0
read(16, 0x817f028, 8000) = -1 EAGAIN (Resource temporarily unavailable)
write(8, "10.10.10.10 - - [24/Sep/2006:17:"..., 153) = 153
close(17) = 0
poll([{fd=0, events=0}], 1, 5000) = 1
read(16, "GET /favicon.ico HTTP/1.1\r\nUser-"..., 8000) = 618
gettimeofday({1852139329, 1293957748}, NULL) = 0
stat("/usr/local/www/apache22/data/favicon.ico", 0xbfbfe820) = -1 ENOENT (No such file or directory)
lstat("/usr", {st_mode=0, st_size=0, ...}) = 0
lstat("/usr/local", {st_mode=0, st_size=0, ...}) = 0
lstat("/usr/local/www", {st_mode=0, st_size=0, ...}) = 0
lstat("/usr/local/www/apache22", {st_mode=0, st_size=0, ...}) = 0
lstat("/usr/local/www/apache22/data", {st_mode=0, st_size=0, ...}) = 0
lstat("/usr/local/www/apache22/data/favicon.ico", 0xbfbfe820) = -1 ENOENT (No such file or directory)
getpid() = 41930 (ppid 1)
gettimeofday({0, 0}, NULL) = 0
write(7, "h\347\16\10\0\0\0\0\320\355\16\0108\362\16\10@\372\16\10"..., 156) = 156
writev(16, [{NULL, 135209656}, {"@*\26\10\0\0\0\0\0\0\0\0\0\0\0\0\2\0\0\0\1\0\0\0LoadMo"..., 135211888}], 2) = 463
read(16, 0x817d028, 8000) = -1 EAGAIN (Resource temporarily unavailable)
write(8, "10.10.10.10 - - [24/Sep/2006:17:"..., 179) = 179
poll([{fd=0, events=0}], 1, 5000) = 0
gettimeofday({0, 0}, NULL) = 0
shutdown(16, 1 /* send */) = 0
poll([{fd=0, events=0}], 1, 2000) = 1
read(16, "", 512) = 0
close(16) = 0
read(5, 0xbfbfeaa3, 1) = -1 EAGAIN (Resource temporarily unavailable)
accept(3, {sa_family=AF_INET6, sin6_port=htons(1583), inet_pton(AF_INET6, "::ffff:10.10.10.10", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 16
====- 8< -===================================================
Here's I'm requesting index2.html (which is getting corrupted). Also browser asks for favicon.ico
automatically.
Can you see anything wrong here?
--
Best regards,
Victor
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Victor Star <gr...@victorstar.com>.
Hi Joshua,
>> I tried it with no luck. I'm far from being a debugger expert, but what's happening doesn't look
>> normal to me. strace starts fine and writes tons of stuff when the server starts. I wait until it's
>> done and make a request to that web page. Page is displayed but nothing is written in the trace log
>> at all, it sits where it stopped after the server start. I've tried -f option on strace but it's no
>> different. Also tried truss instead of strace - same story.
>> Last lines of the strace log are:
>> ====- 8< -===================================================
>> open("/var/run/accept.lock.41858", O_WRONLY) = 12
>> geteuid(0x1) = 0
>> setgid(0x50
>> ====- 8< -===================================================
>> (yes, last line is incomplete)
>> I've also tried strace-ing httpd itself, no difference. It does serve pages, it doesn't show
>> anything. I suspect this might be due to httpd falling down to user and group "www" (last lines hint
>> to that) and it can't be traced thereafter. But Apache doesn't start under root. Do I have any other
>> options here?
> Are you sure you are passing the "-X", which puts it in single-process
> mode? You can try starting under a non-root user to see if the setgid
> is really the problem.
I've tried strace -u www httpd -X
but the result is exactly the same as above.
As far as I know Apache needs to start as root to begin listening http ports and switches to www
user later for securinty reasons.
And yes, I'm starting it with -X option. And server seems to work except strace is not able to trace
anything...
--
Best regards,
Victor
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Joshua Slive <jo...@slive.ca>.
On 9/24/06, Victor Star <gr...@victorstar.com> wrote:
> Hi Joshua,
>
> > And you are sure that the file on the server itself becomes corrupted?
>
> Yes, absolutely. That's the first thing I've checked. It is the file on server.
>
> > Try running "strace apachectl -X" and see if you can track down where
> > stuff is getting written to disk. In the default config, the only
> > thing apache should be writing to disk is logfiles.
>
> I tried it with no luck. I'm far from being a debugger expert, but what's happening doesn't look
> normal to me. strace starts fine and writes tons of stuff when the server starts. I wait until it's
> done and make a request to that web page. Page is displayed but nothing is written in the trace log
> at all, it sits where it stopped after the server start. I've tried -f option on strace but it's no
> different. Also tried truss instead of strace - same story.
> Last lines of the strace log are:
> ====- 8< -===================================================
> open("/var/run/accept.lock.41858", O_WRONLY) = 12
> geteuid(0x1) = 0
> setgid(0x50
> ====- 8< -===================================================
> (yes, last line is incomplete)
> I've also tried strace-ing httpd itself, no difference. It does serve pages, it doesn't show
> anything. I suspect this might be due to httpd falling down to user and group "www" (last lines hint
> to that) and it can't be traced thereafter. But Apache doesn't start under root. Do I have any other
> options here?
Are you sure you are passing the "-X", which puts it in single-process
mode? You can try starting under a non-root user to see if the setgid
is really the problem.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Victor Star <gr...@victorstar.com>.
Hi Joshua,
> And you are sure that the file on the server itself becomes corrupted?
Yes, absolutely. That's the first thing I've checked. It is the file on server.
> Try running "strace apachectl -X" and see if you can track down where
> stuff is getting written to disk. In the default config, the only
> thing apache should be writing to disk is logfiles.
I tried it with no luck. I'm far from being a debugger expert, but what's happening doesn't look
normal to me. strace starts fine and writes tons of stuff when the server starts. I wait until it's
done and make a request to that web page. Page is displayed but nothing is written in the trace log
at all, it sits where it stopped after the server start. I've tried -f option on strace but it's no
different. Also tried truss instead of strace - same story.
Last lines of the strace log are:
====- 8< -===================================================
open("/var/run/accept.lock.41858", O_WRONLY) = 12
geteuid(0x1) = 0
setgid(0x50
====- 8< -===================================================
(yes, last line is incomplete)
I've also tried strace-ing httpd itself, no difference. It does serve pages, it doesn't show
anything. I suspect this might be due to httpd falling down to user and group "www" (last lines hint
to that) and it can't be traced thereafter. But Apache doesn't start under root. Do I have any other
options here?
--
Best regards,
Victor
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Joshua Slive <jo...@slive.ca>.
On 9/24/06, Victor Star <gr...@victorstar.com> wrote:
> That's what I just tried. Re-installed Apache with the default configuration.
> "It works!" index.html really works, doesn't get corrupted. Now I dropped my own html file in there
> without changing any config. And it breaks. I started working from the working file and what I found
> is that it's the file size that matters. Any file over 512 bytes breaks. 512 bytes and under - work
> fine. Totally confused.
And you are sure that the file on the server itself becomes corrupted?
Normally I would suggest that this is something that
EnableSendfile Off
EnableMMAP Off
should cure. But if the file on disk is getting corrupted, there is
something deeper going on.
Try running "strace apachectl -X" and see if you can track down where
stuff is getting written to disk. In the default config, the only
thing apache should be writing to disk is logfiles.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Victor Star <gr...@victorstar.com>.
>> I'm new to Apache. I've just installed Apache 2.2.3 on FreeBSD 6.0. Configured it and got it
>> running. But I'm having something that looks like quite a weird problem. I've done my research on
>> Google, read FAQ and docs but couldn't find anything even closely resembling my situation.
>> Description of the problem: when I'm requesting documents like HTML files or images with the browser
>> - it displays properly the very first time, and after this the original file on the server is
>> corrupted - filled with some binary garbage. And this garbage is what's displayed on every next
>> request. Actually this garbage changes from request to request. So what it looks like - that when
>> Apache grabs the file to send it to the browser, it writes some garbage into it. I've never seen
>> anything like that before.
>> Some additional info: when I request php files (served by php_mod) - everything is fine, nothing
>> gets corrupted.
>>
>> I would greatly appreciate any hints on where to look for a fix.
> I have never ever heard of apache corrupting files on disk, so this
> must be something peculiar to your config. Can you tell us exactly
> what changes you made to the default config. (Or alternatively, go
> back to the default config file and make sure everything works, then
> slowly re-add your customizations, checking each time to figure out
> where things break.)
That's what I just tried. Re-installed Apache with the default configuration.
"It works!" index.html really works, doesn't get corrupted. Now I dropped my own html file in there
without changing any config. And it breaks. I started working from the working file and what I found
is that it's the file size that matters. Any file over 512 bytes breaks. 512 bytes and under - work
fine. Totally confused.
--
Best regards,
Victor
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Corrupted source files
Posted by Joshua Slive <jo...@slive.ca>.
On 9/23/06, Victor Star <gr...@victorstar.com> wrote:
> Hi,
>
> I'm new to Apache. I've just installed Apache 2.2.3 on FreeBSD 6.0. Configured it and got it
> running. But I'm having something that looks like quite a weird problem. I've done my research on
> Google, read FAQ and docs but couldn't find anything even closely resembling my situation.
> Description of the problem: when I'm requesting documents like HTML files or images with the browser
> - it displays properly the very first time, and after this the original file on the server is
> corrupted - filled with some binary garbage. And this garbage is what's displayed on every next
> request. Actually this garbage changes from request to request. So what it looks like - that when
> Apache grabs the file to send it to the browser, it writes some garbage into it. I've never seen
> anything like that before.
> Some additional info: when I request php files (served by php_mod) - everything is fine, nothing
> gets corrupted.
>
> I would greatly appreciate any hints on where to look for a fix.
I have never ever heard of apache corrupting files on disk, so this
must be something peculiar to your config. Can you tell us exactly
what changes you made to the default config. (Or alternatively, go
back to the default config file and make sure everything works, then
slowly re-add your customizations, checking each time to figure out
where things break.)
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
[users@httpd] Re: Corrupted source files
Posted by Joost de Heer <sa...@xs4all.nl>.
Victor Star wrote:
> Hi,
>
> I'm new to Apache. I've just installed Apache 2.2.3 on FreeBSD 6.0.
> Configured it and got it
> running. But I'm having something that looks like quite a weird problem.
> I've done my research on
> Google, read FAQ and docs but couldn't find anything even closely
> resembling my situation.
> Description of the problem: when I'm requesting documents like HTML files
> or images with the browser
> - it displays properly the very first time, and after this the original
> file on the server is
> corrupted - filled with some binary garbage. And this garbage is what's
> displayed on every next
> request. Actually this garbage changes from request to request. So what it
> looks like - that when
> Apache grabs the file to send it to the browser, it writes some garbage
> into it. I've never seen
> anything like that before.
> Some additional info: when I request php files (served by php_mod) -
> everything is fine, nothing
> gets corrupted.
Do you have mod_disk_cache activated? Perhaps you have it configured
wrong, and the cached data is written to the same place as the original
data.
Joost
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org