You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Daniel Watrous <dw...@gmail.com> on 2006/08/27 00:31:53 UTC
[users@httpd] data transfer statistics based on apache logs
Hello,
I have a concern about determining data transfer statistics based on
apache logs. This problem may be more or less severe depending on the
average size of the files being served.
When a web page is requested from apache, each file contained within
the page is requested as it is encountered in the html page. This
means that for each HTML page request the server may receive many hits
or requests. The apache logs keep track of each of these hits or
requests separately.
A scenario might be as follows. An HTML page that is 5kb in size
contains tags which include images totalling 30kb and a movie file
that totals 20MB. When this page is requested, the apache logs will
show the request for the HTML file, a separate entry for each image,
and an entry for the 20MB file. the first 35kb of data downloads
rather quickly, but the video is slower. If the user waits for the
entire 20MB file to download then the log is accurate.
There are two potential problems with this scenario. First, if the
user cancels the download or exits the page before the download is
complete, the apache log still shows a request for the full file,
keeping no information about how much of it was downloaded. The
second problem is if the users revisits the page, and either windows
media player, quicktime, flash, etc. have cached the file, does apache
still receive a request for the content, and delivers nothing?
Using the apache logs to measure data transfer statistics presents
possible inaccuracies. Is there some way to address this? Is there
another solution for measuring data transfer?
Thanks in advance.
Daniel
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] data transfer statistics based on apache logs
Posted by Daniel Watrous <dw...@gmail.com>.
Thanks Joshua,
Your suggestion, mod_logio, addresses both concerns. Thank you.
Daniel
On 8/26/06, Joshua Slive <jo...@slive.ca> wrote:
> On 8/26/06, Daniel Watrous <dw...@gmail.com> wrote:
>
> > There are two potential problems with this scenario. First, if the
> > user cancels the download or exits the page before the download is
> > complete, the apache log still shows a request for the full file,
> > keeping no information about how much of it was downloaded. The
> > second problem is if the users revisits the page, and either windows
> > media player, quicktime, flash, etc. have cached the file, does apache
> > still receive a request for the content, and delivers nothing?
>
> For the first problem, see mod_log_io.
>
> How you deal with the second problem depends on exactly what
> statistics you want and why. There are various cache-busting
> techniques that assure that a new request is made every time a browser
> returns to the content. But they will degrade the browsing experience
> for your users. A middle ground is to cache-bust on just one small
> object on the page. Then you can infer that when people request this
> object, they are also getting the rest of the page from their cache.
>
> Joshua.
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] data transfer statistics based on apache logs
Posted by Joshua Slive <jo...@slive.ca>.
On 8/26/06, Daniel Watrous <dw...@gmail.com> wrote:
> There are two potential problems with this scenario. First, if the
> user cancels the download or exits the page before the download is
> complete, the apache log still shows a request for the full file,
> keeping no information about how much of it was downloaded. The
> second problem is if the users revisits the page, and either windows
> media player, quicktime, flash, etc. have cached the file, does apache
> still receive a request for the content, and delivers nothing?
For the first problem, see mod_log_io.
How you deal with the second problem depends on exactly what
statistics you want and why. There are various cache-busting
techniques that assure that a new request is made every time a browser
returns to the content. But they will degrade the browsing experience
for your users. A middle ground is to cache-bust on just one small
object on the page. Then you can infer that when people request this
object, they are also getting the rest of the page from their cache.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org