RE: [users@httpd] Scaling Apache-> Logging

[Jan van den Berg <ja...@solcon.nl>]

Thanks everyone for the input so far.
Right now we're looking into NAS systems by either DELL or NetApp (any
tips or experiences are welcome). SANs tend to be more expensive, and
don't give us more flexibility than we would need.

But my main question is regarding logging. How to go about it in the
best possible manner.
Right we are using webalizer and keeping error and access logs per
website, and generating reports nightly.
But when using front-ends the logs would be scattered over the different
front-ends; which _could_ later be merged again. Another way would be to
keep the logs on the NAS as well, one central point. But this would
probably highten the NFS traffic, which I would rather keep to a
minimum. Also we could log everything to one big file and later make the
differentiation per user, or we could keep the logs per user(website),
what's smart?
So;
1. What is good way of logging for different websites and different
users? (Centralized or not, one file or per user etc.)
Next question:
2. I heard a lot about AWstats being flexible and all, does anyone have
any experience with this software in regard to my first question? 

--
Regards,

Jan van den Berg

-----Oorspronkelijk bericht-----
Van: Eric [mailto:ef@kwinternet.com] 
Verzonden: donderdag 4 november 2004 18:10
Aan: users@httpd.apache.org
Onderwerp: Re: [users@httpd] Scaling Apache

Hi,

I think that is not really considering OS caching, or how fast his 
connection is. And I don't realy understand what this means, "so it will

realy help you only if you are having very much dynamic content."

Anyway, if he is using an NFS mount over a fiber connection, I bet it is

VERY fast. What we do is to rsync content between servers, but we don't 
have a SAN.. It may be that he has to store a VERY large amount of data.

Then his method of using a SAN is certainly smarter than using local
disks 
if that is the case. It is also probably smarter for other reasons too, 
like being able to grow his storage space easily as needed and
everything 
else that is good about having a SAN. Anyway, he certainly could put a 
couple of proxies in front of his servers if network latency becomes a
big 
problem.

Thanks,

Eric

At 08:59 AM 11/4/2004, you wrote:
>are you sure that you will get any big advantage from using 
>load-balancing. as i understood you, apache has to get the requested
file 
>over the lan from one single server, your master, so it will realy help

>you only if you are having very much dynamic content. in order to realy

>scale the load over the servers, every fe would have to have a complete

>copy of the websites and automatically replicate the data.
>
>Hi,
>
>Im looking for some expertise regarding an Apache setup.
>I have the following in mind:
>
>               LoadBalancer
>      /   |   \ \
>      FE  FE  FE ...
>       \  |  /
>  Master
>    |
>                     SAN
>
>The FEs are the Apache frontends which handle the http requests. All
the
>different websites data is stored on the SAN, which only has the master
>server connected trough GFS (or any other Filesystem). The FEs have
>access to the data trough an NFS mount.
>
>Is this an ideal setup, how can I make this better? What are the things
>I should keep in mind (pitfalls etc.)?
>Im asking here since I cant seem to find info regarding using Apache
>with a SAN and Im looking for a way to scale Apache (and share the
>load). The FEs are easy to pull out and replace or FEs can be added.
>
>Jan
>
>
>---------------------------------------------------------------------
>The official User-To-User support forum of the Apache HTTP Server
Project.
>See <URL:http://httpd.apache.org/userslist.html> for more info.
>To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>   "   from the digest: users-digest-unsubscribe@httpd.apache.org
>For additional commands, e-mail: users-help@httpd.apache.org
>
>
>---------------------------------------------------------------------
>The official User-To-User support forum of the Apache HTTP Server
Project.
>See <URL:http://httpd.apache.org/userslist.html> for more info.
>To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>   "   from the digest: users-digest-unsubscribe@httpd.apache.org
>For additional commands, e-mail: users-help@httpd.apache.org


Lead Programmer
D.M. Contact Management
250.383.8267 ext 229 


---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server
Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org



---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
   "   from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org