[eng007]

Apache & Loging Web Farms

I have multiple load-balanced servers and I want to generate one web report for the whole farm, but each server generates its own log file

Webalizer will not work the log files are 500Gb's each I get about 160 hits a sec

Is there some other software out there what will work with very big logs?

[martouf]

first off, you should read this document written by an admin of a site getting
about 45 million hits a day.

next, read this document written by an admin who has set up a multi-system-one-logfile
configuration. Note the use of a pipe as the log destination.

The general Apache documentation section on logs will round out your knowledge.

Basically, you'll need to have Apache pipe all of the logging to the syslog daemon.

With the logging all going to syslog, you set each system in the farm to send logging
to the one log host you designate, whose own syslog receives the log data via a LAN.