You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Antony Stone <An...@apache.open.source.it> on 2022/04/12 10:04:14 UTC
[users@httpd] Log to syslog?
Hi.
I'd like to have Apache send all log entries to syslog instead of files
(because I run a central syslog aggregator and want to have many servers all
send their log files to this system).
I have found:
https://httpd.apache.org/docs/trunk/mod/mod_syslog.html
However this appears only to be for Error Logs, whereas I would want _all_
logs to be sent to syslog.
Can Apache do this?
I have found some workarounds such as:
https://serverfault.com/questions/1025281
https://kifarunix.com/forward-apache-logs-to-central-log-server-with-rsyslog/
however I would be more comfortable if there were a way to tell Apache I want
it to talk directly to syslog, if this can be done.
Thanks in advance,
Antony.
--
If you were ploughing a field, which would you rather use - two strong oxen or
1024 chickens?
- Seymour Cray, pioneer of supercomputing
Please reply to the list;
please *don't* CC me.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Log to syslog?
Posted by "Paul Kudla (SCOM.CA Internet)" <pa...@scom.ca>.
I have worked on this issue for over ten years
unitil apache will redirect to syslog properly then you need to do the
workarounds
no other options to my knowledge
Happy Tuesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
On 4/12/2022 8:27 AM, Antony Stone wrote:
> On Tuesday 12 April 2022 at 13:32:40, Paul Claridge wrote:
>
>> Did you find the info on loggly.com?
> I foundhttps://www.loggly.com/ultimate-guide/centralizing-apache-logs/ and it
> was essentially a summary of the two mechanisms I had already found elsewhere
> and posted in my original question - telling rsyslog to track file contents
> written by Apache, or using logger in a CustomLog definition.
>
>> Not sure if it covers precisely your requirements.
> They look like they would work for me, however I regard them as "workarounds"
> and wanted to see whether anyone knew of a way to do it natively in Apache.
>
> I'm surprised that it appears not to be possible, but thanks to everyone for
> their responses so far.
>
>
> Antony.
>
>> On 12 Apr 2022, at 11:59, Marc<Ma...@f1-outsourcing.eu> wrote:
>>>>> i went through this issue the hard way
>>>> Urgh - thanks for the comprehensive reply.
>>>>
>>>>> there does not seem to be anything at all as apache seems to be all
>>>>> file related
>>>> I wonder why mod_syslog has not been made more generic?
>>>>
>>>>> redirecting to logger just does not work.
>>>>>
>>>>> i wrote a python script that uses sockets (assuming linux, freebsd etc)
>>>> Yes, I'm on Linux - thanks for the script, and for the comments re
>>>> logger etc.
>>>>
>>>> *If anyone else has a suggestion for how Apache can log to syslog, I'm
>>>> still interested in other possible ways to achieve it!*
>>> I have been asking something similar a while ago, logggin to something
>>> like influx. I know how to redirect syslog to influx. So if I can
>>> redirect eg ip's and 2XX/4XX to syslog, that would be very interesting.
Re: [users@httpd] Log to syslog?
Posted by Antony Stone <An...@apache.open.source.it>.
On Tuesday 12 April 2022 at 13:32:40, Paul Claridge wrote:
> Did you find the info on loggly.com?
I found https://www.loggly.com/ultimate-guide/centralizing-apache-logs/ and it
was essentially a summary of the two mechanisms I had already found elsewhere
and posted in my original question - telling rsyslog to track file contents
written by Apache, or using logger in a CustomLog definition.
> Not sure if it covers precisely your requirements.
They look like they would work for me, however I regard them as "workarounds"
and wanted to see whether anyone knew of a way to do it natively in Apache.
I'm surprised that it appears not to be possible, but thanks to everyone for
their responses so far.
Antony.
> On 12 Apr 2022, at 11:59, Marc <Ma...@f1-outsourcing.eu> wrote:
> >>> i went through this issue the hard way
> >>
> >> Urgh - thanks for the comprehensive reply.
> >>
> >>> there does not seem to be anything at all as apache seems to be all
> >>> file related
> >>
> >> I wonder why mod_syslog has not been made more generic?
> >>
> >>> redirecting to logger just does not work.
> >>>
> >>> i wrote a python script that uses sockets (assuming linux, freebsd etc)
> >>
> >> Yes, I'm on Linux - thanks for the script, and for the comments re
> >> logger etc.
> >>
> >> *If anyone else has a suggestion for how Apache can log to syslog, I'm
> >> still interested in other possible ways to achieve it!*
> >
> > I have been asking something similar a while ago, logggin to something
> > like influx. I know how to redirect syslog to influx. So if I can
> > redirect eg ip's and 2XX/4XX to syslog, that would be very interesting.
--
Please apologise my errors, since I have a very small device.
Please reply to the list;
please *don't* CC me.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Log to syslog?
Posted by Paul Claridge <pa...@vicjen.co.uk>.
Did you find the info on loggly.com?
Not sure if it covers precisely your requirements.
Sent from my iPhone
On 12 Apr 2022, at 11:59, Marc <Ma...@f1-outsourcing.eu> wrote:
>>> i went through this issue the hard way
>>
>> Urgh - thanks for the comprehensive reply.
>>
>>> there does not seem to be anything at all as apache seems to be all file
>>> related
>>
>> I wonder why mod_syslog has not been made more generic?
>>
>>> redirecting to logger just does not work.
>>>
>>> i wrote a python script that uses sockets (assuming linux, freebsd etc)
>>
>> Yes, I'm on Linux - thanks for the script, and for the comments re logger etc.
>>
>> *If anyone else has a suggestion for how Apache can log to syslog, I'm still
>> interested in other possible ways to achieve it!*
>>
>
> I have been asking something similar a while ago, logggin to something like influx. I know how to redirect syslog to influx. So if I can redirect eg ip's and 2XX/4XX to syslog, that would be very interesting.
>
>
> ТÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÐÐ¥FòVç7V'67&–&RÂRÖ֖âW6W'2×Vç7V'67&–&T‡GGBæ6†Ræ÷&pФf÷"FF—F–öæÂ6öÖÖæG2ÂRÖ֖âW6W'2Ö†VLJGGBæ6†Ræ÷&pÐ
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] Log to syslog?
Posted by Marc <Ma...@f1-outsourcing.eu>.
> > i went through this issue the hard way
>
> Urgh - thanks for the comprehensive reply.
>
> > there does not seem to be anything at all as apache seems to be all file
> > related
>
> I wonder why mod_syslog has not been made more generic?
>
> > redirecting to logger just does not work.
> >
> > i wrote a python script that uses sockets (assuming linux, freebsd etc)
>
> Yes, I'm on Linux - thanks for the script, and for the comments re logger etc.
>
> *If anyone else has a suggestion for how Apache can log to syslog, I'm still
> interested in other possible ways to achieve it!*
>
I have been asking something similar a while ago, logggin to something like influx. I know how to redirect syslog to influx. So if I can redirect eg ip's and 2XX/4XX to syslog, that would be very interesting.
Re: [users@httpd] Log to syslog?
Posted by Antony Stone <An...@apache.open.source.it>.
On Tuesday 12 April 2022 at 12:33:01, Paul Kudla (SCOM.CA Internet) wrote:
> i went through this issue the hard way
Urgh - thanks for the comprehensive reply.
> there does not seem to be anything at all as apache seems to be all file
> related
I wonder why mod_syslog has not been made more generic?
> redirecting to logger just does not work.
>
> i wrote a python script that uses sockets (assuming linux, freebsd etc)
Yes, I'm on Linux - thanks for the script, and for the comments re logger etc.
*If anyone else has a suggestion for how Apache can log to syslog, I'm still
interested in other possible ways to achieve it!*
Thanks,
Antony.
--
Why is "dyslexia" so difficult to spell, and why can I never remember "aphasia"
when I want to?
Please reply to the list;
please *don't* CC me.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Log to syslog?
Posted by "Paul Kudla (SCOM.CA Internet)" <pa...@scom.ca>.
i went through this issue the hard way
there does not seem to be anything at all as apache seems to be all file
related
redirecting to logger just does not work.
i wrote a python script that uses sockets (assuming linux, freebsd etc)
nothing for windows
i have to trap all the log files and redirect to a common logging server
(mine was 10.228.0.6 but the useage is pretty straight forward)
example useage
/usr/local/bin/python3 /programs/common/capture -s
/usr/local/apache2/logs/httpd-access.log -l httpd -d 10.228.0.6:514 -p httpd
/usr/local/bin/python3 /programs/common/capture -s
/usr/local/apache2/logs/httpd-error.log -l httpd-err -d 10.228.0.6:514
-p httpd
/usr/local/bin/python3 /programs/common/capture -s
/usr/local/apache2/logs/ssl_request_log -l httpd-ssl -d 10.228.0.6:514
-p httpd
## cat /programs/common/capture
#!/usr/local/bin/python3
# -*- coding: UTF-8 -*-
import os,sys,socket
import datetime,time
from optparse import OptionParser
from lib import *
USAGE_TEXT = '''\
usage: %%prog %s[options]
'''
parser = OptionParser(usage=USAGE_TEXT % '', version='0.4')
parser.add_option("-s", "--socket", dest="socket_file", help="Socket
File to Capture")
parser.add_option("-l", "--label", dest="label", help="Syslog Label to
Insert")
parser.add_option("-d", "--destination", dest="destination",
help="Syslog Destibnation Server:Port")
parser.add_option("-p", "--pid", dest="pid", help="PID Process Name")
#parser.add_option("-e", "--email", dest="email", help="Additional Email
To")
#parser.add_option("-t", "--temp", dest="tempdir", help="Local Temp
Directory")
options, args = parser.parse_args()
print (options.socket_file)
print (options.label)
print (options.destination)
print (options.pid)
if options.socket_file == None :
print ('Missing Socket File Information')
sys.exit()
if options.label == None :
print ('Missing Syslog Label Information')
sys.exit()
if options.destination == None :
print ('Missing Syslog Destination host:[port]')
sys.exit()
if options.pid == None :
print ('Missing Syslog Pid Process Name')
sys.exit()
#try local syslog (/var/run/log)
UDP_IP = options.destination.split(':')
if len(UDP_IP) == 2 : #Set Port
UDP_PORT = int(UDP_IP[1])
else :
UDP_PORT = 514 #Default
UDP_IP = UDP_IP[0] #Server
#MESSAGE = str("<22>Mar 27 04:16:16 es-scom[12345] offsite.scom.ca su:
Hello, World!")
#MESSAGE = str("<183>Mar 27 16:17:41 scom-live[72178]: Hello World")
print("UDP target IP: %s" % UDP_IP)
print("UDP target port: %s" % UDP_PORT)
#print("message: %s" % MESSAGE)
count = 10
#sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
#sock.sendto(MESSAGE, (UDP_IP, UDP_PORT))
#sock.sendto(bytes(MESSAGE, "utf-8"), (UDP_IP, UDP_PORT))
#sock.close()
#sys.exit()
#def read_commands():
try:
print ("Creating read pipe... %s" %options.socket_file )
os.mkfifo(options.socket_file) # Create pipe
print ("Pipe %s created!" %options.socket_file )
except:
print ("Pipe %s already exists" %options.socket_file )
#chmod 777 the file so everyone can talk to it
os.system('/bin/chmod 777 %s' %options.socket_file)
with open(options.socket_file, "r") as pipecmd:
while True:
time.sleep(.001)
try:
line = pipecmd.readline()
if line != '' : #New Data
if line == '\n' :
continue
print ('Raw Text : %s' %line)
encoded_string = line.encode("ascii",
"ignore")
line = encoded_string.decode()
line = create_ascii(line)
line = line.ascii
print ('Line after ASCII : %s' %line)
print ( 'Line Count : %s' %len(line) )
#line = data
#go get my pid
pid_process = '0'
if options.pid == 'postfix' : #its a
diverted postfix process get the actual pid from raw text
pid_process =
line.split('[',1)[1].split(']',1)[0]
else :
command = commands('/bin/ps
-axww | /usr/bin/grep %s' %options.pid)
print ()
#print (command.output)
for n in range
(0,len(command.output)) :
if '/bin/ps -axww |
/usr/bin/grep' not in command.output[n] and '/usr/bin/grep' not in
command.output[n] and '/usr/local/bin/python3' not in command.output[n] :
pid_process = ( command.output.split(' ')[0] ) #whats left should be my
process ?
break
print ('PID Process : %s ' %pid_process )
if options.destination == 'local' :
#Send to log here
print ('Sending to Local Syslog')
log = open ('/var/run/log','w')
log.write ('hello')
log.close()
sys.exit()
else : #Send via socket
#Make the line in freebsd
syslog format
MESSAGE = '<' + str(count) +
'>' + str( time.strftime("%b %d %H:%M:%S ") ) + str(options.label) + '['
+ str(pid_process) + ']: ' + str(line)
print ('Sent : %s' %MESSAGE )
count = count + 1
if count > 255 :
count = 10
# send to udp logger port specified
sock =
socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(bytes(MESSAGE,
"utf-8"), (UDP_IP, UDP_PORT))
sock.close()
else : #No data
pass
except Exception as e:
exc_type, exc_obj, exc_tb = sys.exc_info()
fname =
os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
e = str(e) + '\n\n' + str(exc_type) + '\n' +
str(fname) + '\n' + str(exc_tb.tb_lineno)
print ('\n\nCaught Exception : %s' %e )
print ("Could not read cmd pipe, skipping ...")
sys.exit()
Happy Tuesday !!!
Thanks - paul
Paul Kudla
Scom.ca Internet Services <http://www.scom.ca>
004-1009 Byron Street South
Whitby, Ontario - Canada
L1N 4S3
Toronto 416.642.7266
Main 1.866.411.7266
Fax 1.888.892.7266
On 4/12/2022 6:04 AM, Antony Stone wrote:
> Hi.
>
> I'd like to have Apache send all log entries to syslog instead of files
> (because I run a central syslog aggregator and want to have many servers all
> send their log files to this system).
>
> I have found:
> https://httpd.apache.org/docs/trunk/mod/mod_syslog.html
>
> However this appears only to be for Error Logs, whereas I would want _all_
> logs to be sent to syslog.
>
>
> Can Apache do this?
>
>
> I have found some workarounds such as:
>
> https://serverfault.com/questions/1025281
>
> https://kifarunix.com/forward-apache-logs-to-central-log-server-with-rsyslog/
>
> however I would be more comfortable if there were a way to tell Apache I want
> it to talk directly to syslog, if this can be done.
>
>
> Thanks in advance,
>
>
> Antony.
>