How to include parts of live log files into website

Question

How would I go about having apache pull some lines from log files to be displayed on a website.

I want to pull the last 5 vistiors IPs (possiblly while censoring them slightly) and the last 3 IPs to be banned for trying to connect via SSH (which is currently blocked using fail2ban.)

I really have no clue where to start with this other than I already have apache up and running and my website is hosted in what can be thought of as a "normal" configuration.

The log files are in /var/log/ and I can control the access to them.

Accepted Answer

I wasn't using apache httpd for a long time but if log line begins with IP, you can use

tail -n 5 /path/to/log | awk -F. '{print $1"."$2"."$3".x"}'

to extract 5 last IPs hiding last number or just

tail -n 5 /path/to/log | awk '{print $1}'

to extract 5 last IPs. This may be a cronscript, generating an HTML page can be included in your plain HTML document. Can't describe same process for fail2ban but I guess it's just the same, you'll just have to read the manuals for tools like sed and awk in order to parse logs with shell script.

Just remember: if your logs may contain resolved rDNS hostnames instead of IPs this may be dangerous to display them on website, who knows what it could be set to?

Answer 2

My first thought is that the logs in /var/log are probably owned by Root and thus your web applications won't be able to read them (You can confirm with ls -al)

You could use a (Perl/PHP/Shell) script (cron'd as root) to extract the last X IPs from /var/log/httpd/access_log and /var/log/Fail2Ban.log and write them to a file owned by Apache (or the site user if you're using SuPHP/SuExec) which could then be read by the sites pages and dynamically included.

I can't speak much regarding php but if you use Shell or Perl awk will help you get just the IPs:

Log Excerpt:

221.192.199.xx - - [08/May/2010:08:32:22 +0100] "GET http://www.wantsfly.com/prx2.php?hash=6039A91133E74FD3D454BB8200505327F2E95B294F70 HTTP/1.0" 404 287 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)" 221.192.199.xx - - [08/May/2010:08:57:01 +0100] "GET http://www.wantsfly.com/prx2.php?hash=6039A91133E74FD3D454BB8200505327F2E95B294F70 HTTP/1.0" 404 287 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)" 94.232.9.xx - - [08/May/2010:12:37:04 +0100] "GET /w00tw00t.at.ISC.SANS.DFind:) HTTP/1.1" 400 310 "-" "-"

tail -10 access_log | awk -F"-" {'print $1'}

221.192.199.xx
94.232.9.xx
221.192.199.xx 
221.192.199.xx
94.232.9.xx

Please note that LinuxExchange will be shutting down on December 31st, 2016. Visit this thread for additional information and to provide feedback.

How to include parts of live log files into website

Follow this question

Related questions