extract last 10 minutes from logfile [duplicate]
Introduction
This answer is something long, because there is 3 different way on thinking: 1) perl quick or exact, 2) pure bash and 3) perl script in bash function.
That's a (common) job for perl!:
Simple and efficient:
perl -MDate::Parse -ne 'print if/^(.{15})\s/&&str2time($1)>time-600' /path/log
This version print last 10 minutes event, upto now, by using time
function.
You could test this with:
sudo cat /var/log/syslog | perl -MDate::Parse -ne ' print if /^(\S+\s+\d+\s+\d+:\d+:\d+)\s/ && str2time($1) > time-600'
Note that first representation use only firsts 15 chars from each lines, while second construct use more detailed regexp.
As a perl script: last10m.pl
#!/usr/bin/perl -wnuse strict;use Date::Parse;print if /^(\S+\s+\d+\s+\d+:\d+:\d+)\s/ && str2time($1) > time-600
Strictly: extract last 10 minutes from logfile
Meaning not relative to current time, but to last entry in logfile:
There is two way for retrieving end of period:
date -r logfile +%stail -n1 logfile | perl -MDate::Parse -nE 'say str2time($1) if /^(.{15})/'
Where logically, last modification time of the logfile must be the time of the last entry.
So the command could become:
perl -MDate::Parse -ne 'print if/^(.{15})\s/&&str2time($1)>'$( date -r logfile +%s)
or you could take the last entry as reference:
perl -MDate::Parse -E 'open IN,"<".$ARGV[0];seek IN,-200,2;while (<IN>) { $ref=str2time($1) if /^(\S+\s+\d+\s+\d+:\d+:\d+)/;};seek IN,0,0; while (<IN>) {print if /^(.{15})\s/&&str2time($1)>$ref-600}' logfile
Second version seem stronger, but access to file only once.
As a perl script, this could look like:
#!/usr/bin/perl -wuse strict;use Date::Parse;my $ref; # The only variable I will use in this.open IN,"<".$ARGV[0]; # Open (READ) file submited as 1st argumentseek IN,-200,2; # Jump to 200 character before end of logfile. (This # could not suffice if log file hold very log lines! )while (<IN>) { # Until end of logfile... $ref=str2time($1) if /^(\S+\s+\d+\s+\d+:\d+:\d+)/;}; # store time into $ref variable.seek IN,0,0; # Jump back to the begin of filewhile (<IN>) { print if /^(.{15})\s/&&str2time($1)>$ref-600;}
But if you really wanna use bash
There is a very quick pure bash script:
Warning: This use recent bashisms, require $BASH_VERSION
4.2 or higher.
#!/bin/bashdeclare -A monthfor i in {1..12};do LANG=C printf -v var "%(%b)T" $(((i-1)*31*86400)) month[$var]=$i doneprintf -v now "%(%s)T" -1printf -v ref "%(%m%d%H%M%S)T" $((now-600))while read line;do printf -v crt "%02d%02d%02d%02d%02d" ${month[${line:0:3}]} \ $((10#${line:4:2})) $((10#${line:7:2})) $((10#${line:10:2})) \ $((10#${line:13:2})) # echo " $crt < $ref ??" # Uncomment this line to print each test [ $crt -gt $ref ] && breakdonecat
Store this script and run:
cat >last10min.shchmod +x last10min.shsudo cat /var/log/syslog | ./last10min.sh
Strictly: extract last 10 minutes from logfile
Simply replace line 10, but you have to place filename in the script and not use it as a filter:
#!/bin/bashdeclare -A monthfor i in {1..12};do LANG=C printf -v var "%(%b)T" $(((i-1)*31*86400)) month[$var]=$i doneread now < <(date -d "$(tail -n1 $1|head -c 15)" +%s)printf -v ref "%(%m%d%H%M%S)T" $((now-600))export -A month{ while read line;do printf -v crt "%02d%02d%02d%02d%02d" ${month[${line:0:3}]} \ $((10#${line:4:2})) $((10#${line:7:2})) $((10#${line:10:2})) \ $((10#${line:13:2})) [ $crt -gt $ref ] && break done cat} <$1
A perl script into a bash function
As commented by ajcg
, this could be nice to put efficient perl script into a bash function:
recentLog(){ perl -MDate::Parse -ne ' print if/^(.{'${3:-15}'})\s/ && str2time($1)>time-'$(( 60*${2:-10} )) ${1:-/var/log/daemon.log}}
Usage:
recentLog [filename] [minutes] [time sting length]
filename
of log fileminutes
max before now of lines to showtime sting length
from begin of lines (default15
).
You can match the date range using simple string comparison, for example:
d1=$(date --date="-10 min" "+%b %_d %H:%M")d2=$(date "+%b %_d %H:%M")while read line; do [[ $line > $d1 && $line < $d2 || $line =~ $d2 ]] && echo $linedone
For example if d1='Dec 18 10:19'
and d2='Dec 18 10:27'
then the output will be:
Dec 18 10:19:16Dec 18 10:19:23Dec 18 10:21:03Dec 18 10:22:54Dec 18 10:27:32
Or using awk
if you wish:
awk -v d1="$d1" -v d2="$d2" '$0 > d1 && $0 < d2 || $0 ~ d2'