extract last 10 minutes from logfile [duplicate] extract last 10 minutes from logfile [duplicate] bash bash

extract last 10 minutes from logfile [duplicate]


Here is nice toolrange is any you wish from -10 till now

sed -n "/^$(date --date='10 minutes ago' '+%b %_d %H:%M')/,\$p" /var/log/blaaaa


Introduction

This answer is something long, because there is 3 different way on thinking: 1) quick or exact, 2) pure and 3) script in function.

That's a (common) job for !:

Simple and efficient:

perl -MDate::Parse -ne 'print if/^(.{15})\s/&&str2time($1)>time-600' /path/log

This version print last 10 minutes event, upto now, by using time function.

You could test this with:

sudo cat /var/log/syslog |  perl -MDate::Parse -ne '    print if /^(\S+\s+\d+\s+\d+:\d+:\d+)\s/ && str2time($1) > time-600'

Note that first representation use only firsts 15 chars from each lines, while second construct use more detailed regexp.

As a perl script: last10m.pl

#!/usr/bin/perl -wnuse strict;use Date::Parse;print if /^(\S+\s+\d+\s+\d+:\d+:\d+)\s/ && str2time($1) > time-600

Strictly: extract last 10 minutes from logfile

Meaning not relative to current time, but to last entry in logfile:

There is two way for retrieving end of period:

date -r logfile +%stail -n1 logfile | perl -MDate::Parse -nE 'say str2time($1) if /^(.{15})/'

Where logically, last modification time of the logfile must be the time of the last entry.

So the command could become:

perl -MDate::Parse -ne 'print if/^(.{15})\s/&&str2time($1)>'$(    date -r logfile +%s)

or you could take the last entry as reference:

perl -MDate::Parse -E 'open IN,"<".$ARGV[0];seek IN,-200,2;while (<IN>) {    $ref=str2time($1) if /^(\S+\s+\d+\s+\d+:\d+:\d+)/;};seek IN,0,0;    while (<IN>) {print if /^(.{15})\s/&&str2time($1)>$ref-600}' logfile

Second version seem stronger, but access to file only once.

As a perl script, this could look like:

#!/usr/bin/perl -wuse strict;use Date::Parse;my $ref;                 # The only variable I will use in this.open IN,"<".$ARGV[0];    # Open (READ) file submited as 1st argumentseek IN,-200,2;          # Jump to 200 character before end of logfile. (This                         # could not suffice if log file hold very log lines! )while (<IN>) {           # Until end of logfile...    $ref=str2time($1) if /^(\S+\s+\d+\s+\d+:\d+:\d+)/;};                       # store time into $ref variable.seek IN,0,0;             # Jump back to the begin of filewhile (<IN>) {    print if /^(.{15})\s/&&str2time($1)>$ref-600;}

But if you really wanna use

There is a very quick pure bash script:

Warning: This use recent bashisms, require $BASH_VERSION 4.2 or higher.

#!/bin/bashdeclare -A monthfor i in {1..12};do    LANG=C printf -v var "%(%b)T" $(((i-1)*31*86400))    month[$var]=$i  doneprintf -v now "%(%s)T" -1printf -v ref "%(%m%d%H%M%S)T" $((now-600))while read line;do    printf -v crt "%02d%02d%02d%02d%02d" ${month[${line:0:3}]} \        $((10#${line:4:2})) $((10#${line:7:2})) $((10#${line:10:2})) \        $((10#${line:13:2}))    # echo " $crt < $ref ??"   # Uncomment this line to print each test    [ $crt -gt $ref ] && breakdonecat

Store this script and run:

cat >last10min.shchmod +x last10min.shsudo cat /var/log/syslog | ./last10min.sh

Strictly: extract last 10 minutes from logfile

Simply replace line 10, but you have to place filename in the script and not use it as a filter:

#!/bin/bashdeclare -A monthfor i in {1..12};do    LANG=C printf -v var "%(%b)T" $(((i-1)*31*86400))    month[$var]=$i  doneread now < <(date -d "$(tail -n1 $1|head -c 15)" +%s)printf -v ref "%(%m%d%H%M%S)T" $((now-600))export -A month{    while read line;do        printf -v crt "%02d%02d%02d%02d%02d" ${month[${line:0:3}]} \            $((10#${line:4:2})) $((10#${line:7:2})) $((10#${line:10:2})) \            $((10#${line:13:2}))        [ $crt -gt $ref ] && break    done    cat} <$1

A script into a function

As commented by ajcg, this could be nice to put efficient perl script into a bash function:

recentLog(){     perl -MDate::Parse -ne '        print if/^(.{'${3:-15}'})\s/ &&            str2time($1)>time-'$((                60*${2:-10}            )) ${1:-/var/log/daemon.log}}

Usage:

recentLog [filename] [minutes] [time sting length]

  • filename of log file
  • minutes max before now of lines to show
  • time sting length from begin of lines (default 15).


You can match the date range using simple string comparison, for example:

d1=$(date --date="-10 min" "+%b %_d %H:%M")d2=$(date "+%b %_d %H:%M")while read line; do    [[ $line > $d1 && $line < $d2 || $line =~ $d2 ]] && echo $linedone

For example if d1='Dec 18 10:19' and d2='Dec 18 10:27' then the output will be:

Dec 18 10:19:16Dec 18 10:19:23Dec 18 10:21:03Dec 18 10:22:54Dec 18 10:27:32

Or using awk if you wish:

awk -v d1="$d1" -v d2="$d2" '$0 > d1 && $0 < d2 || $0 ~ d2'