Accurately Calculating CPU Utilization in Linux using /proc/stat

I think iowait/irq/softirq are not counted in one of the first 4 numbers. You can see the comment of irqtime_account_process_tick in kernel code for more detail:

(for Linux kernel 4.1.1)

2815  * Tick demultiplexing follows the order2816  * - pending hardirq update    <-- this is irq2817  * - pending softirq update    <-- this is softirq2818  * - user_time2819  * - idle_time         <-- iowait is included in here, discuss below2820  * - system time2821  *   - check for guest_time2822  *   - else account as system_time

For the idle time handling, see account_idle_time function:

2772 /*2773  * Account for idle time.2774  * @cputime: the cpu time spent in idle wait2775  */2776 void account_idle_time(cputime_t cputime)2777 {2778         u64 *cpustat = kcpustat_this_cpu->cpustat;2779         struct rq *rq = this_rq();27802781         if (atomic_read(&rq->nr_iowait) > 0)2782                 cpustat[CPUTIME_IOWAIT] += (__force u64) cputime;2783         else2784                 cpustat[CPUTIME_IDLE] += (__force u64) cputime;2785 }

If the cpu is idle AND there is some IO pending, it will count the time in CPUTIME_IOWAIT. Otherwise, it is count in CPUTIME_IDLE.

To conclude, I think the jiffies in irq/softirq should be counted as "busy" for cpu because it was actually handling some IRQ or soft IRQ. On the other hand, the jiffies in "iowait" should be counted as "idle" for cpu because it was not doing something but waiting for a pending IO to happen.

from busybox, its top magic is:

static const char fmt[] ALIGN1 = "cp%*s %llu %llu %llu %llu %llu %llu %llu %llu";    int ret;    if (!fgets(line_buf, LINE_BUF_SIZE, fp) || line_buf[0] != 'c' /* not "cpu" */)        return 0;    ret = sscanf(line_buf, fmt,            &p_jif->usr, &p_jif->nic, &p_jif->sys, &p_jif->idle,            &p_jif->iowait, &p_jif->irq, &p_jif->softirq,            &p_jif->steal);    if (ret >= 4) {        p_jif->total = p_jif->usr + p_jif->nic + p_jif->sys + p_jif->idle            + p_jif->iowait + p_jif->irq + p_jif->softirq + p_jif->steal;        /* procps 2.x does not count iowait as busy time */        p_jif->busy = p_jif->total - p_jif->idle - p_jif->iowait;    }