How do I programmatically disable hardware prefetching? How do I programmatically disable hardware prefetching? linux linux

How do I programmatically disable hardware prefetching?


You can enable or disable the hardware prefetchers using msr-toolshttp://www.kernel.org/pub/linux/utils/cpu/msr-tools/.

The following enables the hardware prefetcher (by unsetting bit 9):

[root@... msr-tools-1.2]# ./wrmsr -p 0 0x1a0 0x60628e2089 [root@... msr-tools-1.2]# ./rdmsr 0x1a0 60628e2089

The following disables the hardware prefetcher (by enabling bit 9):

[root@... msr-tools-1.2]# ./wrmsr -p 0 0x1a0 0x60628e2289 [root@... msr-tools-1.2]# ./rdmsr 0x1a0 60628e2289

Programatically, you can do this as root by opening /dev/cpu/<cpunumber>/msr andusing pwrite to write to the msr "file" at the 0x1a0 offset.


From the Intel reference:
This instruction must be executed at privilege level 0 or in real-address mode; otherwise, a general protection exception #GP(0) will be generated. Specifying a reserved or unimplemented MSR address in ECX will also cause a general protection exception.

...
The CPUID instruction should be used to determine whether MSRs are supported (EDX[5]=1)before using this instruction.

So, your fault might be related to a cpu that doesn't support MSRs or using the wrong MSR address.

There are lots of examples of using the MSRs in the kernel source:

In the kernel source, for a single cpu, it demonstrates disabling prefetch for the Xeon in arch/i386/kernel/cpu/intel.c, in the function:

static void __cpuinit Intel_errata_workarounds(struct cpuinfo_x86 *c)

The rdmsr function arguments are the msr number, a pointer to the low 32 bit word, and a pointer to the high 32 bit word.
The wrmsr function arguments are the msr number, the low 32 bit word value, and the high 32 bit word value.

multi-core or smp systems have to pass the cpu struct in as the first argument:
void rdmsr_on_cpu(unsigned int cpu, u32 msr_no, u32 *l, u32 *h);
void wrmsr_on_cpu(unsigned int cpu, u32 msr_no, u32 l, u32 h);


In 2014 Intel published info about h/w prefetcher disabling with 0x1a4 msr (1a4 msr) for Nehalem, Westmere, Sandy Bridge, Ivy Bridge, Haswell, Broadwell (and probably newer cores). Link was found by bholanath here:

https://software.intel.com/en-us/articles/disclosure-of-hw-prefetcher-control-on-some-intel-processors Disclosure of H/W prefetcher control on some Intel processors - Vish Viswanathan (Intel), September 24, 2014

This article discloses the MSR setting that can be used to control the various h/w prefetchers that are available on Intel processors based on the following microarchitectures: Nehalem, Westmere, Sandy Bridge, Ivy Bridge, Haswell, and Broadwell.

The above mentioned processors support 4 types of h/w prefetchers for prefetching data. There are 2 prefetchers associated with L1-data cache (also known as DCU DCU prefetcher, DCU IP prefetcher) and 2 prefetchers associated with L2 cache (L2 hardware prefetcher, L2 adjacent cache line prefetcher).

There is a Model Specific Register (MSR) on every core with address of 0x1A4 that can be used to control these 4 prefetchers. Bits 0-3 in this register can be used to either enable or disable these prefetchers. Other bits of this MSR are reserved.

They are local to every CPU core and can be changed by root with help of msr linux kernel driver. They are used by Intel to measure memory latency in NUMA with Intel MLC tool:

For example, Intel Memory Latency Checker tool (http://www.intel.com/software/mlc) modifies the prefetchers through writes to MSR 0x1a4 to measure accurate latencies and restores them to the original state on exit.