Awesome

kernel_compiler_patch

Why a specific patch?

The kernel uses its own set of CFLAGS, KCFLAGS. For example, see:

• arch/x86/Makefile
• arch/x86/Makefile_32.cpu
• arch/x86/Kconfig.cpu

Alternative way to define a -march= option without this patch

As pointed out by codemac in this topic, one can simply export the value/values for the KCFLAGS and KCPPFLAGS before calling make to achieve the same result, see here.

export KCFLAGS=' -march=znver3'
export KCPPFLAGS=' -march=znver3'
make all

New tunings

These patches adds additional tunings via new x86-64 ISA levels and more micro-architecture options to the Linux kernel in three broad classes.

1. New generic x86-64 ISA levels

When compiling the Generic x86-64 Processor family target, these are selectable under:

 Processor type and features ---> x86-64 compiler ISA level

• x86-64 A value of (1) is the default and builds with the generic x86-64 ISA level
• x86-64-v2 A value of (2) brings support for vector instructions up to Streaming SIMD Extensions 4.2 (SSE4.2) and Supplemental Streaming SIMD Extensions 3(SSSE3), the POPCNT instruction, and CMPXCHG16B.
• x86-64-v3 A value of (3) adds vector instructions up to AVX2, MOVBE, and additional bit-manipulation instructions.

x86-64-v4 does exist but it adds vector instructions from some of the AVX-512 variants which the kernel does not use so including it does not make much sense.

Users of glibc 2.33 and above can see which level is supported by running one of the follownig:

/lib/ld-linux-x86-64.so.2 --help | grep supported
/lib64/ld-linux-x86-64.so.2 --help | grep supported

2. New micro-architectures levels

These are selectable under:

 Processor type and features ---> Processor family

3. Auto-detected micro-architecture levels

These are also selectable under:

 Processor type and features ---> Processor family

They have the ability to compile by passing the '-march=native' option which, according to the GCC manual "selects the CPU to generate code for at compilation time by determining the processor type of the compiling machine. Using -march=native enables all instruction subsets supported by the local machine and will produce code optimized for the local machine under the constraints of the selected instruction set."

Users of Intel CPUs should select the 'Intel-Native' option and users of AMD CPUs should select the 'AMD-Native' option.

Benchmarks

Setup

The test machine measured the time it took to make bzImage of the linux kernel source (.config generated by make x86_64_defconfig prior).

Three separate test machines were evaluated:

1. AMD Ryzen 9 5950X
2. Intel i7-4790K
3. Intel N100

Separate kernels were first compiled from source patched with more-uarches-for-kernel-6.8-rc4+.patch.

• Kernel 1 used the default menu config option for Processor family = Generic x86-64
• Kernel 2 used the menu config option for Processor family = x86-64-v3
• Kernel 3 used the menu config option for Processor family = AMD Zen 3 or Intel Haswell or Intel Alder Lake

The make test

Each machine was booted into its respective kernel and the make test was conducted. Then the next kernel was installed and the machine was booted into it and the make test was again conducted.

The stress-ng benchmark

The AMD 5950X ran stress-ng --taskset 0-1 --metrics-brief -t 30s --foo 2 12 times where foo was one of: af-alg, fork, mmap, or pipe under Kernel 1 and then again under Kernel 3.

Conclusion

Consistently across all three test machines, the kernels built with the optimized processor family options introduced by the patch hosted in this repo ran the make test faster than the kernel compiled with the default processor family option by a small (<1% difference) but statistically significant amount as measured by this make compilation.

The stress-ng testing generally showed small improvements (1-2% faster) and one showing no difference.

What does this mean for real-world usage? Maybe nothing. The intent was to see if something easily automatable could show some value in applying these micro-arch tunings. People have historically gravitated to compilation-based benchmarks so that coupled with ease-of-use point is why I settled on it. If someone has a good kernel-centric benchmark, I am interested to see a controlled comparison.

Discussion

1. All the assumptions for ANOVA are met:
   • Data are normally distributed
   • The population variances are fairly equal
2. The boxplot plot clearly show significance for either pair-wise comparison
   • Pair-wise analysis by Tukey-Kramer data shown for all pairs (see tables)

In other words, x86-64-v3 is significantly different from generic x86-64. The various subtargets are also significantly different from x86-64.

The make test

Stats for Machine 1. AMD Ryzen 9 X5950

Stats for Machine 2. Intel i7-4790K

Stats for Machine 3. Intel N100

Comparing GCC to Clang

The Ryzen 9 5950X was used to compare kernels built with GCC and Clang each with Generic x86-64 and x86-64-v3. The results are consistent for both compilers.

The stress-ng benchmarks

Here, stress-ng microbenchmark improvements or regressions (or neutral changes) were as follows (average from 12 x 30 sec runs):

af-alg: +2.7% (kernel AL_ALG crypto)
fork:     *   (process fork/exit)
mmap:   +1.6% (memory mapping)
pipe:   +1.3% (pipe + context switch)

*no statistically significant difference at p<0.05

Software versions used

All machines ran Arch Linux with the all stock repo packages with the exception of the kernel (see below). At the time of work, the following the toolchain versions were used:

• binutils 2.43+r4+g7999dae6961-1
• clang 18.0.1-1
• gcc 14.2.1+r134+gab884fffe3fc-1
• gcc-libs 14.2.1+r134+gab884fffe3fc-1
• glibc 2.40+r16+gaa533d58ff-2
• linux-api-headers 6.10-1
• stress-ng 0.18.04-1

The kernel packages were built on the official Arch Linux PKGBUILD for kernel version 6.10.10-arch1-1 applying the distro config differing only by the modifications introduced by the aforementioned patch from this repo.

The benchmark was compiling the vanilla Linux kernel version 6.10.10 and as mentioned above, the .config used was generated by running make x86_64_defconfig.

References

• Script to run the benchmark: make_bench.sh
• Data for three machines: results.csv
• Data for GCC vs Clang: results2.csv
• Data for stress-ng tests: stress-ng-data.csv

Credit

• Original author: jeroen AT linuxforge DOT net
• Link to original version: http://www.linuxforge.net/docs/linux/linux-gcc.php
• Box plot generated with statisty.app
• ANOVA stats generated with astatsa.com

Legacy support

Find support for older version of the linux kernel and of gcc in the outdated_versions directory.