Kernel hardening for the rest of us
For years, the best runtime memory protection available for Linux was locked behind a commercial licence. grsecurity’s PaX patchset — the gold standard for mitigating memory safety vulnerabilities — went closed-source in April 2017. If you wanted W^X enforcement, MPROTECT, or meaningful ASLR hardening beyond what the vanilla kernel offers, you paid for a subscription or you went without.
In October 2024, Edera changed this. OpenPaX is an open-source kernel patch under GPLv2 that revives the core PaX protections on modern hardware. I have been running it on both my workstation and my server for several weeks now. It is stable. It works. And it does something grsecurity’s PaX never managed for me: it runs cleanly as a Xen dom0.
Why PaX matters
The principle is simple and old: memory should be either writable or executable, never both at the same time. This is W^X — write XOR execute. If an attacker exploits a buffer overflow to inject code into a writable memory region, that region must not be executable. If a region contains executable code, it must not be writable. The moment you enforce this, an entire class of exploits stops working.
PaX enforces W^X through MPROTECT restrictions: programs cannot change a non-executable memory page to executable after the fact, cannot make read-only executable pages writable again, and cannot create executable pages from anonymous memory. This is not a minor improvement. It eliminates the most common technique attackers use to turn a memory corruption bug into code execution.
The vanilla Linux kernel has made progress here — the Kernel Self-Protection Project (KSPP), led by Kees Cook, has been working since 2015 to upstream hardening features that were pioneered by PaX and grsecurity. But the upstream kernel still does not enforce W^X as strictly as PaX does. OpenPaX closes that gap.
The patchset
OpenPaX is maintained by Ariadne Conill (Alpine Linux maintainer and Edera co-founder) and lives at github.com/edera-dev/linux-openpax. The patchset is exactly 29 commits, rebased on top of vanilla kernel releases. Currently four branches are maintained:
| Branch | Base kernel |
|---|---|
| v6.10.x-openpax | 6.10 (dev snapshot) |
| v6.11.x-openpax | 6.11.5 |
| v6.12.x-openpax | 6.12.19 |
| master | 6.14-rc7/rc8 |
The patches are not identical across branches — they are adapted to upstream API changes between kernel versions. For example, the map_deny_write_exec() function signature changed between 6.11 and 6.12, and the PAXF flag definitions moved from sched/coredump.h to mm_types.h between 6.12 and 6.14. You cannot blindly apply the 6.12 patches to a 6.13 kernel. Pick the branch that matches your kernel’s major.minor series.
Applying it on Gentoo
If you run Gentoo, applying OpenPaX is straightforward thanks to Portage’s user patch system. Clone the repository, check out the branch matching your kernel, and extract the patches:
git clone https://github.com/edera-dev/linux-openpax
cd linux-openpax
git format-patch -29 origin/v6.12.x-openpax
-o /etc/portage/patches/sys-kernel/gentoo-sources-6.12/Replace gentoo-sources-6.12 with whatever your kernel package is — sys-kernel/vanilla-sources, your own custom kernel package, whatever matches. The -29 captures all 29 OpenPaX patches from the tip of the branch. Next time you emerge your kernel sources, Portage applies the patches automatically.
After patching, enable OpenPaX in your kernel configuration:
Security options --->
[*] OpenPaX
[*] Support PaX soft mode
[*] Use filesystem extended attributes to modify OpenPaX featuresSoft mode is useful during initial deployment: it makes OpenPaX protections opt-in per binary rather than enforced globally, giving you time to identify what breaks before switching to enforcement.
Gentoo still knows about PaX
Here is the pleasant surprise: Gentoo never fully forgot about PaX. The infrastructure from the old Hardened Gentoo project is still in the tree. The pax-utils.eclass provides a pax-mark function that ebuilds can call to set per-binary PaX flags during installation. Packages that know they need MPROTECT exceptions — because they use JIT compilation or runtime code generation — already have the right pax-mark -m calls in their ebuilds.
By default, this machinery is dormant. The eclass defaults PAX_MARKINGS to "none". To activate it, add to /etc/portage/make.conf:
PAX_MARKINGS="XT"This tells Portage to apply XATTR_PAX markings — the filesystem extended attributes that OpenPaX reads. With this set, packages like firefox, node, Java, and others that call pax-mark in their ebuilds will automatically get the correct MPROTECT exceptions applied every time they are emerged. No manual intervention, no post-install scripts.
There is also a pax-kernel USE flag. Packages that use it can conditionally adjust their build when they know they are targeting a PaX-hardened kernel — for instance, disabling JIT by default rather than relying on a runtime exception.
What breaks and how to fix it
Any application that relies on JIT (just-in-time) compilation will conflict with MPROTECT. JIT compilers generate executable code at runtime — they write machine instructions into memory and then execute them. This is exactly what MPROTECT is designed to prevent.
The two most common casualties are firefox and node. Both use JavaScript JIT engines that need to mprotect memory regions from writable to executable. Under OpenPaX with MPROTECT enforced, they segfault.
If the ebuild already calls pax-mark -m and you have PAX_MARKINGS="XT" set, the exception is applied automatically during installation. If it does not, or if you are dealing with a binary you installed outside Portage, the manual fix is:
paxctl-ng -m /usr/lib64/firefox/firefox
paxctl-ng -m /usr/bin/nodeThe -m flag disables MPROTECT for that specific binary via extended attributes. Every other binary on the system remains protected. This is the same per-binary exception mechanism that PaX has used for over two decades — it is well understood and the tradeoff is explicit.
If you find a package that needs an exception but the ebuild does not set one, that is worth a bug report — the Gentoo Hardened team has historically been responsive to these.
Xen dom0: what grsecurity could not do
This was the detail that surprised me. In the old grsecurity days, running PaX on a Xen dom0 was a kernel-panic-at-boot affair — the kind of situation where you reach for your rescue USB and reconsider your life choices. The interaction between PaX’s memory protections and Xen’s paravirtualisation layer produced the kind of hard failures that leave no useful logs.
OpenPaX has no such problem. My Xen dom0 boots, runs guests, and has been stable for weeks. For anyone building infrastructure that combines virtualisation with kernel hardening — which is to say, anyone building serious infrastructure — this matters.
Where this fits
This article is intended as a reference for a future series on building your own cloud environment from scratch. In that context, OpenPaX is one piece of host security: the kernel protects itself and the processes running on it from memory exploitation. Combined with KSPP’s recommended kernel configuration settings, Xen isolation between guests, and per-guest hardening, it forms part of a defence-in-depth approach.
The broader point is this: runtime memory protection was a solved problem that became an unsolved problem when grsecurity went proprietary. OpenPaX makes it solved again, for everyone, under GPLv2. If you run Linux in production and you care about the security of your infrastructure, 29 patches is a small investment.
Further reading:
OpenPaX on GitHub: github.com/edera-dev/linux-openpax
Edera’s announcement: edera.dev/stories/edera-restores-security-benefits-for-linux-application-memory-safety-with-openpax
KSPP Recommended Settings: kspp.github.io/Recommended_Settings.html
Gentoo PaX Quickstart (historical, but still useful for understanding per-binary flags): wiki.gentoo.org/wiki/Hardened/PaX_Quickstart