diff --git a/README.md b/README.md index 0d8389d..bd85b35 100644 --- a/README.md +++ b/README.md @@ -68,6 +68,45 @@ sudo ./install.sh --iommu Or add `iommu=pt` to your kernel cmdline manually. IOMMU must also be enabled in BIOS (VT-d on Intel, AMD-Vi / SVM on AMD). +### Surviving Kernel Updates + +The patched modules are built against one specific kernel. Without help, the first kernel update leaves the card on the stock driver — reporting 8GB instead of 64GB — or on nouveau. The installer wires the rebuild into the kernel update path by default, so this does not happen. + +A new kernel triggers a rebuild through the package manager hook for your distro, **before** you reboot: + +| Distro | Hook | +|---|---| +| Fedora, RHEL, openSUSE | `/etc/kernel/install.d/95-cmpunlocker.install` | +| Debian, Ubuntu, HiveOS | `/etc/kernel/postinst.d/cmpunlocker` | +| Arch | `/etc/pacman.d/hooks/95-cmpunlocker.hook` | + +`cmpunlocker-rebuild.service` is the safety net for what hooks cannot see — a hand-built kernel, a restored snapshot, or a hook that ran before the kernel headers were unpacked. It holds the boot until the patched modules exist, because a rig that silently comes up at 8GB is worse than one slow boot. It gives up after three consecutive failures rather than delaying every boot forever. + +Two more things keep the stock driver from winning: + +- `/etc/depmod.d/cmpunlocker.conf` makes the patched modules outrank the stock ones. The distro driver is rebuilt on every kernel update too, into `extra/` (akmod) or `updates/dkms/` (dkms), right next to ours. +- `nvidia-fallback.service` is masked and nouveau is blacklisted, so a driver that fails to load does not hand the card to nouveau. + +**The NVIDIA packages are pinned to their installed version.** A driver upgrade past the versions in `driver/VERSION` makes every later rebuild fail, which is the rollback this is meant to prevent. GPU *firmware* packages are deliberately left unpinned — they belong to `linux-firmware` and holding them back can wedge system upgrades. + +```bash +sudo ./install.sh --no-pin # allow driver upgrades, accept the risk +sudo ./install.sh --no-persist # manage rebuilds yourself +``` + +Checking on it: + +```bash +systemctl status cmpunlocker-rebuild +sudo /usr/lib/cmpunlocker/pin-packages.sh status +cat /var/log/cmpunlocker/rebuild-$(uname -r).log +sudo /usr/lib/cmpunlocker/rebuild.sh # rebuild for the running kernel by hand +``` + +To take a pinned driver upgrade: `sudo /usr/lib/cmpunlocker/pin-packages.sh unpin`, upgrade, then re-run `install.sh` (which re-pins). If the new driver version is not in `driver/VERSION`, the build will refuse it. + +Everything above is undone by `./remove.sh --yes`. + --- ## Verify @@ -117,6 +156,7 @@ cd benchmark && nvcc -O3 -o nvidia_bench nvidia_bench.cu -lnvidia-ml -ldl \ | GPU-to-GPU P2P (`cudaDeviceEnablePeerAccess`) | Opt-in, `--p2p` | | HBM2e memory overclock/downclock | Working | | Persistence across reboot (patched modules) | Working | +| Persistence across kernel updates (auto-rebuild) | Working | --- diff --git a/driver/build.sh b/driver/build.sh index 1374174..0f3ad61 100755 --- a/driver/build.sh +++ b/driver/build.sh @@ -12,7 +12,13 @@ SRC_NAME="open-gpu-kernel-modules-${VERSION}" SRC_DIR="${BUILD_ROOT}/${SRC_NAME}" TARBALL="${BUILD_ROOT}/${SRC_NAME}.tar.gz" TARBALL_URL="https://github.com/NVIDIA/open-gpu-kernel-modules/archive/refs/tags/${VERSION}.tar.gz" -KVER="$(uname -r)" +# +# CMPUNLOCKER_KVER lets the kernel-update hooks build for a kernel that is not +# the running one: dnf/apt unpack the new modules tree before the reboot, so +# the patched driver is already in place the first time that kernel boots. +# +KVER="${CMPUNLOCKER_KVER:-$(uname -r)}" +KRUNNING="$(uname -r)" KSRC="/lib/modules/${KVER}/build" INSTALL_MOD_DIR="/lib/modules/${KVER}/updates/cmpunlocker" @@ -309,6 +315,20 @@ rebuild_initramfs() { } rebuild_initramfs || true + +# +# Building for a kernel that is not running (kernel-update hook): the modules +# are in place for the next boot and there is nothing to swap in now. Trying to +# would unload the driver the current kernel is using and load nothing back. +# +if [[ "${KVER}" != "${KRUNNING}" ]]; then + echo "" + ok "Built and installed for kernel ${KVER} (running: ${KRUNNING})" + info "Takes effect when ${KVER} boots" + echo "" + exit 0 +fi + resolved="$(modprobe -n -v nvidia 2>/dev/null | awk '/insmod/ {print $2; exit}' || true)" if [[ -n "${resolved}" ]]; then info "modprobe will load: ${resolved}" diff --git a/install.sh b/install.sh index 39a725c..3d37a75 100755 --- a/install.sh +++ b/install.sh @@ -12,6 +12,8 @@ CONFIGURE_IOMMU=0 MCLK_NDIV="" MCLK_TIMINGS="" ENABLE_P2P="" +INSTALL_PERSIST=1 +PIN_PACKAGES=1 VERBOSE=0 for arg in "$@"; do case "${arg}" in @@ -19,10 +21,13 @@ for arg in "$@"; do --mclk-ndiv=*) MCLK_NDIV="${arg#*=}" ;; --mclk-timings=*) MCLK_TIMINGS="${arg#*=}" ;; --p2p) ENABLE_P2P=1 ;; + --no-persist) INSTALL_PERSIST=0 ;; + --no-pin) PIN_PACKAGES=0 ;; -v|--verbose) VERBOSE=1 ;; -h|--help) cat <<'EOF' -Usage: sudo ./install.sh [--iommu] [--mclk-ndiv=N] [--mclk-timings=N] [--p2p] [-v] +Usage: sudo ./install.sh [--iommu] [--mclk-ndiv=N] [--mclk-timings=N] [--p2p] + [--no-persist] [--no-pin] [-v] --iommu Add iommu=pt to the kernel command line (see README for details) -v, --verbose Show the full build output instead of a progress bar. The @@ -48,6 +53,27 @@ Usage: sudo ./install.sh [--iommu] [--mclk-ndiv=N] [--mclk-timings=N] [--p2p] [- data silently or wedges the memory controller. Scaled: tRC tRFC tRAS tRP tRCD tWR tFAW tRRD. Never touched: CL, WL, tCCD. + --no-persist Do not survive kernel updates. By default the installer wires + the patched modules into the kernel-update path, so a new + kernel gets them rebuilt automatically instead of booting on + the stock driver (8GB) or falling back to nouveau. Use this + only if you manage module rebuilds yourself. + --no-pin Do not pin the NVIDIA packages. By default they are held at + the currently installed version, because a driver upgrade to + a version cmpunlocker does not support makes every later + rebuild fail and drops the card back to stock. With this flag + the packages upgrade freely and you take that risk on. + +Surviving kernel updates (on by default): + A kernel update rebuilds the patched modules through a package-manager hook + (/etc/kernel/install.d, /etc/kernel/postinst.d, or a pacman hook) before you + reboot. cmpunlocker-rebuild.service is the safety net for anything the hook + misses — it holds boot until the modules exist rather than letting the card + come up unpatched. Check state with: + + systemctl status cmpunlocker-rebuild + sudo /usr/lib/cmpunlocker/pin-packages.sh status + cat /var/log/cmpunlocker/rebuild-$(uname -r).log Memory geometry is selected automatically from PCI device ID: 10de:20c2 → 8GB card → 64GB unlock @@ -89,7 +115,7 @@ warn() { echo -e "${YELLOW}!${NC} $*"; } err() { echo -e "${RED}✗${NC} $*" >&2; } die() { err "$*"; exit 1; } -STEP_TOTAL=5 +STEP_TOTAL=6 STEP_NOW=0 progress_bar() { @@ -390,6 +416,33 @@ CMPUNLOCKER_VERBOSE="${VERBOSE}" \ } ok "Patched modules installed" +step "Surviving kernel updates" +PERSIST_STATUS="skipped" +if (( INSTALL_PERSIST == 0 )); then + info "Persistence not requested (--no-persist)" + warn "A kernel update will boot on the stock driver (8GB) until you re-run this installer" +elif [[ ! -x "${SCRIPT_DIR}/persist/install-persist.sh" ]]; then + warn "persist/install-persist.sh missing — kernel updates will not be survived" +else + # + # The persistence layer is deliberately not fatal: the patched modules for + # the running kernel are already installed and working at this point, and + # failing the whole install over the automation would be a worse outcome. + # + if CMPUNLOCKER_DRIVER_VERSION="${detected}" \ + CMPUNLOCKER_MCLK_NDIV="${MCLK_NDIV}" \ + CMPUNLOCKER_MCLK_TIMINGS="${MCLK_TIMINGS}" \ + CMPUNLOCKER_ENABLE_P2P="${ENABLE_P2P}" \ + CMPUNLOCKER_PIN_PACKAGES="${PIN_PACKAGES}" \ + "${SCRIPT_DIR}/persist/install-persist.sh"; then + PERSIST_STATUS="installed" + else + PERSIST_STATUS="failed" + warn "Could not install kernel-update persistence — see the log" + warn "The patched modules for $(uname -r) are installed and working regardless" + fi +fi + step "Configuring IOMMU (passthrough)" IOMMU_STATUS="skipped" IOMMU_PARAMS="" @@ -575,6 +628,19 @@ fi if [[ -n "${ENABLE_P2P}" ]]; then echo "P2P: forced on (verify with a real peer-to-peer copy)" fi +case "${PERSIST_STATUS}" in + installed) + PIN_RESULT="$(cat /var/lib/cmpunlocker/state/pin-status 2>/dev/null || echo unknown)" + case "${PIN_RESULT}" in + ok) echo "Kernel updates: modules rebuilt automatically, NVIDIA packages pinned" ;; + failed) echo "Kernel updates: modules rebuilt automatically" + echo " NVIDIA packages NOT pinned — see the warning above" ;; + *) echo "Kernel updates: modules rebuilt automatically, packages not pinned" ;; + esac + ;; + failed) echo "Kernel updates: NOT survived (persistence install failed)" ;; + skipped) echo "Kernel updates: NOT survived (--no-persist)" ;; +esac echo "" echo "Next:" echo -e " 1. Cold reboot: ${CYAN}sudo shutdown -h now${NC} (then power on)" diff --git a/persist/boot-check.sh b/persist/boot-check.sh new file mode 100755 index 0000000..57128a9 --- /dev/null +++ b/persist/boot-check.sh @@ -0,0 +1,84 @@ +#!/bin/bash +# +# cmpunlocker - boot-time safety net. +# +# Installed to /usr/lib/cmpunlocker/boot-check.sh, run by +# cmpunlocker-rebuild.service before multi-user.target. +# +# The kernel-update hooks are the normal path and they run before the reboot, +# so in the common case this exits in milliseconds. It exists for the cases the +# hooks cannot cover: a hand-built kernel, a restored snapshot, a distro whose +# hook directory is not one of the three supported ones, or a hook that failed +# because the headers were not unpacked yet. +# +set -uo pipefail + +KVER="$(uname -r)" +MODULE="/lib/modules/${KVER}/updates/cmpunlocker/nvidia.ko" +STATE_DIR="/var/lib/cmpunlocker/state" +ATTEMPTS="${STATE_DIR}/boot-attempts-${KVER}" +MAX_ATTEMPTS=3 + +log() { echo "cmpunlocker: $*"; } + +# Fast path: patched modules already present for this kernel. No boot delay. +if [[ -f "${MODULE}" ]]; then + exit 0 +fi + +mkdir -p "${STATE_DIR}" 2>/dev/null || true + +# +# A build that fails for a permanent reason (unsupported driver version, no +# compiler) would otherwise add its full runtime to every single boot, forever. +# After MAX_ATTEMPTS stop blocking and leave it to the operator. +# +count=0 +[[ -r "${ATTEMPTS}" ]] && count="$(cat "${ATTEMPTS}" 2>/dev/null || echo 0)" +[[ "${count}" =~ ^[0-9]+$ ]] || count=0 + +if (( count >= MAX_ATTEMPTS )); then + log "patched modules missing for ${KVER} and ${count} rebuilds already failed" + log "not blocking boot again — fix the cause and run: sudo /usr/lib/cmpunlocker/rebuild.sh" + log "log: /var/log/cmpunlocker/rebuild-${KVER}.log" + exit 0 +fi + +echo "$((count + 1))" > "${ATTEMPTS}" + +log "patched modules missing for ${KVER} — rebuilding before boot continues" +log "this takes a few minutes; the GPU would otherwise come up unpatched" + +/usr/lib/cmpunlocker/rebuild.sh "${KVER}" + +if [[ ! -f "${MODULE}" ]]; then + log "rebuild did not produce ${MODULE}" + log "booting on the stock driver — see /var/log/cmpunlocker/rebuild-${KVER}.log" + exit 0 +fi + +rm -f "${ATTEMPTS}" + +# +# The modules exist now, but udev may already have autoloaded the stock driver +# during early boot. Nothing has opened /dev/nvidia* this early (no X, no +# persistenced yet), so swapping it out here avoids a second reboot. +# +if lsmod | grep -q '^nvidia'; then + log "swapping the stock driver out for the patched one" + for mod in nvidia_drm nvidia_uvm nvidia_modeset nvidia; do + modprobe -r "${mod}" 2>/dev/null || true + done + sleep 1 +fi + +if modprobe nvidia 2>/dev/null; then + modprobe nvidia-modeset 2>/dev/null || true + modprobe nvidia-uvm 2>/dev/null || true + modprobe nvidia-drm 2>/dev/null || true + log "patched modules loaded for ${KVER}" +else + log "patched modules built but not loaded — reboot to apply" +fi + +exit 0 diff --git a/persist/cmpunlocker-rebuild.service b/persist/cmpunlocker-rebuild.service new file mode 100644 index 0000000..25994e0 --- /dev/null +++ b/persist/cmpunlocker-rebuild.service @@ -0,0 +1,24 @@ +[Unit] +Description=cmpunlocker: ensure patched NVIDIA modules exist for the running kernel +Documentation=https://github.com/asm64-hooligan/cmpunlocker +# /var/lib/cmpunlocker holds the build payload, so the filesystems have to be up. +After=local-fs.target +# Blocks boot from completing until the patched driver is in place. A CMP that +# comes up on the stock driver silently runs at 8 GB, which is worse than a +# one-off slow boot. +Before=multi-user.target +Before=nvidia-persistenced.service +ConditionPathExists=/usr/lib/cmpunlocker/boot-check.sh + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/lib/cmpunlocker/boot-check.sh +# A full rebuild is ~2-3 min on a typical rig; allow for much slower hardware. +TimeoutStartSec=1800 +# The operator needs to see why the boot is taking minutes. +StandardOutput=journal+console +StandardError=journal+console + +[Install] +WantedBy=multi-user.target diff --git a/persist/depmod-cmpunlocker.conf b/persist/depmod-cmpunlocker.conf new file mode 100644 index 0000000..2e24163 --- /dev/null +++ b/persist/depmod-cmpunlocker.conf @@ -0,0 +1,16 @@ +# cmpunlocker - make the patched modules win over the distro's stock nvidia. +# +# The stock driver is rebuilt automatically on every kernel update and lands in +# a directory depmod also searches: +# +# extra/ akmod (Fedora, RHEL) +# updates/dkms/ dkms (Debian, Ubuntu, HiveOS, Arch) +# +# Both sit alongside updates/cmpunlocker/, so which nvidia.ko modprobe resolves +# to would otherwise depend on depmod's search order. These lines make it +# explicit: the patched build always wins. +override nvidia * updates/cmpunlocker +override nvidia-modeset * updates/cmpunlocker +override nvidia-uvm * updates/cmpunlocker +override nvidia-drm * updates/cmpunlocker +override nvidia-peermem * updates/cmpunlocker diff --git a/persist/hooks/kernel-install.sh b/persist/hooks/kernel-install.sh new file mode 100755 index 0000000..abb6048 --- /dev/null +++ b/persist/hooks/kernel-install.sh @@ -0,0 +1,35 @@ +#!/bin/bash +# +# cmpunlocker - systemd kernel-install hook (Fedora, RHEL, openSUSE, any distro +# using /etc/kernel/install.d). +# +# Installed as /etc/kernel/install.d/95-cmpunlocker.install. +# +# kernel-install calls this as: [entry-dir] [image] +# Numbered 95 so it runs after 50-depmod.install and 50-dracut.install have +# populated the new module tree. +# +set -uo pipefail + +COMMAND="${1:-}" +KVER="${2:-}" + +[[ -n "${KVER}" ]] || exit 0 + +case "${COMMAND}" in + add) + [[ -x /usr/lib/cmpunlocker/rebuild.sh ]] || exit 0 + echo "cmpunlocker: building patched NVIDIA modules for ${KVER}..." + /usr/lib/cmpunlocker/rebuild.sh "${KVER}" + ;; + remove) + # + # The kernel is going away, so its patched modules are dead weight. + # Leaving them behind makes /lib/modules grow without bound. + # + rm -rf "/lib/modules/${KVER}/updates/cmpunlocker" + rm -f "/var/lib/cmpunlocker/state/failed-${KVER}" + ;; +esac + +exit 0 diff --git a/persist/hooks/kernel-postinst.sh b/persist/hooks/kernel-postinst.sh new file mode 100755 index 0000000..815a460 --- /dev/null +++ b/persist/hooks/kernel-postinst.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# +# cmpunlocker - Debian/Ubuntu/HiveOS kernel hook. +# +# Installed as /etc/kernel/postinst.d/cmpunlocker. +# +# Called as: +# +# On Debian the headers package is sometimes unpacked after the image, so the +# build here can legitimately fail with "headers not installed yet". +# rebuild.sh records that and cmpunlocker-rebuild.service retries at boot. +# +set -uo pipefail + +KVER="${1:-}" + +[[ -n "${KVER}" ]] || exit 0 +[[ -x /usr/lib/cmpunlocker/rebuild.sh ]] || exit 0 + +echo "cmpunlocker: building patched NVIDIA modules for ${KVER}..." +/usr/lib/cmpunlocker/rebuild.sh "${KVER}" + +exit 0 diff --git a/persist/hooks/kernel-postrm.sh b/persist/hooks/kernel-postrm.sh new file mode 100755 index 0000000..9065f50 --- /dev/null +++ b/persist/hooks/kernel-postrm.sh @@ -0,0 +1,16 @@ +#!/bin/bash +# +# cmpunlocker - Debian/Ubuntu/HiveOS kernel removal hook. +# +# Installed as /etc/kernel/postrm.d/cmpunlocker. Drops the patched modules for +# a kernel that is being removed so /lib/modules does not grow without bound. +# +set -uo pipefail + +KVER="${1:-}" +[[ -n "${KVER}" ]] || exit 0 + +rm -rf "/lib/modules/${KVER}/updates/cmpunlocker" +rm -f "/var/lib/cmpunlocker/state/failed-${KVER}" + +exit 0 diff --git a/persist/hooks/pacman-trigger.sh b/persist/hooks/pacman-trigger.sh new file mode 100755 index 0000000..61fd402 --- /dev/null +++ b/persist/hooks/pacman-trigger.sh @@ -0,0 +1,33 @@ +#!/bin/bash +# +# cmpunlocker - Arch pacman hook trigger. +# +# Installed to /usr/lib/cmpunlocker/pacman-trigger.sh. pacman feeds the matched +# target paths on stdin (usr/lib/modules//vmlinuz), one per line. +# +# A kernel that was removed in the same transaction leaves its path on stdin +# too, so each version is checked for existence rather than assumed present. +# +set -uo pipefail + +[[ -x /usr/lib/cmpunlocker/rebuild.sh ]] || exit 0 + +while read -r target; do + kver="${target#usr/lib/modules/}" + kver="${kver%/vmlinuz}" + [[ -n "${kver}" ]] || continue + + if [[ -d "/usr/lib/modules/${kver}" ]]; then + echo "cmpunlocker: building patched NVIDIA modules for ${kver}..." + /usr/lib/cmpunlocker/rebuild.sh "${kver}" + else + # + # Kernel removed: drop its patched modules so /usr/lib/modules does not + # accumulate trees for kernels that can no longer boot. + # + rm -rf "/usr/lib/modules/${kver}/updates/cmpunlocker" + rm -f "/var/lib/cmpunlocker/state/failed-${kver}" + fi +done + +exit 0 diff --git a/persist/hooks/pacman.hook b/persist/hooks/pacman.hook new file mode 100644 index 0000000..6fb4bad --- /dev/null +++ b/persist/hooks/pacman.hook @@ -0,0 +1,19 @@ +# +# cmpunlocker - Arch Linux pacman hook. +# +# Installed as /etc/pacman.d/hooks/95-cmpunlocker.hook. Fires whenever a kernel +# package drops a new vmlinuz, which is the Arch equivalent of the kernel-install +# and postinst.d hooks used elsewhere. +# +[Trigger] +Type = Path +Operation = Install +Operation = Upgrade +Operation = Remove +Target = usr/lib/modules/*/vmlinuz + +[Action] +Description = cmpunlocker: rebuilding patched NVIDIA modules +When = PostTransaction +Exec = /usr/lib/cmpunlocker/pacman-trigger.sh +NeedsTargets diff --git a/persist/install-persist.sh b/persist/install-persist.sh new file mode 100755 index 0000000..9c4759d --- /dev/null +++ b/persist/install-persist.sh @@ -0,0 +1,196 @@ +#!/bin/bash +# +# cmpunlocker - install the kernel-update persistence layer. +# +# Called by install.sh. Puts everything needed to rebuild the patched modules +# in system locations, so a kernel update rebuilds them automatically and the +# machine never silently falls back to the stock 8 GB driver or to nouveau. +# +# What lands where: +# /var/lib/cmpunlocker/driver/ build payload (sources, patches, tarball) +# /etc/cmpunlocker/build.conf the compile-time flags to reproduce +# /usr/lib/cmpunlocker/*.sh rebuild / boot-check / pin helpers +# /etc/kernel/... or pacman hook rebuild trigger on kernel update +# cmpunlocker-rebuild.service boot-time safety net +# /etc/depmod.d/cmpunlocker.conf patched modules outrank the stock ones +# /etc/modprobe.d/cmpunlocker.conf nouveau stays blacklisted +# +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)" + +PAYLOAD_DIR="/var/lib/cmpunlocker" +CONF_DIR="/etc/cmpunlocker" +LIB_DIR="/usr/lib/cmpunlocker" + +PIN_PACKAGES="${CMPUNLOCKER_PIN_PACKAGES:-1}" + +if [[ -t 1 && -z "${NO_COLOR:-}" ]]; then + GREEN='\033[0;32m'; YELLOW='\033[1;33m'; CYAN='\033[0;36m'; NC='\033[0m' +else + GREEN=""; YELLOW=""; CYAN=""; NC="" +fi +info() { echo -e "${CYAN}==>${NC} $*"; } +ok() { echo -e "${GREEN}✓${NC} $*"; } +warn() { echo -e "${YELLOW}!${NC} $*"; } + +[[ "${EUID}" -eq 0 ]] || { echo "must run as root" >&2; exit 1; } + +# ------------------------------------------------------------------ payload -- +# +# A copy of driver/ rather than a symlink into the user's clone: kernel updates +# have to keep working after the clone is moved, renamed or deleted. +# +info "Installing rebuild payload to ${PAYLOAD_DIR}..." +mkdir -p "${PAYLOAD_DIR}/driver" "${PAYLOAD_DIR}/state" +rm -rf "${PAYLOAD_DIR}/driver/patches" "${PAYLOAD_DIR}/driver/src" +cp -a "${REPO_DIR}/driver/build.sh" "${PAYLOAD_DIR}/driver/build.sh" +cp -a "${REPO_DIR}/driver/VERSION" "${PAYLOAD_DIR}/driver/VERSION" +cp -a "${REPO_DIR}/driver/patches" "${PAYLOAD_DIR}/driver/patches" +cp -a "${REPO_DIR}/driver/src" "${PAYLOAD_DIR}/driver/src" +chmod +x "${PAYLOAD_DIR}/driver/build.sh" + +# +# Carry the downloaded tarball over too. Without it every kernel update needs +# network access to GitHub, which a rig behind a flaky link may not have. +# Only the tarball, not the extracted tree - that is ~1 GB of rebuildable data. +# +mkdir -p "${PAYLOAD_DIR}/driver/.build" +shopt -s nullglob +for tarball in "${REPO_DIR}/driver/.build"/*.tar.gz; do + cp -a "${tarball}" "${PAYLOAD_DIR}/driver/.build/" +done +shopt -u nullglob +if compgen -G "${PAYLOAD_DIR}/driver/.build/*.tar.gz" > /dev/null; then + ok "Payload installed (source tarball cached — rebuilds work offline)" +else + warn "No cached source tarball — kernel-update rebuilds will need network" +fi + +# -------------------------------------------------------------- build.conf -- +# +# The flags are compile-time, so a rebuild that does not replay them produces a +# driver without the user's overclock. Persisting them is what makes an +# automatic rebuild equivalent to re-running install.sh by hand. +# +info "Recording build configuration..." +mkdir -p "${CONF_DIR}" +cat > "${CONF_DIR}/build.conf" </dev/null && [[ -d /etc/kernel/postinst.d ]]; then + install -m 0755 "${SCRIPT_DIR}/hooks/kernel-postinst.sh" /etc/kernel/postinst.d/cmpunlocker + mkdir -p /etc/kernel/postrm.d + install -m 0755 "${SCRIPT_DIR}/hooks/kernel-postrm.sh" /etc/kernel/postrm.d/cmpunlocker + HOOK_KIND="debian" + ok "Installed /etc/kernel/postinst.d/cmpunlocker (Debian/Ubuntu/HiveOS)" +elif command -v pacman &>/dev/null; then + mkdir -p /etc/pacman.d/hooks + install -m 0644 "${SCRIPT_DIR}/hooks/pacman.hook" /etc/pacman.d/hooks/95-cmpunlocker.hook + HOOK_KIND="arch" + ok "Installed /etc/pacman.d/hooks/95-cmpunlocker.hook (Arch)" +elif [[ -d /etc/kernel/install.d ]] || command -v kernel-install &>/dev/null; then + mkdir -p /etc/kernel/install.d + install -m 0755 "${SCRIPT_DIR}/hooks/kernel-install.sh" /etc/kernel/install.d/95-cmpunlocker.install + HOOK_KIND="kernel-install" + ok "Installed /etc/kernel/install.d/95-cmpunlocker.install (Fedora/RHEL)" +fi + +if [[ "${HOOK_KIND}" == "none" ]]; then + warn "No supported kernel hook directory found" + warn "The boot-time service still rebuilds after a kernel update," + warn "it just costs one slow boot instead of rebuilding beforehand." +fi + +# ------------------------------------------------------------------ service -- +# +# The safety net for everything the hooks cannot see: hand-built kernels, +# restored snapshots, a hook that ran before the headers were unpacked. +# +if command -v systemctl &>/dev/null && [[ -d /etc/systemd/system ]]; then + info "Installing boot-time safety net..." + install -m 0644 "${SCRIPT_DIR}/cmpunlocker-rebuild.service" \ + /etc/systemd/system/cmpunlocker-rebuild.service + systemctl daemon-reload 2>/dev/null || true + systemctl enable cmpunlocker-rebuild.service >/dev/null 2>&1 || true + ok "cmpunlocker-rebuild.service enabled" +else + warn "systemd not found — no boot-time safety net installed" +fi + +# ------------------------------------------------------------------- depmod -- +info "Setting module resolution priority..." +mkdir -p /etc/depmod.d +install -m 0644 "${SCRIPT_DIR}/depmod-cmpunlocker.conf" /etc/depmod.d/cmpunlocker.conf +depmod -a 2>/dev/null || true +ok "Patched modules now outrank the stock driver in depmod" + +# ----------------------------------------------------------------- nouveau -- +info "Blocking the nouveau fallback..." +mkdir -p /etc/modprobe.d +install -m 0644 "${SCRIPT_DIR}/modprobe-cmpunlocker.conf" /etc/modprobe.d/cmpunlocker.conf + +# +# This unit exists to load nouveau when nvidia fails to come up. On a CMP that +# turns a fixable driver problem into a card bound to a driver that cannot use +# it, so it is masked rather than left to fire. +# +if command -v systemctl &>/dev/null && \ + systemctl list-unit-files nvidia-fallback.service 2>/dev/null | grep -q nvidia-fallback; then + systemctl mask nvidia-fallback.service >/dev/null 2>&1 || true + ok "Masked nvidia-fallback.service (no automatic switch to nouveau)" +else + ok "nouveau blacklisted (no nvidia-fallback.service on this system)" +fi + +# --------------------------------------------------------------------- pin -- +if [[ "${PIN_PACKAGES}" == "1" ]]; then + info "Pinning NVIDIA package version..." + # + # Recorded rather than assumed: pinning depends on a plugin being present + # and on repo metadata being reachable, so the installer summary has to + # report what actually happened instead of what was asked for. + # + if "${LIB_DIR}/pin-packages.sh" pin; then + echo "ok" > "${PAYLOAD_DIR}/state/pin-status" + else + echo "failed" > "${PAYLOAD_DIR}/state/pin-status" + warn "NVIDIA packages are NOT pinned — a driver upgrade can break rebuilds" + warn "Fix the cause above, then: sudo ${LIB_DIR}/pin-packages.sh pin" + fi +else + echo "skipped" > "${PAYLOAD_DIR}/state/pin-status" + info "NVIDIA packages not pinned (--no-pin)" + warn "A driver upgrade past $(tr '\n' ' ' < "${REPO_DIR}/driver/VERSION" 2>/dev/null || echo 'the supported versions')" + warn "will make kernel-update rebuilds fail." +fi + +echo "" +ok "Kernel-update persistence installed" +echo "" diff --git a/persist/modprobe-cmpunlocker.conf b/persist/modprobe-cmpunlocker.conf new file mode 100644 index 0000000..b65a763 --- /dev/null +++ b/persist/modprobe-cmpunlocker.conf @@ -0,0 +1,12 @@ +# cmpunlocker - keep nouveau from claiming the card. +# +# nouveau cannot drive a CMP 170HX usefully and binding it means the patched +# driver cannot attach. The distro nvidia packages ship the same blacklist; +# this file makes it independent of them, so removing or reinstalling the +# stock package cannot take it away. +# +# Note that blacklisting only blocks automatic (alias-driven) loading. The +# explicit "modprobe nouveau" that nvidia-fallback.service runs is stopped by +# masking that unit, which install.sh does separately. +blacklist nouveau +blacklist nova_core diff --git a/persist/pin-packages.sh b/persist/pin-packages.sh new file mode 100755 index 0000000..f0c648b --- /dev/null +++ b/persist/pin-packages.sh @@ -0,0 +1,239 @@ +#!/bin/bash +# +# cmpunlocker - pin the NVIDIA packages to the version the unlock supports. +# +# Installed to /usr/lib/cmpunlocker/pin-packages.sh. +# +# Usage: pin-packages.sh pin|unpin|status +# +# Why this exists: the patched modules are built from the open-gpu-kernel-modules +# tarball matching the *installed* driver, and driver/VERSION lists the versions +# the unlock has been validated against. If the package manager moves the driver +# to an unsupported version, the next kernel-update rebuild fails and the machine +# falls back to the stock driver - the exact rollback this is meant to prevent. +# +# Everything here is reversible: remove.sh calls "unpin". +# +set -uo pipefail + +ACTION="${1:-status}" +MARK_BEGIN="# BEGIN cmpunlocker" +MARK_END="# END cmpunlocker" + +info() { echo "cmpunlocker: $*"; } + +# +# Blanket-match on the name rather than a curated list: the package split +# differs per distro and per driver flavour (akmod-nvidia-open, nvidia-utils, +# xorg-x11-drv-nvidia-cuda, nvidia-dkms-610, libnvidia-*, ...). Everything +# carrying the driver version has "nvidia" in the name. +# +# Except the GPU firmware packages: those are subpackages of linux-firmware and +# have nothing to do with the driver version — the GSP firmware the unlock needs +# ships inside the NVIDIA driver package itself. Holding them back would block +# linux-firmware updates and can wedge the whole upgrade transaction on a +# dependency conflict. +# +EXCLUDE_RE='^nvidia-gpu-firmware|^firmware-nvidia|^nvidia-firmware' + +filter() { grep -i nvidia | grep -Ev "${EXCLUDE_RE}" | sort -u; } + +rpm_pkgs() { rpm -qa --qf '%{NAME}\n' 2>/dev/null | filter; } +dpkg_pkgs() { dpkg-query -W -f='${Package}\n' 2>/dev/null | filter; } +pac_pkgs() { pacman -Qq 2>/dev/null | filter; } + +# ---------------------------------------------------------------- dnf / rpm -- +# +# "dnf versionlock --help" is not a usable probe: dnf5 accepts the subcommand +# name whether or not the plugin is actually there. Running an real query is, +# and -C keeps it off the network so a rig with no route to the mirrors still +# gets a straight answer. +# +dnf_versionlock_available() { + command -v dnf &>/dev/null || return 1 + dnf -C versionlock list &>/dev/null || dnf versionlock list &>/dev/null +} + +dnf_locked_count() { + { dnf -C versionlock list 2>/dev/null || dnf versionlock list 2>/dev/null; } \ + | grep -ci nvidia || true +} + +pin_dnf() { + local pkgs before after + pkgs="$(rpm_pkgs)" + [[ -n "${pkgs}" ]] || { info "no nvidia packages installed — nothing to pin"; return 0; } + + if ! dnf_versionlock_available; then + info "cannot pin: the dnf versionlock plugin is not available" + info " install it with: sudo dnf install python3-dnf-plugin-versionlock" + info " (dnf5: sudo dnf install dnf5-plugin-versionlock)" + info " then re-run: sudo /usr/lib/cmpunlocker/pin-packages.sh pin" + return 1 + fi + + before="$(dnf_locked_count)" + # shellcheck disable=SC2086 + dnf -C versionlock add ${pkgs} >/dev/null 2>&1 || \ + # shellcheck disable=SC2086 + dnf versionlock add ${pkgs} >/dev/null 2>&1 || true + after="$(dnf_locked_count)" + + # + # Verify rather than trust the exit code: dnf can fail to reach the mirrors + # and still return 0, which would have this claim a pin that does not exist. + # + if (( after > before )) || { (( before > 0 )) && (( after > 0 )); }; then + info "pinned ${after} nvidia package(s) with dnf versionlock" + return 0 + fi + + info "FAILED to pin: dnf versionlock has no nvidia entries after the attempt" + info " most likely the repo metadata could not be refreshed (no network)" + info " retry with: sudo /usr/lib/cmpunlocker/pin-packages.sh pin" + return 1 +} + +unpin_dnf() { + dnf_versionlock_available || return 0 + local pkgs + pkgs="$(rpm_pkgs)" + [[ -n "${pkgs}" ]] || return 0 + # shellcheck disable=SC2086 + dnf -C versionlock delete ${pkgs} >/dev/null 2>&1 || \ + # shellcheck disable=SC2086 + dnf versionlock delete ${pkgs} >/dev/null 2>&1 || true + info "removed dnf versionlock entries for nvidia packages" +} + +status_dnf() { + dnf_versionlock_available || { echo " dnf versionlock plugin not available"; return 0; } + local out + out="$({ dnf -C versionlock list 2>/dev/null || dnf versionlock list 2>/dev/null; } | grep -i nvidia)" + [[ -n "${out}" ]] && echo "${out}" | sed 's/^/ /' || echo " (none)" +} + +# --------------------------------------------------------------- apt / dpkg -- +pin_apt() { + local pkgs held + pkgs="$(dpkg_pkgs)" + [[ -n "${pkgs}" ]] || { info "no nvidia packages installed — nothing to pin"; return 0; } + # shellcheck disable=SC2086 + apt-mark hold ${pkgs} >/dev/null 2>&1 || true + + held="$(apt-mark showhold 2>/dev/null | grep -ci nvidia || true)" + if (( held > 0 )); then + info "held ${held} nvidia package(s) with apt-mark" + return 0 + fi + info "FAILED to pin: apt-mark shows no nvidia holds after the attempt" + return 1 +} + +unpin_apt() { + local pkgs + pkgs="$(apt-mark showhold 2>/dev/null | grep -i nvidia || true)" + [[ -n "${pkgs}" ]] || return 0 + # shellcheck disable=SC2086 + apt-mark unhold ${pkgs} >/dev/null 2>&1 || true + info "released apt-mark holds on nvidia packages" +} + +status_apt() { + apt-mark showhold 2>/dev/null | grep -i nvidia | sed 's/^/ /' || echo " (none)" +} + +# -------------------------------------------------------------------- pacman -- +# +# IgnorePkg only takes effect inside the [options] section, so the entry is +# inserted directly after it rather than appended to the end of the file. +# +pin_pacman() { + local conf="/etc/pacman.conf" pkgs line + pkgs="$(pac_pkgs | tr '\n' ' ')" + pkgs="${pkgs% }" + [[ -n "${pkgs}" ]] || { info "no nvidia packages installed — nothing to pin"; return 0; } + + if grep -q "${MARK_BEGIN}" "${conf}" 2>/dev/null; then + unpin_pacman + fi + + cp -a "${conf}" "${conf}.cmpunlocker.bak" + line="IgnorePkg = ${pkgs}" + awk -v b="${MARK_BEGIN}" -v e="${MARK_END}" -v l="${line}" ' + { print } + /^\[options\]/ && !done { print b; print l; print e; done = 1 } + ' "${conf}" > "${conf}.tmp" && mv "${conf}.tmp" "${conf}" + + # + # An [options] section that awk never matched leaves the file unchanged, + # and IgnorePkg outside that section is silently ignored by pacman. + # + if grep -q "${MARK_BEGIN}" "${conf}" 2>/dev/null; then + info "added IgnorePkg for nvidia packages to ${conf} (backup: ${conf}.cmpunlocker.bak)" + return 0 + fi + info "FAILED to pin: no [options] section found in ${conf}" + return 1 +} + +unpin_pacman() { + local conf="/etc/pacman.conf" + grep -q "${MARK_BEGIN}" "${conf}" 2>/dev/null || return 0 + sed -i "/${MARK_BEGIN}/,/${MARK_END}/d" "${conf}" + info "removed IgnorePkg entry from ${conf}" +} + +status_pacman() { + grep -A1 "${MARK_BEGIN}" /etc/pacman.conf 2>/dev/null | grep IgnorePkg | sed 's/^/ /' || echo " (none)" +} + +# ------------------------------------------------------------------ dispatch -- +detect() { + if command -v dnf &>/dev/null && command -v rpm &>/dev/null; then echo dnf + elif command -v apt-mark &>/dev/null && command -v dpkg &>/dev/null; then echo apt + elif command -v pacman &>/dev/null; then echo pacman + else echo unknown + fi +} + +PM="$(detect)" + +case "${ACTION}" in + pin) + [[ "${EUID}" -eq 0 ]] || { echo "must run as root" >&2; exit 1; } + rc=0 + case "${PM}" in + dnf) pin_dnf || rc=1 ;; + apt) pin_apt || rc=1 ;; + pacman) pin_pacman || rc=1 ;; + *) info "unknown package manager — pin the nvidia packages manually"; rc=1 ;; + esac + exit "${rc}" + ;; + unpin) + [[ "${EUID}" -eq 0 ]] || { echo "must run as root" >&2; exit 1; } + case "${PM}" in + dnf) unpin_dnf ;; + apt) unpin_apt ;; + pacman) unpin_pacman ;; + *) : ;; + esac + ;; + status) + echo "package manager: ${PM}" + echo "pinned nvidia packages:" + case "${PM}" in + dnf) status_dnf ;; + apt) status_apt ;; + pacman) status_pacman ;; + *) echo " (unknown package manager)" ;; + esac + ;; + *) + echo "usage: $0 pin|unpin|status" >&2 + exit 1 + ;; +esac + +exit 0 diff --git a/persist/rebuild.sh b/persist/rebuild.sh new file mode 100755 index 0000000..2d4af98 --- /dev/null +++ b/persist/rebuild.sh @@ -0,0 +1,99 @@ +#!/bin/bash +# +# cmpunlocker - rebuild the patched modules for a given kernel. +# +# Installed to /usr/lib/cmpunlocker/rebuild.sh. Every persistence path calls +# this and nothing else: the kernel-install hook (Fedora), the postinst.d hook +# (Debian/Ubuntu/HiveOS), the pacman hook (Arch), and the boot-time service. +# +# Usage: rebuild.sh [kernel-version] (default: running kernel) +# +# Exits 0 even when the build fails. A package manager hook that returns +# non-zero turns a driver problem into a failed system upgrade, which is worse +# than booting on the stock driver. The failure is recorded in the marker file +# so cmpunlocker-rebuild.service retries on the next boot, where it is visible. +# +set -uo pipefail + +PAYLOAD_DIR="/var/lib/cmpunlocker" +CONF_FILE="/etc/cmpunlocker/build.conf" +LOG_DIR="/var/log/cmpunlocker" +MARKER_DIR="/var/lib/cmpunlocker/state" + +KVER="${1:-$(uname -r)}" +LOG_FILE="${LOG_DIR}/rebuild-${KVER}.log" +MARKER="${MARKER_DIR}/failed-${KVER}" + +mkdir -p "${LOG_DIR}" "${MARKER_DIR}" 2>/dev/null || true + +log() { echo "cmpunlocker: $*"; } + +# +# Everything below is also written to the log, so a failure that scrolled past +# during a dnf upgrade can still be read afterwards. +# +exec > >(tee -a "${LOG_FILE}") 2>&1 + +echo "=== $(date -Is) rebuild for ${KVER} ===" + +fail() { + log "ERROR: $*" + log "Patched modules for ${KVER} were NOT built." + log "Existing modules for other kernels are untouched." + log "Log: ${LOG_FILE}" + : > "${MARKER}" + exit 0 +} + +[[ "${EUID}" -eq 0 ]] || fail "must run as root" + +# +# The payload is a copy of driver/ taken at install time. Rebuilding from it +# rather than from the user's git clone means the clone can be moved or deleted +# without breaking kernel updates. +# +[[ -x "${PAYLOAD_DIR}/driver/build.sh" ]] || \ + fail "payload missing at ${PAYLOAD_DIR}/driver — re-run install.sh" + +# +# The compile-time flags (overclock, timings, P2P, driver version) live here. +# Without them a rebuild would silently produce a stock-clocked driver. +# +if [[ -r "${CONF_FILE}" ]]; then + # shellcheck disable=SC1090 + . "${CONF_FILE}" +else + log "WARN: ${CONF_FILE} missing — building with defaults (no overclock)" +fi + +[[ -d "/lib/modules/${KVER}" ]] || fail "no module tree for ${KVER}" + +# +# Headers are the one thing that is genuinely not ready sometimes: on Debian +# the postinst.d hook can run before linux-headers- is unpacked. Leave the +# marker and let the boot service pick it up once the headers are in place. +# +if [[ ! -d "/lib/modules/${KVER}/build" ]]; then + fail "kernel headers for ${KVER} not installed yet (looked in /lib/modules/${KVER}/build)" +fi + +log "building for ${KVER} (running: $(uname -r))" +[[ -n "${CMPUNLOCKER_MCLK_NDIV:-}" ]] && log " mclk-ndiv=${CMPUNLOCKER_MCLK_NDIV}" +[[ -n "${CMPUNLOCKER_MCLK_TIMINGS:-}" ]] && log " mclk-timings=${CMPUNLOCKER_MCLK_TIMINGS}" +[[ -n "${CMPUNLOCKER_ENABLE_P2P:-}" ]] && log " p2p=on" + +CMPUNLOCKER_KVER="${KVER}" \ +CMPUNLOCKER_DRIVER_VERSION="${CMPUNLOCKER_DRIVER_VERSION:-}" \ +CMPUNLOCKER_MCLK_NDIV="${CMPUNLOCKER_MCLK_NDIV:-}" \ +CMPUNLOCKER_MCLK_TIMINGS="${CMPUNLOCKER_MCLK_TIMINGS:-}" \ +CMPUNLOCKER_ENABLE_P2P="${CMPUNLOCKER_ENABLE_P2P:-}" \ +CMPUNLOCKER_VERBOSE="${CMPUNLOCKER_VERBOSE:-0}" \ + "${PAYLOAD_DIR}/driver/build.sh" || fail "build failed for ${KVER}" + +if [[ ! -f "/lib/modules/${KVER}/updates/cmpunlocker/nvidia.ko" ]]; then + fail "build reported success but nvidia.ko is missing for ${KVER}" +fi + +rm -f "${MARKER}" +log "OK: patched modules installed for ${KVER}" +exit 0 diff --git a/remove.sh b/remove.sh index 2940b01..f9d7855 100755 --- a/remove.sh +++ b/remove.sh @@ -26,6 +26,11 @@ if [[ "${1:-}" != "--yes" && "${1:-}" != "-y" ]]; then echo " - Rebuilds initramfs" echo " - Reloads stock NVIDIA modules (brief display interruption)" echo " - Restores the pre-install kernel command line (reverts IOMMU changes)" + echo " - Removes the kernel-update hooks and the boot-time rebuild service" + echo " - Releases the NVIDIA package version pin" + echo " - Unmasks nvidia-fallback.service and unblocks nouveau" + echo "" + echo " Logs under /var/log/cmpunlocker/ are kept." echo "" echo "Run: sudo ./remove.sh --yes" exit 1 @@ -62,6 +67,44 @@ else info "No IOMMU config backup found — kernel command line left as-is" fi +info "Removing kernel-update persistence..." + +# +# Unpin first: the helper that knows how to undo the hold lives in the +# directory removed a few lines further down. +# +if [[ -x /usr/lib/cmpunlocker/pin-packages.sh ]]; then + /usr/lib/cmpunlocker/pin-packages.sh unpin || true +fi + +if command -v systemctl &>/dev/null; then + systemctl disable --now cmpunlocker-rebuild.service 2>/dev/null || true + # + # Unmask rather than enable: nvidia-fallback is the distro's unit and + # whatever state it was in before the install is the distro's business. + # + systemctl unmask nvidia-fallback.service 2>/dev/null || true +fi + +rm -f /etc/systemd/system/cmpunlocker-rebuild.service +rm -f /etc/kernel/install.d/95-cmpunlocker.install +rm -f /etc/kernel/postinst.d/cmpunlocker +rm -f /etc/kernel/postrm.d/cmpunlocker +rm -f /etc/pacman.d/hooks/95-cmpunlocker.hook +rm -f /etc/depmod.d/cmpunlocker.conf +rm -f /etc/modprobe.d/cmpunlocker.conf +rm -rf /usr/lib/cmpunlocker +rm -rf /var/lib/cmpunlocker +rm -rf /etc/cmpunlocker + +if [[ -f /etc/pacman.conf.cmpunlocker.bak ]]; then + mv -f /etc/pacman.conf.cmpunlocker.bak /etc/pacman.conf + ok "Restored /etc/pacman.conf from pre-install backup" +fi + +command -v systemctl &>/dev/null && systemctl daemon-reload 2>/dev/null || true +ok "Kernel hooks, boot service and package pins removed" + info "Removing patched modules..." mod_removed=0 kernels_touched=() -- 2.51.2 From 89df12f6bf53b4e56cb6c3f53279361de9055e1e Mon Sep 17 00:00:00 2001 From: aboba Date: Thu, 6 Aug 2026 03:39:17 +0300 Subject: [PATCH 2/3] close two rollback paths: firmware-owned dmem.bin, and remove.sh hanging Booter payload override moved out of the firmware package's namespace. cmpunlock.c read it from /lib/firmware/nvidia/ga100/gsp/dmem.bin, which belongs to nvidia-gpu-firmware (linux-firmware on Fedora). That directory already accumulates per-driver blobs on every firmware release, so an update landing a dmem.bin there would have been preferred over the built-in payload and quietly broken the unlock. Pinning that package is not an option - it is a linux-firmware subpackage and holding it back wedges system upgrades on a dependency conflict. Reading from /var/lib/cmpunlocker/dmem.bin instead makes the question moot. The GSP firmware the driver actually loads lives in the versioned directory from the driver package, which is pinned. remove.sh no longer swaps the running driver by default. Loading the stock nvidia-drm against a CMP wedges the machine: traced it to that exact modprobe, after which the kernel still answers pings while userspace stops making progress. Nothing needed the swap - the modules are already off disk, so the next boot picks up the stock driver on its own, and an uninstall ends in a reboot anyway. Behind --reload for anyone who wants it, now guarded with timeouts. Dropped the rmmod -f fallback: forcing out a module that is still referenced can take the kernel with it, which is a bad trade when a reboot finishes the job. remove.sh also syncs after depmod, matching build.sh. Without it a power cut between depmod and the next flush leaves a zero-length modules.dep, and then nothing resolves on the next boot - not the NIC driver, not storage. Hit this three times in testing via hard power-off. --- README.md | 8 ++++- driver/src/cmpunlock.c | 16 ++++++++- remove.sh | 77 +++++++++++++++++++++++++++++++----------- 3 files changed, 79 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index bd85b35..cdf1572 100644 --- a/README.md +++ b/README.md @@ -87,7 +87,9 @@ Two more things keep the stock driver from winning: - `/etc/depmod.d/cmpunlocker.conf` makes the patched modules outrank the stock ones. The distro driver is rebuilt on every kernel update too, into `extra/` (akmod) or `updates/dkms/` (dkms), right next to ours. - `nvidia-fallback.service` is masked and nouveau is blacklisted, so a driver that fails to load does not hand the card to nouveau. -**The NVIDIA packages are pinned to their installed version.** A driver upgrade past the versions in `driver/VERSION` makes every later rebuild fail, which is the rollback this is meant to prevent. GPU *firmware* packages are deliberately left unpinned — they belong to `linux-firmware` and holding them back can wedge system upgrades. +**The NVIDIA packages are pinned to their installed version.** A driver upgrade past the versions in `driver/VERSION` makes every later rebuild fail, which is the rollback this is meant to prevent. This covers the GSP firmware the driver actually loads, from `/lib/firmware/nvidia//`, which ships in the driver package itself. + +GPU *firmware* packages (`nvidia-gpu-firmware` and friends) are deliberately left unpinned — they belong to `linux-firmware`, and holding them back can wedge system upgrades on a dependency conflict. They are also no longer able to affect the unlock: the optional Booter payload override is read from `/var/lib/cmpunlocker/dmem.bin` rather than from `/lib/firmware/nvidia/ga100/gsp/`, a directory that firmware updates add files to and that no amount of pinning could safely protect. ```bash sudo ./install.sh --no-pin # allow driver upgrades, accept the risk @@ -177,6 +179,10 @@ sudo ./remove.sh --yes Then perform a cold reboot (full power off, then boot). +This removes the patched modules from disk, undoes the kernel-update hooks, releases the package pin, and rebuilds the initramfs. The driver already running in memory is left alone — the card comes up on the stock driver at the next boot, which is the safe order. + +`--reload` swaps the running driver for the stock one immediately instead of waiting for the reboot. It is off by default because loading the stock `nvidia-drm` against a CMP 170HX can wedge the machine: the card has no usable display engine, and the kernel keeps answering pings while userspace stops making progress. There is no reason to take that risk during an uninstall you are going to reboot from anyway. + ## Community Join our [Discord community](https://discord.gg/CdHSakKSFv) to discuss with other users. diff --git a/driver/src/cmpunlock.c b/driver/src/cmpunlock.c index b2b9c2f..1a70222 100644 --- a/driver/src/cmpunlock.c +++ b/driver/src/cmpunlock.c @@ -80,7 +80,21 @@ /* Booter payload geometry. */ #define CMP_SIGNATURE_SIZE 0x0000f800ULL #define CMP_PAYLOAD_FILL_DWORD 0x000004a7U -#define CMP_DMEM_PATH "/lib/firmware/nvidia/ga100/gsp/dmem.bin" +/* + * Optional override for the Booter payload. + * + * This used to read /lib/firmware/nvidia/ga100/gsp/dmem.bin, which belongs to + * the distro's GPU firmware package (nvidia-gpu-firmware on Fedora, part of + * linux-firmware). That directory already collects per-driver blobs on every + * firmware release, so a future update dropping a dmem.bin there would have + * been picked up in preference to the built-in payload and quietly broken the + * unlock - with no way to prevent it short of holding back linux-firmware, + * which is not something worth doing to a system. + * + * The path now lives in cmpunlocker's own directory, where nothing else + * writes. Absent - which is the normal case - the payload is generated below. + */ +#define CMP_DMEM_PATH "/var/lib/cmpunlocker/dmem.bin" /* Unlocked framebuffer sizes. */ #define CMP_FB_BYTES_8GB 0x0000001000000000ULL /* 64GB */ diff --git a/remove.sh b/remove.sh index f9d7855..fc7dca6 100755 --- a/remove.sh +++ b/remove.sh @@ -20,19 +20,32 @@ echo -e "${CYAN}║ cmpunlocker ║${NC}" echo -e "${CYAN}╚════════════════════════════════════════╝${NC}" echo "" -if [[ "${1:-}" != "--yes" && "${1:-}" != "-y" ]]; then +CONFIRMED=0 +RELOAD_DRIVER=0 +for arg in "$@"; do + case "${arg}" in + --yes|-y) CONFIRMED=1 ;; + --reload) RELOAD_DRIVER=1 ;; + esac +done + +if (( CONFIRMED == 0 )); then warn "This removes cmpunlocker patched kernel modules:" echo " - Removes /lib/modules/*/updates/cmpunlocker/" echo " - Rebuilds initramfs" - echo " - Reloads stock NVIDIA modules (brief display interruption)" echo " - Restores the pre-install kernel command line (reverts IOMMU changes)" echo " - Removes the kernel-update hooks and the boot-time rebuild service" echo " - Releases the NVIDIA package version pin" echo " - Unmasks nvidia-fallback.service and unblocks nouveau" echo "" + echo " The driver already in memory is left alone — reboot to finish." echo " Logs under /var/log/cmpunlocker/ are kept." echo "" echo "Run: sudo ./remove.sh --yes" + echo "" + echo " --reload also swap the running driver for the stock one now," + echo " instead of at the next boot. On a CMP this can hang the" + echo " machine, so it is off by default — see README." exit 1 fi @@ -114,6 +127,13 @@ for mod_dir in /lib/modules/*/updates/cmpunlocker; do kernel="$(basename "$(dirname "$(dirname "${mod_dir}")")")" rm -rf "${mod_dir}" depmod -a "${kernel}" 2>/dev/null || true + # + # depmod's output sits in the page cache until something flushes it. + # A power cut or hard reset before that leaves a zero-length + # modules.dep, and then nothing resolves on the next boot - not the + # NIC driver, not storage. Cheap insurance against an unbootable box. + # + sync ok "Removed patched modules for kernel ${kernel}" mod_removed=$((mod_removed + 1)) kernels_touched+=("${kernel}") @@ -136,9 +156,26 @@ if [[ ${#kernels_touched[@]} -gt 0 ]]; then ok "initramfs rebuilt" fi -info "Reloading stock NVIDIA driver..." -if lsmod | grep -q '^nvidia'; then - warn "Unloading NVIDIA modules (display may flicker)" +# +# Swapping the running driver is opt-in, because loading the stock one on a +# CMP 170HX can wedge the machine: the card has no usable display engine, and +# nvidia-drm binding to it hangs in a way that leaves the kernel answering +# pings while userspace stops making progress. Nothing needs the swap either - +# the files are already gone, so the next boot comes up on the stock driver by +# itself, and a reboot is the documented last step of an uninstall anyway. +# +if (( RELOAD_DRIVER == 0 )); then + if lsmod | grep -q '^nvidia'; then + info "Leaving the running driver in place" + echo " The patched modules are removed from disk, but the copy already in" + echo " memory keeps running until you reboot. That is the safe order." + echo " ./remove.sh --yes --reload swaps it now instead, at the risk of" + echo " hanging the machine on a CMP." + fi +else + info "Swapping the running driver for the stock one (--reload)..." + warn "This can hang on a CMP 170HX. Reboot instead if it does not return." + for svc in gdm3 sddm lightdm display-manager; do systemctl stop "${svc}" 2>/dev/null || true done @@ -146,24 +183,26 @@ if lsmod | grep -q '^nvidia'; then killall -9 Xorg Xwayland nvidia-persistenced 2>/dev/null || true sleep 1 + # + # No rmmod -f fallback: forcing a module out from under a driver that is + # still referenced is documented as able to take the kernel down with it, + # which is a poor trade when a reboot finishes the job cleanly. + # for mod in nvidia_drm nvidia_uvm nvidia_modeset nvidia; do - modprobe -r "${mod}" 2>/dev/null || true + timeout 30 modprobe -r "${mod}" 2>/dev/null || true done sleep 1 if lsmod | grep -q '^nvidia'; then - for mod in nvidia_uvm nvidia_drm nvidia_modeset nvidia; do - rmmod -f "${mod}" 2>/dev/null || true - done - fi - - if modprobe nvidia 2>/dev/null; then - modprobe nvidia-modeset 2>/dev/null || true - modprobe nvidia-uvm 2>/dev/null || true - modprobe nvidia-drm 2>/dev/null || true - ok "Stock NVIDIA driver reloaded" + warn "Some NVIDIA modules are still in use — reboot to finish cleanup" + elif timeout 60 modprobe nvidia 2>/dev/null; then + timeout 30 modprobe nvidia-modeset 2>/dev/null || true + timeout 30 modprobe nvidia-uvm 2>/dev/null || true + timeout 30 modprobe nvidia-drm 2>/dev/null || \ + warn "nvidia-drm did not load — harmless on a headless card" + ok "Stock NVIDIA driver loaded" else - warn "Could not reload NVIDIA driver — reboot to finish cleanup" + warn "Stock driver did not load — reboot to finish cleanup" fi for svc in gdm3 sddm lightdm display-manager; do @@ -172,14 +211,12 @@ if lsmod | grep -q '^nvidia'; then break fi done -else - warn "NVIDIA modules not loaded — skipping driver reload" fi echo "" ok "cmpunlocker removed" echo "Log saved to: ${LOG_FILE}" echo "" -echo "If the GPU or display is not working, reboot:" +echo "Reboot to finish — the card comes back up on the stock driver:" echo -e " ${CYAN}sudo reboot${NC}" echo "" -- 2.51.2 From 9cf240b10ad176664b9fed0d3dfac4a710581a9d Mon Sep 17 00:00:00 2001 From: aboba Date: Thu, 6 Aug 2026 04:00:57 +0300 Subject: [PATCH 3/3] beautify --- README.md | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index cdf1572..b622ad1 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ Below are memory and performance results after applying the unlock: ### Unlock Results -image +image --- @@ -35,7 +35,9 @@ sudo ./install.sh Then perform a cold reboot (full power off, then boot). The correct memory geometry is selected automatically from the PCI device ID (`0x20C2` = 8GB -> 64GB, `0x2082` = 10GB -> 40GB). -### HBM Memory Clock +### HBM Memory overclock +
+ HBM Memory overclock `--mclk-ndiv=N` sets the FBPA PLL multiplier; the resulting clock is `N * 27` MHz. Any VBIOS works, on both `0x20C2` (8GB) and `0x2082` (10GB). @@ -58,8 +60,12 @@ Without the flag the overclock is compiled out entirely. The multiplier is compi If a value turns out to be unstable - reinstall without `--mclk-ndiv` (or run `./remove.sh`) from a working state. +
+ ### IOMMU +
+ IOMMU NVIDIA recommends `iommu=pt` (passthrough) for all GPUs. The installer does **not** touch the kernel command line by default: ```bash @@ -68,7 +74,12 @@ sudo ./install.sh --iommu Or add `iommu=pt` to your kernel cmdline manually. IOMMU must also be enabled in BIOS (VT-d on Intel, AMD-Vi / SVM on AMD). -### Surviving Kernel Updates +
+ +### Surviving Kernel Updates (Anti-rollback) + +
+ Surviving Kernel Updates The patched modules are built against one specific kernel. Without help, the first kernel update leaves the card on the stock driver — reporting 8GB instead of 64GB — or on nouveau. The installer wires the rebuild into the kernel update path by default, so this does not happen. @@ -108,7 +119,7 @@ sudo /usr/lib/cmpunlocker/rebuild.sh # rebuild for the running kernel b To take a pinned driver upgrade: `sudo /usr/lib/cmpunlocker/pin-packages.sh unpin`, upgrade, then re-run `install.sh` (which re-pins). If the new driver version is not in `driver/VERSION`, the build will refuse it. Everything above is undone by `./remove.sh --yes`. - +
--- ## Verify @@ -150,15 +161,14 @@ cd benchmark && nvcc -O3 -o nvidia_bench nvidia_bench.cu -lnvidia-ml -ldl \ ## What Gets Unlocked -| Feature | Status | -|---------------------------------------------------------|-----------------| -| Full SM compute throughput (SS0/SS1) | Working | -| Memory geometry (64GB on 8GB cards, 40GB on 10GB cards) | Working | -| PCIe Gen 2 speeds | Working | -| GPU-to-GPU P2P (`cudaDeviceEnablePeerAccess`) | Opt-in, `--p2p` | -| HBM2e memory overclock/downclock | Working | -| Persistence across reboot (patched modules) | Working | -| Persistence across kernel updates (auto-rebuild) | Working | +| Feature | Status | +|------------------------------------------------------------------|-------------| +| Full SM compute throughput (SS0/SS1) | Working | +| Memory geometry (64GB on 8GB cards, 40GB on 10GB cards) | Working | +| PCIe Gen 2 speeds | Working | +| GPU-to-GPU P2P (`cudaDeviceEnablePeerAccess`) | In progress | +| HBM2e memory overclock/downclock | Working | +| Persistence across kernel updates (auto-rebuild) (anti-rollback) | Working | ---