Skip to content

Recover Wi-Fi after resume on Apple Silicon Macs - #255

Merged
scottjones merged 7 commits into
omacom:quattrofrom
bodhiblues:fix-wifi-resume
Sep 6, 2026
Merged

scottjones merged 7 commits into
omacom:quattrofrom
bodhiblues:fix-wifi-resume

Conversation

@bodhiblues

Copy link
Copy Markdown

Fixes #197.

The Broadcom firmware on Apple Silicon Macs (BCM4378/BCM4387) wedges across s2idle: scans fail with -52 and every association is rejected with status_code=16, which NetworkManager surfaces as a wrong password. Toggling the radio doesn't reset the chip firmware — only a driver reload clears it. Upstream: AsahiLinux/linux#439.

What this does

  • bin/omarchy-wifi-resume-fix — waits after resume for Wi-Fi to come back on its own, and reloads brcmfmac_wcc/brcmfmac only when it doesn't. The repeating ASSOC-REJECT status_code=16 never appears on a healthy association, so two of them confirm the wedge (~7s after resume, instead of waiting out the timer); a 12s backstop covers anything else. It respects a deliberately disabled radio.
  • install/hardware/apple/fix-wifi-resume.sh — installs and enables omarchy-wifi-resume-fix.service, gated on aarch64 plus PCI ID 14e4:4425|4433 (the same IDs also appear in T2 Intel Macs, where suspend takes a different path — hence the arch gate). Wired into install/hardware/all.sh.
  • migrations/1787753224.sh — runs the same leaf for existing installs.

Design notes

  • A service on suspend.target, not a system-sleep hook. Sleep hooks run synchronously, so anything that waits in one delays resume for every user. suspend.target is only reached after systemd-suspend.service returns, so the service runs on resume without blocking it.
  • Reload only when actually wedged. On machines and kernels where the firmware bug doesn't bite, the service is a no-op — and it stops acting by itself if the bug is ever fixed upstream, rather than reloading the driver on every resume forever.

Testing

Running on the machine from #197 (MacBook Pro 14" M1 Pro, BCM4387, linux-asahi 7.1.6.asahi1-1) since 2026-08-23 — 10 resumes over 3 days, every code path exercised, zero manual interventions:

  • 3 resumes where Wi-Fi came back on its own → correctly no-op ("wifi back after 4s — no reload needed")
  • 4 wedges caught by the status_code=16 signature → reloaded ~4–7s after resume, reconnected ~5s later
  • 3 wedges caught by the backstop timer (no association attempts, so no signature) → reloaded at 12s, reconnected ~6s later

Typical recovery, unattended:

10:29:51 Starting Reload brcmfmac if wifi does not return after resume...
10:29:56 wedged firmware confirmed after 4s (iface=wld0 state=connecting (configuring)) - reloading brcmfmac
10:29:57 brcmfmac reloaded, waiting for NetworkManager to reconnect
10:30:03 reconnected 5s after reload on wld0

Also verified: test/cli passes with the new command's metadata, and the migration no-ops on a machine where the service is already enabled.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FFzQTgH8HymEPL6heXWXv5

The Broadcom firmware on BCM4378/BCM4387 wedges across s2idle: scans
fail with -52 and every association is rejected with status_code=16,
which NetworkManager surfaces as a wrong password. Toggling the radio
does not reset the chip firmware - only a driver reload clears it.
Upstream: AsahiLinux/linux#439

Reload brcmfmac from a service ordered after suspend.target, so it runs
on resume without delaying it, and only when wifi fails to come back on
its own: the repeating status_code=16 rejection never appears on a
healthy association, so it serves as the wedge signature, with a 12s
timer as backstop. That keeps the service a no-op on machines and
kernels where the firmware bug does not bite.

Fixes omacom#197

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FFzQTgH8HymEPL6heXWXv5
@statick88

Copy link
Copy Markdown

Duplicate of #287. The cleaner atomic version with review fixes is at #287.

@malik-na

Copy link
Copy Markdown
Member

Hardened omarchy-wifi-resume-fix with the review fixes from #287: a journal --since fallback (catches the wedge when the clock steps backward across resume) and a dependency guard for nmcli/journalctl/modprobe. Kept the migration, which #287 drops.

@scottjones scottjones left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The design here is careful and I want it in: reloading only when the wedge signature actually appears, respecting a deliberately disabled radio, falling back to a trailing log window when the clock steps backwards across resume, and ordering after suspend.target so it never delays a resume. It also stops acting by itself if the firmware bug is ever fixed, which is the right instinct for a workaround.

I verified your PCI gate on hardware you probably do not have. This machine is an M2 Max with BCM4388 (14e4:4434), driving the same brcmfmac_wcc/brcmfmac stack, so it sits just outside 14e4:(4425|4433). I suspended it to check whether that exclusion is right:

Aug 29 13:51:08  PM: suspend entry (s2idle)
Aug 29 13:56:56  PM: suspend exit

Six minutes of s2idle, and afterwards: wlan0 reconnected on its own, zero CTRL-EVENT-ASSOC-REJECT status_code=16, zero brcmfmac scan -52 errors. So BCM4388 does not wedge, and excluding it is correct rather than an oversight. Worth a line in the leaf saying so — this fork already shipped one brcmfmac ID list that omitted BCM4388 by accident, so the next reader will wonder whether this one did too.

(One concern I had and withdrew: grep -c exits 1 with no matches, but the script sets no -e, so wedged() returns false correctly.)

What I would like before merging: tests. A new bin/ command, an install leaf, an all.sh entry and a migration, with no coverage — while every sibling in install/hardware/apple/ has one (asahi-audio-install-test.sh, brcmfmac-supplicant-test.sh). It matters more than usual here because neither the maintainers nor I have hardware that reproduces the bug, so stubs are the only way anyone can check the logic holds.

Two things worth pinning, both stub-testable the way the sibling tests already do it:

  • The gate: stub lspci, assert the service is installed for 14e4:4425 and 14e4:4433 and not for 14e4:4434, and not on x86 with the same IDs (your comment notes T2 Macs carry them).
  • The wedge detection: stub journalctl, assert wedged() is true at two status_code=16 events and false at one, and that a disabled radio exits without touching the driver.

Three bugs in this repo this week were invisible precisely because nothing exercised the path — a predicate that could never fire, a redirect that silently emptied a variable, a fixture asserting the wrong filesystem layout. This is the same shape of code: a gate and a signature match, both easy to get subtly wrong and impossible to notice without hardware.

Everything else looks right to me, and I will merge as soon as there is coverage.

bodhiblues and others added 3 commits August 30, 2026 18:50
Migrations run under pipefail, where grep -q exiting at the match kills
lspci with SIGPIPE and the failed pipeline reads as "no such hardware".
Read all of the output instead, and note that BCM4388 is excluded on
purpose: an M2 Max rode out a six-minute s2idle with no rejects.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019sJiLMSd7SL39X1ir2SP63
journalctl prints "-- No entries --" to stdout on an empty window, so
the empty-output check never fired and the trailing-log fallback was
dead code - and reviving it would count stale rejects from the last
genuine wedge. A cursor captured at start is immune to the clock
stepping backwards across resume and to pre-suspend history alike; a
quiet --since window remains as the degraded path when cursor capture
fails. Also anchor the status code so 160-169 do not match, and use
omarchy-cmd-missing for the dependency guard per repo style.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019sJiLMSd7SL39X1ir2SP63
Stub lspci, uname, systemctl, nmcli, journalctl and modprobe to pin the
PCI/architecture gate (BCM4378 and BCM4387 on aarch64 only - BCM4388
and T2 Intel Macs excluded), migration wiring and idempotence, and the
recovery command's decisions: signature-confirmed reload, backstop,
clock-step immunity, stale-history rejection, and loud failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019sJiLMSd7SL39X1ir2SP63
@bodhiblues

Copy link
Copy Markdown
Author

Thanks for the thorough review — and especially for suspending your own M2 Max to verify the BCM4388 exclusion. That's now recorded in the leaf: a comment on the gate says the omission is deliberate and cites your six-minute s2idle with zero rejects, so the next reader won't wonder.

Coverage is in (test/shell.d/wifi-resume-fix-test.sh, 22 assertions in the style of brcmfmac-supplicant-test.sh), pinning both things you called out:

  • The gate: stubbed lspci/uname/systemctl assert the service installs for 14e4:4425 and 14e4:4433 on aarch64 only — not for 14e4:4434, and not on x86_64 with the same IDs (the T2 case). Also pinned: After=/WantedBy= on the unit, ExecStart resolving to a command the repo actually ships, and the migration's wiring and idempotence.
  • The wedge detection: stubbed journalctl/nmcli/modprobe assert two status_code=16 rejects confirm a wedge and one does not (it waits out the backstop), and that a disabled radio exits without touching the driver.

Writing the stubs flushed out two real bugs, which rather proves your point about this shape of code:

  1. The migration's gate had the Follow up to #6559 - update fix-t2.sh and ‎migrations/1785944594.sh to not use grep -q omarchy#6608 SIGPIPE shape. Migrations run under pipefail, and lspci | grep -qE reads as "no such hardware" when grep exits at the match and a chatty lspci dies of SIGPIPE — silently skipping affected machines. The chatty-lspci stub catches it (verified by mutation: the old form fails the test); the gate now uses the full-read grep -E ... >/dev/null form the sibling migration uses.

  2. The clock-step fallback was dead code. journalctl prints -- No entries -- to stdout on an empty --since window, so the [[ -z $output ]] check never fired — and reviving it as written would have been worse, since the unwindowed tail -n 200 counts stale rejects from the last genuine wedge and would reload on healthy resumes. Wedge detection now pins a journal cursor at script start (--after-cursor polling), which is immune to both the clock stepping backwards and pre-suspend history; a -q --since window remains as the degraded path if cursor capture fails, where an empty window falls through to the backstop rather than guessing. The stub models journalctl's real -- No entries -- contract so this can't regress silently, and there are cases for clock-step immunity and stale-history rejection.

Also anchored the signature so status_code=160169 don't match, and switched the dependency guard to omarchy-cmd-missing (the raw command -v was failing bin-style-test).

One thing I left as-is, flagged for a deliberate decision rather than sneaking a change in: the backstop still reloads on any resume where Wi-Fi isn't connected within 12s, so a machine with the radio on but deliberately unassociated (out of range, unmanaged) gets a reload plus a failed unit each resume. If you'd rather gate the backstop on some post-resume wpa_supplicant activity, happy to add that — the test scaffolding makes it a small change now.

@scottjones scottjones left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested this on an M2 Max (apple,j414c, t6021) running Asahi. The rework addresses what I raised last time — the journal cursor is a real fix for the clock-stepping problem, not a workaround, and the guards around a disabled radio and missing tools are right.

The test suite is the strongest in this batch. I mutation-tested it — reverting a production behavior and checking the suite goes red:

REJECTS 2 -> 1                    caught
remove cursor, always use --since  caught
remove the radio-disabled guard    caught
skip the reload entirely           caught
WAIT_BEFORE 12 -> 0                caught
drop \b from the reject signature  NOT caught

Five of six. That is unusual — most suites I run this against catch one or two.

What I validated on hardware

My concern going in was that there is no chip gate: the install leaf puts the unit on every Apple Silicon Mac, and this machine has a BCM4388, which isn't in the BCM4378/BCM4387 set the header names. If M2 resume were slower than the 12s backstop, this would reload the driver after every resume on hardware that never had the bug.

It doesn't. Real suspend, 215 seconds, sampling nmcli every 500ms across the cycle:

resume +0.0s  unavailable
       +0.5s  disconnected
       +4.6s  connecting
       +5.7s  connected

Wi-Fi recovered on its own at 5.7s, comfortably inside the 12s backstop, so the script exits "no reload needed" at ~6s. I confirmed that path directly — run in the connected state it returns wifi back after 0s on wlan0 - no reload needed, exit 0.

So the missing chip gate turns out not to matter, and I'd argue it shouldn't be added: gating on observed behavior rather than on a chip allowlist is the better design, and it means the fix reaches BCM4388 machines too if they ever wedge.

Two small things

  1. The \b in REJECT_SIGNATURE is the one documented behavior with no test — the comment says it keeps status_code=160-169 from matching, and that mutation was the only one to survive. Cheap to pin.
  2. The 6.3s margin came from one sample on a strong-signal PSK network. A weak-signal or enterprise-auth reconnect could exceed 12s. If Wi-Fi genuinely isn't up at 12s a reload isn't unreasonable, so I don't think this needs changing — worth being aware of.

One thing I could not test

I can't reproduce the wedge on a BCM4388, so what I validated is the absence of a regression on unaffected hardware — not the recovery itself. Nothing on this PR records that the reload actually recovers a wedged BCM4378/4387. If you've confirmed that on your own machine, could you say so on the PR? That's the claim the change rests on, and it would be good to have it written down.

@bodhiblues

Copy link
Copy Markdown
Author

Thanks for running the mutation pass and for the M2 Max resume timings — that 5.7s sample is a useful number to have on record next to the 12s backstop.

The \b mutation is caught on the current head. Test status_code=160 does not count as status_code=16 (wifi-resume-fix-test.sh:328) stubs two ASSOC-REJECT ... status_code=160 events and asserts the wedge is not confirmed. Dropping the \b from REJECT_SIGNATURE turns it red:

$ sed -i 's/status_code=16\\b/status_code=16/' bin/omarchy-wifi-resume-fix
$ bash test/shell.d/wifi-resume-fix-test.sh
not ok - status_code=160 does not count as status_code=16

So that makes six of six. I suspect the mutation run predates 26ef217, which is the commit that added the suite.

On the chip gate: the leaf and the migration both still gate on uname -m == aarch64 and lspci matching 14e4:(4425|4433) — the same gate you verified excludes your BCM4388 in the first round. I have not removed it, though I agree with the point that the behavioural check is what does the real work; the gate is there so unaffected machines never run the unit at all.

Confirmation that the reload recovers a wedged BCM4387. My machine is a 14-inch MacBook Pro M1 (2021) with a BCM4387 (14e4:4433), and it has been running this recovery as a systemd unit since 23 Aug. It wedges on essentially every s2idle resume. Today's three cycles, from the journal:

09:34:42  kernel: PM: suspend exit
09:34:46  kernel: brcmf_cfg80211_scan: scan error (-52)
09:34:46  wpa_supplicant: wld0: CTRL-EVENT-ASSOC-REJECT ... status_code=16
09:34:46  wpa_supplicant: wld0: CTRL-EVENT-ASSOC-REJECT ... status_code=16
09:34:47  wifi-resume-fix: wedged firmware confirmed after 4s (iface=wld0) - reloading brcmfmac
09:34:48  wifi-resume-fix: brcmfmac reloaded, waiting for NetworkManager to reconnect
09:34:53  wpa_supplicant: wld0: CTRL-EVENT-CONNECTED
09:34:54  wifi-resume-fix: reconnected 5s after reload on wld0

14:28:12  kernel: PM: suspend exit
14:28:16  kernel: brcmf_cfg80211_scan: scan error (-52)
14:28:16  wpa_supplicant: CTRL-EVENT-ASSOC-REJECT status_code=16  (x2)
14:28:17  wifi-resume-fix: wedged firmware confirmed after 4s - reloading brcmfmac
14:28:24  wifi-resume-fix: reconnected 5s after reload on wld0

20:58:26  kernel: PM: suspend exit
20:58:29  wpa_supplicant: CTRL-EVENT-ASSOC-REJECT status_code=16  (x2)
20:58:30  wifi-resume-fix: wedged firmware confirmed after 4s - reloading brcmfmac
20:58:38  wifi-resume-fix: reconnected 5s after reload on wld0

Across every boot the unit has logged on this machine:

Outcome Count
wedge confirmed by signature, reloaded 8
backstop expired, reloaded 5
reconnected after reload 12
Wi-Fi came back on its own, no reload 4

Every reconnect landed 5–6s after the reload. The one caveat: the copy running on my machine is the pre-cursor revision of the script (it still uses --since), so what is validated on hardware is the signature match, the reload, and the reconnect. The cursor path is covered by the stub tests only.

Happy to add a line to the PR description recording the above if you would like it somewhere more durable than a comment.

@scottjones scottjones left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving.

First, a correction: my mutation result was wrong. I reported the \b in REJECT_SIGNATURE as the one surviving mutation, five of six. You are right that it is caught, and I re-ran it on the current head to be sure:

$ sed -i 's/status_code=16\\b/status_code=16/' bin/omarchy-wifi-resume-fix
$ bash test/shell.d/wifi-resume-fix-test.sh
not ok - status_code=160 does not count as status_code=16

Six of six. Your explanation is almost certainly right — I ran that pass against a head that predated the commit adding the suite. That is my error, not a gap in your tests, and it is worth saying plainly because "the one documented behaviour with no test" was the only concrete thing left in my last review.

Thank you for the BCM4387 log. That was the claim the whole change rests on and it had not been written down anywhere:

Outcome Count
wedge confirmed by signature, reloaded 8
backstop expired, reloaded 5
reconnected after reload 12
Wi-Fi came back on its own, no reload 4

Twelve reconnects, all landing 5-6s after the reload. And flagging yourself that your running copy predates the cursor change, so what hardware validates is the signature match, the reload and the reconnect, while the cursor path is stub-tested only — that is exactly the right way to report it. Please do add it to the PR description; a comment is easy to lose and this is the evidence the change stands on.

Re-verified on the M2 Max. The gate correctly excludes this machine:

01:00.0 Broadcom BCM4388 802.11ax [14e4:4434]     <- not in 14e4:(4425|4433)

So the migration exits before touching anything here, which matches what I saw in the first round. 21 assertions pass, omarchy commands --check clean at 457, CI green, no unresolved threads, merges into quattro without conflict.

The migration's guards read well too — the lspci | grep without -q with its comment about SIGPIPE under pipefail (omacom#6608), and the is-enabled check for a machine another user already repaired.

One note for after this lands, not a blocker and not specific to this PR. ExecStart=/usr/bin/omarchy-wifi-resume-fix resolves to the omarchy package, which will not carry the new command until it is rebuilt:

/usr/bin/omarchy-wifi-resume-fix : MISSING

On affected BCM4378/4387 hardware the unit gets written and enabled, and until the package ships the command the service will fail on resume rather than silently skipping — no regression, since those machines were already wedging, but a failed unit is more visible than nothing. #298 and #303 are both in the same position right now, so it is worth someone tracking the package rebuild rather than each PR carrying the concern separately.

Good work on this one, and sorry for the bad mutation report.

@scottjones
scottjones merged commit 2cd173a into omacom:quattro Sep 6, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wi-Fi wedges on every resume from suspend on Apple Silicon (BCM4387); needs a driver reload

4 participants