WireGuard cleanup and epoch-publication recovery โ September 14, 2026
This release was superseded by the process-worker deployment at 23:16 UTC on September 14. The observations below are historical.
The enclave hotfix 659fc731e5afb25e70f8421d339e972de1f4b824 launched on the
existing c9g.8xlarge at 2026-09-14 22:46:40 UTC (15:46 PDT), with the original
24 CPUs and 8 GiB allocation. It is warming, not yet ready for application
requests. Existing archives, pastes and solver checkpoints are preserved.
Failure and fix
The previous enclave logged puzzle publication failed; service remains closed
at 2026-09-14 04:20:45 UTC. Its third puzzle had completed all 465M hashes.
Mullvad was also down. The parent held 1,015 UDP associations and 2,043 file
descriptors. Retired WireGuard stacks dropped task handles without cancelling
readers blocked on their parent connection. A deterministic regression test
reproduced that retained connection before the fix and passed after cleanup.
This explains the observed accumulation and is consistent with exhaustion of
the enclave's descriptor limit; the old static diagnostic did not expose the
individual failed syscall.
The enclave now aborts both reader and writer tasks when the stack exits. Publication retries preserve the candidate's original disclosure timestamp and monotonic lifetime. Lost acknowledgements cannot rebase an already-public puzzle as fresh. If a candidate expires, its key is discarded and a new candidate is generated rather than permanently exiting the generation loop. Evidence failures before puzzle disclosure retry without throwing away the completed puzzle.
The parent had the same detached-task pattern. Its separate host patch
021d07e4c06c70307873e29bfdbc73ed4f4c4e0b cancels the other UDP direction when
one closes. Host patch. The parent is outside the measured enclave; updating it does not
change the enclave PCR0 or its in-memory generation.
Verification
- 30 enclave unit tests passed, including the retired-tunnel regression.
- 13 host unit tests passed, including idle UDP association closure.
- Seven integration tests passed; the optional external HTTPS test was skipped.
- Failure injection covers retrying a bundle after puzzle disclosure, replacing an expired candidate, refusing late acknowledgements, normal rollover, POST, pastes, and offline recovery of short-work encrypted records.
- Two fresh GitHub ARM builds reproduced PCR0/1/2 and the Docker image. Run 34905620395.
- The report's Sigstore signature verified against the exact CI revision
aa983c2547f7ff28cdf5fea7b8333faecee2c48e, rejecting self-hosted runners. - Fresh public Nitro evidence verified the exact new PCR0, nonce, TLS key, Graviton5, full 465M policy and Mullvad. The normal SDK correctly refuses warming state; forwarding returns 503 until the first puzzle is ready.
Release inputs and PCRs, signed reproduction report, portable signature bundle, fresh Nitro evidence, enclave launch, parent installation, test summary, public source.
Remote reconnect check
After installing the separate parent fix at 22:54 UTC, the enclave ID and TLS identity remained unchanged. The parent logged tunnel establishment at 22:54:06 and a replacement tunnel at 22:57:06. Samples before and after that reconnect remained at one UDP association and 18โ20 parent descriptors, while generation advanced. The old process had accumulated 1,015 associations and 2,043 descriptors. Reconnect observations.
Operations and remaining checks
A separate new-PCR host follower is enabled alongside existing recovery jobs.
A one-shot acceptance service waits for full readiness, then checks a 100 KiB
POST, paste isolation, anonymously retrieved S3 artifacts and a checkpointing
solver. It does not retry application writes or restart the enclave. Its results
are retained on the parent in /home/ec2-user/rollover-fix-deployment-20260914/acceptance.
Boot startup verifies and starts the pinned image only when no enclave exists; it refuses to replace an existing differently measured enclave. Enabling it on the running deployment preserved the current enclave. A reboot was not performed.
Real full-work completion and the next daily rollover remain pending. These checks do not establish an exact seven-day delay or prove the code bug-free. Automatic publication of recovered keys/plaintext remains a separate unfinished feature; existing solvers retain recovered keys locally.