Merged
Conversation
Recorded ahead of Zig 0.15.2 → 0.16.0 migration to enable per-task regression detection during the migration phases. Reference values (ReleaseSafe / vm backend / macOS): binary size: 4.65 MB startup: 4.4 ms ± 0.3 ms (10 runs) RSS: 7.6 MB
Working document for the Zig 0.15.2 → 0.16.0 migration. Records: - existing -Dwasm=false infrastructure (already complete in build.zig) - what fails under -Dwasm=false (test runners, e2e wasm tests, wasm benches) - source files referencing WasmModule (for io threading awareness) - scoped Phase 0 plan Key finding: build/test infrastructure is mostly ready. Main Phase 0 work is teaching the test runners (run_all.sh, run_e2e.sh, wasm_bench.sh, run_bench.sh) a --no-wasm flag. Will be deleted after Phase 7 completion.
Build infrastructure (build.zig -Dwasm=false) was already in place but
test/bench runners assumed wasm was always enabled. Add --no-wasm flag
to four runners so we can validate the rest of the system while zwasm
is detached during the Zig 0.15.2 → 0.16.0 migration:
- test/run_all.sh: --no-wasm propagates -Dwasm=false to zig builds
and forwards --no-wasm to e2e runner
- test/e2e/run_e2e.sh: --no-wasm filters out test/e2e/wasm/ files
(returns 0 when filter empties the test set)
- bench/run_bench.sh: --no-wasm filters wasm_* benchmarks and builds
ReleaseSafe with -Dwasm=false
- bench/wasm_bench.sh: --no-wasm exits 0 immediately
Verified: `zig build -Dwasm=false && bash test/run_all.sh --quick --no-wasm`
passes 4/4 (zig tests, cljw 83 namespaces, e2e non-wasm, deps e2e).
`bash bench/run_bench.sh --no-wasm --quick` runs 22 of 31 benchmarks
(9 wasm_* filtered out).
…ts (Phase 0b) Toolchain pins (build.zig.zon, flake.nix/lock, .github/workflows/*) and version-mention strings (README badge, CLAUDE.md intro, etc.) are deferred to Phase 7 to avoid a window where neither 0.15.2 nor 0.16.0 builds cleanly. This commit makes only the changes that are safe NOW: - baselines.md: temporarily relax binary size ceiling 5.0 MB → 5.5 MB (zwasm v1.10.0+ link_libc adds ~150 KB on macOS / ~290 KB on Linux). Will be reset to the actual measured post-migration value in Phase 7, with a follow-up F## task to strip libc back out (cf. zwasm W46). - zig-016-migration.md: complete file inventory for the Phase 7 atomic toolchain flip. Lists what to touch, what to leave alone (archived phase notes, immutable D## entries), and post-flip validation steps.
… (Phase 0c) Mirror zwasm's flake.nix pattern (sha256 sourced from zwasm v1.10.0+). After this commit, `nix develop` provides Zig 0.16.0 — existing 0.15-style code (std.fs.cwd(), main() with no init arg, etc.) will fail to compile, which is the intended starting point for Phase 1. Other changes: - Drop unused zig-overlay input (was a tracking-only input; the github ref `0.16.0` is not a valid SHA, and zwasm doesn't carry this input). - nixpkgs auto-bumped via `nix flake update`. `build.zig.zon` minimum_zig_version stays at 0.15.2 (will be flipped in Phase 7 alongside README/CLAUDE.md/CI). The `>=` comparison admits 0.16.0 during the migration.
Replan: include zwasm v1.11.0 (first 0.16-compatible tag) from the start of the migration instead of detaching it. Migrating CW + the wasm bridge together keeps wasm tests green throughout (a cleaner signal) and avoids a separate Phase 6 reattach step. Changes: - build.zig.zon: zwasm v1.9.1 → v1.11.0 (hash via `zig fetch --save`) - .gitignore: add zig-pkg/ (Zig 0.16 dependency cache directory) Phase 6 is repurposed to a lightweight bridge-validation step.
Zig 0.16 rejects `|_|` in switch prongs that don't otherwise need the capture — the rule is to omit the capture clause entirely. Two sites: - analyzer.zig:3323 `.regex => |_| Value.nil_val` → `.regex => Value.nil_val` - node.zig:359 `.constant => |_| "constant"` → `.constant => "constant"` Other `|_|` uses in the codebase (catch handlers, for-loop discards) remain valid in 0.16.
Entry points: pub fn main(init: std.process.Init) — replaces argsAlloc with init.minimal.args.toSlice(arena), uses init.gpa for the GPA, and exposes init.io for downstream wiring. Both src/main.zig and src/cache_gen.zig follow zwasm v1.10.0+'s pattern (D135). GC mutex: std.Thread.Mutex was removed in 0.16; the replacement std.Io.Mutex requires an io for lock/unlock. To avoid plumbing io through all 30+ MarkSweepGc.init() call sites (mostly tests), gc.zig keeps a process-wide std.Io.Threaded.init_single_threaded as the default io. Production callers (main, cache_gen — and REPL once Phase 2 lands) overwrite gc.io with init.io after construction so the thread_pool path gets the real cancelable mutex. Tests inherit the single-threaded default unchanged. Build is intentionally not green at this commit — many other 0.15-only stdlib APIs (std.fs.cwd, std.time.nanoTimestamp, std.posix.getenv, std.mem.trimRight, etc.) still need migration in subsequent commits. The build resumes compiling once Phases 2-4 land.
Mechanical rename. Three sites: eval.zig (×2), strings.zig (×1).
Zig 0.16 removed std.Thread.Mutex; std.Io.Mutex's lock/unlock now require
an io argument. CW carries many module-level mutexes that don't have
access to a per-call io value. Introduce src/runtime/io_default.zig with
a process-wide std.Io that defaults to a single-threaded io for tests
and is overwritten with init.io by production entry points (main,
cache_gen).
Migrated mutexes (8 sites):
- src/runtime/gc.zig: gc_mutex (was already on Io.Mutex via init field;
now reads io_default to avoid threading it)
- src/runtime/keyword_intern.zig: module-level intern table mutex
- src/runtime/lifecycle.zig: hook_mutex (shutdown hook list)
- src/runtime/wasm_types.zig: context_mutex (host trampoline registry)
- src/lang/builtins/arithmetic.zig: prng_mutex
- src/lang/builtins/ns_ops.zig: ns_mutex (load tracking)
- src/main.zig, src/cache_gen.zig: call io_default.set(init.io) at startup
Build is still not green: thread_pool.zig (Future, ThreadPool, with
Conditions and timedWait), stm.zig (~20 sites with Conditions), value.zig
(AgentInner, RefInner), atom.zig (nanoTimestamp), nrepl.zig, http_server.zig
remain. Those land in subsequent commits.
Extends io_default with helper wrappers (lockMutex/unlockMutex/condWait/
condTimedWait/condSignal/condBroadcast/sleep) so call sites can replace
the old std.Thread.{Mutex,Condition} method calls with a similar shape
and no per-call io plumbing.
condTimedWait mirrors zwasm's helper (D135): the deadline is computed
once outside the wait loop so spurious wake-ups don't extend the wait.
Migrated:
- runtime/value.zig: AgentInner (mutex+await_cond), RefInner (lock)
field types switched to std.Io.{Mutex,Condition}
- runtime/thread_pool.zig: FutureResult, ThreadPool (queue_mutex+queue_cond),
global pool_mutex; std.Thread.sleep → io_default.sleep
- runtime/stm.zig: ~20 RefInner.lock callers
- lang/builtins/atom.zig: ~10 inner.{mutex,lock,await_cond} callers,
including await/await-for (timed wait)
Build still has fs.cwd/fs.File/nanoTimestamp/posix.getenv/Child.init/
crypto.random/std.net errors in lang/builtins/{io,system,eval,http_server,
shell,collections}, lang/interop/classes/, and engine/vm/jit.zig (macho
vm_prot_t). Those are subsequent commits.
Zig 0.16 removed std.time.{nano,milli}Timestamp, std.posix.getenv, and
std.Thread.sleep. Replacements all need io context; centralize via the
io_default module so call sites don't need to plumb io individually.
io_default additions:
- nanoTimestamp() / milliTimestamp() — wrap std.Io.Timestamp.now(.real)
- getEnv(name) / setEnvironMap(*Environ.Map) — borrow init.environ_map
from main/cache_gen
- sleep(ns) — wrap std.Io.sleep on the awake clock
main/cache_gen: pass init.environ_map to io_default.setEnvironMap.
Migrated:
- lang/builtins/system.zig: nanoTime, currentTimeMillis, getenv, getProperty
user.{dir,home,name,...}, java.io.tmpdir, Thread/sleep
user.dir uses std.c.getcwd (libc is linked via zwasm v1.11.0+).
- lang/builtins/collections.zig: shuffle PRNG seed
- runtime/stm.zig: RefInner.lock initializer adjusted to .init
Remaining: jit.zig macho.vm_prot_t, http_server.zig std.net + Client.io,
io.zig many fs.cwd, eval.zig fs.File, shell.zig process.Child.init,
{buffered_writer,file}.zig fs.cwd, uuid.zig crypto.random,
cljw_wasm_builtins.zig fs.cwd, clojure_java_browse.zig.
std.fs.cwd() / std.fs.File were replaced by std.Io.Dir / std.Io.File
in 0.16, with all I/O methods now requiring an io argument.
Functions migrated (slurp, spit, read-line, load-file, line-seq,
delete-file, make-parents, copy, resource, plus the writeOutput stdout
shortcut and the in-file tests):
- std.fs.cwd().openFile + readToEndAlloc → readFileAlloc(io, path, alloc, .limited(N))
- std.fs.cwd().createFile / writeAll → createFile(io, ...) + writeStreamingAll(io, bytes)
- std.fs.cwd().deleteFile / deleteDir → deleteFile/deleteDir(io, path)
- std.fs.cwd().makePath → createDirPath(io, path)
- std.fs.cwd().statFile → statFile(io, path, .{})
- {STDOUT,STDIN}_FILENO File literal → std.Io.File.{stdout,stdin}()
- file.read(buf) → file.readStreaming(io, &[_][]u8{&buf})
spit's :append mode now reads the existing content and rewrites
(file.seekFromEnd was removed in 0.16; createFile + write is the
straightforward equivalent for typical append-once workloads).
Build still has: jit macho vm_prot_t, eval fs.File, http_server std.net,
ns_ops + buffered_writer + file + cljw_wasm_builtins fs.cwd, shell +
clojure_java_browse Child.init, uuid crypto.random.
…lasses/{file,buffered_writer,uuid}, lang/lib/{cljw_wasm_builtins,clojure_java_browse}
Same fs.cwd / process.Child / crypto.random / etc. patterns as the
previous lang/builtins/io.zig commit. Notable choices:
- shell.zig: split into two paths — std.process.run for no-stdin
(concise: spawn+collect+wait in one call), std.process.spawn for
the input case (write stdin, then read stdout/stderr via
file.reader().interface.allocRemaining). Term variants are now
lowercase (.exited/.signal/.stopped/.unknown) and signal/stopped
carry std.posix.SIG instead of raw integers.
- file.zig getAbsolutePath: std.fs.cwd().realpath was removed; resolve
via std.c.realpath with libc, fall back to a manual cwd-join.
- file.zig listDir: dir.iterate() now returns an Iterator that takes
io on next(); dir.close also takes io.
- uuid.zig randomUUID: std.crypto.random.bytes was removed; use
std.Io.randomSecure (preferred) with a fall-through to std.Io.random.
- eval.zig stdin reads: std.fs.File literal { .handle = STDIN_FILENO }
→ std.Io.File.stdin(); read(buf) → readStreaming(io, &[_][]u8{&buf}).
- ns_ops.zig load path probe + loadResource: openDir → openDir(io, ...);
openFile + readToEndAlloc → readFileAlloc(io, path, alloc, .limited(N)).
- buffered_writer.zig flush in append mode: file.seekFromEnd was removed;
read existing content, rewrite with new bytes appended.
Build still has: jit.zig macho vm_prot_t and http_server.zig std.net +
std.http.Client.io field.
…y off)
Two remaining issues unblock the build:
1. engine/vm/jit.zig — std.posix.PROT became a packed struct in 0.16
(and the macOS variant uses macho.vm_prot_t). Replace bitwise OR with
struct literals (`.{ .READ = true, .WRITE = true }`). std.posix.mprotect
was also removed; call libc's mprotect directly via std.c.mprotect.
2. lang/builtins/http_server.zig — std.net.{Address,Server,Stream} were all
removed in 0.16 (replaced by std.Io.net), and std.http.Client now requires
an `.io` field. Migrating both is substantial: stdlib accept loop, futex-
based shutdown, stream reader/writer interfaces, and the Client API
reshape. Defer the network rewrite to a Phase 7 follow-up F## task and
stub the runtime here:
- run-server, get/post/put/delete return a clear runtime error message
("temporarily disabled while the std.{net,http} migration is in progress")
- parseHttpRequest, ParsedRequest, statusText (and tests) stay intact
- sendRingResponse rewritten as a buffer-formatting helper so it keeps
compiling and is ready to be re-wired once std.Io.net.Stream lands
- ServerState's listener field dropped along with handleConnection /
acceptLoop; bg_server is stubbed
Port the remaining lang-agnostic CLI/runner/test_runner code to 0.16:
- std.fs.File literal { .handle = STDxxx_FILENO } → std.Io.File.{stdout,stderr,stdin}()
- file.write(bytes) → file.writeStreamingAll(io, bytes) (return type
changed from !usize to !void; drop the `_ = ... catch {}` shape)
- std.fs.cwd() → std.Io.Dir.cwd() with io threading on
readFileAlloc/openFile/createFile/createDir/createDirPath/access/
deleteTree/openDir
- std.process.Child.run(.{...}) → std.process.run(allocator, io, .{...});
Term variants are now lowercase (.exited)
- std.posix.getenv → io_default.getEnv
- std.posix.isatty → std.Io.File.stderr().isTty(io)
- std.io.fixedBufferStream + .writer() + .getWritten() → std.Io.Writer.fixed
+ .buffered()
- cli.zig `cljw new` rewritten using std.fmt.bufPrint (the old
fixedBufferStream pattern is gone)
Stubbed for Phase 7 follow-up F##:
- runner.zig readEmbeddedSource: std.fs.selfExePath + openFileAbsolute
were both removed in 0.16; needs argv[0] + std.c.realpath rework
- runner.zig handleBuildCommand: same self-path issue + bundled binary
write loop. `cljw build` now prints a "temporarily disabled" notice.
Remaining: nrepl.zig Thread.Mutex (last mutex site), lifecycle.zig
signal handler + posix.write (signal handler signature change in 0.16).
…green build
After this commit the full Zig binary builds cleanly under Zig 0.16.0 for
the first time since the migration started. Smoke tests:
cljw -e '(+ 1 2 3)' → 6
cljw -e '(println "hello 0.16!")' → hello 0.16!
cljw <file.clj> with side effects → ok
Final batch of changes:
- runtime/lifecycle.zig:
- Sigaction handler signature: fn(_: i32) → fn(_: std.posix.SIG)
- std.posix.write removed → std.c.write (libc) for the async-signal-safe
newline emit on shutdown
- acceptWithShutdownCheck stubbed: std.net.Server / std.posix.poll both
gone in 0.16. Tracked as Phase 7 follow-up alongside http_server +
nrepl. The two callers (http_server, nrepl) are themselves stubbed.
- app/repl/nrepl.zig: ~1818 lines collapsed to a 50-line stub. The full
implementation (bencode dispatch, sessions, eval/load-file/info/eldoc/
lookup/ns-list ops, accept loop) was built on std.net.Server / Stream /
std.Thread.Mutex / std.posix.poll — all gone in 0.16. Original stays in
git history; restored in Phase 7 after the std.Io.net pattern lands.
- app/repl/line_editor.zig: not modified yet — kept compiling by routing
runRepl unconditionally to runReplSimple. The fancy raw-mode line
editor (termios + fs.File + std.io.fixedBufferStream) is non-essential
for the test gate; full port is a Phase 7 follow-up.
- engine/pipeline.zig + app/runner.zig: trailing fs.File literal /
posix.isatty / posix.write sites flushed.
Final batch needed to compile and pass all 1324 unit tests under Zig 0.16:
- runtime/concurrency_test.zig: gc.gc_mutex.lock/unlock → io_default
helpers; std.Thread.sleep → io_default.sleep.
- lang/builtins/ns_ops.zig (test code): std.fs.cwd().makePath /
deleteTree / makeOpenPath / writeFile → std.Io.Dir equivalents with
io threading. makeOpenPath replaced by createDirPathOpen (renamed in
0.16). The require test now closes the dir handle explicitly.
- lang/builtins/io.zig (test): missed cwd.createFile call site picked
up the io argument it was lacking.
- engine/{analyzer,compiler,reader} fuzz tests: std.testing.fuzz now
expects fn(ctx, *std.testing.Smith) instead of fn(ctx, []const u8).
Bridge it via `const input = smith.in orelse return;` (matches
zwasm 1c1526d).
- lang/builtins/shell.zig tests: io_default's default single-threaded
io has `.allocator = .failing`, fine for mutex paths but unable to
back std.process.spawn's Future allocator. Each shell test now sets
up a local std.Io.Threaded with the test's arena allocator before
calling shFn (mirrors zwasm's pattern from src/instance.zig).
- runtime/io_default.zig: getEnv now falls back to std.c.getenv when
setEnvironMap hasn't been called (tests, pre-init). We link libc
anyway, so this is just a thin wrapper. Fixes the getenv "PATH"
test that was returning nil.
Result: 1324/1324 unit tests pass on macOS aarch64 / Zig 0.16.0.
…w-ups
Zig 0.16.0 migration is complete. Flip every remaining version mention
in a single commit so the project advertises 0.16.0 cleanly:
- build.zig.zon: minimum_zig_version 0.15.2 → 0.16.0
- README.md: badge + install link → 0.16.0
- .claude/CLAUDE.md: intro line + "Pitfalls" section header + lib path
- .claude/references/zig-tips.md: title + std.Io.File.stdout() in the
buffered-writer example (with io_default.get())
- .dev/CONTRIBUTING.md: install requirement → 0.16.0
- .dev/baselines.md: re-baselined to post-migration measurements:
4.12 MB binary / 4.1 ms startup / 8.2 MB RSS. Threshold left at
5.5 MB to give headroom for restoring the four 0.16-stubbed
features (HTTP, nREPL, line editor, cljw build) before stripping
libc.
- .dev/references/{setup-orbstack,ubuntu-testing-guide}.md: CI / VM
zig version → 0.16.0; the Rosetta `--seed 0` note now reads as
"needs re-verification on 0.16.0" (line numbers in std/Random.zig
may have shifted) — tracked as F145.
- docs/differences.md: runtime row → 0.16.0
- .dev/future.md: WasmGC paragraph → 0.16.0
- .github/workflows/{ci,nightly,release}.yml: setup-zig version pin
→ 0.16.0 (3 + 2 + 1 sites)
Other meta:
- .dev/decisions.md: D111 records the migration approach (zwasm-first,
io_default centralization, libc linkage, intentionally stubbed
network/build/repl pieces, and the green test gate).
- .dev/checklist.md: F140-F146 capture the post-migration follow-ups
(HTTP server / client, nREPL, line editor, cljw build self-bundling,
OrbStack re-validation, libc strip).
- bench/history.yaml: post-zig-016 entry recorded — no benchmark
regressed beyond noise vs pre-zig-016.
- .dev/zig-016-migration.md → .dev/archive/zig-016-migration.md
(working doc archived; not deleted because the table of Phase 7 flip
targets and zwasm CHANGELOG questions are still useful as historical
reference).
Final gate: bash test/run_all.sh --quick = 4/4 PASS.
Sweep all non-archived docs for the four features stubbed during the Zig 0.16 migration (HTTP server/client, nREPL server, raw-mode line editor, `cljw build` self-bundling). Each user-facing description now carries a "temporarily disabled — see F140-F144" marker pointing at the .dev/checklist.md follow-ups, so anyone reading the docs after this commit knows the features are coming back, not gone. Files: - README.md: marker on the Highlights bullet for `cljw build`, marker on the Build Standalone Binary section, marker on the Server & Networking block, marker on the nREPL/CIDER section; startup figure refreshed 5ms → 4ms to match the new baseline. - ARCHITECTURE.md: status block after the nREPL Server section pointing at F142 + git history pre-`e9b65f3` for the original. - docs/cli.md: top-of-file status note covering all four features, marker on the Build Standalone Binary section, marker on the --nrepl-server option row. - docs/differences.md: CW-Specific Features table — added the HTTP server row (was previously missing) and tagged HTTP server / HTTP client / `cljw build` / nREPL server with their F## numbers. - .dev/memo.md: Current State + Current Task + Task Queue refreshed for the migration. Binary/startup/RSS use the post-migration values. - CHANGELOG.md: added an `Unreleased` section narrating the 0.16 migration (toolchain, zwasm v1.11.0 bump, link_libc rationale, performance numbers, the four stubs, the misc renames). Ready to promote to v0.5.0 once the branch lands on main. Verification: `zig build test` still green (1324/1324). No live doc references stale APIs (`std.fs.cwd`, `std.fs.File`, `fixedBufferStream`, `std.posix.getenv`, `std.posix.write`, `Thread.Mutex`, `Thread.sleep`, `nanoTimestamp`, `milliTimestamp`). The remaining "0.15.2" mentions outside `.dev/archive/` are all intentional historical context (D111 narrative, F145 workaround origin, D258/D442 still-true Zig observations, baselines.md "smaller than 0.15.2" note).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
std.Ioreshapes are centralized behindruntime/io_default.zigso the existing module-level mutexes/condvars/sleeps/env lookups don't have to threadiothrough every call site.build.zig.zonminimum_zig_version,flake.nix/flake.lock,.github/workflows/{ci,nightly,release}.yml, README badge, CLAUDE.md, baselines, CONTRIBUTING, ubuntu/orbstack guides, docs/differences, and theUnreleasedCHANGELOG entry all flip to 0.16.0 in one shot.Test plan
zig build test— 1324/1324 unit tests pass on macOS aarch64./zig-out/bin/cljw test— 83 namespaces, 0 failures, 0 errorsbash test/e2e/run_e2e.sh— 6/6 wasm e2e PASS, deps.edn e2e PASSbash test/run_all.sh --quick— 4/4 PASScljw -e '(+ 1 2 3)'→6,cljw <file.clj>→ okbench/history.yamlrecordspre-zig-016andpost-zig-016entries; no individual benchmark regressed beyond noise (lazy_chainactually improved).dev/baselines.md)Stubs (Phase 7 follow-ups, tracked in
.dev/checklist.md)Four features were collapsed to runtime-error stubs to keep the migration scope tight. Each prints a clear error pointing at the F## item, and the original code is preserved in source or git history.
cljw.http/run-server) — needsstd.Io.net.Serverrewritehttp/get|post|put|delete) — needsstd.http.Client.iofield--nrepl-server) — samestd.Io.network plusstd.posix.pollreplacementrunReplfalls through torunReplSimpleuntil portedcljw buildself-bundling — needsstd.fs.selfExePathreplacementlink_libc = trueoncestd.c.*shims have pure-Zig replacementsCommits (19)
f752739Phase -1 audit …798d794doc audit + CHANGELOG. See.dev/decisions.mdD111 for the full migration narrative.