Skip to content

Commit 11a1342

Browse files
hyperpolymathclaude
hyperpolymath
and
claude
committed
feat(zig-api): initial implementation — Idris2 ABI + Zig FFI replacing V-lang connectors
Adds developer-ecosystem/zig-api/ — the Idris2 ABI + Zig FFI library that replaces all V-lang API and connector code across the hyperpolymath estate (V-lang banned 2026-04-10). Idris2 ABI layer (src/abi/): - Types.idr — Result (11 variants), Handle, Slot, roundtrip + injective proofs - Http.idr — Method (7), ServerState (4), HealthStatus, RouteDescriptor, ValidRoute - Process.idr — IsSafePath dependent type, GnosisRenderCmd/ContextCmd with proof args - Connector.idr — ServiceId (11), ConnectorState (6), GrooveManifest, defaultPort - Foreign.idr — %foreign declarations + IO wrappers for all uapi_* symbols Zig FFI layer (ffi/zig/src/): - core.zig — Result enum, thread-local error storage, generic Pool(T, N) - process.zig — runProcess, runGnosis, safe-path validation (DEFAULT_ALLOWLIST) - gnosis.zig — HTTP/1.1 server pool; handles /render, /context, /health, / and gRPC-style /gnosis.GnosisService/* aliases (replaces v-grpc + v-rest) - connector.zig — 64-slot service connector pool; uapi_connector_* for all 11 services - lib.zig — root entry; uapi_init / uapi_teardown / uapi_version Infrastructure: - build.zig (Zig 0.15.2 module API), integration tests, generated/abi/zig_api.h, Justfile, EXPLAINME.adoc, .machine_readable/6a2/STATE.a2ml All 14 unit tests pass (zig build test). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 28cace4 commit 11a1342

17 files changed

Lines changed: 3144 additions & 0 deletions

File tree

‎zig-api/.machine_readable/6a2/STATE.a2ml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
;; SPDX-License-Identifier: PMPL-1.0-or-later
2+
;; unified-zig-api STATE.a2ml
3+
;; Updated: 2026-04-11
4+
5+
(state
6+
(version "0.1.0")
7+
(project "unified-zig-api")
8+
(phase "initial-implementation")
9+
(status "in-progress")
10+
11+
(completed
12+
(item "Idris2 ABI layer — Types, Http, Process, Connector, Foreign")
13+
(item "Zig FFI — core.zig (Result, Pool, error storage)")
14+
(item "Zig FFI — process.zig (runProcess, runGnosis, safe-path)")
15+
(item "Zig FFI — gnosis.zig (HTTP/1.1 server pool, uapi_gnosis_*)")
16+
(item "Zig FFI — connector.zig (64-slot connector pool, uapi_connector_*)")
17+
(item "Zig FFI — lib.zig (root entry, uapi_init/teardown/version)")
18+
(item "Integration test — ffi/zig/test/integration_test.zig")
19+
(item "C header — generated/abi/zig_api.h")
20+
(item "Justfile")
21+
(item "EXPLAINME.adoc"))
22+
23+
(pending
24+
(item "src/abi/Layout.idr — memory layout proofs")
25+
(item "src/abi/Proofs.idr — combined roundtrip proof entry point")
26+
(item "V-lang estate sweep — aerie, ambientops, verisimdb connectors, proven, 007, standards/lol, opsm_ex")
27+
(item "First compile + test run — zig build test in ffi/zig/"))
28+
29+
(blockers
30+
(item "gnosis binary must be on PATH for integration tests"))
31+
32+
(v-lang-migration
33+
(repos-to-migrate
34+
"developer-ecosystem/v-grpc"
35+
"developer-ecosystem/v-rest"
36+
"ambientops connectors"
37+
"nextgen-databases/verisimdb connectors"
38+
"proven V connector stubs"
39+
"007 V helper modules"
40+
"standards/lol V stubs"
41+
"odds-and-sods-package-manager/opsm_ex"))
42+
43+
(session-history
44+
(session
45+
(date "2026-04-11")
46+
(author "Jonathan D.A. Jewell + Claude")
47+
(work "Initial implementation: all Idris2 ABI files, all Zig FFI files, C header, integration tests, Justfile, EXPLAINME"))))

‎zig-api/EXPLAINME.adoc‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
3+
= unified-zig-api — EXPLAINME
4+
:toc:
5+
:toc-placement: preamble
6+
:doctype: article
7+
8+
== What this is
9+
10+
`unified-zig-api` is the Idris2 ABI + Zig FFI replacement for all V-lang API
11+
and connector code in the hyperpolymath estate. It produces a single shared
12+
library (`libzig_api.so`) and static library (`libzig_api.a`) that expose a
13+
stable C ABI consumed by any language (Idris2, Rust, ReScript, Julia, etc.).
14+
15+
V-lang was banned estate-wide on 2026-04-10. This library is the designated
16+
replacement for the 10 repos that still had V-lang connector or API server code.
17+
18+
== Why it's built this way
19+
20+
=== Idris2 ABI layer (`src/abi/`)
21+
22+
Idris2's dependent types let us _prove_ things about the interface at
23+
compile time:
24+
25+
* **Tag round-trips**: every `Result`, `ServiceId`, `ConnectorState`, and
26+
`Method` tag encodes and decodes back to the same value, proved by exhaustive
27+
`Refl` cases.
28+
* **Safe-path soundness**: `IsSafePath path allowlist` is a structural proof
29+
that `path` starts with an allowlist prefix. The `GnosisRenderCmd` record
30+
carries this proof as an auto-implicit argument — you cannot construct a
31+
render command for an unsafe path without discharging the proof.
32+
* **ABI injective**: `resultTagInjective` proves that equal Bits8 tags imply
33+
equal Result values, so the decode side cannot silently alias two variants.
34+
35+
=== Zig FFI layer (`ffi/zig/src/`)
36+
37+
Zig gives us native C ABI compatibility without overhead, comptime generics
38+
(the `Pool(T, capacity)` in `core.zig`), and `std.process.Child` for safe
39+
subprocess management.
40+
41+
The four modules:
42+
43+
[cols="1,3"]
44+
|===
45+
| Module | Responsibility
46+
47+
| `core.zig`
48+
| `Result` enum, thread-local error storage, generic `Pool(T, N)` with mutex
49+
50+
| `process.zig`
51+
| Safe-path validation, `runProcess`, `runGnosis` (shells out to the `gnosis` binary)
52+
53+
| `gnosis.zig`
54+
| HTTP/1.1 server pool (replaces `v-grpc` + `v-rest`). Handles `/render`,
55+
`/context`, `/health`, `/`, and gRPC-style `/gnosis.GnosisService/*` paths.
56+
57+
| `connector.zig`
58+
| 64-slot HTTP connector pool for all 11 hyperpolymath services.
59+
`uapi_connector_call` is the universal request dispatcher.
60+
|===
61+
62+
== Routes served by the gnosis server
63+
64+
[cols="2,1,3"]
65+
|===
66+
| Path | Method | Description
67+
68+
| `/render` | POST | Render a template via the `gnosis` binary
69+
| `/gnosis.GnosisService/Render` | POST | gRPC-style compat alias
70+
| `/context` | GET/POST | Retrieve SCM context
71+
| `/gnosis.GnosisService/Context` | POST | gRPC-style compat alias
72+
| `/health` | GET | 200 if serving, 503 if not
73+
| `/gnosis.GnosisService/Health` | POST | gRPC-style compat alias
74+
| `/` | GET | Service discovery JSON
75+
|===
76+
77+
== What it replaces
78+
79+
`developer-ecosystem/v-grpc/` — gRPC server calling the gnosis binary +
80+
`developer-ecosystem/v-rest/` — REST server calling the gnosis binary
81+
82+
Both upstream callers call the same `gnosis` binary. A single HTTP interface
83+
with both path families covers all existing call sites.
84+
85+
== Directory layout
86+
87+
----
88+
zig-api/
89+
├── src/abi/ Idris2 ABI definitions (proofs live here)
90+
│ ├── Types.idr Result codes, Handle, Slot
91+
│ ├── Http.idr Method, ServerState, HealthStatus, RouteDescriptor
92+
│ ├── Process.idr IsSafePath, ExecResult, GnosisRenderCmd
93+
│ ├── Connector.idr ServiceId, ConnectorState, GrooveManifest
94+
│ └── Foreign.idr %foreign declarations + IO wrappers
95+
├── ffi/zig/
96+
│ ├── build.zig Produces shared + static libs, test runner
97+
│ └── src/
98+
│ ├── lib.zig Root entry point, uapi_init / teardown / version
99+
│ ├── core.zig Result, error storage, Pool(T, N)
100+
│ ├── process.zig runProcess, runGnosis, safe-path validation
101+
│ ├── gnosis.zig HTTP/1.1 server pool (uapi_gnosis_*)
102+
│ └── connector.zig Service connector pool (uapi_connector_*)
103+
├── generated/abi/
104+
│ └── zig_api.h C header (installed alongside the libraries)
105+
└── Justfile Build, test, and ABI-check recipes
106+
----
107+
108+
== Quick start
109+
110+
[source,bash]
111+
----
112+
# Build shared + static library
113+
just build
114+
115+
# Run all unit tests
116+
just test
117+
118+
# Check Idris2 proofs
119+
just check-abi
120+
121+
# Run integration tests (requires gnosis binary on $PATH)
122+
just test-integration
123+
----
124+
125+
== Consuming from Idris2
126+
127+
[source,idris]
128+
----
129+
import ZigApi.ABI.Foreign
130+
131+
main : IO ()
132+
main = do
133+
Ok <- uapiInit | _ => putStrLn "init failed"
134+
h <- gnosisCreate 8091
135+
Ok <- gnosisStart h | _ => putStrLn "start failed"
136+
-- ... use the server ...
137+
gnosisStop h
138+
gnosisDestroy h
139+
uapiTeardown
140+
----
141+
142+
== Consuming from C
143+
144+
[source,c]
145+
----
146+
#include "zig_api.h"
147+
148+
int main(void) {
149+
uapi_init();
150+
uint64_t h = uapi_gnosis_create(8091);
151+
uapi_gnosis_start(h);
152+
/* ... */
153+
uapi_gnosis_stop(h);
154+
uapi_gnosis_destroy(h);
155+
uapi_teardown();
156+
return 0;
157+
}
158+
----
159+
160+
== V-lang estate migration status
161+
162+
Repos to migrate from V-lang to this library:
163+
164+
* `developer-ecosystem/v-grpc` — replaced by `gnosis.zig` server
165+
* `developer-ecosystem/v-rest` — replaced by `gnosis.zig` server
166+
* `ambientops/` connectors — replace with `uapi_connector_create(UAPI_SERVICE_AMBIENT_OPS, ...)`
167+
* `nextgen-databases/verisimdb/` connectors — `UAPI_SERVICE_VERISIMDB`
168+
* `proven/` V connector stubs — replace with connector pool calls
169+
* `007/` V helper modules — replace with `uapi_connector_*` or process calls
170+
* `standards/lol/` V stubs — replace per sub-project
171+
* `odds-and-sods-package-manager/opsm_ex/` — replace V HTTP client
172+
173+
See `.machine_readable/6a2/STATE.a2ml` for detailed migration tracking.

‎zig-api/Justfile‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# SPDX-License-Identifier: PMPL-1.0-or-later
2+
# Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
3+
#
4+
# Justfile — Build recipes for unified-zig-api
5+
#
6+
# Requires:
7+
# zig >= 0.15.2
8+
# idris2 (for ABI proof checking)
9+
10+
zig_dir := "ffi/zig"
11+
12+
# Default: build everything
13+
default: build
14+
15+
# Build shared + static libraries
16+
build:
17+
cd {{zig_dir}} && zig build
18+
19+
# Build with release optimisation
20+
build-release:
21+
cd {{zig_dir}} && zig build -Doptimize=ReleaseSafe
22+
23+
# Run unit tests (core, process, gnosis, connector)
24+
test:
25+
cd {{zig_dir}} && zig build test
26+
27+
# Run integration tests (spawns a local gnosis server)
28+
test-integration:
29+
cd {{zig_dir}} && zig build test-integration
30+
31+
# Run all tests (unit + integration)
32+
test-all: test test-integration
33+
34+
# Check Idris2 ABI proofs (requires idris2 on PATH)
35+
check-abi:
36+
idris2 --check src/abi/Types.idr
37+
idris2 --check src/abi/Http.idr
38+
idris2 --check src/abi/Process.idr
39+
idris2 --check src/abi/Connector.idr
40+
idris2 --check src/abi/Foreign.idr
41+
42+
# Build + check everything
43+
ci: check-abi test
44+
45+
# Remove build artefacts
46+
clean:
47+
rm -rf {{zig_dir}}/zig-out {{zig_dir}}/.zig-cache
48+
49+
# Show the ABI version embedded in the library
50+
version:
51+
cd {{zig_dir}} && zig build && ./zig-out/bin/zig_api_version 2>/dev/null || \
52+
strings zig-out/lib/libzig_api.so | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | head -1

‎zig-api/ffi/zig/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
.zig-cache/
2+
zig-out/

‎zig-api/ffi/zig/build.zig‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
3+
//
4+
// build.zig — unified-zig-api build configuration (Zig 0.15.2)
5+
//
6+
// Produces:
7+
// libzig_api.so — shared library (runtime linking)
8+
// libzig_api.a — static library (embed in binaries)
9+
// unit tests — all modules under zig build test
10+
// integration — spawns a local gnosis server (zig build test-integration)
11+
12+
const std = @import("std");
13+
14+
pub fn build(b: *std.Build) void {
15+
const target = b.standardTargetOptions(.{});
16+
const optimize = b.standardOptimizeOption(.{});
17+
18+
// -------------------------------------------------------------------------
19+
// Root module — shared by both library artifacts and the test runner
20+
// -------------------------------------------------------------------------
21+
const root_mod = b.createModule(.{
22+
.root_source_file = b.path("src/lib.zig"),
23+
.target = target,
24+
.optimize = optimize,
25+
.link_libc = true,
26+
});
27+
28+
// -------------------------------------------------------------------------
29+
// Shared library — libzig_api.so / libzig_api.dylib / libzig_api.dll
30+
// -------------------------------------------------------------------------
31+
const shared = b.addLibrary(.{
32+
.name = "zig_api",
33+
.root_module = root_mod,
34+
.linkage = .dynamic,
35+
.version = .{ .major = 0, .minor = 1, .patch = 0 },
36+
});
37+
b.installArtifact(shared);
38+
39+
// -------------------------------------------------------------------------
40+
// Static library — libzig_api.a
41+
// -------------------------------------------------------------------------
42+
const static_mod = b.createModule(.{
43+
.root_source_file = b.path("src/lib.zig"),
44+
.target = target,
45+
.optimize = optimize,
46+
.link_libc = true,
47+
});
48+
const static = b.addLibrary(.{
49+
.name = "zig_api",
50+
.root_module = static_mod,
51+
.linkage = .static,
52+
});
53+
b.installArtifact(static);
54+
55+
// -------------------------------------------------------------------------
56+
// Install C header alongside the libraries
57+
// -------------------------------------------------------------------------
58+
const install_header = b.addInstallHeaderFile(
59+
b.path("../../generated/abi/zig_api.h"),
60+
"zig_api.h",
61+
);
62+
b.getInstallStep().dependOn(&install_header.step);
63+
64+
// -------------------------------------------------------------------------
65+
// Unit test runner — covers all modules via lib.zig's `test { refAllDecls }`
66+
// -------------------------------------------------------------------------
67+
const test_mod = b.createModule(.{
68+
.root_source_file = b.path("src/lib.zig"),
69+
.target = target,
70+
.optimize = optimize,
71+
.link_libc = true,
72+
});
73+
const unit_tests = b.addTest(.{
74+
.root_module = test_mod,
75+
});
76+
77+
const run_unit_tests = b.addRunArtifact(unit_tests);
78+
const test_step = b.step("test", "Run all unified-zig-api unit tests");
79+
test_step.dependOn(&run_unit_tests.step);
80+
81+
// -------------------------------------------------------------------------
82+
// Integration test — spawns a local gnosis server and probes it
83+
// -------------------------------------------------------------------------
84+
const integration_mod = b.createModule(.{
85+
.root_source_file = b.path("test/integration_test.zig"),
86+
.target = target,
87+
.optimize = optimize,
88+
.link_libc = true,
89+
});
90+
// Make `@import("zig_api")` available inside the integration test.
91+
integration_mod.addAnonymousImport("zig_api", .{
92+
.root_source_file = b.path("src/lib.zig"),
93+
.target = target,
94+
.optimize = optimize,
95+
.link_libc = true,
96+
});
97+
98+
const integration_tests = b.addTest(.{
99+
.root_module = integration_mod,
100+
});
101+
102+
const run_integration = b.addRunArtifact(integration_tests);
103+
const integration_step = b.step("test-integration", "Run integration tests");
104+
integration_step.dependOn(&run_integration.step);
105+
}

0 commit comments

Comments
 (0)