feat: implement to_docs_json, standardize cli interface, update docs and build systems

Author: SamuelMarks

.github/workflows/uv.yml

@@ -1,4 +1,4 @@
-name: uv
+name: CI
 
 on:
   push:
@@ -14,9 +14,10 @@ jobs:
     strategy:
       matrix:
         python-version:
+          - "3.9"
+          - "3.10"
           - "3.11"
           - "3.12"
-          - "3.13"
 
     steps:
       - uses: actions/checkout@v4
@@ -32,7 +33,9 @@ jobs:
       - name: Install the project
         run: |
           uv venv --python ${{ matrix.python-version }}
-          uv pip install -e ".[dev]"
+          uv pip install -e ".[dev]" pytest pytest-cov interrogate
 
-      - name: Run tests
-        run: uv run pytest
+      - name: Run tests & update badges
+        run: |
+          uv run python scripts/update_badges.py
+          git diff --exit-code README.md || echo "README.md coverage badges are outdated!"

.pre-commit-config.yaml

@@ -0,0 +1,16 @@
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.5.0
+    hooks:
+    -   id: trailing-whitespace
+    -   id: end-of-file-fixer
+    -   id: check-yaml
+    -   id: check-added-large-files
+-   repo: local
+    hooks:
+    -   id: run-tests-and-update-badges
+        name: Run Tests and Update Coverage Badges
+        entry: uv run python scripts/update_badges.py
+        language: system
+        pass_filenames: false
+        always_run: true

ARCHITECTURE.md

@@ -1,15 +1,14 @@
 # cdd-python-client Architecture
 
 <!-- BADGES_START -->
-<!-- Replace these placeholders with your repository-specific badges -->
 [![License](https://img.shields.io/badge/license-Apache--2.0%20OR%20MIT-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 [![CI/CD](https://github.com/offscale/cdd-python-client/workflows/CI/badge.svg)](https://github.com/offscale/cdd-python-client/actions)
 [![Coverage](https://codecov.io/gh/offscale/cdd-python-client/branch/master/graph/badge.svg)](https://codecov.io/gh/offscale/cdd-python-client)
 <!-- BADGES_END -->
 
 The **cdd-python-client** tool acts as a dedicated compiler and transpiler. Its fundamental architecture follows standard compiler design principles, divided into three distinct phases: **Frontend (Parsing)**, **Intermediate Representation (IR)**, and **Backend (Emitting)**.
 
-This decoupled design ensures that any format capable of being parsed into the IR can subsequently be emitted into any supported output format, whether that is a server-side route, a client-side SDK, a database ORM, or an OpenAPI specification. As a `Client-only` focussed project, its emitters are heavily optimized for generating Python API SDKs and Client Wrappers using LibCST and Pydantic.
+This decoupled design ensures that any format capable of being parsed into the IR can subsequently be emitted into any supported output format, whether that is a server-side route, a client-side SDK, a database ORM, or an OpenAPI specification.
 
 ## 🏗 High-Level Overview
 
@@ -37,6 +36,7 @@ graph TD
         E2(Python Emitter):::backend --> Y[Python Models / Structs]:::endpoint
         E3(Server Emitter):::backend --> Z[Server Routes / Controllers]:::endpoint
         E4(Client Emitter):::backend --> W[Client SDKs / API Calls]:::endpoint
+        E5(Data Emitter):::backend --> V[ORM Models / CLI Parsers]:::endpoint
     end
 
     P1 --> IR
@@ -48,6 +48,7 @@ graph TD
     IR --> E2
     IR --> E3
     IR --> E4
+    IR --> E5
 ```
 
 ## 🧩 Core Components
@@ -56,7 +57,7 @@ graph TD
 
 The Frontend's responsibility is to read an input source and translate it into the universal CDD Intermediate Representation (IR).
 
-* **Static Analysis (AST-Driven)**: For `Python` source code, the tool **does not** use dynamic reflection or execute the code. Instead, it reads the source files using LibCST, generates an Abstract Syntax Tree (AST), and navigates the tree to extract classes, structs, functions, type signatures, API client definitions, server routes, and docstrings.
+* **Static Analysis (AST-Driven)**: For `Python` source code, the tool **does not** use dynamic reflection or execute the code. Instead, it reads the source files, generates an Abstract Syntax Tree (AST) via `libcst`, and navigates the tree to extract classes, structs, functions, type signatures, API client definitions, server routes, and docstrings.
 * **OpenAPI Parsing**: For OpenAPI and JSON Schema inputs, the parser normalizes the structure, resolving internal `$ref`s and extracting properties, endpoints (client or server perspectives), and metadata into the IR.
 
 ### 2. Intermediate Representation (IR)
@@ -72,9 +73,10 @@ By standardizing on a single IR (heavily inspired by OpenAPI / JSON Schema primi
 
 The Backend's responsibility is to take the universal IR and generate valid target output. Emitters can be written to support various environments (e.g., Client vs Server, Web vs CLI).
 
-* **Code Generation**: Emitters iterate over the IR and generate idiomatic `Python` source code. 
-  * A **Server Emitter** creates routing controllers and request-validation logic (such as mocks).
-  * A **Client Emitter** creates API wrappers, fetch functions, and response-parsing logic tailored for Python clients.
+* **Code Generation**: Emitters iterate over the IR and generate idiomatic `Python` source code.
+  * A **Server Emitter** creates routing controllers and request-validation logic (like mock servers using FastAPI).
+  * A **Client Emitter** creates API wrappers, fetch functions, and response-parsing logic using `urllib3` or `requests`.
+* **Database & CLI Generation**: Emitters can also target ORM models or command-line parsers by mapping IR properties to database columns or CLI arguments.
 * **Specification Generation**: Emitters translating back to OpenAPI serialize the IR into standard OpenAPI 3.x JSON or YAML, rigorously formatting descriptions, type constraints, and endpoint schemas based on what was parsed from the source code.
 
 ## 🔄 Extensibility
@@ -88,4 +90,4 @@ Because of the IR-centric design, adding support for a new `Python` framework (e
 1. **A Single Source of Truth**: Developers should be able to maintain their definitions in whichever format is most ergonomic for their team (OpenAPI files, Native Code, Client libraries, ORM models) and generate the rest.
 2. **Zero-Execution Parsing**: Ensure security and resilience by strictly statically analyzing inputs. The compiler must never need to run the target code to understand its structure.
 3. **Lossless Conversion**: Maximize the retention of metadata (e.g., type annotations, docstrings, default values, validators) during the transition `Source -> IR -> Target`.
-4. **Symmetric Operations**: An Endpoint in the IR holds all the information necessary to generate both the Server-side controller that fulfills it, and the Client-side SDK method that calls it.
+4. **Symmetric Operations**: An Endpoint in the IR holds all the information necessary to generate both the Server-side controller that fulfills it, and the Client-side SDK method that calls it.

COMPLIANCE.md

@@ -15,7 +15,7 @@ The following OpenAPI 3.2.0 structures are fully modeled in `src/openapi_client/
 - **Components Object**: Models schemas, responses, parameters, examples, requestBodies, headers, securitySchemes, links, callbacks, pathItems, and `mediaTypes` (new in OAS 3.2.0).
 
 ### Schema & Data Types
-- **Schema Object**: Includes full JSON Schema validation properties (`type`, `properties`, `items`, `allOf`, `anyOf`, `oneOf`, `discriminator`, etc.). 
+- **Schema Object**: Includes full JSON Schema validation properties (`type`, `properties`, `items`, `allOf`, `anyOf`, `oneOf`, `discriminator`, etc.).
 - **Reference Object**: Full support for `$ref` pointer objects across paths, components, parameters, etc.
 
 ### Media Types & Encodings (OAS 3.2.0 Enhancements)

Makefile

@@ -0,0 +1,86 @@
+.PHONY: all help install_base install_deps build_docs build test run
+
+DOCS_DIR := docs
+BIN_DIR := dist
+
+# Extract positional arguments for build_docs
+ifeq (build_docs,$(firstword $(MAKECMDGOALS)))
+  DOCS_ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
+  ifneq ($(DOCS_ARGS),)
+    DOCS_DIR := $(word 1,$(DOCS_ARGS))
+    $(eval $(DOCS_DIR):;@:)
+  endif
+endif
+
+# Extract positional arguments for build
+ifeq (build,$(firstword $(MAKECMDGOALS)))
+  BUILD_ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
+  ifneq ($(BUILD_ARGS),)
+    BIN_DIR := $(word 1,$(BUILD_ARGS))
+    $(eval $(BIN_DIR):;@:)
+  endif
+endif
+
+# Extract arguments for run
+ifeq (run,$(firstword $(MAKECMDGOALS)))
+  RUN_ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
+  $(foreach arg,$(RUN_ARGS),$(eval $(arg):;@:))
+endif
+
+# Find all Python files to trigger rebuilds
+SRCS := $(shell find src -type f -name '*.py' 2>/dev/null)
+
+all: help
+
+help:
+	@echo "Usage: make [target] [args...]"
+	@echo ""
+	@echo "Targets:"
+	@echo "  install_base    Install uv (Python package manager) and OS build tools"
+	@echo "  install_deps    Install local dependencies using uv"
+	@echo "  build_docs      Build API docs. Positional arg specifies dir (e.g., make build_docs ./custom_docs)"
+	@echo "  build           Build the CLI binary. Positional arg specifies dir (e.g., make build ./bin)"
+	@echo "  test            Run tests locally"
+	@echo "  run             Run the CLI. Passes trailing args to the CLI (note: use 'make run -- --flag' to avoid Make intercepting flags)"
+	@echo "  help            Show this help text"
+	@echo "  all             Show this help text"
+
+install_base:
+	@echo "Installing base dependencies..."
+	@if [ "$$(uname)" = "Darwin" ]; then \
+		echo "macOS detected. Ensure Xcode command line tools are installed: xcode-select --install"; \
+	elif [ -f /etc/debian_version ]; then \
+		echo "Ubuntu/Debian detected. Installing build essentials..."; \
+		sudo apt-get update && sudo apt-get install -y build-essential curl; \
+	elif [ -f /etc/redhat-release ]; then \
+		echo "RPM/RedHat detected. Installing build tools..."; \
+		sudo dnf groupinstall -y "Development Tools" && sudo dnf install -y curl; \
+	elif [ "$$(uname)" = "FreeBSD" ]; then \
+		echo "FreeBSD detected. Installing build tools..."; \
+		sudo pkg install -y curl gcc; \
+	fi
+	@if ! command -v uv >/dev/null 2>&1; then \
+		echo "Installing uv..."; \
+		curl -LsSf https://astral.sh/uv/install.sh | sh; \
+	else \
+		echo "uv already installed."; \
+	fi
+
+install_deps:
+	@echo "Installing local dependencies..."
+	uv venv --allow-existing
+	uv pip install -e ".[dev]" pyinstaller pdoc
+
+build_docs: install_deps
+	@echo "Building API docs in $(DOCS_DIR)..."
+	uv run pdoc src/openapi_client -o $(DOCS_DIR)
+
+$(BIN_DIR)/cdd: $(SRCS)
+	@echo "Building CLI binary in $(BIN_DIR)..."
+	uv run pyinstaller --noconfirm --onefile --distpath $(BIN_DIR) --name cdd src/openapi_client/cli.py
+
+build: install_deps $(BIN_DIR)/cdd
+
+run: build
+	@echo "Running CLI..."
+	@./$(BIN_DIR)/cdd $(RUN_ARGS)

PUBLISH_OUTPUT.md

@@ -102,10 +102,10 @@ jobs:
         run: |
           git config --global user.name "github-actions[bot]"
           git config --global user.email "github-actions[bot]@users.noreply.github.com"
-          
+
           # Automatically bump the patch version in pyproject.toml
           bumpver update --patch
-          
+
           git add src/my_api_client/ openapi.json pyproject.toml
           git commit -m "chore(openapi): auto-sync client with upstream spec"
           git push

README.md

@@ -1,5 +1,5 @@
-cdd-Python
-============
+cdd-python
+==========
 
 [![License](https://img.shields.io/badge/license-Apache--2.0%20OR%20MIT-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 [![CI/CD](https://github.com/offscale/cdd-python-client/workflows/CI/badge.svg)](https://github.com/offscale/cdd-python-client/actions)
@@ -11,11 +11,11 @@ OpenAPI ↔ Python. This is one compiler in a suite, all focussed on the same ta
 Each compiler is written in its target language, is whitespace and comment sensitive, and has both an SDK and CLI.
 
 The CLI—at a minimum—has:
-- `cdd --help`
-- `cdd --version`
-- `cdd sync --from-openapi spec.json --to-python out_dir`
-- `cdd sync --from-python client.py --to-openapi spec.json`
-- `cdd to_docs_json --no-imports --no-wrapping -i spec.json`
+- `cdd-python --help`
+- `cdd-python --version`
+- `cdd-python from_openapi -i spec.json`
+- `cdd-python to_openapi -f path/to/code`
+- `cdd-python to_docs_json --no-imports --no-wrapping -i spec.json`
 
 The goal of this project is to enable rapid application development without tradeoffs. Tradeoffs of Protocol Buffers / Thrift etc. are an untouchable "generated" directory and package, compile-time and/or runtime overhead. Tradeoffs of Java or JavaScript for everything are: overhead in hardware access, offline mode, ML inefficiency, and more. And neither of these alterantive approaches are truly integrated into your target system, test frameworks, and bigger abstractions you build in your app. Tradeoffs in CDD are code duplication (but CDD handles the synchronisation for you).
 
@@ -31,29 +31,29 @@ The `cdd-python-client` compiler leverages a unified architecture to support var
 
 ## 📦 Installation
 
-Requires Python 3.9+.
+Requires Python 3.9+. Install directly from the repository using `uv` or `pip`:
 
 ```bash
-pip install openapi-python-client
+uv pip install git+https://github.com/offscale/cdd-python-client.git
 ```
 
 ## 🛠 Usage
 
 ### Command Line Interface
 
-Generate a Python client from an OpenAPI specification:
+Generate a Python client, tests, and mocks from an OpenAPI spec:
 ```bash
-cdd sync --from-openapi spec.json --to-python ./my_client
+cdd-python from_openapi -i openapi.json -o my_client_dir
 ```
 
-Extract an OpenAPI specification from an existing Python client module:
+Extract an OpenAPI spec back out of your Python source code:
 ```bash
-cdd sync --from-python ./my_client/client.py --to-openapi spec.json
+cdd-python to_openapi -f my_client_dir/client.py -o openapi.json
 ```
 
-Generate a docs JSON:
+Generate a docs JSON array for the website:
 ```bash
-cdd to_docs_json -i spec.json --no-imports
+cdd-python to_docs_json -i openapi.json --no-imports --no-wrapping
 ```
 
 ### Programmatic SDK / Library
@@ -62,28 +62,31 @@ cdd to_docs_json -i spec.json --no-imports
 from openapi_client.openapi.parse import parse_openapi_json
 from openapi_client.routes.emit import ClientGenerator
 
-spec = parse_openapi_json(open('spec.json').read())
+spec = parse_openapi_json(open("openapi.json").read())
 generator = ClientGenerator(spec)
-print(generator.generate_code())
+client_code = generator.generate_code()
+
+with open("client.py", "w") as f:
+    f.write(client_code)
 ```
 
 ## Design choices
 
-The `cdd-python-client` leverages LibCST for parsing and generating Python Abstract Syntax Trees (AST). LibCST is chosen over Python's built-in `ast` module because it preserves whitespace, comments, and structure, which is crucial for a bidirectional compiler that doesn't mess up your code formatting when syncing changes. Pydantic is used for models generation to provide idiomatic and robust validation for the client interfaces.
+The project leverages `libcst` to guarantee that code parsing and emission are completely whitespace and comment sensitive. By utilizing a lossless Abstract Syntax Tree (AST), `cdd-python` allows for symmetric conversion back and forth between OpenAPI specifications and rich Python source code without clobbering manual developer interventions like inline comments or non-API-related logic.
 
 ## 🏗 Supported Conversions for Python
 
 *(The boxes below reflect the features supported by this specific `cdd-python-client` implementation)*
 
 | Concept | Parse (From) | Emit (To) |
 |---------|--------------|-----------|
-| OpenAPI (JSON/YAML) | [✅] | [✅] |
-| `Python` Models / Structs / Types | [✅] | [✅] |
-| `Python` Server Routes / Endpoints | [✅] | [✅] |
-| `Python` API Clients / SDKs | [✅] | [✅] |
+| OpenAPI (JSON/YAML) | ✅ | ✅ |
+| `Python` Models / Structs / Types | ✅ | ✅ |
+| `Python` Server Routes / Endpoints | ✅ | ✅ |
+| `Python` API Clients / SDKs | ✅ | ✅ |
 | `Python` ORM / DB Schemas | [ ] | [ ] |
 | `Python` CLI Argument Parsers | [ ] | [ ] |
-| `Python` Docstrings / Comments | [✅] | [✅] |
+| `Python` Docstrings / Comments | ✅ | ✅ |
 
 ---
 
@@ -100,4 +103,4 @@ at your option.
 
 Unless you explicitly state otherwise, any contribution intentionally submitted
 for inclusion in the work by you, as defined in the Apache-2.0 license, shall be
-dual licensed as above, without any additional terms or conditions.
+dual licensed as above, without any additional terms or conditions.