Skip to content

Commit 4ba7424

Browse files
committed
Initial commit
Hyper: a Python framework for hypermedia-driven applications. Write templates in .hyper syntax, compile to type-safe Python.
0 parents  commit 4ba7424

402 files changed

Lines changed: 53604 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ci.yml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
8+
concurrency:
9+
group: ${{ github.workflow }}-${{ github.ref }}
10+
cancel-in-progress: true
11+
12+
jobs:
13+
check:
14+
runs-on: ubuntu-latest
15+
steps:
16+
- uses: actions/checkout@v4
17+
- uses: dtolnay/rust-toolchain@stable
18+
with:
19+
components: clippy, rustfmt
20+
- uses: Swatinem/rust-cache@v2
21+
with:
22+
workspaces: rust
23+
- name: Format
24+
run: cd rust && cargo fmt --check
25+
- name: Lint
26+
run: cd rust && cargo clippy -- -D warnings
27+
- name: Test
28+
run: cd rust && cargo test

.gitignore

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Python
2+
__pycache__/
3+
*.py[cod]
4+
*.egg-info/
5+
.eggs/
6+
dist/
7+
build/
8+
.venv/
9+
10+
# Virtual/generated files
11+
*.virtual.py
12+
playground/templates/**/*.py
13+
rust/playground/
14+
15+
# Generated .py files in test directories (only .expected.py should be tracked)
16+
rust/transpiler/tests/**/*.py
17+
!rust/transpiler/tests/**/*.expected.py
18+
19+
# Rust build artifacts
20+
rust/target/
21+
22+
# IDE
23+
.idea/
24+
*.iml
25+
.vscode/
26+
.claude/
27+
28+
# JetBrains plugin build artifacts
29+
editors/jetbrains-plugin/.intellijPlatform/
30+
editors/jetbrains-plugin/.gradle/
31+
editors/jetbrains-plugin/.sandbox/
32+
editors/jetbrains-plugin/build/
33+
editors/jetbrains-plugin/src/main/gen/
34+
35+
# Worktrees
36+
.worktrees/
37+
38+
# Temp files
39+
*.log
40+
.DS_Store

.pre-commit-config.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
repos:
2+
- repo: https://github.com/doublify/pre-commit-rust
3+
rev: v1.0
4+
hooks:
5+
- id: fmt

CHANGELOG.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Changelog
2+
3+
## 0.1.0
4+
5+
Initial release.
6+
7+
- Rust transpiler compiling `.hyper` templates to type-safe Python
8+
- Component model with props, slots, and `@html` decorator
9+
- Full Python control flow (`if`, `for`, `while`, `match`, `try`, `with`, `async`)
10+
- Streaming generator output
11+
- Content collections with Markdown, JSON, YAML, TOML support
12+
- JetBrains IDE plugin with Python code intelligence
13+
- TextMate syntax bundle

CLAUDE.md

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Project Overview
6+
7+
Hyper is a Python template framework. Write templates in `.hyper` syntax, compile to type-safe Python code.
8+
9+
Monorepo with 3 components:
10+
- **Rust transpiler** (`rust/transpiler/`) — Compiles `.hyper``.py`
11+
- **Python runtime** (`python/`) — Runtime helpers, CLI, optional content collections
12+
- **JetBrains plugin** (`editors/jetbrains-plugin/`) — IDE support via language injection
13+
14+
## Build and Test Commands
15+
16+
```bash
17+
# Transpiler
18+
just build transpiler # Release build
19+
just test transpiler # Run all tests (or: cd rust && cargo test)
20+
just compile <files-or-dirs> # Build + run in one step (debug, suppresses warnings)
21+
just test-accept # Regenerate all .expected.* files from current output
22+
just test-accept basic # Regenerate only files matching "basic"
23+
24+
# Plugin
25+
just build plugin # Build transpiler + bundle binary + build plugin
26+
just run plugin # Launch sandbox IDE
27+
just test plugin # Run plugin tests
28+
29+
# Python
30+
uv sync # Setup workspace
31+
pytest # Run Python tests
32+
33+
# Generate .py from .hyper (uses release binary)
34+
just generate <files>
35+
```
36+
37+
The `compile` recipe accepts directories — it walks them for `.hyper` files.
38+
39+
## Transpiler Architecture
40+
41+
Three-stage pipeline in `rust/transpiler/src/lib.rs`:
42+
43+
1. **Parser** (`parser/`) — `tokenizer.rs` lexes into tokens, `tree_builder.rs` builds AST. Line-based tokenizer with `after_structural` flag for comment detection, `is_control_flow()` requiring trailing `:` to distinguish Python keywords from content text.
44+
45+
2. **Transformer** (`transform/`) — Visitor-pattern plugins (`HelperDetectionPlugin`, `AsyncDetectionPlugin`, `SlotDetectionPlugin`) analyze the AST and produce `TransformMetadata`.
46+
47+
3. **Generator** (`generate/`) — `python.rs` combines consecutive text/expression/element nodes into single f-strings, emits control flow as separate statements. `injection_analyzer.rs` produces IDE language injection ranges. `output.rs` tracks source positions with UTF-16 mapping.
48+
49+
### Compile-time HTML validation (`html.rs` + `tree_builder.rs`)
50+
51+
The parser validates HTML at parse time:
52+
- **Void elements**`<br>`, `<img>`, etc. cannot have children or closing tags
53+
- **Duplicate attributes** — Same attribute name twice on one element
54+
- **Invalid nesting** — Block elements inside `<p>`, nested interactive elements (`<a>` inside `<button>`)
55+
56+
Validation uses an `element_stack` in `TreeBuilder` for parent context.
57+
58+
### Error system (`error.rs`)
59+
60+
`ParseError` renders with ANSI colors when stderr is a TTY:
61+
- `render()` — plain text (for piped output, tests)
62+
- `render_color()` — colored (red errors, blue line numbers, cyan related spans, yellow help)
63+
- `Display` impl outputs just the message (no redundant kind prefix)
64+
- Errors support `related_span` with custom `related_label` and multiline `help` text
65+
66+
## Testing
67+
68+
**Expected output tests** (`rust/transpiler/tests/expected_tests.rs`):
69+
- Test files: `rust/transpiler/tests/<category>/<name>.hyper`
70+
- Expected output: `<name>.expected.py` (compiled Python), `<name>.expected.json` (injection ranges/injections), `<name>.expected.err` (error messages)
71+
- Error tests live in `tests/errors/` and are auto-detected
72+
73+
**Expected output workflow:**
74+
```bash
75+
cargo test # Run tests, see failures
76+
cargo run --bin accept_expected # Regenerate all .expected.* files from current compiler output
77+
cargo run --bin accept_expected basic # Regenerate only files matching "basic"
78+
```
79+
80+
**IMPORTANT — expected output review:** `cargo run --bin accept_expected` blindly stamps the compiler's current output as correct. Never run it after a change without manually reviewing the diffs (`git diff`) to confirm the new output is actually what you expect. A bug in the compiler will silently become the blessed expected output otherwise. When changing parser or codegen logic, always spot-check at least the directly affected `.expected.py` and `.expected.json` files before considering the change done.
81+
82+
**CRITICAL — injection range validation:** Every Python injection range `source[start:end]` must extract to meaningful text from the source file (not mid-word garbage). After accepting expected output, verify that source positions in `.expected.json` files map to the correct source text. The test suite includes semantic validation (range text extraction checks) — if these fail, the ranges are wrong, do NOT blindly accept. Common mistakes: off-by-one in span calculations, stale expected files accepted without review, substring-matching tests that pass accidentally.
83+
84+
**Invariant tests** (`rust/transpiler/tests/invariants/`):
85+
- Property-based checks that run across ALL `.hyper` test files automatically
86+
- Each module validates one structural invariant (roundtrip, monotonicity, bounds, html_completeness, etc.)
87+
- New invariants go in their own module file under `invariants/`
88+
- Adding a new `.hyper` test file automatically gets invariant coverage with zero extra work
89+
90+
**Kitchen sink smoke test** (`tests/basic/kitchen_sink.hyper`):
91+
- Exercises every syntax construct in one file (elements, components, slots, control flow, decorators, attributes, expressions, comments)
92+
- After any injection change, open this file in JetBrains and visually verify highlighting
93+
- All 8 invariants run against it automatically
94+
95+
## CLI Modes (`main.rs`)
96+
97+
- `hyper generate <files|dirs>` — Compile to `.py` files, walks directories
98+
- `hyper generate --stdin` — Read from stdin, write to stdout
99+
- `hyper generate --json` — JSON output with source mappings
100+
- `hyper generate --daemon` — Length-prefixed JSON protocol for IDE integration
101+
102+
## Gotchas
103+
104+
- Transpiler binary must be rebuilt and re-bundled for plugin changes: `just build`
105+
- Plugin requires JDK 17+
106+
- Expected output files (`.expected.py`, `.expected.json`, `.expected.err`) are managed by `cargo run --bin accept_expected` — never edit them by hand
107+
- The tokenizer is line-based; multiline Python expressions (paren/bracket spanning lines) are a known limitation
108+
- `is_control_flow()` uses trailing `:` heuristic — content text that starts with a Python keyword and ends with `:` inside an element is an edge case
109+
110+
## Bug Fix Workflow
111+
112+
When the user reports a bug, visual issue, or incorrect behavior — especially phrases like "not highlighted", "looks wrong", "should be X but is Y", "missing", "broken" — invoke the `/red-green-fix` skill before making any code changes. Never fix a bug without first writing a failing test that captures the expected behavior.
113+

CONTRIBUTING.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Contributing
2+
3+
## Prerequisites
4+
5+
- [Rust toolchain](https://rustup.rs/)
6+
- [uv](https://docs.astral.sh/uv/)
7+
- [just](https://github.com/casey/just)
8+
- JDK 17+ (only for JetBrains plugin work)
9+
10+
## Setup
11+
12+
```bash
13+
git clone https://github.com/scriptogre/hyper.git
14+
cd hyper
15+
uv sync
16+
just build transpiler
17+
```
18+
19+
## Running Tests
20+
21+
```bash
22+
just test transpiler # Rust transpiler tests
23+
pytest # Python runtime tests
24+
just test plugin # JetBrains plugin tests
25+
```
26+
27+
## Linting
28+
29+
```bash
30+
cd rust && cargo fmt --check
31+
cd rust && cargo clippy -- -D warnings
32+
```

Justfile

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
# Default: build transpiler, bundle it in plugin, and build plugin
2+
default: build
3+
4+
# Build transpiler
5+
build-transpiler:
6+
cd {{justfile_directory()}}/rust && cargo build --release
7+
8+
# Build plugin (always rebuilds + bundles transpiler first)
9+
build-plugin: build-transpiler _bundle
10+
#!/usr/bin/env bash
11+
set -e
12+
ROOT="{{justfile_directory()}}"
13+
echo "Building JetBrains plugin..."
14+
cd "$ROOT/editors/jetbrains-plugin" && ./gradlew clean buildPlugin
15+
cp "$ROOT/editors/jetbrains-plugin/build/distributions"/*.zip "$ROOT/editors/jetbrains-plugin/hyper-plugin.zip"
16+
echo ""
17+
echo "✅ Plugin built!"
18+
echo "📦 Install: editors/jetbrains-plugin/hyper-plugin.zip"
19+
20+
# Build everything (or a specific target)
21+
build target="":
22+
#!/usr/bin/env bash
23+
set -e
24+
case "{{target}}" in
25+
transpiler) just build-transpiler ;;
26+
plugin) just build-plugin ;;
27+
"") just build-transpiler && just _bundle && just build-plugin ;;
28+
*) echo "Usage: just build [transpiler|plugin]"; exit 1 ;;
29+
esac
30+
31+
# Run transpiler or plugin
32+
run target *args:
33+
#!/usr/bin/env bash
34+
set -e
35+
ROOT="{{justfile_directory()}}"
36+
37+
case "{{target}}" in
38+
transpiler)
39+
"$ROOT/rust/target/release/hyper" {{args}}
40+
;;
41+
plugin)
42+
just build-plugin
43+
cd "$ROOT/editors/jetbrains-plugin" && ./gradlew runIde
44+
;;
45+
*)
46+
echo "Usage: just run [transpiler|plugin] [args...]"
47+
exit 1
48+
;;
49+
esac
50+
51+
# Test transpiler or plugin
52+
test target:
53+
#!/usr/bin/env bash
54+
set -e
55+
ROOT="{{justfile_directory()}}"
56+
57+
case "{{target}}" in
58+
transpiler)
59+
cd "$ROOT/rust" && cargo test
60+
;;
61+
plugin)
62+
echo "Building transpiler (debug)..."
63+
cd "$ROOT/rust" && cargo build -q
64+
cd "$ROOT/editors/jetbrains-plugin" && ./gradlew test
65+
;;
66+
*)
67+
echo "Usage: just test [transpiler|plugin]"
68+
exit 1
69+
;;
70+
esac
71+
72+
# Update all expected test files from current transpiler output
73+
test-accept *filter:
74+
cd {{justfile_directory()}}/rust/transpiler && cargo run --example accept_expected -- {{filter}}
75+
76+
# Show diff between expected and actual transpiler output
77+
test-diff *filter:
78+
#!/usr/bin/env bash
79+
cd "{{justfile_directory()}}/rust" && cargo test --test expected_tests 2>&1 | head -100
80+
81+
# Run expected tests only (faster than full test suite)
82+
test-expected:
83+
cd {{justfile_directory()}}/rust && cargo test --test expected_tests
84+
85+
# Bundle transpiler binary into plugin resources (internal helper)
86+
_bundle:
87+
#!/usr/bin/env bash
88+
set -e
89+
ROOT="{{justfile_directory()}}"
90+
91+
# Detect platform
92+
OS=$(uname -s | tr '[:upper:]' '[:lower:]')
93+
ARCH=$(uname -m)
94+
case "$OS" in
95+
darwin) OS_NAME="darwin" ;;
96+
linux) OS_NAME="linux" ;;
97+
*) OS_NAME="$OS" ;;
98+
esac
99+
case "$ARCH" in
100+
arm64|aarch64) ARCH_NAME="arm64" ;;
101+
x86_64|amd64) ARCH_NAME="x64" ;;
102+
*) ARCH_NAME="$ARCH" ;;
103+
esac
104+
105+
BINARY_NAME="hyper-${OS_NAME}-${ARCH_NAME}"
106+
SRC="$ROOT/rust/target/release/hyper"
107+
DEST="$ROOT/editors/jetbrains-plugin/src/main/resources/bin/${BINARY_NAME}"
108+
109+
mkdir -p "$(dirname "$DEST")"
110+
cp "$SRC" "$DEST"
111+
echo "✓ Bundled: $DEST"
112+
113+
# Generate Python from .hyper files
114+
generate *files:
115+
{{justfile_directory()}}/rust/target/release/hyper generate {{files}}
116+
117+
# Compile .hyper file(s) (build + run, for quick iteration)
118+
compile *files:
119+
cd {{justfile_directory()}}/rust && RUSTFLAGS="-Awarnings" cargo run -q -- generate {{files}}

LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026 hyper contributors
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

0 commit comments

Comments
 (0)