feat(writer): Added frame post processing. Frame post processing allo…

CHANGELOG.md

@@ -7,6 +7,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- **writer/api**: writer-side frame post-processor pipeline.  A new hook fires inside `ReplayWriter::RecordFrame` after timer validation and **before** `Serializer::FromNative` consumes the native `Frame`, so whatever the processor mutates is what gets serialised to the on-disk `.vtx`.  Three new public headers and three new facade methods materialise the feature:
+  - **`sdk/include/vtx/writer/core/vtx_frame_post_processor.h`** -- `IFramePostProcessor` interface (`Init` / `Process` / `Clear` / `PrintInfo`), `FramePostProcessorChain` composable container, `FramePostProcessorInitContext` (frame_accessor + total_frames + schema/format version) and `FramePostProcessContext` (per-frame: global_frame_index, schema_version, frame_accessor) carriers.  Chain execution: `Init`/`Process`/`PrintInfo` in registration order; `Clear` in reverse (destructor-like teardown); last writer wins on shared property mutations.
+  - **`sdk/include/vtx/writer/core/vtx_frame_mutation_view.h`** -- write-side mirror of `EntityView` / `FrameAccessor`.  `EntityMutator` (non-owning wrapper over `PropertyContainer*` with `Get<T>` + `Set<T>` + `GetMutableView` + `GetMutableArray<T>`); `BucketMutator` (mutable iteration + structural mutation: `AddEntity` / `RemoveEntity` / `RemoveIf` / `Clear`); `FrameMutationView` (entry point the processor receives -- wraps `Frame&` + borrows a `FrameAccessor*` so processors can resolve schema names without coupling to reader internals).  Hot-path cost is identical to `EntityView::Get` -- single non-owning pointer indirection, fully inlinable.
+  - **`IVtxWriterFacade::SetPostProcessor(std::shared_ptr<IFramePostProcessor>)` / `GetPostProcessor()` / `ClearPostProcessor()`** -- registration API on the writer facade, forwarded to both `FlatBuffersWriterFacadeImpl` and `ProtobuffWriterFacadeImpl`.  `Init()` runs synchronously inside `SetPostProcessor` BEFORE the new processor becomes visible to any `RecordFrame()` -- this is the right place to resolve every `PropertyKey<T>` upfront since the schema is constant for the recording session.  The writer is single-threaded by design (`RecordFrame` called sequentially from the capture loop) so no mutex is needed on `post_processor_`.  The destructor invokes `Clear()` on whatever is currently registered.  `SetPostProcessor` does NOT call `Clear` on the previously-registered processor; the caller keeps the `shared_ptr` and calls `Clear` explicitly if they need outgoing teardown -- use `ClearPostProcessor()` for the common case of explicit pre-destruction reset
+- **scripts/codegen**: `scripts/vtx_codegen.py` extended to emit, per schema struct, in addition to the existing `XView` read-only wrapper:
+  - **`XMutator`** -- write-capable wrapper around `EntityMutator`.  All `Get*` methods identical to the View; adds `Set*(value)` for scalars and `GetMutable*()` returning `std::span<T>` (arrays) or `EntityMutator` (nested structs).  `PropertyKey<T>` resolution stays cached in `static` locals per-method on first use, so registering a processor doesn't trigger a one-time hash sweep.
+  - **`ForEachX(BucketMutator&, FrameAccessor&, Fn)`** -- template helper that filters a bucket by `entity_type_id` (matching `EntityType::X`) and invokes the lambda with an `XMutator&`.  Read-only counterpart `ForEachXView(const Bucket&, FrameAccessor&, Fn)` paralleled.  Result: processors operate on strongly-typed views (`p.SetHealth(...)`) with zero hardcoded schema strings, zero `PropertyKey<T>` members on the processor, and no manual `entity_type_id` gating -- if the schema changes, regenerating the header makes new properties available; if a property is renamed or removed, code fails to compile early instead of silently mismatching at runtime
+- **samples**: `vtx_sample_post_process_write` target (`samples/post_process_write.cpp`) -- minimum end-to-end demo of the writer-side post-processor.  Builds synthetic frames with intentionally out-of-range Health values via the codegen-generated `PlayerMutator`, registers a `PlayerHealthProcessor` (clamp `[0, 100]`, derive `IsAlive=false` when `Health<=0`, cross-frame low-health counter, lifecycle hooks), records 30 frames, then re-opens the `.vtx` with `OpenReplayFile` and uses `ForEachPlayerView` (also codegen-generated) to print the persisted values -- proving the on-disk bytes contain the post-processed state, not the raw input
+- **samples**: `samples/advance_write.cpp` extended to register an `ArenaConsistencyProcessor` on each of the three pipelines (JSON / Protobuf / FlatBuffers source).  Same processor instance per pipeline using `VTX::ArenaSchema::ForEachPlayer` -- demonstrates that frame post-processing is **orthogonal to the source format**: the same logic runs on the canonical `VTX::Frame` regardless of whether it came from JSON, Protobuf, or FlatBuffers
+- **tests**: `tests/writer/test_frame_post_processor.cpp` with 10 cases:
+  - `WriterPostProcessor_MutationViewUnit.SetThenGetRoundTrips` and `WriterPostProcessor_ChainUnit.OrderAndRemove` -- standalone unit smokes for the mutation view + chain primitives.
+  - `WriterPostProcessorTest.NoProcessorBaselineUnchanged` -- behaviour identical when no processor is registered.
+  - `WriterPostProcessorTest.DoubleHealthIsPersistedToDisk` -- `Init` resolves the Health key, processor doubles values pre-serialise, readback confirms 200.0f on disk.
+  - `WriterPostProcessorTest.ChainLastWriterWinsOnDisk` and `.ChainRemoveDropsAndOtherStillFires` -- chain ordering + `Remove` semantics from disk.
+  - `WriterPostProcessorTest.GhostInjectorEntityIsOnDisk` -- `BucketMutator::AddEntity` injects a synthetic entity with `entity_type_id` set explicitly, readback confirms it persisted.
+  - `WriterPostProcessorTest.TeamTwoFilterDropsEntitiesFromDisk` -- `BucketMutator::RemoveIf` filters entities pre-serialise.
+  - `WriterPostProcessorTest.GlobalFrameIndexIsMonotonic` -- `ctx.global_frame_index` monotonically increments across `RecordFrame` calls.
+  - `WriterPostProcessorTest.ClearPostProcessorCallsClearAndUnregisters` -- explicit teardown invokes `Clear` and subsequent `RecordFrame` calls bypass the processor entirely
+- **docs**: new **`docs/POST_PROCESSING.md`** -- dedicated reference covering the feature pipeline diagram, lifecycle (`Init` synchronous before first `Process`, `Clear` on destructor / explicit teardown), threading model (single-threaded writer, no mutex needed), two ways to write a processor (generic with raw `PropertyKey<T>` vs codegen-driven strongly-typed `XMutator` / `ForEachX`), patterns for cross-frame state / chains / replay-level metadata / schema-version branching / structural mutation, error handling (exceptions swallowed at hook boundary), performance characteristics (zero overhead when unused; same hot path as `EntityView` when active), gotchas (the FlatBuffers serialiser drops entities with `entity_type_id < 0`, the writer renames bucket[0] to "data" / bucket[1] to "bone_data" / drops bucket[2+] silently, `type_ranges` invalidated after `RemoveIf` but rebuilt by the serializer), and pointers to the runnable demos
+- **docs**: **`docs/SDK_API.md`** new "Frame Post-Processor" section between "Writing Replays" and "Diffing Frames" -- API cheat-sheet covering processor implementation, registration on the writer, chain composition, the strongly-typed codegen alternative, and the mutation view API surface.  Links to `POST_PROCESSING.md` for the full reference
+- **docs**: **`docs/SAMPLES.md`** updated for the two new sample targets (`vtx_sample_post_process_write` and the post-processor addition in `vtx_sample_advance_write`) plus the extended `arena_generated.h` codegen output (now includes `*Mutator` classes + `ForEachX` helpers).  "What each sample teaches" table gains four new rows covering the post-processor and codegen-driven typed accessor patterns
+- **docs**: **`README.md`** "Write a replay" snippet gains a sub-section showing a minimal `IFramePostProcessor` implementation and `writer->SetPostProcessor` registration.  In-tree docs list updated to include `POST_PROCESSING.md`
+
+- **scripts**: `scripts/check_clang_format.py` (+ `.sh` and `.bat` wrappers) -- local mirror of the CI clang-format diff-gate.  Validates only the lines you've modified vs a base ref (default `origin/main`), matching the CI's exclusion list (`thirdparty/`, `*generated/`, `arena_generated.h`, `portable-file-dialogs.h`) and scope (`.cpp` / `.cc` / `.h` / `.hpp`).  Three modes via `--fix` / `--base <ref>`: read-only check, apply fixes in place, or check against a different ref.  Auto-detects `clang-format-diff.py` under `Program Files\LLVM\share\clang\` on Windows when it's not on `PATH`.  Cross-platform wrappers delegate to the Python implementation.  Exit codes match CI semantics: 0 clean, 1 violations, 2 tooling missing
+- **scripts**: `scripts/git-hooks/pre-push` -- versioned pre-push hook that runs `check_clang_format.py` and aborts the push on violation.  Opt-in per clone via `git config core.hooksPath scripts/git-hooks` (built-in to git ≥ 2.9, no Husky / pre-commit dependency).  Bypass for a one-off push with `git push --no-verify`
+- **docs**: **`docs/BUILD.md`** "Formatting gate" subsection extended with the local helper script usage (read-only check, `--fix`, `--base` arg) and the pre-push hook activation one-liner.  Same coverage in **`CONTRIBUTING.md`** under "Validate formatting before pushing" + "Pre-push hook" so contributors landing on either doc find the workflow
+
+### Changed
+
+- **sdk/include layout**: `vtx_frame_accessor.h` moved from `sdk/include/vtx/reader/core/` to `sdk/include/vtx/common/`.  The header is fundamentally a schema utility (`FrameAccessor` resolves names against `PropertyAddressCache`, `EntityView` is a generic read-only wrapper over `PropertyContainer`); pre-move it lived under `reader/` for historical reasons, which forced the new writer-side post-processor headers (`vtx_frame_mutation_view.h`, `vtx_frame_post_processor.h`) to either re-implement the schema-name resolution or include from `reader/` (creating a writer→reader cross-dependency).  Post-move `writer/core/` and `reader/core/` share `common/vtx_frame_accessor.h` directly and have zero include-path edges between each other.  11 include sites updated to the new path (benchmarks, tests, samples, the codegen script, `vtx_reader.h`, the two new writer-side headers, and `writer.h`); the codegen template emits the new path so regenerating `arena_generated.h` produces a correct include
+
 ## [0.1.1] - 2026-04-28
 
 ### Added

CONTRIBUTING.md

@@ -33,6 +33,32 @@ scripts\build_sdk.bat
 
 A `.clang-format` file is provided for automatic formatting of new code. Do not mass-reformat existing files.
 
+### Validate formatting before pushing
+
+CI runs a clang-format diff-gate on every PR: only the **lines** you've modified vs the base branch get checked (pre-existing files predate the style file and are exempt). To catch violations locally before pushing, use the helper script:
+
+```bash
+# Check (read-only) -- same logic as CI
+python scripts/check_clang_format.py
+
+# Auto-fix in place
+python scripts/check_clang_format.py --fix
+```
+
+There are `.sh` and `.bat` wrappers if you prefer them (`scripts/check_clang_format.sh` / `.bat`). The script auto-detects `clang-format-diff.py` on Windows under `Program Files\LLVM\share\clang\` if it's not on `PATH`.
+
+### Pre-push hook (optional, opt-in)
+
+To run the format check automatically before every `git push`, activate the versioned hook directory once in your clone:
+
+```bash
+git config core.hooksPath scripts/git-hooks
+```
+
+That activates `scripts/git-hooks/pre-push`, which calls the format check and aborts the push if it fails. Bypass for a one-off push with `git push --no-verify`. Deactivate with `git config --unset core.hooksPath`.
+
+No external dependencies -- `core.hooksPath` is built into git ≥ 2.9.
+
 ### Naming Conventions
 
 | Element | Convention | Example |
@@ -88,9 +114,10 @@ These use C++20 `std::format` syntax (`{}`), **not** printf-style (`%s`, `%d`).
 
 1. Create a feature branch from `main`
 2. Make focused, incremental changes
-3. Ensure the project builds without warnings
-4. Test with sample replay files when possible
-5. Write clear commit messages describing _why_, not just _what_
+3. Run `python scripts/check_clang_format.py` before pushing (or set up the pre-push hook -- see [Validate formatting before pushing](#validate-formatting-before-pushing))
+4. Ensure the project builds without warnings
+5. Test with sample replay files when possible
+6. Write clear commit messages describing _why_, not just _what_
 
 ## Project Structure
 

README.md

@@ -53,6 +53,32 @@ writer->Flush();
 writer->Stop();
 ```
 
+#### Optional: post-process every frame before it lands on disk
+
+`writer->SetPostProcessor(...)` installs a hook that runs after timer validation and before serialization. Whatever it mutates is what gets persisted. Use it to sanitize values, derive consistency state, filter or inject entities, accumulate cross-frame stats, or branch by schema version.
+
+```cpp
+#include "vtx/writer/core/vtx_frame_post_processor.h"
+
+class HealthClamp : public VTX::IFramePostProcessor {
+public:
+    void Init(const VTX::FramePostProcessorInitContext& ctx) override {
+        health_key_ = ctx.frame_accessor->Get<float>("Player", "Health");
+    }
+    void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext&) override {
+        for (auto entity : view.GetBucket("entity")) {
+            if (entity.Get(health_key_) < 0.0f) entity.Set(health_key_, 0.0f);
+        }
+    }
+private:
+    VTX::PropertyKey<float> health_key_ {-1};
+};
+
+writer->SetPostProcessor(std::make_shared<HealthClamp>());
+```
+
+For string-free, schema-driven processors with type-safety in compile time, use `scripts/vtx_codegen.py` to generate per-struct mutators and iteration helpers (`PlayerMutator`, `ForEachPlayer`, etc.) -- full reference in [`docs/POST_PROCESSING.md`](docs/POST_PROCESSING.md), runnable end-to-end demo in [`samples/post_process_write.cpp`](samples/post_process_write.cpp).
+
 ### Read a replay
 
 ```cpp
@@ -134,7 +160,7 @@ Dependencies are pulled via `vcpkg.json` on Windows or system packages on Linux.
 - **[Performance](https://github.com/ZenosInteractive/VTX/wiki/Performance)** — full numbers, methodology, and sizing guidance.
 - **[Use Cases](https://github.com/ZenosInteractive/VTX/wiki/Use-Cases)** — what people build on top of VTX.
 
-In-tree reference: [`docs/`](docs/) includes `ARCHITECTURE.md`, `BUILD.md`, `FILE_FORMAT.md`, `PERFORMANCE.md`, `SAMPLES.md`, `SDK_API.md`.
+In-tree reference: [`docs/`](docs/) includes `ARCHITECTURE.md`, `BUILD.md`, `FILE_FORMAT.md`, `PERFORMANCE.md`, `POST_PROCESSING.md`, `SAMPLES.md`, `SDK_API.md`.
 
 ## License
 

benchmarks/bench_accessor.cpp

@@ -23,7 +23,7 @@
 //                                  resident
 
 #include "vtx/reader/core/vtx_reader_facade.h"
-#include "vtx/reader/core/vtx_frame_accessor.h"
+#include "vtx/common/vtx_frame_accessor.h"
 #include "vtx/common/vtx_logger.h"
 #include "vtx/common/vtx_types.h"
 

benchmarks/bench_accessor_key_resolution.cpp

@@ -9,7 +9,7 @@
 // directly.  This file probes the public-facing API instead.
 
 #include "vtx/reader/core/vtx_reader_facade.h"
-#include "vtx/reader/core/vtx_frame_accessor.h"
+#include "vtx/common/vtx_frame_accessor.h"
 #include "vtx/common/vtx_logger.h"
 
 #include <benchmark/benchmark.h>

benchmarks/bench_cs.cpp

@@ -23,7 +23,7 @@
 #include "bench_utils.h"
 
 #include "vtx/reader/core/vtx_reader_facade.h"
-#include "vtx/reader/core/vtx_frame_accessor.h"
+#include "vtx/common/vtx_frame_accessor.h"
 #include "vtx/differ/core/vtx_differ_facade.h"
 #include "vtx/common/vtx_logger.h"
 #include "vtx/common/vtx_types.h"

benchmarks/bench_rl.cpp

@@ -16,7 +16,7 @@
 #include "bench_utils.h"
 
 #include "vtx/reader/core/vtx_reader_facade.h"
-#include "vtx/reader/core/vtx_frame_accessor.h"
+#include "vtx/common/vtx_frame_accessor.h"
 #include "vtx/differ/core/vtx_differ_facade.h"
 #include "vtx/common/vtx_logger.h"
 #include "vtx/common/vtx_types.h"

benchmarks/bench_scenarios.cpp

@@ -23,7 +23,7 @@
 #include "bench_utils.h"
 
 #include "vtx/reader/core/vtx_reader_facade.h"
-#include "vtx/reader/core/vtx_frame_accessor.h"
+#include "vtx/common/vtx_frame_accessor.h"
 #include "vtx/differ/core/vtx_differ_facade.h"
 #include "vtx/common/vtx_logger.h"
 #include "vtx/common/vtx_types.h"

docs/BUILD.md

@@ -10,6 +10,39 @@ Every push to `main` / `master` and every pull request triggers the CI workflow
 
 A dedicated `clang-format` job checks every C++ file **added or modified** by the PR / push against `.clang-format`. Pre-existing files are not checked -- about 90% of the codebase predates the style file, so a strict full-repo check would be permanently red. The gate catches *new* formatting regressions without forcing a one-time blame-destroying style sweep.
 
+#### Validate locally before pushing
+
+The same diff-gate logic is shipped as a local helper so you can catch violations before the CI does:
+
+```bash
+# Check (read-only) -- exits 1 if there are violations on changed lines
+python scripts/check_clang_format.py
+
+# Auto-fix the offending lines in place
+python scripts/check_clang_format.py --fix
+
+# Different base ref (default is origin/main)
+python scripts/check_clang_format.py --base HEAD~1
+```
+
+Cross-platform wrappers:
+- `scripts/check_clang_format.sh` -- Linux / macOS / WSL / Git Bash
+- `scripts/check_clang_format.bat` -- Windows `cmd`
+
+Exit codes: `0` clean, `1` violations, `2` tooling missing. The script auto-detects `clang-format-diff.py` under `Program Files\LLVM\share\clang\` on Windows when it's not on `PATH`.
+
+#### Pre-push hook (opt-in, versioned in the repo)
+
+A versioned pre-push hook lives at `scripts/git-hooks/pre-push` and runs the same check before every `git push`. Activate it in your clone once:
+
+```bash
+git config core.hooksPath scripts/git-hooks
+```
+
+No external dependency (Husky, pre-commit, etc.) -- `core.hooksPath` is built into git ≥ 2.9. To bypass for a one-off push: `git push --no-verify`. To deactivate: `git config --unset core.hooksPath`.
+
+#### Sweeping the whole tree
+
 If you want to apply the style to the whole tree in one commit (consider recording that SHA in `.git-blame-ignore-revs` afterwards):
 
 ```bash