diff --git a/CHANGELOG.md b/CHANGELOG.md index 81ba678..8643ca0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **writer/api**: writer-side frame post-processor pipeline. A new hook fires inside `ReplayWriter::RecordFrame` after timer validation and **before** `Serializer::FromNative` consumes the native `Frame`, so whatever the processor mutates is what gets serialised to the on-disk `.vtx`. Three new public headers and three new facade methods materialise the feature: + - **`sdk/include/vtx/writer/core/vtx_frame_post_processor.h`** -- `IFramePostProcessor` interface (`Init` / `Process` / `Clear` / `PrintInfo`), `FramePostProcessorChain` composable container, `FramePostProcessorInitContext` (frame_accessor + total_frames + schema/format version) and `FramePostProcessContext` (per-frame: global_frame_index, schema_version, frame_accessor) carriers. Chain execution: `Init`/`Process`/`PrintInfo` in registration order; `Clear` in reverse (destructor-like teardown); last writer wins on shared property mutations. + - **`sdk/include/vtx/writer/core/vtx_frame_mutation_view.h`** -- write-side mirror of `EntityView` / `FrameAccessor`. `EntityMutator` (non-owning wrapper over `PropertyContainer*` with `Get` + `Set` + `GetMutableView` + `GetMutableArray`); `BucketMutator` (mutable iteration + structural mutation: `AddEntity` / `RemoveEntity` / `RemoveIf` / `Clear`); `FrameMutationView` (entry point the processor receives -- wraps `Frame&` + borrows a `FrameAccessor*` so processors can resolve schema names without coupling to reader internals). Hot-path cost is identical to `EntityView::Get` -- single non-owning pointer indirection, fully inlinable. + - **`IVtxWriterFacade::SetPostProcessor(std::shared_ptr)` / `GetPostProcessor()` / `ClearPostProcessor()`** -- registration API on the writer facade, forwarded to both `FlatBuffersWriterFacadeImpl` and `ProtobuffWriterFacadeImpl`. `Init()` runs synchronously inside `SetPostProcessor` BEFORE the new processor becomes visible to any `RecordFrame()` -- this is the right place to resolve every `PropertyKey` upfront since the schema is constant for the recording session. The writer is single-threaded by design (`RecordFrame` called sequentially from the capture loop) so no mutex is needed on `post_processor_`. The destructor invokes `Clear()` on whatever is currently registered. `SetPostProcessor` does NOT call `Clear` on the previously-registered processor; the caller keeps the `shared_ptr` and calls `Clear` explicitly if they need outgoing teardown -- use `ClearPostProcessor()` for the common case of explicit pre-destruction reset +- **scripts/codegen**: `scripts/vtx_codegen.py` extended to emit, per schema struct, in addition to the existing `XView` read-only wrapper: + - **`XMutator`** -- write-capable wrapper around `EntityMutator`. All `Get*` methods identical to the View; adds `Set*(value)` for scalars and `GetMutable*()` returning `std::span` (arrays) or `EntityMutator` (nested structs). `PropertyKey` resolution stays cached in `static` locals per-method on first use, so registering a processor doesn't trigger a one-time hash sweep. + - **`ForEachX(BucketMutator&, FrameAccessor&, Fn)`** -- template helper that filters a bucket by `entity_type_id` (matching `EntityType::X`) and invokes the lambda with an `XMutator&`. Read-only counterpart `ForEachXView(const Bucket&, FrameAccessor&, Fn)` paralleled. Result: processors operate on strongly-typed views (`p.SetHealth(...)`) with zero hardcoded schema strings, zero `PropertyKey` members on the processor, and no manual `entity_type_id` gating -- if the schema changes, regenerating the header makes new properties available; if a property is renamed or removed, code fails to compile early instead of silently mismatching at runtime +- **samples**: `vtx_sample_post_process_write` target (`samples/post_process_write.cpp`) -- minimum end-to-end demo of the writer-side post-processor. Builds synthetic frames with intentionally out-of-range Health values via the codegen-generated `PlayerMutator`, registers a `PlayerHealthProcessor` (clamp `[0, 100]`, derive `IsAlive=false` when `Health<=0`, cross-frame low-health counter, lifecycle hooks), records 30 frames, then re-opens the `.vtx` with `OpenReplayFile` and uses `ForEachPlayerView` (also codegen-generated) to print the persisted values -- proving the on-disk bytes contain the post-processed state, not the raw input +- **samples**: `samples/advance_write.cpp` extended to register an `ArenaConsistencyProcessor` on each of the three pipelines (JSON / Protobuf / FlatBuffers source). Same processor instance per pipeline using `VTX::ArenaSchema::ForEachPlayer` -- demonstrates that frame post-processing is **orthogonal to the source format**: the same logic runs on the canonical `VTX::Frame` regardless of whether it came from JSON, Protobuf, or FlatBuffers +- **tests**: `tests/writer/test_frame_post_processor.cpp` with 10 cases: + - `WriterPostProcessor_MutationViewUnit.SetThenGetRoundTrips` and `WriterPostProcessor_ChainUnit.OrderAndRemove` -- standalone unit smokes for the mutation view + chain primitives. + - `WriterPostProcessorTest.NoProcessorBaselineUnchanged` -- behaviour identical when no processor is registered. + - `WriterPostProcessorTest.DoubleHealthIsPersistedToDisk` -- `Init` resolves the Health key, processor doubles values pre-serialise, readback confirms 200.0f on disk. + - `WriterPostProcessorTest.ChainLastWriterWinsOnDisk` and `.ChainRemoveDropsAndOtherStillFires` -- chain ordering + `Remove` semantics from disk. + - `WriterPostProcessorTest.GhostInjectorEntityIsOnDisk` -- `BucketMutator::AddEntity` injects a synthetic entity with `entity_type_id` set explicitly, readback confirms it persisted. + - `WriterPostProcessorTest.TeamTwoFilterDropsEntitiesFromDisk` -- `BucketMutator::RemoveIf` filters entities pre-serialise. + - `WriterPostProcessorTest.GlobalFrameIndexIsMonotonic` -- `ctx.global_frame_index` monotonically increments across `RecordFrame` calls. + - `WriterPostProcessorTest.ClearPostProcessorCallsClearAndUnregisters` -- explicit teardown invokes `Clear` and subsequent `RecordFrame` calls bypass the processor entirely +- **docs**: new **`docs/POST_PROCESSING.md`** -- dedicated reference covering the feature pipeline diagram, lifecycle (`Init` synchronous before first `Process`, `Clear` on destructor / explicit teardown), threading model (single-threaded writer, no mutex needed), two ways to write a processor (generic with raw `PropertyKey` vs codegen-driven strongly-typed `XMutator` / `ForEachX`), patterns for cross-frame state / chains / replay-level metadata / schema-version branching / structural mutation, error handling (exceptions swallowed at hook boundary), performance characteristics (zero overhead when unused; same hot path as `EntityView` when active), gotchas (the FlatBuffers serialiser drops entities with `entity_type_id < 0`, the writer renames bucket[0] to "data" / bucket[1] to "bone_data" / drops bucket[2+] silently, `type_ranges` invalidated after `RemoveIf` but rebuilt by the serializer), and pointers to the runnable demos +- **docs**: **`docs/SDK_API.md`** new "Frame Post-Processor" section between "Writing Replays" and "Diffing Frames" -- API cheat-sheet covering processor implementation, registration on the writer, chain composition, the strongly-typed codegen alternative, and the mutation view API surface. Links to `POST_PROCESSING.md` for the full reference +- **docs**: **`docs/SAMPLES.md`** updated for the two new sample targets (`vtx_sample_post_process_write` and the post-processor addition in `vtx_sample_advance_write`) plus the extended `arena_generated.h` codegen output (now includes `*Mutator` classes + `ForEachX` helpers). "What each sample teaches" table gains four new rows covering the post-processor and codegen-driven typed accessor patterns +- **docs**: **`README.md`** "Write a replay" snippet gains a sub-section showing a minimal `IFramePostProcessor` implementation and `writer->SetPostProcessor` registration. In-tree docs list updated to include `POST_PROCESSING.md` + +- **scripts**: `scripts/check_clang_format.py` (+ `.sh` and `.bat` wrappers) -- local mirror of the CI clang-format diff-gate. Validates only the lines you've modified vs a base ref (default `origin/main`), matching the CI's exclusion list (`thirdparty/`, `*generated/`, `arena_generated.h`, `portable-file-dialogs.h`) and scope (`.cpp` / `.cc` / `.h` / `.hpp`). Three modes via `--fix` / `--base `: read-only check, apply fixes in place, or check against a different ref. Auto-detects `clang-format-diff.py` under `Program Files\LLVM\share\clang\` on Windows when it's not on `PATH`. Cross-platform wrappers delegate to the Python implementation. Exit codes match CI semantics: 0 clean, 1 violations, 2 tooling missing +- **scripts**: `scripts/git-hooks/pre-push` -- versioned pre-push hook that runs `check_clang_format.py` and aborts the push on violation. Opt-in per clone via `git config core.hooksPath scripts/git-hooks` (built-in to git ≥ 2.9, no Husky / pre-commit dependency). Bypass for a one-off push with `git push --no-verify` +- **docs**: **`docs/BUILD.md`** "Formatting gate" subsection extended with the local helper script usage (read-only check, `--fix`, `--base` arg) and the pre-push hook activation one-liner. Same coverage in **`CONTRIBUTING.md`** under "Validate formatting before pushing" + "Pre-push hook" so contributors landing on either doc find the workflow + +### Changed + +- **sdk/include layout**: `vtx_frame_accessor.h` moved from `sdk/include/vtx/reader/core/` to `sdk/include/vtx/common/`. The header is fundamentally a schema utility (`FrameAccessor` resolves names against `PropertyAddressCache`, `EntityView` is a generic read-only wrapper over `PropertyContainer`); pre-move it lived under `reader/` for historical reasons, which forced the new writer-side post-processor headers (`vtx_frame_mutation_view.h`, `vtx_frame_post_processor.h`) to either re-implement the schema-name resolution or include from `reader/` (creating a writer→reader cross-dependency). Post-move `writer/core/` and `reader/core/` share `common/vtx_frame_accessor.h` directly and have zero include-path edges between each other. 11 include sites updated to the new path (benchmarks, tests, samples, the codegen script, `vtx_reader.h`, the two new writer-side headers, and `writer.h`); the codegen template emits the new path so regenerating `arena_generated.h` produces a correct include + ## [0.1.1] - 2026-04-28 ### Added diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0121f8f..dd456c0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -33,6 +33,32 @@ scripts\build_sdk.bat A `.clang-format` file is provided for automatic formatting of new code. Do not mass-reformat existing files. +### Validate formatting before pushing + +CI runs a clang-format diff-gate on every PR: only the **lines** you've modified vs the base branch get checked (pre-existing files predate the style file and are exempt). To catch violations locally before pushing, use the helper script: + +```bash +# Check (read-only) -- same logic as CI +python scripts/check_clang_format.py + +# Auto-fix in place +python scripts/check_clang_format.py --fix +``` + +There are `.sh` and `.bat` wrappers if you prefer them (`scripts/check_clang_format.sh` / `.bat`). The script auto-detects `clang-format-diff.py` on Windows under `Program Files\LLVM\share\clang\` if it's not on `PATH`. + +### Pre-push hook (optional, opt-in) + +To run the format check automatically before every `git push`, activate the versioned hook directory once in your clone: + +```bash +git config core.hooksPath scripts/git-hooks +``` + +That activates `scripts/git-hooks/pre-push`, which calls the format check and aborts the push if it fails. Bypass for a one-off push with `git push --no-verify`. Deactivate with `git config --unset core.hooksPath`. + +No external dependencies -- `core.hooksPath` is built into git ≥ 2.9. + ### Naming Conventions | Element | Convention | Example | @@ -88,9 +114,10 @@ These use C++20 `std::format` syntax (`{}`), **not** printf-style (`%s`, `%d`). 1. Create a feature branch from `main` 2. Make focused, incremental changes -3. Ensure the project builds without warnings -4. Test with sample replay files when possible -5. Write clear commit messages describing _why_, not just _what_ +3. Run `python scripts/check_clang_format.py` before pushing (or set up the pre-push hook -- see [Validate formatting before pushing](#validate-formatting-before-pushing)) +4. Ensure the project builds without warnings +5. Test with sample replay files when possible +6. Write clear commit messages describing _why_, not just _what_ ## Project Structure diff --git a/README.md b/README.md index e96ae50..e476321 100644 --- a/README.md +++ b/README.md @@ -53,6 +53,32 @@ writer->Flush(); writer->Stop(); ``` +#### Optional: post-process every frame before it lands on disk + +`writer->SetPostProcessor(...)` installs a hook that runs after timer validation and before serialization. Whatever it mutates is what gets persisted. Use it to sanitize values, derive consistency state, filter or inject entities, accumulate cross-frame stats, or branch by schema version. + +```cpp +#include "vtx/writer/core/vtx_frame_post_processor.h" + +class HealthClamp : public VTX::IFramePostProcessor { +public: + void Init(const VTX::FramePostProcessorInitContext& ctx) override { + health_key_ = ctx.frame_accessor->Get("Player", "Health"); + } + void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext&) override { + for (auto entity : view.GetBucket("entity")) { + if (entity.Get(health_key_) < 0.0f) entity.Set(health_key_, 0.0f); + } + } +private: + VTX::PropertyKey health_key_ {-1}; +}; + +writer->SetPostProcessor(std::make_shared()); +``` + +For string-free, schema-driven processors with type-safety in compile time, use `scripts/vtx_codegen.py` to generate per-struct mutators and iteration helpers (`PlayerMutator`, `ForEachPlayer`, etc.) -- full reference in [`docs/POST_PROCESSING.md`](docs/POST_PROCESSING.md), runnable end-to-end demo in [`samples/post_process_write.cpp`](samples/post_process_write.cpp). + ### Read a replay ```cpp @@ -134,7 +160,7 @@ Dependencies are pulled via `vcpkg.json` on Windows or system packages on Linux. - **[Performance](https://github.com/ZenosInteractive/VTX/wiki/Performance)** — full numbers, methodology, and sizing guidance. - **[Use Cases](https://github.com/ZenosInteractive/VTX/wiki/Use-Cases)** — what people build on top of VTX. -In-tree reference: [`docs/`](docs/) includes `ARCHITECTURE.md`, `BUILD.md`, `FILE_FORMAT.md`, `PERFORMANCE.md`, `SAMPLES.md`, `SDK_API.md`. +In-tree reference: [`docs/`](docs/) includes `ARCHITECTURE.md`, `BUILD.md`, `FILE_FORMAT.md`, `PERFORMANCE.md`, `POST_PROCESSING.md`, `SAMPLES.md`, `SDK_API.md`. ## License diff --git a/benchmarks/bench_accessor.cpp b/benchmarks/bench_accessor.cpp index b6e2a59..5324e2a 100644 --- a/benchmarks/bench_accessor.cpp +++ b/benchmarks/bench_accessor.cpp @@ -23,7 +23,7 @@ // resident #include "vtx/reader/core/vtx_reader_facade.h" -#include "vtx/reader/core/vtx_frame_accessor.h" +#include "vtx/common/vtx_frame_accessor.h" #include "vtx/common/vtx_logger.h" #include "vtx/common/vtx_types.h" diff --git a/benchmarks/bench_accessor_key_resolution.cpp b/benchmarks/bench_accessor_key_resolution.cpp index fa56987..86ea366 100644 --- a/benchmarks/bench_accessor_key_resolution.cpp +++ b/benchmarks/bench_accessor_key_resolution.cpp @@ -9,7 +9,7 @@ // directly. This file probes the public-facing API instead. #include "vtx/reader/core/vtx_reader_facade.h" -#include "vtx/reader/core/vtx_frame_accessor.h" +#include "vtx/common/vtx_frame_accessor.h" #include "vtx/common/vtx_logger.h" #include diff --git a/benchmarks/bench_cs.cpp b/benchmarks/bench_cs.cpp index 46a8644..ca7f69f 100644 --- a/benchmarks/bench_cs.cpp +++ b/benchmarks/bench_cs.cpp @@ -23,7 +23,7 @@ #include "bench_utils.h" #include "vtx/reader/core/vtx_reader_facade.h" -#include "vtx/reader/core/vtx_frame_accessor.h" +#include "vtx/common/vtx_frame_accessor.h" #include "vtx/differ/core/vtx_differ_facade.h" #include "vtx/common/vtx_logger.h" #include "vtx/common/vtx_types.h" diff --git a/benchmarks/bench_rl.cpp b/benchmarks/bench_rl.cpp index 466a011..e382695 100644 --- a/benchmarks/bench_rl.cpp +++ b/benchmarks/bench_rl.cpp @@ -16,7 +16,7 @@ #include "bench_utils.h" #include "vtx/reader/core/vtx_reader_facade.h" -#include "vtx/reader/core/vtx_frame_accessor.h" +#include "vtx/common/vtx_frame_accessor.h" #include "vtx/differ/core/vtx_differ_facade.h" #include "vtx/common/vtx_logger.h" #include "vtx/common/vtx_types.h" diff --git a/benchmarks/bench_scenarios.cpp b/benchmarks/bench_scenarios.cpp index 648a0f4..eb5da40 100644 --- a/benchmarks/bench_scenarios.cpp +++ b/benchmarks/bench_scenarios.cpp @@ -23,7 +23,7 @@ #include "bench_utils.h" #include "vtx/reader/core/vtx_reader_facade.h" -#include "vtx/reader/core/vtx_frame_accessor.h" +#include "vtx/common/vtx_frame_accessor.h" #include "vtx/differ/core/vtx_differ_facade.h" #include "vtx/common/vtx_logger.h" #include "vtx/common/vtx_types.h" diff --git a/docs/BUILD.md b/docs/BUILD.md index 4d08257..509b6ca 100644 --- a/docs/BUILD.md +++ b/docs/BUILD.md @@ -10,6 +10,39 @@ Every push to `main` / `master` and every pull request triggers the CI workflow A dedicated `clang-format` job checks every C++ file **added or modified** by the PR / push against `.clang-format`. Pre-existing files are not checked -- about 90% of the codebase predates the style file, so a strict full-repo check would be permanently red. The gate catches *new* formatting regressions without forcing a one-time blame-destroying style sweep. +#### Validate locally before pushing + +The same diff-gate logic is shipped as a local helper so you can catch violations before the CI does: + +```bash +# Check (read-only) -- exits 1 if there are violations on changed lines +python scripts/check_clang_format.py + +# Auto-fix the offending lines in place +python scripts/check_clang_format.py --fix + +# Different base ref (default is origin/main) +python scripts/check_clang_format.py --base HEAD~1 +``` + +Cross-platform wrappers: +- `scripts/check_clang_format.sh` -- Linux / macOS / WSL / Git Bash +- `scripts/check_clang_format.bat` -- Windows `cmd` + +Exit codes: `0` clean, `1` violations, `2` tooling missing. The script auto-detects `clang-format-diff.py` under `Program Files\LLVM\share\clang\` on Windows when it's not on `PATH`. + +#### Pre-push hook (opt-in, versioned in the repo) + +A versioned pre-push hook lives at `scripts/git-hooks/pre-push` and runs the same check before every `git push`. Activate it in your clone once: + +```bash +git config core.hooksPath scripts/git-hooks +``` + +No external dependency (Husky, pre-commit, etc.) -- `core.hooksPath` is built into git ≥ 2.9. To bypass for a one-off push: `git push --no-verify`. To deactivate: `git config --unset core.hooksPath`. + +#### Sweeping the whole tree + If you want to apply the style to the whole tree in one commit (consider recording that SHA in `.git-blame-ignore-revs` afterwards): ```bash diff --git a/docs/POST_PROCESSING.md b/docs/POST_PROCESSING.md new file mode 100644 index 0000000..ff276f6 --- /dev/null +++ b/docs/POST_PROCESSING.md @@ -0,0 +1,336 @@ +# Frame Post-Processor + +A writer-side hook that runs on every `RecordFrame()` call, **after** timer validation succeeds and **before** the frame is handed to the serializer. Whatever the processor mutates is what ends up in the on-disk `.vtx` file. + +This is the canonical extension point for game-specific or integration-specific transformations: clamp out-of-range values, derive consistency state (e.g. `IsAlive` from `Health`), filter or inject entities, accumulate cross-frame statistics, branch behaviour by schema version. + +The SDK provides the **pipeline** (interface, chain, mutation view, lifecycle). Each integration provides the **logic** (processor classes) and, optionally, **strongly-typed accessors** via `scripts/vtx_codegen.py`. + +## Concepts + +``` + +-----------------------+ +RecordFrame --->| timer validation | + +-----------+-----------+ + | + v + +-----------------------+ + | POST-PROCESSOR HOOK | <-- IFramePostProcessor::Process + +-----------+-----------+ + | + v + +-----------------------+ + | Serializer::FromNat | <-- mutations land in wire bytes + +-----------+-----------+ + | + v + (.vtx file) +``` + +| Concept | Where it lives | What it does | +|---|---|---| +| `IFramePostProcessor` | `vtx/writer/core/vtx_frame_post_processor.h` | Interface that integrators implement. Four hooks: `Init`, `Process`, `Clear`, `PrintInfo`. | +| `FramePostProcessorChain` | same | Composable container of processors. Implements `IFramePostProcessor` itself, so a chain is registered just like a single processor. | +| `FrameMutationView` | `vtx/writer/core/vtx_frame_mutation_view.h` | Write-side mirror of `EntityView`/`FrameAccessor`. Entry point the processor receives per frame. | +| `EntityMutator` / `BucketMutator` | same | Cheap non-owning wrappers. `Get` (read) + `Set` (write) + iteration + structural mutation. | +| `FramePostProcessorInitContext` | `vtx_frame_post_processor.h` | Passed to `Init` once per registration: schema accessor, total frames, version info. | +| `FramePostProcessContext` | same | Passed to `Process` per frame: `global_frame_index`, schema version, frame accessor. | + +## Lifecycle + +| Method | When the writer calls it | Override? | +|---|---|---| +| `Init(InitContext)` | Once, synchronously inside `SetPostProcessor()` -- BEFORE the new processor becomes visible to any `RecordFrame()` | Optional. Resolve every `PropertyKey` upfront and load external resources here. | +| `Process(view, ctx)` | Once per `RecordFrame()`, on the writer's calling thread | **Mandatory.** The hot path. | +| `Clear()` | Writer's destructor, OR explicit `ClearPostProcessor()` | Optional. Reset cross-frame accumulators here. | +| `PrintInfo() const` | Whenever the caller invokes it | Optional. Telemetry dump. | + +`Clear` is **not** called on `Stop()` -- the writer can be re-used between `Stop` and destruction (e.g. caller does `Stop`, inspects telemetry, then drops the writer). `Clear` is the destructor's job, or use `ClearPostProcessor()` for explicit teardown. + +## Threading model + +The writer is **single-threaded by design**: `RecordFrame()` is called sequentially from the user's capture loop. No mutex is needed on `post_processor_`. `SetPostProcessor` and `RecordFrame` are expected to be called from the same thread. + +This is different from the reader-side processor model some other VTX features use (e.g. async chunk loads). The writer's simplicity is deliberate. + +## API at a glance + +```cpp +namespace VTX { + + struct FramePostProcessorInitContext { + const FrameAccessor* frame_accessor = nullptr; + int32_t total_frames = 0; + uint32_t schema_version = 0; + uint32_t format_major = 0; + uint32_t format_minor = 0; + }; + + struct FramePostProcessContext { + int32_t global_frame_index = 0; + int32_t chunk_local_frame_index = 0; + int32_t chunk_index = 0; + uint32_t schema_version = 0; + const FrameAccessor* frame_accessor = nullptr; + const Frame* previous_frame = nullptr; // null on writer-side + }; + + class IFramePostProcessor { + public: + virtual ~IFramePostProcessor() = default; + virtual void Init(const FramePostProcessorInitContext&) {} + virtual void Clear() {} + virtual void PrintInfo() const {} + virtual void Process(FrameMutationView& view, + const FramePostProcessContext& ctx) = 0; + }; + + class FramePostProcessorChain final : public IFramePostProcessor { + public: + void Add(std::shared_ptr); + bool Remove(const std::shared_ptr&); + size_t size() const noexcept; + // Init / Process / PrintInfo run in registration order. + // Clear runs in REVERSE order (destructor-like teardown). + }; + +} // namespace VTX +``` + +Registering on a writer: + +```cpp +auto writer = VTX::CreateFlatBuffersWriterFacade(config); +auto processor = std::make_shared(); +writer->SetPostProcessor(processor); // Init() runs here + +// ... RecordFrame() loop ... + +writer->ClearPostProcessor(); // explicit teardown +// OR let writer destructor handle it +``` + +`SetPostProcessor` does **not** invoke `Clear()` on whatever was previously registered. The previous processor is simply dropped; if you want explicit teardown of an outgoing processor, keep your own `shared_ptr` and call `Clear()` yourself. + +## Two ways to write a processor + +### A. Generic API (no codegen) + +Resolve `PropertyKey` by name once in `Init`, then use them in the hot path: + +```cpp +class HealthClampProcessor : public VTX::IFramePostProcessor { +public: + void Init(const VTX::FramePostProcessorInitContext& ctx) override { + health_key_ = ctx.frame_accessor->Get("Player", "Health"); + } + + void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext&) override { + if (!view.HasBucket("entity") || !health_key_.IsValid()) return; + auto bucket = view.GetBucket("entity"); + for (auto entity : bucket) { + if (entity.raw() && entity.raw()->entity_type_id != /*Player type_id*/ 0) continue; + const float hp = entity.Get(health_key_); + if (hp < 0) entity.Set(health_key_, 0); + } + } + +private: + VTX::PropertyKey health_key_ {-1}; +}; +``` + +Works on any schema. Drawback: you carry strings and you have to gate on `entity_type_id` yourself. + +### B. Strongly-typed via codegen (recommended) + +Run `scripts/vtx_codegen.py` over your schema JSON to produce typed views, mutators, and per-struct `ForEachX` helpers. The processor stops carrying strings entirely: + +```cpp +#include "arena_generated.h" // produced by vtx_codegen.py + +class HealthClampProcessor : public VTX::IFramePostProcessor { +public: + void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext&) override { + if (!view.HasBucket("entity")) return; + auto bucket = view.GetBucket("entity"); + + VTX::ArenaSchema::ForEachPlayer(bucket, *view.accessor(), [](auto& p) { + if (p.GetHealth() < 0) p.SetHealth(0); + }); + } +}; +``` + +- No `PropertyKey` members. +- No `Init()` (the codegen-generated `PlayerMutator` caches keys in `static` locals on first use). +- No manual `entity_type_id` gate (`ForEachPlayer` filters by type id internally). +- No `"Player"` / `"Health"` strings. +- If the schema gains a new property, regenerate the header; if a property is removed or renamed, the code fails to compile -- regression caught early. + +The codegen emits: + +| Per struct | What it gives you | +|---|---| +| `EntityType::X` enum | Strongly-typed id (`Player = 0`, `Projectile = 1`, ...) | +| `X::PropertyName` `constexpr const char*` | String constants matching the schema | +| `XView` class | Read-only wrapper (`GetHealth()`, `GetPosition()`, ...) | +| `XMutator` class | Read + write wrapper (`SetHealth(value)`, `GetMutableYyy()` for nested / arrays) | +| `ForEachX(bucket, accessor, fn)` | Filters a `BucketMutator` by entity_type_id, calls `fn(XMutator&)` | +| `ForEachXView(bucket, accessor, fn)` | Read-only iteration counterpart over `const Bucket&` | + +Run the codegen: + +```bash +python scripts/vtx_codegen.py path/to/schema.json path/to/generated.h +``` + +See `samples/arena_generated.h` for a complete example produced from `samples/content/writer/arena/arena_schema.json`. + +## Patterns + +### Cross-frame state + +Processors are regular C++ objects. Cross-frame state lives on member variables -- no API ceremony needed: + +```cpp +class KillStreakProcessor : public VTX::IFramePostProcessor { +public: + void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext& ctx) override { + // ... read current frame, compare to previous_*, update ... + ++frames_seen_; + last_frame_index_ = ctx.global_frame_index; + } + void Clear() override { frames_seen_ = 0; last_frame_index_ = -1; } + +private: + int frames_seen_ = 0; + int32_t last_frame_index_ = -1; + std::unordered_map streaks_; +}; +``` + +`ctx.previous_frame` is always `nullptr` on the writer side -- the writer drops the native form after serialise. Cache what you need on the instance. + +### Composition: chain of processors + +When you have multiple independent transformations: + +```cpp +auto chain = std::make_shared(); +chain->Add(std::make_shared()); +chain->Add(std::make_shared()); +chain->Add(std::make_shared()); +writer->SetPostProcessor(chain); +``` + +| Phase | Order | +|---|---| +| `Init`, `Process`, `PrintInfo` | Registration order (A, B, C) | +| `Clear` | Reverse order (C, B, A) -- destructor semantics | +| Last writer wins | If A and B both `Set` the same property, B's value persists | + +### Replay-level metadata (KeyFrames, stats) + +If your processor accumulates artefacts that don't belong on individual frames (e.g. a `std::vector` extracted from death events), expose them via your own getters and let the caller read after the run: + +```cpp +class StatsProcessor : public VTX::IFramePostProcessor { +public: + void Process(VTX::FrameMutationView&, const VTX::FramePostProcessContext& ctx) override { + // ... accumulate into key_frames_ / order_stats_ / chaos_stats_ ... + } + const std::vector& GetKeyFrames() const { return key_frames_; } + const GlobalStats& GetOrderStats() const { return order_stats_; } + const GlobalStats& GetChaosStats() const { return chaos_stats_; } +private: + std::vector key_frames_; + GlobalStats order_stats_, chaos_stats_; +}; + +auto stats = std::make_shared(); +writer->SetPostProcessor(stats); +// ... record frames ... +writer->Stop(); + +// Caller reads after the run. No SDK plumbing required. +for (const auto& kf : stats->GetKeyFrames()) { ... } +``` + +The processor is just an object you also hold a `shared_ptr` to. + +### Schema-version branching + +Read `ctx.schema_version` per frame and branch: + +```cpp +void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext& ctx) override { + if (ctx.schema_version == /*v13.8*/ 138) { + ProcessV138(view); + } else { + ProcessLatest(view); + } +} +``` + +`ctx.schema_version` is currently always `0` on the writer side (the version lives in the sink's header). Reader-side processors -- if added in a future release -- would see the actual version. Document any branching assumption explicitly. + +### Structural mutation + +`BucketMutator` exposes four operations beyond per-entity reads/writes: + +```cpp +EntityMutator AddEntity(); // appends a default-constructed PropertyContainer; returns mutator +void RemoveEntity(uint32_t entity_index); +template size_t RemoveIf(P pred); // bulk filter -- pred is callable +void Clear(); +``` + +Gotchas worth knowing: + +- **`AddEntity` and `entity_type_id`**: a freshly-added `PropertyContainer` has `entity_type_id = -1` by default. The FlatBuffers serializer **silently drops** entities with `entity_type_id < 0`. Always set it explicitly: + + ```cpp + auto ghost = bucket.AddEntity(); + ghost.raw()->entity_type_id = static_cast(MySchema::EntityType::Player); + ghost.raw()->int32_properties.resize(3, 0); // size to schema's expected slot count + ghost.raw()->float_properties.resize(2, 0.0f); + ghost.Set(health_key_, 100.0f); + ``` + +- **`RemoveIf` invalidates `type_ranges`**: the bucket's typed-range index is wiped to zero after a bulk filter. The serializer rebuilds them from `entity_type_id`, so on-disk output is correct; in-memory tooling that reads `type_ranges` after the processor runs must recompute them or rely on `entity_type_id` directly. + +- **Writer bucket limits**: the FlatBuffers serializer only persists `buckets[0]` (renamed to `"data"` on disk) and `buckets[1]` (renamed to `"bone_data"`). Additional buckets created by a processor via `view.raw()->CreateBucket(...)` are **silently dropped**. The Protobuf serializer preserves all buckets but applies the type-id reordering only to bucket 0. + +## Error handling + +The hook in `RecordFrame()` wraps `processor->Process()` in a try/catch: + +- `std::exception` and unknown exceptions are caught and swallowed (the recording pipeline keeps running). +- Whatever the processor managed to mutate before throwing **stays mutated**. +- Out-of-range or invalid keys on `EntityMutator::Set` are silent no-ops -- matching the tolerance of `EntityView::Get`. + +If `Init()` throws, the exception propagates to the caller and the half-initialized processor is **not** installed. + +## Performance + +When **no** processor is set, the hot path cost in `RecordFrame()` is one `shared_ptr` null-check + one untaken branch. Effectively zero overhead. + +When a processor is set, the cost is dominated by the processor's own logic. The mutation view layer (`EntityMutator::Get` / `Set`) compiles to the same indexed access you'd write by hand against `PropertyContainer::*_properties` -- the wrappers are header-only and inlined. + +Avoid resolving `PropertyKey` per `Process()` call -- the lookup hashes by name. Resolve once in `Init` (or rely on the codegen-generated `Mutator` which caches in `static` locals). + +## Out of scope + +- **Re-serializing mutated bytes back to disk on the reader side.** Not this feature. Reader-side mutations would diverge from on-disk wire bytes -- documented contract. +- **Mutating raw byte spans.** `GetRawFrameBytes` returns on-disk truth by design. +- **Async processors.** `Process` is synchronous on the writer's calling thread. Long-running work blocks `RecordFrame`. +- **Replay-level metadata emission API.** Processors that produce per-replay artefacts (KeyFrames, stats) expose them via their own getters -- the SDK does not add a metadata channel. + +## See also + +- `samples/post_process_write.cpp` -- minimal end-to-end demo: register processor, record synthetic frames, re-open the `.vtx`, verify persisted values. +- `samples/advance_write.cpp` -- the `ArenaConsistencyProcessor` runs over all three data sources (JSON / Proto / FBS), proving the post-processor is orthogonal to source format. +- `samples/arena_generated.h` -- example codegen output (`PlayerMutator`, `ForEachPlayer`, etc.). +- `scripts/vtx_codegen.py` -- the codegen itself. +- `tests/writer/test_frame_post_processor.cpp` -- coverage including value mutation, chain ordering, structural mutation, lifecycle. diff --git a/docs/SAMPLES.md b/docs/SAMPLES.md index d586cd9..b4fe84e 100644 --- a/docs/SAMPLES.md +++ b/docs/SAMPLES.md @@ -1,17 +1,19 @@ # Samples -The `samples/` directory contains five example programs, from a one-shot hello-world to a full data-source-driven pipeline. Each is built as an independent CMake target so you can open any one in isolation. +The `samples/` directory contains seven example programs, from a one-shot hello-world to a full data-source-driven pipeline with frame post-processing. Each is built as an independent CMake target so you can open any one in isolation. ``` samples/ - basic_read.cpp vtx_sample_read reader smoke test - basic_write.cpp vtx_sample_write writer smoke test - basic_diff.cpp vtx_sample_diff hash-based diff - generate_replay.cpp vtx_sample_generate arena simulator -> 3 data sources - advance_write.cpp vtx_sample_advance_write 3 data sources -> 3 .vtx replays + basic_read.cpp vtx_sample_read reader smoke test + basic_write.cpp vtx_sample_write writer smoke test + basic_diff.cpp vtx_sample_diff hash-based diff + ready_api.cpp vtx_sample_ready_api chunk-0 ready signalling + post_process_write.cpp vtx_sample_post_process_write writer-side post-processor + generate_replay.cpp vtx_sample_generate arena simulator -> 3 data sources + advance_write.cpp vtx_sample_advance_write 3 data sources -> 3 .vtx replays (with post-processor) arena_mappings.h JSON data model + JsonMapping + ArenaToVtx::MapFrame() - arena_generated.h autogenerated schema constants + typed views + arena_generated.h autogenerated schema constants + typed Views + Mutators + ForEachX helpers schemas/ arena_data.proto Protobuf game schema (namespace arena_pb) arena_data.fbs FlatBuffers game schema (namespace arena_fb) @@ -87,12 +89,12 @@ The simulation models player patrols, projectile fire, kill/respawn cycles, and ### 5. `advance_write` -- data-source consumer + writer driver -Reads the three data-source files produced by `generate_replay`, maps each record into a `VTX::Frame` using SDK-native integration primitives, and writes one `.vtx` replay per source. +Reads the three data-source files produced by `generate_replay`, maps each record into a `VTX::Frame` using SDK-native integration primitives, **runs a writer-side post-processor** (`ArenaConsistencyProcessor` using the codegen-generated typed API), and writes one `.vtx` replay per source. The same processor instance is registered per pipeline -- demonstrating that frame post-processing is **orthogonal to the source format**: the same logic applies whether the frame came from JSON, Protobuf, or FlatBuffers. | | | |---|---| | **Links against** | `vtx_writer`, `vtx_common` | -| **Needs codegen** | Yes -- arena schemas + `arena_generated.h` | +| **Needs codegen** | Yes -- arena schemas + `arena_generated.h` (now also includes `*Mutator` classes and `ForEachX` helpers) | | **Reads** | all three `content/writer/arena/arena_replay_data.*` files | | **Produces** | `content/reader/arena/arena_from_json_ds.vtx`
`content/reader/arena/arena_from_proto_ds.vtx`
`content/reader/arena/arena_from_fbs_ds.vtx` | @@ -116,6 +118,22 @@ class IFrameDataSource { That's the same interface a real ingestion pipeline would implement -- see `tools/integrations/rl/rl15/rl15_data_source.h` for an example over a streaming protobuf file. +### 6. `post_process_write` -- post-processor end-to-end + +The minimum end-to-end demo of the writer-side **frame post-processor pipeline**. Builds synthetic frames, registers a `PlayerHealthProcessor` (clamp Health to `[0, 100]`, derive `IsAlive`, count cross-frame stats), records 30 frames, then re-opens the `.vtx` with the reader to prove the persisted bytes contain the post-processed values. + +| | | +|---|---| +| **Links against** | `vtx_writer`, `vtx_reader` | +| **Needs codegen** | Yes -- `arena_generated.h` (uses `PlayerMutator` + `ForEachPlayer` + `ForEachPlayerView`) | +| **argv[1]** | schema JSON path (default: `content/writer/arena/arena_schema.json`) | +| **argv[2]** | output `.vtx` path (default: `post_processed.vtx`) | +| **argv[3]** | frame count (default: 30) | + +Key APIs exercised: `VTX::IFramePostProcessor` (`Init` / `Process` / `Clear` / `PrintInfo`), `writer->SetPostProcessor`, `FrameMutationView::GetBucket`, `VTX::ArenaSchema::ForEachPlayer`/`ForEachPlayerView` (codegen-generated), `PlayerMutator::SetHealth` / `SetIsAlive` (codegen-generated). + +The processor has **zero hardcoded strings**, **zero `PropertyKey` members**, and **no manual `entity_type_id` gating** -- everything comes from the codegen-generated wrappers in `arena_generated.h`. Full feature reference in [`POST_PROCESSING.md`](POST_PROCESSING.md). + ## Arena schema codegen The advanced samples depend on generated C++ code for the two arena wire schemas: @@ -127,13 +145,21 @@ The advanced samples depend on generated C++ code for the two arena wire schemas Both codegen steps are declared in `samples/CMakeLists.txt` as `add_custom_command` rules, so re-editing either schema triggers a rebuild automatically. -`arena_generated.h` (hand-checked into the repo, under `samples/`) is a **separate** autogenerated file: it contains schema-field name constants (`ArenaSchema::Player::UniqueID`, `ArenaSchema::Player::StructName`, ...) and typed view classes (`PlayerView`, `ProjectileView`, `MatchStateView`). It is produced from the VTX property schema JSON by: +`arena_generated.h` (hand-checked into the repo, under `samples/`) is a **separate** autogenerated file. From the VTX property schema JSON it emits, per struct: + +- **Constants**: `ArenaSchema::Player::UniqueID`, `ArenaSchema::Player::StructName`, etc. +- **`EntityType` enum**: `ArenaSchema::EntityType::Player = 0`, `Projectile = 1`, ... +- **Read-only views**: `PlayerView`, `ProjectileView`, `MatchStateView` -- wrap `EntityView`, cache `PropertyKey` in `static` locals. +- **Mutators**: `PlayerMutator`, `ProjectileMutator`, `MatchStateMutator` -- wrap `EntityMutator`, expose `Get*` (same as Views) **plus** `Set*` for scalars and `GetMutable*` for arrays / nested structs. +- **Iteration helpers**: `ForEachPlayer(bucket, accessor, fn)` / `ForEachPlayerView(bucket, accessor, fn)` -- filter a `BucketMutator` (or `const Bucket&`) by `entity_type_id` and invoke `fn` with the strongly-typed mutator/view. + +Produced by: ```bat python scripts/vtx_codegen.py samples/content/writer/arena/arena_schema.json samples/arena_generated.h ArenaSchema ``` -Regenerate it whenever `arena_schema.json` changes. +Regenerate it whenever `arena_schema.json` changes. The same script handles any schema -- the same pattern applies to League, CS, Rocket League, or any other integration that ships its own JSON. The SDK doesn't know about the game; the codegen output knows. ## What each sample teaches @@ -142,10 +168,14 @@ Regenerate it whenever `arena_schema.json` changes. | Open a `.vtx` and print its contents | `basic_read.cpp` | | Build `PropertyContainer`s by hand and write them | `basic_write.cpp` | | Compare two frames at all | `basic_diff.cpp` | +| Wait for the reader's first chunk to land (poll / block / callback) | `ready_api.cpp` | | Simulate game data and persist it as a raw data source | `generate_replay.cpp` | | Implement `IFrameDataSource` over **JSON** using `JsonMapping` | `advance_write.cpp` -- `ArenaJsonDataSource` + `arena_mappings.h` | | Implement `IFrameDataSource` over **Protobuf** using `ProtoBinding` | `advance_write.cpp` -- `ArenaProtoDataSource` + `ProtoBinding` specialisations | | Implement `IFrameDataSource` over **FlatBuffers** using `FlatBufferBinding` | `advance_write.cpp` -- `ArenaFbsDataSource` + `FlatBufferBinding` specialisations | +| Hook a writer-side frame post-processor with full lifecycle | `post_process_write.cpp` -- `PlayerHealthProcessor` | +| Run the same post-processor across multiple data sources | `advance_write.cpp` -- `ArenaConsistencyProcessor` registered per pipeline | +| Use codegen-generated typed mutators (`PlayerMutator`, `ForEachPlayer`) | `post_process_write.cpp` + `advance_write.cpp` | | See the same patterns in a production integration | `tools/integrations/rl/rl15/` | ## Running the samples diff --git a/docs/SDK_API.md b/docs/SDK_API.md index 552071c..9c82b55 100644 --- a/docs/SDK_API.md +++ b/docs/SDK_API.md @@ -187,6 +187,119 @@ writer->Stop(); // Write footer and close file --- +## Frame Post-Processor + +A writer-side hook that runs on every `RecordFrame()` call, **after** timer validation and **before** the serializer touches the native `Frame`. Whatever the processor mutates lands on disk -- callers who re-read the `.vtx` see the post-processed values. + +See [`POST_PROCESSING.md`](POST_PROCESSING.md) for the full reference (lifecycle, threading, structural mutation, error handling, performance, codegen integration). This section is the API cheat-sheet. + +### Implementing a processor + +```cpp +#include "vtx/writer/core/vtx_frame_post_processor.h" +#include "vtx/writer/core/vtx_frame_mutation_view.h" + +class HealthClampProcessor : public VTX::IFramePostProcessor { +public: + // Called once, synchronously, inside SetPostProcessor. Resolve every + // PropertyKey here -- the schema is constant for the recording session. + void Init(const VTX::FramePostProcessorInitContext& ctx) override { + health_key_ = ctx.frame_accessor->Get("Player", "Health"); + } + + // Called once per RecordFrame, on the writer's calling thread. + void Process(VTX::FrameMutationView& view, + const VTX::FramePostProcessContext& ctx) override { + if (!view.HasBucket("entity") || !health_key_.IsValid()) return; + auto bucket = view.GetBucket("entity"); + for (auto entity : bucket) { + if (entity.raw()->entity_type_id != /*Player*/ 0) continue; + if (entity.Get(health_key_) < 0.0f) entity.Set(health_key_, 0.0f); + } + } + + // Optional: reset cross-frame accumulators. Called by writer destructor + // or explicit ClearPostProcessor(). + void Clear() override {} + + // Optional: telemetry dump. Called when the caller invokes it. + void PrintInfo() const override {} + +private: + VTX::PropertyKey health_key_ {-1}; +}; +``` + +### Registering on the writer + +```cpp +auto writer = VTX::CreateFlatBuffersWriterFacade(config); +auto processor = std::make_shared(); +writer->SetPostProcessor(processor); // Init() runs here + +for (...) { writer->RecordFrame(frame, time); } // Process() runs per frame +writer->Stop(); +// Writer's destructor calls processor->Clear() +// OR call writer->ClearPostProcessor() explicitly first. +``` + +### Composing multiple processors + +```cpp +auto chain = std::make_shared(); +chain->Add(std::make_shared()); +chain->Add(std::make_shared()); +chain->Add(std::make_shared()); +writer->SetPostProcessor(chain); +``` + +`Init`, `Process`, and `PrintInfo` run in registration order. `Clear` runs in reverse (destructor-like teardown). If two processors `Set` the same property, the last one wins. + +### Strongly-typed accessors via codegen + +`scripts/vtx_codegen.py` produces per-struct `XMutator` classes and `ForEachX` helpers for any schema. The processor becomes string-free: + +```cpp +#include "arena_generated.h" // produced by vtx_codegen.py + +class HealthClampProcessor : public VTX::IFramePostProcessor { +public: + void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext&) override { + if (!view.HasBucket("entity")) return; + auto bucket = view.GetBucket("entity"); + VTX::ArenaSchema::ForEachPlayer(bucket, *view.accessor(), [](auto& p) { + if (p.GetHealth() < 0) p.SetHealth(0); + }); + } +}; +``` + +No `Init`, no `PropertyKey` members, no `entity_type_id` gating, no schema strings. See [`POST_PROCESSING.md`](POST_PROCESSING.md#strongly-typed-via-codegen-recommended) and [`SAMPLES.md`](SAMPLES.md#6-post_process_write----post-processor-end-to-end). + +### Mutation view API + +```cpp +// Per-entity reads / writes (writer-side mirror of EntityView) +template T Get(PropertyKey key) const; +template void Set(PropertyKey key, T value); + +// Nested struct mutation (Champion.Spells[i].Cooldown style) +EntityMutator GetMutableView(PropertyKey key); + +// Array mutation +template std::span GetMutableArray(PropertyKey key); + +// Bucket-level (BucketMutator) +EntityMutator AddEntity(); // inject synthetic entity +void RemoveEntity(uint32_t entity_index); +template size_t RemoveIf(P predicate); // bulk filter; P : bool(EntityView) +void Clear(); +``` + +Out-of-range / invalid-key writes are **silent no-ops**, matching the read-side tolerance. The serializer drops entities whose `entity_type_id < 0`, so newly-added entities must have their type id set explicitly before the chunk is flushed (see [`POST_PROCESSING.md` gotchas](POST_PROCESSING.md#structural-mutation)). + +--- + ## Diffing Frames ### Creating a Differ diff --git a/samples/CMakeLists.txt b/samples/CMakeLists.txt index 6dd39a3..1b8a80f 100644 --- a/samples/CMakeLists.txt +++ b/samples/CMakeLists.txt @@ -61,6 +61,14 @@ add_executable(vtx_sample_ready_api ready_api.cpp) target_link_libraries(vtx_sample_ready_api PRIVATE vtx_reader) vtx_configure_sample(vtx_sample_ready_api) +# --- post_process_write (frame post-processor: writer-side hook) --- +add_executable(vtx_sample_post_process_write post_process_write.cpp) +target_link_libraries(vtx_sample_post_process_write PRIVATE vtx_writer vtx_reader) +target_include_directories(vtx_sample_post_process_write PRIVATE + "${VTX_COMMON_GENERATED_DIR}" +) +vtx_configure_sample(vtx_sample_post_process_write) + # ============================================================================== # Arena-specific codegen (arena_data.proto + arena_data.fbs) # diff --git a/samples/advance_write.cpp b/samples/advance_write.cpp index 8d47519..1080a90 100644 --- a/samples/advance_write.cpp +++ b/samples/advance_write.cpp @@ -38,6 +38,8 @@ #include "vtx/writer/core/vtx_writer_facade.h" #include "vtx/writer/core/vtx_data_source.h" +#include "vtx/writer/core/vtx_frame_post_processor.h" +#include "vtx/writer/core/vtx_frame_mutation_view.h" #include "vtx/common/adapters/json/json_adapter.h" #include "vtx/common/vtx_types_helpers.h" #include "vtx/common/readers/frame_reader/flatbuffer_loader.h" @@ -51,8 +53,10 @@ #include "arena_data.pb.h" #include "arena_data_generated.h" +#include #include #include +#include #include #include @@ -439,6 +443,60 @@ class ArenaFbsDataSource : public VTX::IFrameDataSource { size_t cursor_ = 0; }; +class ArenaConsistencyProcessor : public VTX::IFramePostProcessor { +public: + void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext& /*ctx*/) override { + if (!view.HasBucket("entity")) + return; + + auto bucket = view.GetBucket("entity"); + VTX::ArenaSchema::ForEachPlayer(bucket, *view.accessor(), [&](auto& p) { + const float hp = p.GetHealth(); + const float clamped_hp = std::clamp(hp, 0.0f, 100.0f); + if (clamped_hp != hp) { + p.SetHealth(clamped_hp); + ++health_mutations_; + } + + const float armor = p.GetArmor(); + const float clamped_armor = std::clamp(armor, 0.0f, 100.0f); + if (clamped_armor != armor) { + p.SetArmor(clamped_armor); + ++armor_mutations_; + } + + if (clamped_hp <= 0.0f && p.GetIsAlive()) { + p.SetIsAlive(false); + ++is_alive_mutations_; + } + ++players_inspected_; + }); + ++frames_seen_; + } + + void PrintInfo() const override { + VTX_INFO( + " [post-process] frames={} players_inspected={} health_clamps={} armor_clamps={} is_alive_corrections={}", + frames_seen_, players_inspected_, health_mutations_, armor_mutations_, is_alive_mutations_); + } + + void Clear() override { + frames_seen_ = 0; + players_inspected_ = 0; + health_mutations_ = 0; + armor_mutations_ = 0; + is_alive_mutations_ = 0; + } + +private: + int frames_seen_ = 0; + int players_inspected_ = 0; + int health_mutations_ = 0; + int armor_mutations_ = 0; + int is_alive_mutations_ = 0; +}; + + // =================================================================== // Pipeline driver -- initialise source, stream frames into the writer // =================================================================== @@ -467,6 +525,9 @@ static bool RunPipeline(VTX::IFrameDataSource& source, const std::string& output return false; } + auto processor = std::make_shared(); + writer->SetPostProcessor(processor); + VTX::Frame frame; VTX::GameTime::GameTimeRegister time; int count = 0; @@ -477,6 +538,8 @@ static bool RunPipeline(VTX::IFrameDataSource& source, const std::string& output writer->Flush(); writer->Stop(); + processor->PrintInfo(); + VTX_INFO(" -> wrote {} frames to {}", count, output_path); return true; } @@ -494,6 +557,8 @@ int main() { VTX_INFO("=== advance_write - IFrameDataSource pattern ==="); VTX_INFO("Demonstrates JSON, Protobuf and FlatBuffers sources using integration-style bindings."); + VTX_INFO("Each writer also runs ArenaConsistencyProcessor (frame post-processor)"); + VTX_INFO("on every recorded frame; the persisted .vtx contains the post-processed values."); VTX_INFO("--- 1. JSON data source ---"); ArenaJsonDataSource json_ds(writer_dir + "/arena_replay_data.json"); diff --git a/samples/arena_generated.h b/samples/arena_generated.h index d23770a..df3a1e9 100644 --- a/samples/arena_generated.h +++ b/samples/arena_generated.h @@ -6,7 +6,8 @@ #include #include #include "vtx/common/vtx_property_cache.h" -#include "vtx/reader/core/vtx_frame_accessor.h" +#include "vtx/common/vtx_frame_accessor.h" +#include "vtx/writer/core/vtx_frame_mutation_view.h" namespace VTX::ArenaSchema { @@ -102,6 +103,130 @@ namespace VTX::ArenaSchema { }; + class PlayerMutator { + private: + VTX::EntityMutator data_mut; + const VTX::FrameAccessor& accessor; + + public: + PlayerMutator(VTX::EntityMutator m, const VTX::FrameAccessor& acc) + : data_mut(m), accessor(acc) {} + + PlayerMutator(VTX::PropertyContainer& container, const VTX::FrameAccessor& acc) + : data_mut(container), accessor(acc) {} + + inline const std::string& GetUniqueID() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::UniqueID); + return data_mut.Get(cached_key); + } + + inline void SetUniqueID(const std::string& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::UniqueID); + data_mut.Set(cached_key, value); + } + + inline const std::string& GetName() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Name); + return data_mut.Get(cached_key); + } + + inline void SetName(const std::string& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Name); + data_mut.Set(cached_key, value); + } + + inline int32_t GetTeam() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Team); + return data_mut.Get(cached_key); + } + + inline void SetTeam(int32_t value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Team); + data_mut.Set(cached_key, value); + } + + inline float GetHealth() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Health); + return data_mut.Get(cached_key); + } + + inline void SetHealth(float value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Health); + data_mut.Set(cached_key, value); + } + + inline float GetArmor() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Armor); + return data_mut.Get(cached_key); + } + + inline void SetArmor(float value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Armor); + data_mut.Set(cached_key, value); + } + + inline const VTX::Vector& GetPosition() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Position); + return data_mut.Get(cached_key); + } + + inline void SetPosition(const VTX::Vector& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Position); + data_mut.Set(cached_key, value); + } + + inline const VTX::Quat& GetRotation() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Rotation); + return data_mut.Get(cached_key); + } + + inline void SetRotation(const VTX::Quat& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Rotation); + data_mut.Set(cached_key, value); + } + + inline const VTX::Vector& GetVelocity() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Velocity); + return data_mut.Get(cached_key); + } + + inline void SetVelocity(const VTX::Vector& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Velocity); + data_mut.Set(cached_key, value); + } + + inline bool GetIsAlive() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::IsAlive); + return data_mut.Get(cached_key); + } + + inline void SetIsAlive(bool value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::IsAlive); + data_mut.Set(cached_key, value); + } + + inline int32_t GetScore() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Score); + return data_mut.Get(cached_key); + } + + inline void SetScore(int32_t value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Score); + data_mut.Set(cached_key, value); + } + + inline int32_t GetDeaths() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Deaths); + return data_mut.Get(cached_key); + } + + inline void SetDeaths(int32_t value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Player, Player::Deaths); + data_mut.Set(cached_key, value); + } + + }; + namespace Projectile { constexpr const char* StructName = "Projectile"; constexpr const char* UniqueID = "UniqueID"; @@ -156,6 +281,80 @@ namespace VTX::ArenaSchema { }; + class ProjectileMutator { + private: + VTX::EntityMutator data_mut; + const VTX::FrameAccessor& accessor; + + public: + ProjectileMutator(VTX::EntityMutator m, const VTX::FrameAccessor& acc) + : data_mut(m), accessor(acc) {} + + ProjectileMutator(VTX::PropertyContainer& container, const VTX::FrameAccessor& acc) + : data_mut(container), accessor(acc) {} + + inline const std::string& GetUniqueID() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::UniqueID); + return data_mut.Get(cached_key); + } + + inline void SetUniqueID(const std::string& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::UniqueID); + data_mut.Set(cached_key, value); + } + + inline const std::string& GetOwnerID() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::OwnerID); + return data_mut.Get(cached_key); + } + + inline void SetOwnerID(const std::string& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::OwnerID); + data_mut.Set(cached_key, value); + } + + inline const VTX::Vector& GetPosition() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::Position); + return data_mut.Get(cached_key); + } + + inline void SetPosition(const VTX::Vector& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::Position); + data_mut.Set(cached_key, value); + } + + inline const VTX::Vector& GetVelocity() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::Velocity); + return data_mut.Get(cached_key); + } + + inline void SetVelocity(const VTX::Vector& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::Velocity); + data_mut.Set(cached_key, value); + } + + inline float GetDamage() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::Damage); + return data_mut.Get(cached_key); + } + + inline void SetDamage(float value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::Damage); + data_mut.Set(cached_key, value); + } + + inline const std::string& GetType() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::Type); + return data_mut.Get(cached_key); + } + + inline void SetType(const std::string& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::Projectile, Projectile::Type); + data_mut.Set(cached_key, value); + } + + }; + namespace MatchState { constexpr const char* StructName = "MatchState"; constexpr const char* UniqueID = "UniqueID"; @@ -210,4 +409,155 @@ namespace VTX::ArenaSchema { }; + class MatchStateMutator { + private: + VTX::EntityMutator data_mut; + const VTX::FrameAccessor& accessor; + + public: + MatchStateMutator(VTX::EntityMutator m, const VTX::FrameAccessor& acc) + : data_mut(m), accessor(acc) {} + + MatchStateMutator(VTX::PropertyContainer& container, const VTX::FrameAccessor& acc) + : data_mut(container), accessor(acc) {} + + inline const std::string& GetUniqueID() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::UniqueID); + return data_mut.Get(cached_key); + } + + inline void SetUniqueID(const std::string& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::UniqueID); + data_mut.Set(cached_key, value); + } + + inline int32_t GetScoreTeam1() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::ScoreTeam1); + return data_mut.Get(cached_key); + } + + inline void SetScoreTeam1(int32_t value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::ScoreTeam1); + data_mut.Set(cached_key, value); + } + + inline int32_t GetScoreTeam2() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::ScoreTeam2); + return data_mut.Get(cached_key); + } + + inline void SetScoreTeam2(int32_t value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::ScoreTeam2); + data_mut.Set(cached_key, value); + } + + inline int32_t GetRound() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::Round); + return data_mut.Get(cached_key); + } + + inline void SetRound(int32_t value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::Round); + data_mut.Set(cached_key, value); + } + + inline const std::string& GetPhase() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::Phase); + return data_mut.Get(cached_key); + } + + inline void SetPhase(const std::string& value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::Phase); + data_mut.Set(cached_key, value); + } + + inline float GetTimeRemaining() const { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::TimeRemaining); + return data_mut.Get(cached_key); + } + + inline void SetTimeRemaining(float value) { + static VTX::PropertyKey cached_key = accessor.Get(EntityType::MatchState, MatchState::TimeRemaining); + data_mut.Set(cached_key, value); + } + + }; + + // ---------------------------------------------------------------- + // Strongly-typed iteration helpers + // ---------------------------------------------------------------- + // ForEachX(bucket, accessor, fn) walks a BucketMutator and calls + // fn(XMutator&) only for entities whose entity_type_id matches the + // struct. Read-only counterparts (XView) are available via + // ForEachXView. + + template + void ForEachPlayer(VTX::BucketMutator& bucket, const VTX::FrameAccessor& accessor, Fn fn) { + constexpr int32_t kTypeId = static_cast(EntityType::Player); + for (auto entity : bucket) { + const auto* raw = entity.raw(); + if (raw && raw->entity_type_id == kTypeId) { + PlayerMutator obj(entity, accessor); + fn(obj); + } + } + } + + template + void ForEachPlayerView(const VTX::Bucket& bucket, const VTX::FrameAccessor& accessor, Fn fn) { + constexpr int32_t kTypeId = static_cast(EntityType::Player); + for (const auto& container : bucket.entities) { + if (container.entity_type_id == kTypeId) { + PlayerView obj(container, accessor); + fn(obj); + } + } + } + + template + void ForEachProjectile(VTX::BucketMutator& bucket, const VTX::FrameAccessor& accessor, Fn fn) { + constexpr int32_t kTypeId = static_cast(EntityType::Projectile); + for (auto entity : bucket) { + const auto* raw = entity.raw(); + if (raw && raw->entity_type_id == kTypeId) { + ProjectileMutator obj(entity, accessor); + fn(obj); + } + } + } + + template + void ForEachProjectileView(const VTX::Bucket& bucket, const VTX::FrameAccessor& accessor, Fn fn) { + constexpr int32_t kTypeId = static_cast(EntityType::Projectile); + for (const auto& container : bucket.entities) { + if (container.entity_type_id == kTypeId) { + ProjectileView obj(container, accessor); + fn(obj); + } + } + } + + template + void ForEachMatchState(VTX::BucketMutator& bucket, const VTX::FrameAccessor& accessor, Fn fn) { + constexpr int32_t kTypeId = static_cast(EntityType::MatchState); + for (auto entity : bucket) { + const auto* raw = entity.raw(); + if (raw && raw->entity_type_id == kTypeId) { + MatchStateMutator obj(entity, accessor); + fn(obj); + } + } + } + + template + void ForEachMatchStateView(const VTX::Bucket& bucket, const VTX::FrameAccessor& accessor, Fn fn) { + constexpr int32_t kTypeId = static_cast(EntityType::MatchState); + for (const auto& container : bucket.entities) { + if (container.entity_type_id == kTypeId) { + MatchStateView obj(container, accessor); + fn(obj); + } + } + } + } // namespace VTX::ArenaSchema \ No newline at end of file diff --git a/samples/content/reader/arena/arena_from_fbs_ds.vtx b/samples/content/reader/arena/arena_from_fbs_ds.vtx index 7139e94..fad0da8 100644 Binary files a/samples/content/reader/arena/arena_from_fbs_ds.vtx and b/samples/content/reader/arena/arena_from_fbs_ds.vtx differ diff --git a/samples/content/reader/arena/arena_from_json_ds.vtx b/samples/content/reader/arena/arena_from_json_ds.vtx index 7e4460a..fc7e6ad 100644 Binary files a/samples/content/reader/arena/arena_from_json_ds.vtx and b/samples/content/reader/arena/arena_from_json_ds.vtx differ diff --git a/samples/content/reader/arena/arena_from_proto_ds.vtx b/samples/content/reader/arena/arena_from_proto_ds.vtx index 75c6c74..a8361a5 100644 Binary files a/samples/content/reader/arena/arena_from_proto_ds.vtx and b/samples/content/reader/arena/arena_from_proto_ds.vtx differ diff --git a/samples/post_process_write.cpp b/samples/post_process_write.cpp new file mode 100644 index 0000000..9a92f3f --- /dev/null +++ b/samples/post_process_write.cpp @@ -0,0 +1,223 @@ +// post_process_write.cpp -- Record synthetic frames with a frame post-processor. +// +// Purpose +// Demonstrates the writer-side frame post-processor pipeline using the +// strongly-typed API generated by scripts/vtx_codegen.py. The processor +// has zero hardcoded strings, zero PropertyKey members, and no manual +// entity_type_id gating -- the codegen takes care of all of that. +// +// This sample drives four behaviours through a single processor: +// 1. Clamp Health to [0, 100] (value mutation) +// 2. Force IsAlive=false when Health drops to 0 (derived state) +// 3. Count frames where a player is "low health" (cross-frame state) +// 4. Dump that count via PrintInfo() on teardown (lifecycle hook) +// +// After the .vtx is written we re-open it with the reader and print the +// first few frames to prove the persisted bytes reflect the mutations. +// +// Args +// argv[1] -- schema JSON path (default: content/writer/arena/arena_schema.json) +// argv[2] -- output .vtx path (default: post_processed.vtx) +// argv[3] -- frame count (default: 30) +// +// Build +// Link against vtx_writer + vtx_reader (vtx_common is transitive). + +#include "vtx/writer/core/vtx_writer_facade.h" +#include "vtx/writer/core/vtx_frame_post_processor.h" +#include "vtx/writer/core/vtx_frame_mutation_view.h" +#include "vtx/reader/core/vtx_reader_facade.h" +#include "vtx/common/vtx_types.h" +#include "vtx/common/vtx_logger.h" + +#include "arena_generated.h" + +#include +#include +#include + +// --------------------------------------------------------------------------- +// PlayerHealthProcessor +// --------------------------------------------------------------------------- +// Uses the codegen-generated PlayerMutator + ForEachPlayer helpers. +// * No PropertyKey members. +// * No Init() to resolve them (the Mutator caches keys in static locals). +// * No manual entity_type_id gate (ForEachPlayer does it). +// * No "Player" / "Health" string literals in the processor body. +class PlayerHealthProcessor : public VTX::IFramePostProcessor { +public: + void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext& ctx) override { + if (!view.HasBucket("entity")) + return; + + auto bucket = view.GetBucket("entity"); + VTX::ArenaSchema::ForEachPlayer(bucket, *view.accessor(), [&](auto& p) { + // (1) Clamp Health to [0, 100]. + const float hp = p.GetHealth(); + const float clamped_hp = std::clamp(hp, 0.0f, 100.0f); + if (clamped_hp != hp) + p.SetHealth(clamped_hp); + + // (2) Derived state: IsAlive=false when Health hits 0. + if (clamped_hp <= 0.0f && p.GetIsAlive()) + p.SetIsAlive(false); + + // (3) Cross-frame accumulator: count "low health" frame-entities. + // Demonstrates state lives on the processor instance. + if (clamped_hp < 50.0f) + ++low_health_frame_entities_; + }); + + ++frames_seen_; + last_global_frame_ = ctx.global_frame_index; + } + + void PrintInfo() const override { + VTX_INFO("PlayerHealthProcessor: frames_seen={}, last_global_frame={}, " + "low_health_frame_entities={}", + frames_seen_, last_global_frame_, low_health_frame_entities_); + } + + void Clear() override { + VTX_INFO("PlayerHealthProcessor::Clear -- resetting state"); + frames_seen_ = 0; + last_global_frame_ = -1; + low_health_frame_entities_ = 0; + } + +private: + int frames_seen_ = 0; + int32_t last_global_frame_ = -1; + int low_health_frame_entities_ = 0; +}; + + +// --------------------------------------------------------------------------- +// Helpers: build a synthetic Player frame with two entities. +// --------------------------------------------------------------------------- +// Builds entities using PlayerMutator. Health is intentionally driven out +// of range (negative on even frames, >100 on multiples of 7) so the +// processor's clamp has something to do. +// +// Note: PlayerMutator writes into pre-sized typed vectors of the +// PropertyContainer. The writer's schema layout dictates how big each +// vector is -- we resize them here based on the schema struct definition +// (counted from arena_generated.h). +static VTX::Frame MakeSyntheticFrame(int frame_index, const VTX::FrameAccessor& accessor) { + VTX::Frame frame; + auto& bucket = frame.CreateBucket("entity"); + + auto push_player = [&](int player_id, int team, float health) { + VTX::PropertyContainer pc; + pc.entity_type_id = static_cast(VTX::ArenaSchema::EntityType::Player); + // Pre-size the typed vectors so PlayerMutator::SetX can index them. + // Player schema: UniqueID, Name (strings); Team, Score, Deaths (int32); + // Health, Armor (floats); Position, Velocity (vectors); Rotation (quat); + // IsAlive (bool). + pc.string_properties.resize(2); + pc.int32_properties.resize(3); + pc.float_properties.resize(2); + pc.vector_properties.resize(2); + pc.quat_properties.resize(1); + pc.bool_properties.resize(1); + + VTX::ArenaSchema::PlayerMutator p(pc, accessor); + p.SetUniqueID("player_" + std::to_string(player_id)); + p.SetName("Player_" + std::to_string(player_id)); + p.SetTeam(team); + p.SetHealth(health); + p.SetArmor(50.0f); + p.SetPosition({double(frame_index), double(player_id), 0.0}); + p.SetVelocity({}); + p.SetRotation({0.0f, 0.0f, 0.0f, 1.0f}); + p.SetIsAlive(true); + p.SetScore(0); + p.SetDeaths(0); + + bucket.unique_ids.push_back("player_" + std::to_string(player_id)); + bucket.entities.push_back(std::move(pc)); + }; + + // Two players. P0 drifts into negative health on even frames; P1 drifts + // over 100 every 7th frame. Both will be clamped by the processor. + push_player(0, 1, (frame_index % 2 == 0) ? -5.0f - float(frame_index) : 80.0f); + push_player(1, 2, (frame_index % 7 == 0) ? 150.0f : 60.0f - float(frame_index) * 0.5f); + return frame; +} + + +int main(int argc, char* argv[]) { + const std::string schema_path = (argc > 1) ? argv[1] : "content/writer/arena/arena_schema.json"; + const std::string output_path = (argc > 2) ? argv[2] : "post_processed.vtx"; + const int frame_count = (argc > 3) ? std::max(1, std::atoi(argv[3])) : 30; + + // --- 1. Build the writer + register the processor BEFORE the first frame. + VTX::WriterFacadeConfig config; + config.output_filepath = output_path; + config.schema_json_path = schema_path; + config.replay_name = "PostProcessSample"; + config.replay_uuid = "sample-postproc-0001"; + config.default_fps = 60.0f; + config.chunk_max_frames = 16; + config.use_compression = true; + + auto writer = VTX::CreateFlatBuffersWriterFacade(config); + if (!writer) { + VTX_ERROR("Failed to create writer. Schema path: {}", schema_path); + return 1; + } + + // Build a FrameAccessor from the writer's loaded schema so MakeSyntheticFrame + // can use the strongly-typed PlayerMutator. + VTX::FrameAccessor accessor; + accessor.InitializeFromCache(writer->GetSchema().GetPropertyCache()); + + auto processor = std::make_shared(); + writer->SetPostProcessor(processor); + + // --- 2. Record synthetic frames. The processor runs on each one. + for (int i = 0; i < frame_count; ++i) { + VTX::Frame frame = MakeSyntheticFrame(i, accessor); + VTX::GameTime::GameTimeRegister game_time; + game_time.game_time = static_cast(i) / 60.0f; + writer->RecordFrame(frame, game_time); + } + + // Dump telemetry from the processor before tearing it down. + processor->PrintInfo(); + + writer->Flush(); + writer->Stop(); + // Writer's destructor will invoke processor->Clear(). + writer.reset(); + + VTX_INFO("Wrote {} post-processed frames to {}", frame_count, output_path); + + // --- 3. Re-open the .vtx and prove the persisted bytes are post-processed. + auto ctx = VTX::OpenReplayFile(output_path); + if (!ctx) { + VTX_ERROR("Re-open failed: {}", ctx.GetError()); + return 1; + } + if (!ctx.WaitUntilReady(std::chrono::seconds(2))) { + VTX_ERROR("Reader never became ready"); + return 1; + } + + VTX_INFO("Reader sees {} frames", ctx.reader->GetTotalFrames()); + const VTX::FrameAccessor read_accessor = ctx.reader->CreateAccessor(); + const int frames_to_print = std::min(frame_count, 5); + for (int i = 0; i < frames_to_print; ++i) { + const VTX::Frame* frame = ctx.reader->GetFrameSync(i); + if (!frame || frame->GetBuckets().empty()) + continue; + const auto& bucket = frame->GetBuckets()[0]; + // Use the codegen-generated PlayerView on the reader side too -- + // zero hardcoded strings, mirrors the writer-side processor. + VTX::ArenaSchema::ForEachPlayerView(bucket, read_accessor, [&](auto& p) { + VTX_INFO("frame {} player {} -> Health={} IsAlive={}", i, p.GetName(), p.GetHealth(), p.GetIsAlive()); + }); + } + + return 0; +} diff --git a/samples/post_processed.vtx b/samples/post_processed.vtx new file mode 100644 index 0000000..2d81e19 Binary files /dev/null and b/samples/post_processed.vtx differ diff --git a/scripts/vtx_codegen.py b/scripts/vtx_codegen.py index ec38f3a..59a9811 100644 --- a/scripts/vtx_codegen.py +++ b/scripts/vtx_codegen.py @@ -72,20 +72,146 @@ def normalize_schema_entries(data): return normalized def clean_enum_name(name): - """Convierte strings como 'smoke_grenade' o 'he' a PascalCase 'SmokeGrenade', 'He'""" + """Converts strings like 'smoke_grenade' or 'he' to PascalCase 'SmokeGrenade', 'He'""" parts = str(name).replace(" ", "_").split("_") return "".join(p.capitalize() for p in parts) + +def resolve_property_types(val, struct_name, known_structs, type_map): + raw_type_field = val.get("type", "") + is_list_enum = isinstance(raw_type_field, list) + raw_type = "implicit_enum" if is_list_enum else str(raw_type_field).lower() + + struct_type = val.get("structType", "").lower() + container = val.get("container", "").lower() + is_array = val.get("array", False) or (container == "array") + b_is_enum = val.get("bIsEnum", False) + b_is_actor = val.get("bIsActor", False) + + if raw_type in ["struct", "complexobject", "object", "none", ""] and struct_type: + lookup_type = struct_type + else: + lookup_type = raw_type + lookup_type = lookup_type.replace("vtx::", "").replace("std::", "") + + if is_list_enum: + accessor_type = "int32_t" + user_ret_type = f"{struct_name}::E{val.get('name', '')}" + elif b_is_enum or lookup_type in ["byte", "uint8"]: + accessor_type = "int32_t" + user_ret_type = "uint8_t" + elif b_is_actor: + accessor_type = "int64_t" + user_ret_type = "int64_t" + elif lookup_type in known_structs: + accessor_type = "VTX::EntityView" + user_ret_type = "VTX::EntityView" + else: + accessor_type = type_map.get(lookup_type) + if not accessor_type: + print(f" [WARNING] Unknown type '{lookup_type}' in property '{val.get('name', '')}'. Using float by default.") + accessor_type = "float" + user_ret_type = accessor_type + + return { + "is_array": is_array, + "is_list_enum": is_list_enum, + "accessor_type": accessor_type, + "user_ret_type": user_ret_type, + } + + +def emit_getter(lines, val, struct_name, info, complex_types, source_member): + prop_name = val.get("name", "") + accessor_type = info["accessor_type"] + user_ret_type = info["user_ret_type"] + is_array = info["is_array"] + + if accessor_type == "VTX::EntityView": + if is_array: + span_type = "VTX::PropertyContainer" + lines.append(f" /** @brief Returns a span of nested structures */") + lines.append(f" inline std::span Get{prop_name}() const {{") + lines.append(f" static VTX::PropertyKey> cached_key = accessor.GetViewArrayKey(EntityType::{struct_name}, {struct_name}::{prop_name});") + lines.append(f" return {source_member}.AsView().GetViewArray(cached_key);" if source_member == "data_mut" else f" return {source_member}.GetViewArray(cached_key);") + lines.append(" }") + else: + lines.append(f" /** @brief Returns a nested view */") + lines.append(f" inline VTX::EntityView Get{prop_name}() const {{") + lines.append(f" static VTX::PropertyKey cached_key = accessor.GetViewKey(EntityType::{struct_name}, {struct_name}::{prop_name});") + lines.append(f" return {source_member}.AsView().GetView(cached_key);" if source_member == "data_mut" else f" return {source_member}.GetView(cached_key);") + lines.append(" }") + else: + if is_array: + span_ret_type = "uint8_t" if accessor_type == "bool" else accessor_type + lines.append(f" /** @brief Returns a span of {accessor_type} */") + lines.append(f" inline std::span Get{prop_name}() const {{") + lines.append(f" static VTX::PropertyKey<{accessor_type}> cached_key = accessor.GetArray<{accessor_type}>(EntityType::{struct_name}, {struct_name}::{prop_name});") + lines.append(f" return {source_member}.AsView().GetArray<{accessor_type}>(cached_key);" if source_member == "data_mut" else f" return {source_member}.GetArray<{accessor_type}>(cached_key);") + lines.append(" }") + else: + ret_ref = "&" if user_ret_type in complex_types else "" + const_ref = f"const {user_ret_type}{ret_ref}" if user_ret_type in complex_types else user_ret_type + lines.append(f" inline {const_ref} Get{prop_name}() const {{") + lines.append(f" static VTX::PropertyKey<{accessor_type}> cached_key = accessor.Get<{accessor_type}>(EntityType::{struct_name}, {struct_name}::{prop_name});") + if user_ret_type != accessor_type: + lines.append(f" return static_cast<{user_ret_type}>({source_member}.Get<{accessor_type}>(cached_key));") + else: + lines.append(f" return {source_member}.Get<{accessor_type}>(cached_key);") + lines.append(" }") + lines.append("") + + +def emit_setter(lines, val, struct_name, info, complex_types): + prop_name = val.get("name", "") + accessor_type = info["accessor_type"] + user_ret_type = info["user_ret_type"] + is_array = info["is_array"] + + if accessor_type == "VTX::EntityView": + # Nested struct: writeable accessor returns a mutator. + if is_array: + lines.append(f" /** @brief Returns a mutable span of nested PropertyContainers */") + lines.append(f" inline std::span GetMutable{prop_name}() {{") + lines.append(f" static VTX::PropertyKey> cached_key = accessor.GetViewArrayKey(EntityType::{struct_name}, {struct_name}::{prop_name});") + lines.append(f" return data_mut.GetMutableViewArray(cached_key);") + lines.append(" }") + else: + lines.append(f" /** @brief Returns a mutator over the nested struct */") + lines.append(f" inline VTX::EntityMutator GetMutable{prop_name}() {{") + lines.append(f" static VTX::PropertyKey cached_key = accessor.GetViewKey(EntityType::{struct_name}, {struct_name}::{prop_name});") + lines.append(f" return data_mut.GetMutableView(cached_key);") + lines.append(" }") + else: + if is_array: + span_ret_type = "uint8_t" if accessor_type == "bool" else accessor_type + lines.append(f" /** @brief Returns a mutable span of {accessor_type} */") + lines.append(f" inline std::span<{span_ret_type}> GetMutable{prop_name}() {{") + lines.append(f" static VTX::PropertyKey<{accessor_type}> cached_key = accessor.GetArray<{accessor_type}>(EntityType::{struct_name}, {struct_name}::{prop_name});") + lines.append(f" return data_mut.GetMutableArray<{accessor_type}>(cached_key);") + lines.append(" }") + else: + param_type = f"const {user_ret_type}&" if user_ret_type in complex_types else user_ret_type + lines.append(f" inline void Set{prop_name}({param_type} value) {{") + lines.append(f" static VTX::PropertyKey<{accessor_type}> cached_key = accessor.Get<{accessor_type}>(EntityType::{struct_name}, {struct_name}::{prop_name});") + if user_ret_type != accessor_type: + lines.append(f" data_mut.Set<{accessor_type}>(cached_key, static_cast<{accessor_type}>(value));") + else: + lines.append(f" data_mut.Set<{accessor_type}>(cached_key, value);") + lines.append(" }") + lines.append("") + + def generate_cpp_header(json_path, output_path, namespace="Schema"): print(f"[VTX CodeGen] Reading schema from: {json_path}") - + try: with open(json_path, 'r', encoding='utf-8') as f: data = json.load(f) except Exception as e: print(f"[VTX CodeGen] Error reading JSON: {e}") sys.exit(1) - + type_map = { "int": "int32_t", "int8": "int32_t", @@ -116,61 +242,57 @@ def generate_cpp_header(json_path, output_path, namespace="Schema"): "#include ", "#include ", "#include \"vtx/common/vtx_property_cache.h\"", - "#include \"vtx/reader/core/vtx_frame_accessor.h\"", + "#include \"vtx/common/vtx_frame_accessor.h\"", + "#include \"vtx/writer/core/vtx_frame_mutation_view.h\"", "", f"namespace VTX::{namespace} {{", "" ] schema_list = normalize_schema_entries(data) - + lines.append(" /** @brief Strongly typed IDs for all entities in the schema */") lines.append(" enum class EntityType : int32_t {") - - known_structs = {} - + + known_structs = {} for i, struct_def in enumerate(schema_list): name = struct_def.get("struct", "") if name: known_structs[name.lower()] = name lines.append(f" {name} = {i},") - + lines.append(" Unknown = -1") lines.append(" };") lines.append("") for struct_def in schema_list: struct_name = struct_def.get("struct", "") - if not struct_name: continue - + if not struct_name: + continue + + # 1. Constexpr names + nested enums for list-typed properties. lines.append(f" namespace {struct_name} {{") lines.append(f" constexpr const char* StructName = \"{struct_name}\";") - + values = struct_def.get("values", []) - - # 1. Generate Constexpr names and Enums for val in values: prop_name = val.get("name", "") - if not prop_name: continue - + if not prop_name: + continue lines.append(f" constexpr const char* {prop_name} = \"{prop_name}\";") - - # Enum generation logic for list-based types raw_type_field = val.get("type", "") if isinstance(raw_type_field, list): lines.append(f" enum class E{prop_name} : int32_t {{") for enum_idx, enum_str in enumerate(raw_type_field): - clean_name = clean_enum_name(enum_str) - lines.append(f" {clean_name} = {enum_idx},") + clean = clean_enum_name(enum_str) + lines.append(f" {clean} = {enum_idx},") lines.append(" };") - lines.append(" }") lines.append("") - # 2. View Class + # 2. View class -- read-only. lines.append(f" class {struct_name}View {{") lines.append(" private:") - lines.append(" VTX::EntityView data_view;") lines.append(" const VTX::FrameAccessor& accessor;") lines.append("") @@ -181,97 +303,75 @@ def generate_cpp_header(json_path, output_path, namespace="Schema"): lines.append(f" {struct_name}View(const VTX::PropertyContainer& container, const VTX::FrameAccessor& acc) ") lines.append(f" : data_view(container), accessor(acc) {{}}") lines.append("") - - # 3. Generate Getters for val in values: - prop_name = val.get("name", "") - if not prop_name: continue - - raw_type_field = val.get("type", "") - is_list_enum = isinstance(raw_type_field, list) - - if is_list_enum: - raw_type = "implicit_enum" - else: - raw_type = str(raw_type_field).lower() - - struct_type = val.get("structType", "").lower() - container = val.get("container", "").lower() - - is_array = val.get("array", False) or (container == "array") - b_is_enum = val.get("bIsEnum", False) - b_is_actor = val.get("bIsActor", False) - - if raw_type in ["struct", "complexobject", "object", "none", ""] and struct_type: - lookup_type = struct_type - else: - lookup_type = raw_type - - lookup_type = lookup_type.replace("vtx::", "").replace("std::", "") - - accessor_type = "" - user_ret_type = "" - - if is_list_enum: - accessor_type = "int32_t" - user_ret_type = f"{struct_name}::E{prop_name}" - elif b_is_enum or lookup_type in ["byte", "uint8"]: - accessor_type = "int32_t" - user_ret_type = "uint8_t" - elif b_is_actor: - accessor_type = "int64_t" - user_ret_type = "int64_t" - elif lookup_type in known_structs: - accessor_type = "VTX::EntityView" - user_ret_type = "VTX::EntityView" - else: - accessor_type = type_map.get(lookup_type) - if not accessor_type: - print(f" [WARNING] Unknown type '{lookup_type}' in property '{prop_name}'. Using float by default.") - accessor_type = "float" - user_ret_type = accessor_type - - # Render Method - if accessor_type == "VTX::EntityView": - if is_array: - span_type = "VTX::PropertyContainer" - lines.append(f" /** @brief Returns a span of nested structures */") - lines.append(f" inline std::span Get{prop_name}() const {{") - lines.append(f" static VTX::PropertyKey> cached_key = accessor.GetViewArrayKey(EntityType::{struct_name}, {struct_name}::{prop_name});") - lines.append(f" return data_view.GetViewArray(cached_key);") - lines.append(" }") - else: - lines.append(f" /** @brief Returns a nested view */") - lines.append(f" inline VTX::EntityView Get{prop_name}() const {{") - lines.append(f" static VTX::PropertyKey cached_key = accessor.GetViewKey(EntityType::{struct_name}, {struct_name}::{prop_name});") - lines.append(f" return data_view.GetView(cached_key);") - lines.append(" }") - else: - if is_array: - span_ret_type = "uint8_t" if accessor_type == "bool" else accessor_type - lines.append(f" /** @brief Returns a span of {accessor_type} */") - lines.append(f" inline std::span Get{prop_name}() const {{") - lines.append(f" static VTX::PropertyKey<{accessor_type}> cached_key = accessor.GetArray<{accessor_type}>(EntityType::{struct_name}, {struct_name}::{prop_name});") - lines.append(f" return data_view.GetArray<{accessor_type}>(cached_key);") - lines.append(" }") - else: - ret_ref = "&" if user_ret_type in complex_types else "" - const_ref = f"const {user_ret_type}{ret_ref}" if user_ret_type in complex_types else user_ret_type - - lines.append(f" inline {const_ref} Get{prop_name}() const {{") - lines.append(f" static VTX::PropertyKey<{accessor_type}> cached_key = accessor.Get<{accessor_type}>(EntityType::{struct_name}, {struct_name}::{prop_name});") - - # Apply static cast if we need to return the generated Enum or a uint8_t - if user_ret_type != accessor_type: - lines.append(f" return static_cast<{user_ret_type}>(data_view.Get<{accessor_type}>(cached_key));") - else: - lines.append(f" return data_view.Get<{accessor_type}>(cached_key);") - lines.append(" }") - lines.append("") + if not val.get("name", ""): + continue + info = resolve_property_types(val, struct_name, known_structs, type_map) + emit_getter(lines, val, struct_name, info, complex_types, "data_view") + lines.append(" };") + lines.append("") + # 3. Mutator class -- read + write. + lines.append(f" class {struct_name}Mutator {{") + lines.append(" private:") + lines.append(" VTX::EntityMutator data_mut;") + lines.append(" const VTX::FrameAccessor& accessor;") + lines.append("") + lines.append(" public:") + lines.append(f" {struct_name}Mutator(VTX::EntityMutator m, const VTX::FrameAccessor& acc) ") + lines.append(f" : data_mut(m), accessor(acc) {{}}") + lines.append("") + lines.append(f" {struct_name}Mutator(VTX::PropertyContainer& container, const VTX::FrameAccessor& acc) ") + lines.append(f" : data_mut(container), accessor(acc) {{}}") + lines.append("") + for val in values: + if not val.get("name", ""): + continue + info = resolve_property_types(val, struct_name, known_structs, type_map) + emit_getter(lines, val, struct_name, info, complex_types, "data_mut") + emit_setter(lines, val, struct_name, info, complex_types) lines.append(" };") lines.append("") - + + # 4. Strongly-typed iteration helpers. Filter a BucketMutator by + # entity_type_id and yield the matching strongly-typed mutator. + lines.append(" // ----------------------------------------------------------------") + lines.append(" // Strongly-typed iteration helpers") + lines.append(" // ----------------------------------------------------------------") + lines.append(" // ForEachX(bucket, accessor, fn) walks a BucketMutator and calls") + lines.append(" // fn(XMutator&) only for entities whose entity_type_id matches the") + lines.append(" // struct. Read-only counterparts (XView) are available via") + lines.append(" // ForEachXView.") + lines.append("") + for struct_def in schema_list: + struct_name = struct_def.get("struct", "") + if not struct_name: + continue + lines.append(f" template ") + lines.append(f" void ForEach{struct_name}(VTX::BucketMutator& bucket, const VTX::FrameAccessor& accessor, Fn fn) {{") + lines.append(f" constexpr int32_t kTypeId = static_cast(EntityType::{struct_name});") + lines.append(f" for (auto entity : bucket) {{") + lines.append(f" const auto* raw = entity.raw();") + lines.append(f" if (raw && raw->entity_type_id == kTypeId) {{") + lines.append(f" {struct_name}Mutator obj(entity, accessor);") + lines.append(f" fn(obj);") + lines.append(f" }}") + lines.append(f" }}") + lines.append(f" }}") + lines.append("") + + lines.append(f" template ") + lines.append(f" void ForEach{struct_name}View(const VTX::Bucket& bucket, const VTX::FrameAccessor& accessor, Fn fn) {{") + lines.append(f" constexpr int32_t kTypeId = static_cast(EntityType::{struct_name});") + lines.append(f" for (const auto& container : bucket.entities) {{") + lines.append(f" if (container.entity_type_id == kTypeId) {{") + lines.append(f" {struct_name}View obj(container, accessor);") + lines.append(f" fn(obj);") + lines.append(f" }}") + lines.append(f" }}") + lines.append(f" }}") + lines.append("") + lines.append(f"}} // namespace VTX::{namespace}") output_content = "\n".join(lines) diff --git a/sdk/include/vtx/reader/core/vtx_frame_accessor.h b/sdk/include/vtx/common/vtx_frame_accessor.h similarity index 100% rename from sdk/include/vtx/reader/core/vtx_frame_accessor.h rename to sdk/include/vtx/common/vtx_frame_accessor.h diff --git a/sdk/include/vtx/reader/core/vtx_reader.h b/sdk/include/vtx/reader/core/vtx_reader.h index 8a50b19..375b137 100644 --- a/sdk/include/vtx/reader/core/vtx_reader.h +++ b/sdk/include/vtx/reader/core/vtx_reader.h @@ -20,7 +20,7 @@ #include "vtx/common/vtx_types.h" #include "vtx/common/vtx_concepts.h" #include "vtx/reader/core/vtx_schema_adapter.h" -#include "vtx_frame_accessor.h" +#include "vtx/common/vtx_frame_accessor.h" #include "vtx/reader/serialization/flatbuffers_to_vtx.h" namespace VTX { @@ -406,10 +406,6 @@ namespace VTX { } void ReadFooter(std::ifstream& stream) { - // Fix A1: the original code declared `uint32_t footer_size;` and - // used it after `stream.read()` without checking `gcount()`. If - // the file is too short to contain the trailing 8-byte footer - // size, we would seek to a garbage offset and crash. const int64_t file_size = GetStreamSize(stream); if (file_size < 8) { throw std::runtime_error("VTX file too small to contain footer"); @@ -422,9 +418,6 @@ namespace VTX { throw std::runtime_error("VTX file truncated while reading footer size"); } - // Fix A3: guard against an implausible footer_size (either - // malicious UINT32_MAX or garbage from a corrupt trailer) that - // would seek before the beginning of the file. if (footer_size == 0 || static_cast(footer_size) + 8 > file_size) { throw std::runtime_error("VTX file has implausible footer size"); } @@ -567,7 +560,6 @@ namespace VTX { } const bool load_succeeded = thread_survived && !data.native_frames.empty(); - { std::lock_guard lock(cache_mutex_); if (!stop_token.stop_requested()) { diff --git a/sdk/include/vtx/writer/core/vtx_frame_mutation_view.h b/sdk/include/vtx/writer/core/vtx_frame_mutation_view.h new file mode 100644 index 0000000..ce0b89e --- /dev/null +++ b/sdk/include/vtx/writer/core/vtx_frame_mutation_view.h @@ -0,0 +1,261 @@ +#pragma once + +#include +#include +#include +#include + +#include "vtx/common/vtx_concepts.h" +#include "vtx/common/vtx_frame_accessor.h" +#include "vtx/common/vtx_types.h" + +namespace VTX { + + class EntityMutator { + public: + EntityMutator() = default; + explicit EntityMutator(PropertyContainer& data) + : data_(&data) {} + + template + typename EntityView::template ScalarRetType Get(PropertyKey key) const { + return AsView().Get(key); + } + + template + void Set(PropertyKey key, T value) { + if (!data_ || !key.IsValid()) + return; + constexpr auto MemberPtr = EntityView::GetContainerMember(); + auto& values = data_->*MemberPtr; + if (static_cast(key.index) >= values.size()) + return; + values[key.index] = std::move(value); + } + + EntityMutator GetMutableView(PropertyKey key) { + if (!data_ || !key.IsValid()) + return {}; + if (static_cast(key.index) >= data_->any_struct_properties.size()) + return {}; + return EntityMutator(data_->any_struct_properties[key.index]); + } + + std::span GetMutableViewArray(PropertyKey> key) { + if (!data_ || !key.IsValid()) + return {}; + return data_->any_struct_arrays.GetMutableSubArray(key.index); + } + + template + auto GetMutableArray(PropertyKey key) { + constexpr auto MemberPtr = EntityView::GetArrayContainerMember(); + using SpanType = decltype((data_->*MemberPtr).GetMutableSubArray(0)); + if (!data_ || !key.IsValid()) + return SpanType {}; + return (data_->*MemberPtr).GetMutableSubArray(key.index); + } + + EntityView AsView() const { return data_ ? EntityView(*data_) : EntityView(); } + + PropertyContainer* raw() noexcept { return data_; } + const PropertyContainer* raw() const noexcept { return data_; } + bool valid() const noexcept { return data_ != nullptr; } + + private: + PropertyContainer* data_ = nullptr; + }; + + class BucketMutator { + public: + BucketMutator() = default; + explicit BucketMutator(Bucket& bucket) + : bucket_(&bucket) {} + + size_t entity_count() const noexcept { return bucket_ ? bucket_->entities.size() : 0; } + + EntityMutator entity(uint32_t i) { + if (!bucket_ || i >= bucket_->entities.size()) + return {}; + return EntityMutator(bucket_->entities[i]); + } + + EntityView entity_view(uint32_t i) const { + if (!bucket_ || i >= bucket_->entities.size()) + return {}; + return EntityView(bucket_->entities[i]); + } + + class iterator { + public: + iterator() = default; + iterator(Bucket* b, size_t i) + : bucket_(b) + , idx_(i) {} + + EntityMutator operator*() const { return EntityMutator(bucket_->entities[idx_]); } + iterator& operator++() { + ++idx_; + return *this; + } + bool operator==(const iterator& o) const noexcept { return bucket_ == o.bucket_ && idx_ == o.idx_; } + bool operator!=(const iterator& o) const noexcept { return !(*this == o); } + + private: + Bucket* bucket_ = nullptr; + size_t idx_ = 0; + }; + + iterator begin() { return bucket_ ? iterator {bucket_, 0} : iterator {}; } + iterator end() { return bucket_ ? iterator {bucket_, bucket_->entities.size()} : iterator {}; } + + std::span entities_of_type(int32_t type_id) { + if (!bucket_) + return {}; + if (type_id < 0 || static_cast(type_id) >= bucket_->type_ranges.size()) + return {}; + const auto& range = bucket_->type_ranges[type_id]; + if (range.count <= 0) + return {}; + return std::span(bucket_->entities.data() + range.start_index, + static_cast(range.count)); + } + + EntityMutator AddEntity() { + if (!bucket_) + return {}; + bucket_->entities.emplace_back(); + if (bucket_->unique_ids.size() + 1 == bucket_->entities.size()) + bucket_->unique_ids.emplace_back(); + return EntityMutator(bucket_->entities.back()); + } + + void RemoveEntity(uint32_t entity_index) { + if (!bucket_ || entity_index >= bucket_->entities.size()) + return; + bucket_->entities.erase(bucket_->entities.begin() + entity_index); + if (entity_index < bucket_->unique_ids.size()) + bucket_->unique_ids.erase(bucket_->unique_ids.begin() + entity_index); + for (auto& r : bucket_->type_ranges) { + if (r.count <= 0) + continue; + const int32_t end = r.start_index + r.count; + if (static_cast(entity_index) < r.start_index) { + --r.start_index; + } else if (static_cast(entity_index) < end) { + --r.count; + } + } + } + + template + size_t RemoveIf(Predicate pred) { + if (!bucket_) + return 0; + const size_t before = bucket_->entities.size(); + std::vector keep(before, 1); + for (size_t i = 0; i < before; ++i) { + if (pred(EntityView(bucket_->entities[i]))) + keep[i] = 0; + } + size_t write = 0; + for (size_t read = 0; read < before; ++read) { + if (!keep[read]) + continue; + if (write != read) { + bucket_->entities[write] = std::move(bucket_->entities[read]); + if (read < bucket_->unique_ids.size() && write < bucket_->unique_ids.size()) + bucket_->unique_ids[write] = std::move(bucket_->unique_ids[read]); + } + ++write; + } + bucket_->entities.resize(write); + if (bucket_->unique_ids.size() > write) + bucket_->unique_ids.resize(write); + for (auto& r : bucket_->type_ranges) { + r.start_index = 0; + r.count = 0; + } + return before - write; + } + + void Clear() { + if (!bucket_) + return; + bucket_->entities.clear(); + bucket_->unique_ids.clear(); + for (auto& r : bucket_->type_ranges) { + r.start_index = 0; + r.count = 0; + } + } + + Bucket* raw() noexcept { return bucket_; } + const Bucket* raw() const noexcept { return bucket_; } + bool valid() const noexcept { return bucket_ != nullptr; } + + private: + Bucket* bucket_ = nullptr; + }; + + class FrameMutationView { + public: + FrameMutationView() = default; + FrameMutationView(Frame& frame, const FrameAccessor& accessor) + : frame_(&frame) + , accessor_(&accessor) {} + + bool HasBucket(const std::string& name) const { + if (!frame_) + return false; + return frame_->bucket_map.contains(name); + } + + BucketMutator GetBucket(const std::string& name) { + if (!frame_) + return {}; + auto it = frame_->bucket_map.find(name); + if (it == frame_->bucket_map.end()) + return {}; + return BucketMutator(frame_->buckets[it->second]); + } + + BucketMutator GetBucket(int32_t bucket_index) { + if (!frame_ || bucket_index < 0 || static_cast(bucket_index) >= frame_->buckets.size()) + return {}; + return BucketMutator(frame_->buckets[static_cast(bucket_index)]); + } + + size_t bucket_count() const noexcept { return frame_ ? frame_->buckets.size() : 0; } + + template + PropertyKey ResolveKey(const std::string& struct_name, const std::string& prop_name) const { + if (!accessor_) + return PropertyKey {-1}; + return accessor_->Get(struct_name, prop_name); + } + + template + PropertyKey ResolveArrayKey(const std::string& struct_name, const std::string& prop_name) const { + if (!accessor_) + return PropertyKey {-1}; + return accessor_->GetArray(struct_name, prop_name); + } + + PropertyKey ResolveViewKey(const std::string& struct_name, const std::string& prop_name) const { + if (!accessor_) + return PropertyKey {-1}; + return accessor_->GetViewKey(struct_name, prop_name); + } + + const Frame* AsConstFrame() const noexcept { return frame_; } + Frame* raw() noexcept { return frame_; } + const FrameAccessor* accessor() const noexcept { return accessor_; } + bool valid() const noexcept { return frame_ != nullptr && accessor_ != nullptr; } + + private: + Frame* frame_ = nullptr; + const FrameAccessor* accessor_ = nullptr; + }; + +} // namespace VTX diff --git a/sdk/include/vtx/writer/core/vtx_frame_post_processor.h b/sdk/include/vtx/writer/core/vtx_frame_post_processor.h new file mode 100644 index 0000000..633f53c --- /dev/null +++ b/sdk/include/vtx/writer/core/vtx_frame_post_processor.h @@ -0,0 +1,100 @@ +#pragma once + +#include +#include +#include + +#include "vtx/common/vtx_frame_accessor.h" +#include "vtx/writer/core/vtx_frame_mutation_view.h" + +namespace VTX { + + struct FramePostProcessorInitContext { + const FrameAccessor* frame_accessor = nullptr; + int32_t total_frames = 0; + uint32_t schema_version = 0; + uint32_t format_major = 0; + uint32_t format_minor = 0; + }; + + struct FramePostProcessContext { + int32_t global_frame_index = 0; + int32_t chunk_local_frame_index = 0; + int32_t chunk_index = 0; + uint32_t schema_version = 0; + + const FrameAccessor* frame_accessor = nullptr; + const Frame* previous_frame = nullptr; + }; + + class IFramePostProcessor { + public: + virtual ~IFramePostProcessor() = default; + + virtual void Init(const FramePostProcessorInitContext& context) {} + virtual void Clear() {} + virtual void PrintInfo() const {} + + virtual void Process(FrameMutationView& view, const FramePostProcessContext& ctx) = 0; + }; + + class FramePostProcessorChain final : public IFramePostProcessor { + public: + void Add(std::shared_ptr p) { + if (p) + processors_.push_back(std::move(p)); + } + + bool Remove(const std::shared_ptr& p) { + for (auto it = processors_.begin(); it != processors_.end(); ++it) { + if (*it == p) { + processors_.erase(it); + return true; + } + } + return false; + } + + size_t size() const noexcept { return processors_.size(); } + bool empty() const noexcept { return processors_.empty(); } + + void Init(const FramePostProcessorInitContext& ctx) override { + for (auto& p : processors_) { + if (p) + p->Init(ctx); + } + } + + void Process(FrameMutationView& view, const FramePostProcessContext& ctx) override { + for (auto& p : processors_) { + if (p) + p->Process(view, ctx); + } + } + + void PrintInfo() const override { + for (const auto& p : processors_) { + if (p) + p->PrintInfo(); + } + } + + void Clear() override { + for (auto it = processors_.rbegin(); it != processors_.rend(); ++it) { + if (*it) + (*it)->Clear(); + } + } + + void ClearChain() noexcept { processors_.clear(); } + + void ClearMembers() { + Clear(); + ClearChain(); + } + + private: + std::vector> processors_; + }; + +} // namespace VTX diff --git a/sdk/include/vtx/writer/core/vtx_writer_facade.h b/sdk/include/vtx/writer/core/vtx_writer_facade.h index f10501a..cf7dc26 100644 --- a/sdk/include/vtx/writer/core/vtx_writer_facade.h +++ b/sdk/include/vtx/writer/core/vtx_writer_facade.h @@ -6,6 +6,7 @@ #include "schema_sanitizer.h" #include "vtx/common/vtx_types.h" #include "vtx/common/readers/schema_reader/schema_registry.h" +#include "vtx/writer/core/vtx_frame_post_processor.h" namespace VTX { @@ -20,6 +21,15 @@ namespace VTX { virtual void Flush() = 0; virtual void Stop() = 0; virtual VTX::SchemaRegistry& GetSchema() = 0; + + // The processor's Process() runs on every RecordFrame() call, + // after timer validation and BEFORE serialisation to disk: its + // mutations are what end up in the .vtx file. Call before + // RecordFrame to take effect; safe to swap mid-recording (new + // processor takes over on the next RecordFrame). + virtual void SetPostProcessor(std::shared_ptr processor) = 0; + virtual std::shared_ptr GetPostProcessor() const = 0; + virtual void ClearPostProcessor() = 0; }; struct WriterFacadeConfig { diff --git a/sdk/include/vtx/writer/core/writer.h b/sdk/include/vtx/writer/core/writer.h index b465198..7902be9 100644 --- a/sdk/include/vtx/writer/core/writer.h +++ b/sdk/include/vtx/writer/core/writer.h @@ -3,6 +3,9 @@ #include #include #include "vtx/common/vtx_types.h" +#include "vtx/common/vtx_frame_accessor.h" +#include "vtx/writer/core/vtx_frame_mutation_view.h" +#include "vtx/writer/core/vtx_frame_post_processor.h" namespace VTX { @@ -45,6 +48,15 @@ namespace VTX { registry_.LoadFromJson(config.schema_json_path); auto schema = Serializer::CreateSchema(registry_); sink_.OnSessionStart(schema); + frame_accessor_.InitializeFromCache(registry_.GetPropertyCache()); + } + + ~ReplayWriter() { + if (post_processor_) { + try { + post_processor_->Clear(); + } catch (...) {} + } } void RecordFrame(VTX::Frame& native_frame, const VTX::GameTime::GameTimeRegister& game_time_register) { @@ -61,6 +73,22 @@ namespace VTX { return; } + if (post_processor_) { + FrameMutationView view(native_frame, frame_accessor_); + FramePostProcessContext ctx; + ctx.global_frame_index = total_frames_; + ctx.chunk_local_frame_index = static_cast(pending_frames_.size()); + ctx.chunk_index = chunks_flushed_; + ctx.schema_version = 0; + ctx.frame_accessor = &frame_accessor_; + ctx.previous_frame = nullptr; + try { + post_processor_->Process(view, ctx); + } catch (const std::exception& e) { + (void)e; + } catch (...) {} + } + std::unique_ptr sink_frame = Serializer::FromNative(std::move(native_frame)); size_t frameSize = Serializer::GetFrameSize(*sink_frame); @@ -84,6 +112,7 @@ namespace VTX { pending_frames_.clear(); current_chunk_bytes_ = 0; + ++chunks_flushed_; timer_.UpdateChunkStartIndex(); } @@ -108,6 +137,35 @@ namespace VTX { VTX::SchemaRegistry& GetRegistry() { return registry_; } + void SetPostProcessor(std::shared_ptr processor) { + if (processor) { + FramePostProcessorInitContext init_ctx; + init_ctx.frame_accessor = &frame_accessor_; + init_ctx.total_frames = 0; + init_ctx.schema_version = 0; + init_ctx.format_major = 0; + init_ctx.format_minor = 0; + try { + processor->Init(init_ctx); + } catch (...) { + throw; + } + } + post_processor_ = std::move(processor); + } + + std::shared_ptr GetPostProcessor() const { return post_processor_; } + + void ClearPostProcessor() { + auto outgoing = std::move(post_processor_); + post_processor_.reset(); + if (outgoing) { + try { + outgoing->Clear(); + } catch (...) {} + } + } + private: SinkPolicy sink_; ChunkingPolicy chunker_; @@ -118,5 +176,9 @@ namespace VTX { VTX::GameTime::VTXGameTimes timer_; size_t current_chunk_bytes_ = 0; int32_t total_frames_ = 0; + int32_t chunks_flushed_ = 0; + + FrameAccessor frame_accessor_ = {}; + std::shared_ptr post_processor_; }; } // namespace VTX \ No newline at end of file diff --git a/sdk/src/vtx_reader/src/vtx/reader/core/vtx_reader_facade.cpp b/sdk/src/vtx_reader/src/vtx/reader/core/vtx_reader_facade.cpp index 30c98e7..da7197b 100644 --- a/sdk/src/vtx_reader/src/vtx/reader/core/vtx_reader_facade.cpp +++ b/sdk/src/vtx_reader/src/vtx/reader/core/vtx_reader_facade.cpp @@ -110,12 +110,17 @@ namespace VTX { } bool IsReady() const override { return InternalReader.IsReady(); } + bool IsReadyFailed() const override { return InternalReader.IsReadyFailed(); } + std::string GetReadyError() const override { return InternalReader.GetReadyError(); } + bool WaitUntilReady() override { return InternalReader.WaitUntilReady(); } + bool WaitUntilReady(std::chrono::milliseconds timeout) override { return InternalReader.WaitUntilReady(timeout); } + void MarkReadyVacuous() override { InternalReader.MarkReadyVacuous(); } private: @@ -180,12 +185,17 @@ namespace VTX { } bool IsReady() const override { return InternalReader.IsReady(); } + bool IsReadyFailed() const override { return InternalReader.IsReadyFailed(); } + std::string GetReadyError() const override { return InternalReader.GetReadyError(); } + bool WaitUntilReady() override { return InternalReader.WaitUntilReady(); } + bool WaitUntilReady(std::chrono::milliseconds timeout) override { return InternalReader.WaitUntilReady(timeout); } + void MarkReadyVacuous() override { InternalReader.MarkReadyVacuous(); } private: @@ -254,22 +264,6 @@ namespace VTX { }; result.reader->SetEvents(events); - // Eagerly warm chunk 0 so callers can poll IsReady(), block - // via WaitUntilReady(), or register OnReady / OnReadyFailed - // on a subsequent SetEvents. WarmAt() dispatches the load - // asynchronously, so OpenReplayFile's own return latency - // is unchanged. Empty replays (0 frames) get a vacuous - // "ready" flip so waiters / pollers don't hang forever. - // - // We narrow the cache window to (0, 0) around the eager - // warm so ONLY chunk 0 is loaded -- not the default-window - // forward neighbours. Loading extra chunks on every open - // would quietly break callers that set a narrow cache - // window immediately after OpenReplayFile() (e.g. memory- - // constrained tools, tests that isolate a single chunk). - // The window is restored to the reader's default right - // after, and nothing external holds a handle to the reader - // yet, so the temporary narrow is unobservable. if (result.reader->GetTotalFrames() > 0) { result.reader->SetCacheWindow(0, 0); result.reader->WarmAt(0); diff --git a/sdk/src/vtx_writer/src/vtx/writer/core/vtx_writer_facade.cpp b/sdk/src/vtx_writer/src/vtx/writer/core/vtx_writer_facade.cpp index f04fe3d..88dd460 100644 --- a/sdk/src/vtx_writer/src/vtx/writer/core/vtx_writer_facade.cpp +++ b/sdk/src/vtx_writer/src/vtx/writer/core/vtx_writer_facade.cpp @@ -16,9 +16,7 @@ namespace VTX { WriterFacadeImpl(typename ReplayWriter::Config internal_config) : writer_(internal_config) {} - // RecordFrame / Flush / Stop are no-ops after the first Stop(). The - // on-disk file was finalised (header + chunks + footer) there, and - // letting further writes through would rewrite/corrupt the footer. + void RecordFrame(VTX::Frame& native_frame, const VTX::GameTime::GameTimeRegister& game_time_register) override { if (stopped_) return; @@ -33,13 +31,19 @@ namespace VTX { void Stop() override { if (stopped_) - return; // idempotent: second Stop() is a no-op + return; // second Stop() is a no-op writer_.Stop(); stopped_ = true; } VTX::SchemaRegistry& GetSchema() override { return writer_.GetRegistry(); } + void SetPostProcessor(std::shared_ptr processor) override { + writer_.SetPostProcessor(std::move(processor)); + } + std::shared_ptr GetPostProcessor() const override { return writer_.GetPostProcessor(); } + void ClearPostProcessor() override { writer_.ClearPostProcessor(); } + private: ReplayWriter writer_; bool stopped_ = false; @@ -81,4 +85,4 @@ namespace VTX { internal_cfg.sink_config.b_use_compression = config.use_compression; return std::make_unique>(internal_cfg); } -} // namespace VTX \ No newline at end of file +} // namespace VTX diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 13ca183..ad40cce 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -68,6 +68,7 @@ add_executable(vtx_tests writer/test_writer_edges.cpp writer/test_roundtrip.cpp writer/test_writer_advanced.cpp + writer/test_frame_post_processor.cpp reader/test_reader_context.cpp reader/test_corrupt_files.cpp diff --git a/tests/common/test_frame_accessor.cpp b/tests/common/test_frame_accessor.cpp index d10c470..3c94d59 100644 --- a/tests/common/test_frame_accessor.cpp +++ b/tests/common/test_frame_accessor.cpp @@ -5,7 +5,7 @@ #include #include -#include "vtx/reader/core/vtx_frame_accessor.h" +#include "vtx/common/vtx_frame_accessor.h" namespace { diff --git a/tests/writer/test_frame_post_processor.cpp b/tests/writer/test_frame_post_processor.cpp new file mode 100644 index 0000000..f23dd08 --- /dev/null +++ b/tests/writer/test_frame_post_processor.cpp @@ -0,0 +1,336 @@ +#include + +#include +#include +#include +#include + +#include "vtx/common/vtx_types.h" +#include "vtx/writer/core/vtx_frame_post_processor.h" +#include "vtx/writer/core/vtx_frame_mutation_view.h" +#include "vtx/reader/core/vtx_reader_facade.h" +#include "vtx/writer/core/vtx_writer_facade.h" + +#include "util/test_fixtures.h" + +namespace { + + constexpr int kPlayersPerFrame = 3; + + VTX::Frame MakePlayerFrame(int frame_index) { + VTX::Frame f; + auto& bucket = f.CreateBucket("entity"); + for (int p = 0; p < kPlayersPerFrame; ++p) { + VTX::PropertyContainer pc; + pc.entity_type_id = 0; + pc.string_properties = {"player_" + std::to_string(p), "Alpha_" + std::to_string(p)}; + pc.int32_properties = {p % 2 == 0 ? 1 : 2, frame_index * 10 + p, 0}; + pc.float_properties = {100.0f - float(frame_index), 50.0f}; + pc.vector_properties = {VTX::Vector {double(frame_index), double(p), 0.0}, VTX::Vector {}}; + pc.quat_properties = {VTX::Quat {0.0f, 0.0f, 0.0f, 1.0f}}; + pc.bool_properties = {true}; + + bucket.unique_ids.push_back("player_" + std::to_string(p)); + bucket.entities.push_back(std::move(pc)); + } + return f; + } + + std::string OutPath(const std::string& uuid) { + return VtxTest::OutputPath("writer_post_processor_" + uuid + ".vtx"); + } + + VTX::WriterFacadeConfig MakeConfig(const std::string& uuid) { + VTX::WriterFacadeConfig cfg; + cfg.output_filepath = OutPath(uuid); + cfg.schema_json_path = VtxTest::FixturePath("test_schema.json"); + cfg.replay_name = "WriterPostProcessorTest"; + cfg.replay_uuid = uuid; + cfg.default_fps = 60.0f; + cfg.chunk_max_frames = 8; + cfg.use_compression = true; + return cfg; + } + + void RunWriter(VTX::IVtxWriterFacade& writer, int frames) { + for (int i = 0; i < frames; ++i) { + auto frame = MakePlayerFrame(i); + VTX::GameTime::GameTimeRegister t; + t.game_time = float(i) / 60.0f; + writer.RecordFrame(frame, t); + } + writer.Flush(); + writer.Stop(); + } + + + class DoubleHealthProcessor : public VTX::IFramePostProcessor { + public: + void Init(const VTX::FramePostProcessorInitContext& ctx) override { + init_called_ = true; + ASSERT_NE(ctx.frame_accessor, nullptr); + health_key_ = ctx.frame_accessor->Get("Player", "Health"); + } + void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext& ctx) override { + ++process_calls_; + last_global_frame_ = ctx.global_frame_index; + if (!health_key_.IsValid()) + return; + auto bucket = view.GetBucket("entity"); + for (auto entity : bucket) { + entity.Set(health_key_, entity.Get(health_key_) * 2.0f); + } + } + void Clear() override { clear_called_ = true; } + + bool init_called_ = false; + bool clear_called_ = false; + int process_calls_ = 0; + int32_t last_global_frame_ = -1; + VTX::PropertyKey health_key_ {-1}; + }; + + class ScoreSetter : public VTX::IFramePostProcessor { + public: + explicit ScoreSetter(int32_t tag) + : tag_(tag) {} + void Init(const VTX::FramePostProcessorInitContext& ctx) override { + score_key_ = ctx.frame_accessor->Get("Player", "Score"); + } + void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext&) override { + if (!score_key_.IsValid()) + return; + auto bucket = view.GetBucket("entity"); + for (auto entity : bucket) { + entity.Set(score_key_, tag_); + } + } + int32_t tag_; + VTX::PropertyKey score_key_ {-1}; + }; + + class GhostInjector : public VTX::IFramePostProcessor { + public: + void Init(const VTX::FramePostProcessorInitContext& ctx) override { + team_key_ = ctx.frame_accessor->Get("Player", "Team"); + health_key_ = ctx.frame_accessor->Get("Player", "Health"); + } + void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext&) override { + auto bucket = view.GetBucket("entity"); + auto ghost = bucket.AddEntity(); + if (auto* raw = ghost.raw()) { + raw->entity_type_id = 0; + raw->int32_properties.resize(3, 0); + raw->float_properties.resize(2, 0.0f); + } + if (team_key_.IsValid()) + ghost.Set(team_key_, 99); + if (health_key_.IsValid()) + ghost.Set(health_key_, 1.0f); + } + VTX::PropertyKey team_key_ {-1}; + VTX::PropertyKey health_key_ {-1}; + }; + + class TeamTwoFilter : public VTX::IFramePostProcessor { + public: + void Init(const VTX::FramePostProcessorInitContext& ctx) override { + team_key_ = ctx.frame_accessor->Get("Player", "Team"); + } + void Process(VTX::FrameMutationView& view, const VTX::FramePostProcessContext&) override { + if (!team_key_.IsValid()) + return; + auto bucket = view.GetBucket("entity"); + bucket.RemoveIf([this](VTX::EntityView e) { return e.Get(team_key_) == 2; }); + } + VTX::PropertyKey team_key_ {-1}; + }; + + class FrameIndexRecorder : public VTX::IFramePostProcessor { + public: + void Process(VTX::FrameMutationView&, const VTX::FramePostProcessContext& ctx) override { + indices_.push_back(ctx.global_frame_index); + } + std::vector indices_; + }; + +} // namespace + + +TEST(WriterPostProcessor_MutationViewUnit, SetThenGetRoundTrips) { + VTX::PropertyContainer pc; + pc.float_properties = {0.0f}; + VTX::EntityMutator m(pc); + VTX::PropertyKey key {0}; + m.Set(key, 42.5f); + EXPECT_FLOAT_EQ(m.Get(key), 42.5f); +} + +TEST(WriterPostProcessor_ChainUnit, OrderAndRemove) { + VTX::FramePostProcessorChain chain; + auto a = std::make_shared(1); + auto b = std::make_shared(2); + chain.Add(a); + chain.Add(b); + EXPECT_EQ(chain.size(), 2u); + EXPECT_TRUE(chain.Remove(a)); + EXPECT_EQ(chain.size(), 1u); + EXPECT_FALSE(chain.Remove(a)); +} + + +class WriterPostProcessorTest : public ::testing::Test { +protected: + std::string TestUuid() const { + const auto* info = ::testing::UnitTest::GetInstance()->current_test_info(); + return VtxTest::SanitizePathComponent(info->name()); + } +}; + + +TEST_F(WriterPostProcessorTest, NoProcessorBaselineUnchanged) { + const auto cfg = MakeConfig(TestUuid()); + { + auto writer = VTX::CreateFlatBuffersWriterFacade(cfg); + RunWriter(*writer, 5); + } + auto ctx = VTX::OpenReplayFile(cfg.output_filepath); + ASSERT_TRUE(ctx) << ctx.error; + const auto* frame = ctx.reader->GetFrameSync(0); + ASSERT_NE(frame, nullptr); + EXPECT_FLOAT_EQ(frame->GetBuckets()[0].entities[0].float_properties[0], 100.0f); +} + +TEST_F(WriterPostProcessorTest, DoubleHealthIsPersistedToDisk) { + const auto cfg = MakeConfig(TestUuid()); + auto proc = std::make_shared(); + { + auto writer = VTX::CreateFlatBuffersWriterFacade(cfg); + writer->SetPostProcessor(proc); + EXPECT_TRUE(proc->init_called_); + EXPECT_TRUE(proc->health_key_.IsValid()) << "Init must have resolved the Health key against the loaded schema"; + RunWriter(*writer, 5); + } + EXPECT_TRUE(proc->clear_called_); + EXPECT_EQ(proc->process_calls_, 5); + + auto rctx = VTX::OpenReplayFile(cfg.output_filepath); + ASSERT_TRUE(rctx) << rctx.error; + const auto* frame = rctx.reader->GetFrameSync(0); + ASSERT_NE(frame, nullptr); + EXPECT_FLOAT_EQ(frame->GetBuckets()[0].entities[0].float_properties[0], 200.0f); + EXPECT_FLOAT_EQ(frame->GetBuckets()[0].entities[1].float_properties[0], 200.0f); +} + +TEST_F(WriterPostProcessorTest, ChainLastWriterWinsOnDisk) { + const auto cfg = MakeConfig(TestUuid()); + { + auto writer = VTX::CreateFlatBuffersWriterFacade(cfg); + auto chain = std::make_shared(); + chain->Add(std::make_shared(11)); + chain->Add(std::make_shared(22)); + writer->SetPostProcessor(chain); + RunWriter(*writer, 3); + } + auto rctx = VTX::OpenReplayFile(cfg.output_filepath); + ASSERT_TRUE(rctx) << rctx.error; + const auto* frame = rctx.reader->GetFrameSync(0); + ASSERT_NE(frame, nullptr); + EXPECT_EQ(frame->GetBuckets()[0].entities[0].int32_properties[1], 22); +} + +TEST_F(WriterPostProcessorTest, ChainRemoveDropsAndOtherStillFires) { + const auto cfg = MakeConfig(TestUuid()); + { + auto writer = VTX::CreateFlatBuffersWriterFacade(cfg); + auto chain = std::make_shared(); + auto a = std::make_shared(11); + auto b = std::make_shared(22); + chain->Add(a); + chain->Add(b); + chain->Remove(a); + writer->SetPostProcessor(chain); + RunWriter(*writer, 3); + } + auto rctx = VTX::OpenReplayFile(cfg.output_filepath); + ASSERT_TRUE(rctx) << rctx.error; + const auto* frame = rctx.reader->GetFrameSync(0); + ASSERT_NE(frame, nullptr); + EXPECT_EQ(frame->GetBuckets()[0].entities[0].int32_properties[1], 22); +} + +TEST_F(WriterPostProcessorTest, GhostInjectorEntityIsOnDisk) { + const auto cfg = MakeConfig(TestUuid()); + { + auto writer = VTX::CreateFlatBuffersWriterFacade(cfg); + writer->SetPostProcessor(std::make_shared()); + RunWriter(*writer, 3); + } + auto rctx = VTX::OpenReplayFile(cfg.output_filepath); + ASSERT_TRUE(rctx) << rctx.error; + const auto* frame = rctx.reader->GetFrameSync(0); + ASSERT_NE(frame, nullptr); + const auto& entities = frame->GetBuckets()[0].entities; + EXPECT_EQ(entities.size(), kPlayersPerFrame + 1); + EXPECT_EQ(entities.back().int32_properties[0], 99); + EXPECT_FLOAT_EQ(entities.back().float_properties[0], 1.0f); +} + +TEST_F(WriterPostProcessorTest, TeamTwoFilterDropsEntitiesFromDisk) { + const auto cfg = MakeConfig(TestUuid()); + { + auto writer = VTX::CreateFlatBuffersWriterFacade(cfg); + writer->SetPostProcessor(std::make_shared()); + RunWriter(*writer, 3); + } + auto rctx = VTX::OpenReplayFile(cfg.output_filepath); + ASSERT_TRUE(rctx) << rctx.error; + const auto* frame = rctx.reader->GetFrameSync(0); + ASSERT_NE(frame, nullptr); + const auto& entities = frame->GetBuckets()[0].entities; + EXPECT_EQ(entities.size(), 2u); + for (const auto& e : entities) { + EXPECT_NE(e.int32_properties[0], 2); + } +} + +TEST_F(WriterPostProcessorTest, GlobalFrameIndexIsMonotonic) { + const auto cfg = MakeConfig(TestUuid()); + auto recorder = std::make_shared(); + { + auto writer = VTX::CreateFlatBuffersWriterFacade(cfg); + writer->SetPostProcessor(recorder); + RunWriter(*writer, 7); + } + ASSERT_EQ(recorder->indices_.size(), 7u); + for (size_t i = 0; i < recorder->indices_.size(); ++i) { + EXPECT_EQ(recorder->indices_[i], static_cast(i)); + } +} + +TEST_F(WriterPostProcessorTest, ClearPostProcessorCallsClearAndUnregisters) { + const auto cfg = MakeConfig(TestUuid()); + auto proc = std::make_shared(); + auto writer = VTX::CreateFlatBuffersWriterFacade(cfg); + writer->SetPostProcessor(proc); + + for (int i = 0; i < 2; ++i) { + auto f = MakePlayerFrame(i); + VTX::GameTime::GameTimeRegister t; + t.game_time = float(i) / 60.0f; + writer->RecordFrame(f, t); + } + writer->ClearPostProcessor(); + EXPECT_TRUE(proc->clear_called_); + EXPECT_EQ(writer->GetPostProcessor(), nullptr); + + const int calls_before = proc->process_calls_; + auto f = MakePlayerFrame(2); + VTX::GameTime::GameTimeRegister t; + t.game_time = 2.0f / 60.0f; + writer->RecordFrame(f, t); + EXPECT_EQ(proc->process_calls_, calls_before); + + writer->Flush(); + writer->Stop(); +}