feat(writer): Added frame post processing. Frame post processing allo…#12
Merged
Merged
Conversation
…ws to sanitize, mofidy, infer or remove data before it goes inside of VTX. It runs in the writer during the ProcessFrame, implemented as single post process or a a chain of post process. Call SetPostPRocessor from the writer facade and pass the passing a FramePostProcessorChain previusly filled with IFramePostProcessor To achive this vtx_codegen.py is extended to generate entityview mutators to easily modify values by SetHealth,SetArmor,etc. Removing the need of knowing and harcoding strings in cpp. Added a unit test, bencharmk and samples of the new frame post processing benchmark
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Writer-side hook that runs on every
RecordFrame()after timer validation and before the serializer touches the nativeFrame. Whatever the processor mutates is what gets persisted to disk -- callers who re-read the.vtxsee the post-processed values.What's in this PR
Core API (writer-side)
sdk/include/vtx/writer/core/vtx_frame_post_processor.h(new)IFramePostProcessorinterface withInit/Process/Clear/PrintInfolifecycle;FramePostProcessorChainfor composition;FramePostProcessorInitContext+FramePostProcessContextcarriers.sdk/include/vtx/writer/core/vtx_frame_mutation_view.h(new)Write-side mirror of
EntityView/FrameAccessor.EntityMutator(Get+Set+ nested + array),BucketMutator(mutable iteration + structural mutation:AddEntity/RemoveEntity/RemoveIf/Clear),FrameMutationView(entry point withFrameAccessor*for schema resolution).sdk/include/vtx/writer/core/writer.h(modified)Hook in
RecordFrame()after timer validation, beforeSerializer::FromNative. Destructor invokesprocessor->Clear().sdk/include/vtx/writer/core/vtx_writer_facade.h+.cpp(modified)Three new virtual methods on
IVtxWriterFacade:SetPostProcessor/GetPostProcessor/ClearPostProcessor, forwarded to both FlatBuffers and Protobuf impls.Init()runs synchronously onSetPostProcessorBEFORE the new processor becomes visible to anyRecordFrame().Structural refactor (sustaining the feature)
vtx_frame_accessor.hmoved fromsdk/include/vtx/reader/core/tosdk/include/vtx/common/.Pre-move it lived under
reader/for historical reasons; the new writer-side headers would have created a writer→reader cross-dependency. Post-move bothwriter/core/andreader/core/sharecommon/vtx_frame_accessor.hwith zero include edges between them.11 include sites updated: benchmarks, tests, samples, scripts, reader, writer headers.
Codegen extension
scripts/vtx_codegen.py(modified)Emits per struct alongside the existing
XView:XMutator-- write-capable wrapper;Get*(same as View) +Set*for scalars +GetMutable*for arrays / nested.ForEachX(bucket, accessor, fn)-- filters aBucketMutatorbyentity_type_id, invokesfnwith a strongly-typedXMutator&.ForEachXView(bucket, accessor, fn)-- read-only counterpart.samples/arena_generated.hregenerated; now includesPlayerMutator,ProjectileMutator,MatchStateMutator, plus all theForEachXhelpers.Tests (
tests/writer/test_frame_post_processor.cpp, 10 cases)MutationViewUnit.SetThenGetRoundTrips-- unit smoke for the mutation view primitives.ChainUnit.OrderAndRemove-- chain composition + removal semantics.NoProcessorBaselineUnchanged-- zero-overhead path identical to baseline.DoubleHealthIsPersistedToDisk--Initresolves the Health key; processor doubles values pre-serialise; readback confirms200.0fon disk.ChainLastWriterWinsOnDisk-- ordering semantics.ChainRemoveDropsAndOtherStillFires--Chain::Removecorrectness.GhostInjectorEntityIsOnDisk--BucketMutator::AddEntity+ explicitentity_type_idpersists into the file.TeamTwoFilterDropsEntitiesFromDisk--BucketMutator::RemoveIfpre-serialise filter.GlobalFrameIndexIsMonotonic--ctx.global_frame_indexmonotonically increments.ClearPostProcessorCallsClearAndUnregisters-- explicit teardown + subsequentRecordFramebypasses processor.Samples
samples/post_process_write.cpp(new) -- minimal end-to-end demo. Synthetic frames with out-of-range Health,PlayerHealthProcessorclamps viaPlayerMutatorfrom the codegen output, re-opens the.vtxwith the reader to verify persisted bytes.samples/advance_write.cpp(modified) -- registersArenaConsistencyProcessoron every pipeline (JSON / Protobuf / FlatBuffers). The same processor instance per pipeline -- proves post-processing is orthogonal to the source format.Documentation
docs/POST_PROCESSING.md(new) -- dedicated reference: pipeline diagram, lifecycle, threading model, two ways to write a processor (generic vs codegen), patterns (cross-frame state, chains, replay-level metadata, schema-version branching, structural mutation), gotchas, performance, out-of-scope.docs/SDK_API.md-- new "Frame Post-Processor" section between "Writing Replays" and "Diffing Frames".docs/SAMPLES.md-- two new samples documented (post_process_writestandalone + post-processor inadvance_write), codegen output extended description.README.md-- "Optional: post-process every frame" snippet under "Write a replay" +POST_PROCESSING.mdin the docs list.Design highlights
IFramePostProcessoroperates on genericFrame/PropertyContainer. Each integration plugs in via codegen-generated types from their own schema JSON. The SDK doesn't know about Player / Champion / Projectile -- the codegen output does.post_processor_(writer is the user's capture thread).Setis silent no-op (matchesEntityView::Gettolerance); processor exceptions are swallowed withVTX_ERROR.