diff --git a/api-reference/server/frames/overview.mdx b/api-reference/server/frames/overview.mdx index 4c437e6d..a6a93737 100644 --- a/api-reference/server/frames/overview.mdx +++ b/api-reference/server/frames/overview.mdx @@ -54,6 +54,11 @@ Every frame has these properties set automatically: Presentation timestamp in nanoseconds. Used for audio/video synchronization. + + ID of the paired frame when a frame is broadcast in both directions. Set + automatically by `broadcast_frame()` and `broadcast_frame_instance()`. + + Dictionary for arbitrary frame metadata. diff --git a/api-reference/server/rtvi/rtvi-processor.mdx b/api-reference/server/rtvi/rtvi-processor.mdx index 2d951ab1..aed1e4ef 100644 --- a/api-reference/server/rtvi/rtvi-processor.mdx +++ b/api-reference/server/rtvi/rtvi-processor.mdx @@ -235,8 +235,9 @@ class MyProcessor(FrameProcessor): ``` - `RTVIServerMessageFrame` is a `SystemFrame`, so it propagates immediately - through the pipeline and is not affected by interruptions or queuing. + `RTVIServerMessageFrame` is a `SystemFrame`, so it uses the high-priority + SystemFrame lane, remains ordered with other SystemFrames, and is not + discarded by interruptions. ### Client-side handling diff --git a/pipecat/learn/pipeline-termination.mdx b/pipecat/learn/pipeline-termination.mdx index 3bb5f877..12b05225 100644 --- a/pipecat/learn/pipeline-termination.mdx +++ b/pipecat/learn/pipeline-termination.mdx @@ -105,8 +105,8 @@ async def on_client_disconnected(transport, client): 1. An event triggers the cancellation (like client disconnection) 2. `task.cancel()` pushes a `CancelFrame` downstream from the PipelineTask -3. `CancelFrame`s are `SystemFrame`s and bypass queues for immediate processing -4. Processors handle the `CancelFrame` and shut down immediately +3. `CancelFrame`s are `SystemFrame`s, so they use the high-priority input queue and are processed before queued non-system frames +4. Processors handle the `CancelFrame` and shut down without waiting for pending non-system work to drain 5. Any pending frames are discarded during shutdown @@ -252,7 +252,7 @@ await self.push_frame(CancelTaskFrame(), FrameDirection.UPSTREAM) - **Choose the right method** - graceful for natural endings, immediate for disconnections - **Event handlers enable automatic termination** - respond to user disconnections cleanly - **Idle detection provides safety net** - prevents hanging processes and resource waste -- **SystemFrames bypass queues** - CancelFrames process immediately for fast shutdown +- **SystemFrames have priority** - CancelFrames are processed before queued non-system frames for fast shutdown - **Resource cleanup is automatic** - proper termination ensures clean resource disposal ## What's Next diff --git a/pipecat/learn/pipeline.mdx b/pipecat/learn/pipeline.mdx index ae6ad362..5345fa85 100644 --- a/pipecat/learn/pipeline.mdx +++ b/pipecat/learn/pipeline.mdx @@ -59,7 +59,7 @@ Frames in Pipecat have different base classes that determine how they're process ```python @dataclass class SystemFrame(Frame): - """System frames are processed immediately, bypassing queues.""" + """System frames are queued with high priority.""" pass @dataclass @@ -75,13 +75,13 @@ class ControlFrame(Frame): **Key differences:** -- **SystemFrames**: Processed immediately (interruptions, pipeline control, user input) +- **SystemFrames**: High-priority and ordered with other SystemFrames; interruptions do not discard them (interruptions, pipeline control, user input) - **DataFrames & ControlFrames**: Queued and processed in order (audio output, text, images) **Examples by type:** ```python -# SystemFrames (processed immediately) +# SystemFrames (high-priority and ordered with other SystemFrames) InputAudioRawFrame # User audio input UserStartedSpeakingFrame # Speech detection events InterruptionFrame # Interruption control @@ -103,7 +103,7 @@ LLMFullResponseStartFrame # LLM response boundaries ### Frame Processing Order -**Frames are processed in guaranteed order**, even across ParallelPipelines. This enables reliable sequencing. For example, you can push two frames in order and the order will be respected. Additionally, the corresponding processing will finish before allowing the next frame to be processed. Let's look at an example where we push two frames—`TTSSpeakFrame` and `EndFrame`—in order to say goodbye then end the pipeline: +**Frames are processed in guaranteed order within their processing lane**, even across ParallelPipelines. SystemFrames are queued in a high-priority lane and ordered with other SystemFrames; DataFrames and ControlFrames are queued together in the non-system lane and ordered with each other. This enables reliable sequencing. For example, you can push two non-system frames in order and the order will be respected. Additionally, the corresponding processing will finish before allowing the next non-system frame to be processed. Let's look at an example where we push two frames—`TTSSpeakFrame` and `EndFrame`—in order to say goodbye then end the pipeline: ```python from pipecat.frames.frames import EndFrame, TTSSpeakFrame @@ -214,10 +214,10 @@ ParallelPipelines are traditionally paired with filters or gates to control whic Frame processors have internal queues that ensure ordered processing: -- **SystemFrames bypass queues** for immediate processing (interruptions, errors) -- **DataFrames and ControlFrames are queued** and processed in order +- **SystemFrames use a high-priority input queue** and are processed in order with other SystemFrames (interruptions, errors, input audio) +- **DataFrames and ControlFrames use the non-system process queue** and are processed in order with each other - **Queuing is managed automatically** by the pipeline infrastructure -- **Order is guaranteed** even across complex pipeline structures +- **Order is guaranteed within each lane** even across complex pipeline structures Learn more about frame flow patterns in the [Custom Frame Processor @@ -372,8 +372,8 @@ These configuration options enable you to optimize performance, debug issues, an - **Order matters** - arrange processors so each gets the frames it needs - **Processors push frames** - processors pass frames downstream, not consume them -- **Frame types determine processing** - SystemFrames bypass queues, others are ordered -- **Queuing ensures reliability** - frames are processed in guaranteed order +- **Frame types determine processing** - SystemFrames use the high-priority input queue, while DataFrames and ControlFrames use the non-system process queue +- **Queuing ensures reliability** - frames are processed in guaranteed order within their processing lane - **Parallel processing** enables conditional logic and multi-modal handling - **PipelineTask** configures execution parameters and monitoring - **PipelineRunner** manages the complete pipeline lifecycle