Skip to content

PublishTrack: populate SimulcastCodecs with primary codec for video tracks (fixes H.265 single-stream forwarding)#901

Open
aphexcx wants to merge 1 commit intolivekit:mainfrom
aphexcx:fix/single-stream-codec-mime
Open

PublishTrack: populate SimulcastCodecs with primary codec for video tracks (fixes H.265 single-stream forwarding)#901
aphexcx wants to merge 1 commit intolivekit:mainfrom
aphexcx:fix/single-stream-codec-mime

Conversation

@aphexcx
Copy link
Copy Markdown

@aphexcx aphexcx commented May 6, 2026

Summary

PublishTrack for video tracks without an explicit backup codec was leaving AddTrackRequest.SimulcastCodecs empty. The server's TrackPublishedResponse therefore echoed back an empty TrackInfo.MimeType (the comment at publication.go:78-105 describes this dependency), so trackPublicationBase.MimeType() returned "" for every single-stream video publish.

setPublishingCodecsQuality (publication.go:485-528) then misclassified every subscribed codec as backup — strings.HasSuffix("", "h265") is always false — and emitted "subscriber requested backup codec but no track found" instead of running the primary-codec branch.

For H.264 the SFU still forwarded RTP via defaults, so the bug was invisible; for H.265 (and presumably other non-default codecs) the SFU never started forwarding, the subscriber's track.muted stayed true, and zero bytes were received even though SDP negotiation completed cleanly.

Fix

Add a single else if branch in PublishTrack that populates SimulcastCodecs[0] with {Codec: primaryCodec.MimeType, Cid: track.ID()} when:

  • there's no explicit backup codec (existing branch already handles that case at localparticipant.go:159-176)
  • kind == TrackKindVideo
  • primaryCodec.MimeType != ""

This mirrors what PublishSimulcastTrack already does at localparticipant.go:311-328. After the fix:

  • MimeType() returns the actual codec MIME (video/H265, video/VP8, etc.)
  • setPublishingCodecsQuality correctly takes the primary-codec branch
  • The spurious subscriber requested backup codec but no track found warnings stop firing for normal single-stream publishes
  • H.265 subscribers actually receive RTP

Audio is correctly excluded — audio codec updates dispatch through handleSubscribedAudioCodecUpdate / setAudioCodecSubscribed, not setPublishingCodecsQuality.

Test

Adds TestPublishTrackSingleStreamCodecMime covering H.264, H.265, and VP8 single-stream PublishTrack. For each:

  1. Asserts trackPub.MimeType() == codec.MimeType after publish
  2. Subscribes from a second participant and asserts the track is received within 10s (regression: H.265 was previously stuck on track.muted=true and never delivered)

Test follows the existing TestSimulcastCodec pattern (same createAgent / pubNullTrack helpers, same Docker-server mage test runner). Compiles cleanly; functional verification requires the LiveKit server harness.

Context / how this was found

Discovered while spiking LiveKit as a candidate SFU for low-latency robot teleop. The camera produces hardware H.265 (Rockchip mpph265enc); re-encoding is too slow for our latency budget. lk room join --publish h265://host:port (livekit-cli v2.16.2) negotiated SDP, fired TrackSubscribed in Chrome 147, but the receiver's track stayed muted and zero RTP arrived. Same pipeline with H.264 worked.

Initial issue reported against livekit-cli: livekit/livekit-cli#837. After tracing through the SDK with help from a code-review agent, the actual bug is here in server-sdk-go's PublishTrack AddTrackRequest assembly — livekit-cli is just a caller that happens to hit it. Fixing it here also helps publishFile and any other PublishTrack caller.

I verified the build with go build ./..., go vet ./..., and go test -c . (test binary builds clean). I haven't run the integration test against a live LiveKit server in CI — happy to iterate on the test if maintainers want a different shape.

Notes for reviewers

  • The new branch only runs when pubOptions.backupCodecTrack == nil (existing branch handles the backup case) and primaryCodec.MimeType != "" (skips the case where the SDK can't infer a codec).
  • No encryption guard, mirroring how PublishSimulcastTrack already populates SimulcastCodecs unconditionally — the existing E2EE guard in the backup-codec branch is specifically about setBackupCodecTrack, not about SimulcastCodecs[0] metadata.
  • The change is metadata-only; no behavior change to RTP packetization, RID extension, or transceiver setup.

…racks

Previously, PublishTrack only set AddTrackRequest.SimulcastCodecs when an
explicit backup codec was supplied via WithBackupCodec. With no backup
codec, the request omitted SimulcastCodecs entirely, so the server's
TrackPublishedResponse echoed back an empty TrackInfo.MimeType (and
empty TrackInfo.Codecs[0].MimeType — see comment at publication.go:78-105).

trackPublicationBase.MimeType() then returned "" for any single-stream
PublishTrack video track, which made setPublishingCodecsQuality
(publication.go:485-528) fail the primary-codec check
(strings.HasSuffix("", subscribedCodec.Codec) is always false for any
non-empty codec). Every subscribed codec fell into the backup branch and
emitted "subscriber requested backup codec but no track found", and the
primary track's per-quality muted state was never toggled.

For default codecs like H.264 the SFU still forwarded RTP from defaults,
so the bug was invisible. For non-default codecs like H.265 the
publisher's track stayed unconfigured, the SFU never started forwarding,
and subscribers were stuck on track.muted=true with zero bytes received
even though SDP negotiation completed cleanly.

The fix adds an else-if to PublishTrack that populates
SimulcastCodecs[0] with the primary codec MimeType + track CID for
video tracks with a known primary codec MIME, mirroring what
PublishSimulcastTrack already does at localparticipant.go:300.

A new integration test, TestPublishTrackSingleStreamCodecMime, covers
H.264, H.265, and VP8 single-stream PublishTrack and asserts that:
  1. trackPub.MimeType() returns the codec MIME after publish
  2. A subscriber actually receives the track (regression for H.265
     stuck-muted)

Reported in livekit/livekit-cli#837, where lk room join --publish
h265://host:port produced the symptom; the bug is in the SDK rather
than the CLI.
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented May 6, 2026

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants