PublishTrack: populate SimulcastCodecs with primary codec for video tracks (fixes H.265 single-stream forwarding)#901
Open
aphexcx wants to merge 1 commit intolivekit:mainfrom
Open
Conversation
…racks
Previously, PublishTrack only set AddTrackRequest.SimulcastCodecs when an
explicit backup codec was supplied via WithBackupCodec. With no backup
codec, the request omitted SimulcastCodecs entirely, so the server's
TrackPublishedResponse echoed back an empty TrackInfo.MimeType (and
empty TrackInfo.Codecs[0].MimeType — see comment at publication.go:78-105).
trackPublicationBase.MimeType() then returned "" for any single-stream
PublishTrack video track, which made setPublishingCodecsQuality
(publication.go:485-528) fail the primary-codec check
(strings.HasSuffix("", subscribedCodec.Codec) is always false for any
non-empty codec). Every subscribed codec fell into the backup branch and
emitted "subscriber requested backup codec but no track found", and the
primary track's per-quality muted state was never toggled.
For default codecs like H.264 the SFU still forwarded RTP from defaults,
so the bug was invisible. For non-default codecs like H.265 the
publisher's track stayed unconfigured, the SFU never started forwarding,
and subscribers were stuck on track.muted=true with zero bytes received
even though SDP negotiation completed cleanly.
The fix adds an else-if to PublishTrack that populates
SimulcastCodecs[0] with the primary codec MimeType + track CID for
video tracks with a known primary codec MIME, mirroring what
PublishSimulcastTrack already does at localparticipant.go:300.
A new integration test, TestPublishTrackSingleStreamCodecMime, covers
H.264, H.265, and VP8 single-stream PublishTrack and asserts that:
1. trackPub.MimeType() returns the codec MIME after publish
2. A subscriber actually receives the track (regression for H.265
stuck-muted)
Reported in livekit/livekit-cli#837, where lk room join --publish
h265://host:port produced the symptom; the bug is in the SDK rather
than the CLI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PublishTrackfor video tracks without an explicit backup codec was leavingAddTrackRequest.SimulcastCodecsempty. The server'sTrackPublishedResponsetherefore echoed back an emptyTrackInfo.MimeType(the comment atpublication.go:78-105describes this dependency), sotrackPublicationBase.MimeType()returned""for every single-stream video publish.setPublishingCodecsQuality(publication.go:485-528) then misclassified every subscribed codec as backup —strings.HasSuffix("", "h265")is always false — and emitted"subscriber requested backup codec but no track found"instead of running the primary-codec branch.For H.264 the SFU still forwarded RTP via defaults, so the bug was invisible; for H.265 (and presumably other non-default codecs) the SFU never started forwarding, the subscriber's
track.mutedstayedtrue, and zero bytes were received even though SDP negotiation completed cleanly.Fix
Add a single
else ifbranch inPublishTrackthat populatesSimulcastCodecs[0]with{Codec: primaryCodec.MimeType, Cid: track.ID()}when:localparticipant.go:159-176)kind == TrackKindVideoprimaryCodec.MimeType != ""This mirrors what
PublishSimulcastTrackalready does atlocalparticipant.go:311-328. After the fix:MimeType()returns the actual codec MIME (video/H265,video/VP8, etc.)setPublishingCodecsQualitycorrectly takes the primary-codec branchsubscriber requested backup codec but no track foundwarnings stop firing for normal single-stream publishesAudio is correctly excluded — audio codec updates dispatch through
handleSubscribedAudioCodecUpdate/setAudioCodecSubscribed, notsetPublishingCodecsQuality.Test
Adds
TestPublishTrackSingleStreamCodecMimecovering H.264, H.265, and VP8 single-streamPublishTrack. For each:trackPub.MimeType() == codec.MimeTypeafter publishtrack.muted=trueand never delivered)Test follows the existing
TestSimulcastCodecpattern (samecreateAgent/pubNullTrackhelpers, same Docker-servermage testrunner). Compiles cleanly; functional verification requires the LiveKit server harness.Context / how this was found
Discovered while spiking LiveKit as a candidate SFU for low-latency robot teleop. The camera produces hardware H.265 (Rockchip
mpph265enc); re-encoding is too slow for our latency budget.lk room join --publish h265://host:port(livekit-cli v2.16.2) negotiated SDP, firedTrackSubscribedin Chrome 147, but the receiver's track stayed muted and zero RTP arrived. Same pipeline with H.264 worked.Initial issue reported against
livekit-cli: livekit/livekit-cli#837. After tracing through the SDK with help from a code-review agent, the actual bug is here inserver-sdk-go'sPublishTrackAddTrackRequestassembly —livekit-cliis just a caller that happens to hit it. Fixing it here also helpspublishFileand any otherPublishTrackcaller.I verified the build with
go build ./...,go vet ./..., andgo test -c .(test binary builds clean). I haven't run the integration test against a live LiveKit server in CI — happy to iterate on the test if maintainers want a different shape.Notes for reviewers
pubOptions.backupCodecTrack == nil(existing branch handles the backup case) andprimaryCodec.MimeType != ""(skips the case where the SDK can't infer a codec).PublishSimulcastTrackalready populatesSimulcastCodecsunconditionally — the existing E2EE guard in the backup-codec branch is specifically aboutsetBackupCodecTrack, not aboutSimulcastCodecs[0]metadata.