RTMP Frame callback

[xiuminga]

Hi, @pedroSG94

Does RootEncoder provide any kind of frame callback mechanism similar to WebRTC?

Specifically, I’m looking for a way to access video frames during the pipeline (e.g., raw frames, processed frames after filters, or encoded frames) so I can perform custom processing on each frame.

Is there any existing API for this, or a recommended way to achieve it?

Thanks!

[pedroSG94]

Hello,

Currently you have access to encoded frames (video and audio), raw audio frames, video frames using photo (after filters) and video frames using filters.
You can do it like this:

• Using a custom RecordController class you can access to encoded frames (audio and video). You can copy AndroidMuxerRecordController class and create a callback in onWriteFrame method. This way you will receive frames when you call startRecord method. Now, you only need set the class to the library using setRecordController method.

• You can get video frames after filters using takePhoto method as a bitmap. It is not recommended to use in a loop because the performance is bad. It is supposed to be use to take an occasional screenshoot.

• You can use custom filters to get video frames. You can create a filter to render the frame in an external surface, canvas or as you need. This is maybe the more complex way because you need know about opengl.

• Using CustomAudioEffect you can get raw audio frames. This allow you access to the byte array of the pcm audio frame in a callback. You can create your own class and set the effect to the library like this:

//assuming you are using MicrophoneSource
(genericStream.audioSource as MicrophoneSource).setAudioEffect()

Let me know if you need more info about it.

[xiuminga]

The GL filter approach works for me, so I’m going to close this issue. Thanks for your help.