noVNC or similar video straming?

[MammothsHub]

Hi,
I'm very new to niceGUI. I've created programs in python/bash/c++ before but doing anything web related is relatively new for me. Therefore NiceGui seems to be a good library for someone in my position.

Long term I might want to use this to stream a vnc session from my raspberry pi cluster. Probably using noVNC. Would that be possible in NiceGui? Some of the examples are promising such as the web xterm example which shows how a continually interactive session similar to noVNC might be integrated. Further, the performance of the ui.video example is encouraging.

But I also note that the xterm example uses a built in wrapper constructed by niceGui itself. So is there a way to potentially have a "viewport" of some description that shows another hosted webservice such as noVNC? Or would a wrapper have to be constructed for noVNC.js?

Apologies if this question reveals my lack of experience on these matters.

[falkoschindler]

Welcome to NiceGUI, @MammothsHub! Great question. I have only little experience with VNC myself, so I asked Claude to chime in:

Embedding noVNC (or a similar web-based VNC viewer) is definitely possible, and you don't necessarily need a custom wrapper element.

Simplest approach: embed via iframe

If noVNC is already running as a standalone web service (e.g., on http://raspberrypi:6080/vnc.html), you can embed it directly:

from nicegui import ui

ui.html('<iframe src="http://raspberrypi:6080/vnc.html" style="width: 100%; height: 80vh; border: none;"></iframe>')

ui.run()

This works well because noVNC is a self-contained web app – it handles keyboard/mouse input and WebSocket connections internally, so an iframe is all you need.

Alternative: custom JavaScript element

If you want tighter integration (e.g., controlling the VNC connection from Python or reacting to VNC events), you could wrap the noVNC JavaScript library using ui.element with custom __init__.js / Vue component code – similar to how the xterm element works. But for a "just show me the remote desktop" use case, the iframe approach is much simpler and should work out of the box.

Tips for your Raspberry Pi setup:

• Run noVNC with websockify pointing at your VNC server (e.g., websockify --web /usr/share/novnc 6080 localhost:5900).
• If NiceGUI and noVNC run on different hosts/ports, you may need to handle CORS or put both behind a reverse proxy.
• For a polished look, you can use noVNC's vnc_lite.html (minimal UI) or pass query parameters to auto-connect.

Hope that helps – feel free to share your progress!

[MammothsHub]

Hi @falkoschindler,
Thank you very much for the reply. I can confirm this works!

I just went with the iframe approach. There is no drop off in performance even when using heavier loads such as youTube videos or text editing. A little bit of frame stutter but it is a raspberry pi after all.

Claude was almost correct in its suggestions I needed to add the following:
ui.html(<claude suggestion>, sanitize=False)
I'm not sure why that was needed for it to work. But without santize=False it's just a white screen.

Also, for anyone else who may happen upon this page Claude also doesn't mention port forwarding from the raspberry pi cluster or gaining access to its public ip address. Whilst this is obvious it did trip me up when I first tried to connect to it. So either port forward the NoVnc web port or on the pi cluster you can do the following:
ifconfig | grep 19 | grep inet
The viz node of your raspberry pi cluster (or standalone raspberry pi) will be displayed as inet 19x.xxx.x.xxx.

Finally I changed the style to be in pixels as that's more familiar to me when I'm streaming a desktop.

The end solution looked something like this (I chose to run the NoVnc web port on 8059, to avoid 8080 where nice gui was running):

from nicegui import ui

# Example using the public ip address of the rasperry pi viz node 
#ui.html('<iframe src="http://19x.xxx.x.xxx:8059/vnc.html" style="width: 1600px; height: 900px; border: none;"></iframe>', sanitize=False)

# Example where the user portforwards the NoVnc port on the raspberry pi viz node to loacl host
ui.html('<iframe src="http://localhost:8059/vnc.html" style="width: 1600px; height: 900px; border: none;"></iframe>', sanitize=False)

ui.run()

And viola!:

I do note that full screen button in noVNC doesn't work as I get the following:

TypeError: Disallowed by permissions policy
    at HTMLInputElement.toggleFullscreen (http://localhost:8059/app/ui.js:1317:42)

But this is a minor issue and its not surprising.

If people are interested I can also post how I got NoVnc working. But I'm not sure if that's relevant for your discussion board as its more about NoVnc than NiceGui. What do you think? Would people be Interested in that?

[falkoschindler]

Glad to hear it's working well! And yes, feel free to share your noVNC setup – since that's what this thread is about, I'm sure others who stumble upon it will appreciate having the full picture in one place. The sanitize=False tip alone is already a valuable addition. Just to explain why it's needed: ui.html sanitizes content by default to protect against cross-site scripting (XSS) attacks, which means it strips tags like <iframe> and <script>. Passing sanitize=False tells NiceGUI to trust the HTML as-is.

[MammothsHub]

Hi @falkoschindler,
Thank you for the explanation! That makes sense and as the source of the noVNC is trusted the santization is not required in this case.

For the sake of other users who may come to this post afterwards and want to know how to setup noVNC (at least on raspberry pi) I will go over the process now.

The full setup behind the noVNC setup sounds more involved than it is. Really there's just a few things to keep in mind. But this is a touch verbose, don't be intimidated its quite easy really!

Firstly you'll need/want a vnc server that can scale the video stream dynamically I personally used tigervnc:

sudo apt install tigervnc-standalone-server

Secondly you'll of course need an xstartup file copied to your $HOME/.vnc, the contents of which for gnome raspberry pi would need to look like this:

export XKL_XMODMAP_DISABLE=1
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &

gnome-panel &
gnome-settings-daemon &
metacity &
nautilus &
gnome-terminal &

Please note that the default xstartup supplied will fail (on rpi anyway) comment that out and replace it with the above contents.

Thirdly we'll actually install noVNC. This can be done in a few different ways but I'm lazy and just used a python venv like so:

python -m venv noVncVenv
. noVncVenv/bin/activate 
pip install novnc 

Fourthly, we can now run noVNC by running our tigervncserver activating our noVNC python venv and then running novnc. Here I've chosen display 1 (remember display 0 is likely taken by your non-vnc x/wayland session), the VNC port as 5901 and the webport as 8059. We don't want NiceGUI and noVNC fighting over the same port so I let NiceGUI have the 8080 port but its a matter of preference of course. (See my above comment for my NiceGUI code).

# Start tiger vnc on a particular port
tigervncserver -geometry 1600x900 :1

# Activate vnenv for novnc
. <full-path-to-noVNC-venv>/bin/activate

# Then do a novnc targeting the web port 
novnc --listen 0.0.0.0:8059 --target 127.0.0.1:5901

Please note you'll need to port forward 8059 to your localhost (on vs-code you can go to your ports section, in the same panel as the terminal, and type 8059) then go to localhost:8059 in your web browser of choice and you'll be able to see your noVNC session. Or you can figure out your pi's IP address on your local network through ifconfig | grep 19 | grep inet and go to that instead of port forwarding through local host.

In the options of your noVNC side bar you'll probably want to set the scaling to remote resizing so you can alter the size of the window and have your desktop dynamically adjust to its new dimensions.

Fifthly (optional) you can copy this script (I called it startNoVnc) to your /bin and do chmod ugo+x so that you can just type startNoVnc <DIPSLAY_NUM> <WEB_PORT>. Then port forward to your localhost. The crtl+c is trapped so the vnc server is automatically killed when you cancel the script:

#!/bin/bash

# Set this incase user hits control+c before vncserver is up
export DISPLAY_NUM=-1

# Setup trap command, assign what to call and what is the trigger (SIGINT)
trap ctrl_c INT

# Function of what trap command calls    
function ctrl_c() {

    if (( $DISPLAY_NUM < 0 )); then
        printf "\n No VNC Server Needs killing \n"
    else 
        printf "\n killing vncserver :$DISPLAY_NUM \n"
        vncserver -kill :$DISPLAY_NUM
    fi

    exit 0
}

# Default values for webport and display number for NoVNC
# User changes them by entering startNoVNC <DISPLAY_NUM> <WEB_PORT>
if [[ -z "$1"  ]]; then
    export DISPLAY_NUM=1
else 
    export DISPLAY_NUM=$1
fi

if [[ -z "$2"  ]]; then
    export WEB_PORT=8080
else 
    export WEB_PORT=$2
fi

# Change VNC port to match DISPLAY_NUM 
if (( $DISPLAY_NUM < 10 )); then
    export VNC_PORT="590$DISPLAY_NUM"
else
    export VNC_PORT="59$DISPLAY_NUM"
fi

# Start tiger vnc on a particular port
tigervncserver -geometry 1600x900 :$DISPLAY_NUM

# Activate vnenv for novnc
. <full-path-to-no-venv>/bin/activate

# Then do a novnc targeting the web port 
novnc --listen 0.0.0.0:$WEB_PORT --target 127.0.0.1:$VNC_PORT 

[MammothsHub]

Hi Future Users,
Incase anyone comes across this and want to know how to use NiceGUI and noVNC together and have the noVNC window auto scale and fullscreen in NiceGUI. This is for you.

My above post has the following error when trying to use full screen:

TypeError: Disallowed by permissions policy
    at HTMLInputElement.toggleFullscreen (http://localhost:8059/app/ui.js:1317:42)

This is not an error with NiceGUI, instead it is and issue with the iframe approach. iframe or NiceGUI, by default, has very strict permissions (sensible). But it means that in order to allow the fullscreen button in noVNC the permission allowfullscreen has to be enabled in the ui.html options. So to enable full screen it would look something like this:

ui.html('<iframe src="http://localhost:8059/vnc.html" allowfullscreen></iframe>', sanitize=False)

And now when we click on the full screen option we get something that looks like this:

Source mozillia allowFullscreen

That's one issue down and one to go.

Now lets talk about resizing. This issue is also considered a "full screen" issue. But instead it might be better considered a "full web page" issue. The iframe element isn't taking up the full web page. Instead, its only tacking up a small portion and has white boarders around it.

Like with the fullscreen button for noVNC this issue is once again solved by adding additional iframe arguments. But this time we're going to add elements to the style. In particular the arguments mentioned in this stack overflow post:

position:fixed; top:0; left:0; bottom:0; right:0; width:100%; height:100%; border:none; margin:0; padding:0; overflow:hidden;

Putting this options with our other iframe options we now have the following code

from nicegui import ui

# Portforwarding through localhost here, but the previous approach by using rpi ipadress still works
ui.html('<iframe src="http://localhost:8059/vnc.html" style="position:fixed; top:0; left:0; bottom:0; right:0; width:100%; height:100%; border:none; margin:0; padding:0; overflow:hidden;" allowfullscreen></iframe>', sanitize=False)

ui.run()

Finally we just need to remember to hit the Remote resizing option in the settings menu of noVNC to ensure that the noVNC fills the available space on the webPage we have allotted for it. And hey presto!:

And with that I'll stop posting to this page. After this there is very little to discuss regarding noVNC and NiceGUI. Suffice to say if you want to have a noVNC virtual desktop as part of your NiceGUI web application it would appear to work very well!