How to handle large data downloads with using ui.table?

[wasabismile]

First Check

• I added a very descriptive title here.
• I have checked the FAQs for a similar question and didn't find it.
• I used the GitHub search to find a similar question and didn't find it.
• I searched the NiceGUI documentation, with the integrated search.
• I already searched online "How to X in NiceGUI" and didn't find any information.

Example Code

from nicegui import  ui
import random
columns = [
    {'name': f'col{i}', 'label': f'Col{i}', 'field': f'col{i}'}
 for i in range(40)
]
rows = [
    {f'col{i}': ''.join(random.choices('abcdefghijklmnopqrstuvwxyz',k=4)) for i in range(40)}
    for r in range(2000)
]
async def download_data() -> None:
    rows=await table.get_filtered_sorted_rows(timeout=2.0)
    # print(rows)
table = ui.table(columns=columns, rows=rows,pagination=20,row_key='name').props('dense').classes('w-full')

ui.input('Search by name/age').bind_value(table, 'filter')
ui.button('download',on_click=download_data)
ui.run()

Description

I am trying to download data from a table using the get_filtered_sorted_rows method. However, when the dataset exceeds a certain size, an error occurs indicating that the payload is too large for WebSocket transmission. Could you advise how to resolve this issue, or suggest alternative approaches for handling large data downloads?

NiceGUI Version

3.6.1

Python Version

3.11

Browser

Chrome

Operating System

Windows

Additional Context

[evnchn]

The WebSocket limits how big of an update can be delivered to the browser via the NiceGUI underlying messaging system. However, you can circumvent by:

1. Server-side pagination to send less stuff to the browser for the table
2. For the download, use other download functions https://nicegui.io/documentation/download

[wasabismile]

@evnchn Based on your suggestion to reduce the data volume by using server-side pagination, I modified the code. Could you please help confirm whether I correctly understood your suggestion?

async def download_data() -> None:
    rowscount=await table.get_computed_rows_number()
    rowsperpage=table.pagination['rowsPerPage']
    pagenumber=rowscount//rowsperpage
    remainder = rowscount % rowsperpage
    allrows=[]
    if remainder>0:
        pagenumber=pagenumber+1
    for i in range(1,pagenumber+1):
        prows=await table.get_computed_rows()
        table.pagination['page']=i+1
        allrows.extend(prows)
    table.pagination['page']=1
    print(len(allrows))

[evnchn]

Oh so the table dump is too large. I though the table itself was too large (that we need pagination to avoid crashing the browser).

If that's the case, then it's about right, but will take N round trips for N pages though.

Now, I'm thinking, specifically for big data back-flow, we MIGHT implement APIs which helps you absorb the chunk on server-side to bypass the WebSocket limit. @falkoschindler

However, indeed yes, this code, at least similar ones like it, is the best way forward for now.