Summary
Once Python 3.10 support is dropped, several places in the codebase can benefit from asyncio.TaskGroup (introduced in Python 3.11) for proper structured concurrency.
High Priority
src/apify/request_loaders/_apify_request_list.py:145-161 — Remote URL fetching
Current code uses create_task + add_done_callback + gather. The callbacks spawn fire-and-forget asyncio.create_task calls inside lambdas — those secondary tasks are never awaited. If create_requests_from_response fails, the error is silently lost.
Wins:
- Eliminates fire-and-forget tasks that can silently swallow errors.
- All tasks (fetch + process) are managed under one scope.
- Errors propagate properly via
ExceptionGroup.
Medium Priority
src/apify/_actor.py:1187 — Reboot listener dispatch
asyncio.gather fires persist-state and migrating listeners concurrently. If one listener raises, the others are left running unmanaged.
Wins:
- All listener tasks are cancelled on first failure instead of being left dangling.
- Produces an
ExceptionGroup with all errors, improving debuggability.
Low Priority
src/apify/events/_apify_event_manager.py:76 — Background WebSocket task
A long-running create_task with manual cancel + suppress(CancelledError) in __aexit__. Could be replaced by a TaskGroup owned by the context manager, but the task must outlive __aenter__, so it would require restructuring the class.
Wins:
- Cleaner lifecycle management (no manual cancel/suppress boilerplate).
- Requires class restructuring, so benefit-to-effort ratio is lower.
Not Applicable
src/apify/scrapy/_async_thread.py:103-113 — Shutdown tasks
Cancels pre-existing tasks and awaits them with gather(*tasks, return_exceptions=True). TaskGroup is for spawning new tasks, not for wrangling already-running ones. Leave as-is.
Summary
Once Python 3.10 support is dropped, several places in the codebase can benefit from
asyncio.TaskGroup(introduced in Python 3.11) for proper structured concurrency.High Priority
src/apify/request_loaders/_apify_request_list.py:145-161— Remote URL fetchingCurrent code uses
create_task+add_done_callback+gather. The callbacks spawn fire-and-forgetasyncio.create_taskcalls inside lambdas — those secondary tasks are never awaited. Ifcreate_requests_from_responsefails, the error is silently lost.Wins:
ExceptionGroup.Medium Priority
src/apify/_actor.py:1187— Reboot listener dispatchasyncio.gatherfires persist-state and migrating listeners concurrently. If one listener raises, the others are left running unmanaged.Wins:
ExceptionGroupwith all errors, improving debuggability.Low Priority
src/apify/events/_apify_event_manager.py:76— Background WebSocket taskA long-running
create_taskwith manual cancel +suppress(CancelledError)in__aexit__. Could be replaced by aTaskGroupowned by the context manager, but the task must outlive__aenter__, so it would require restructuring the class.Wins:
Not Applicable
src/apify/scrapy/_async_thread.py:103-113— Shutdown tasksCancels pre-existing tasks and awaits them with
gather(*tasks, return_exceptions=True).TaskGroupis for spawning new tasks, not for wrangling already-running ones. Leave as-is.