Skip to content

API stability fixes and dependency modernisation#36

Merged
dpgiakatos merged 2 commits into
mainfrom
security-fixes
Mar 31, 2026
Merged

API stability fixes and dependency modernisation#36
dpgiakatos merged 2 commits into
mainfrom
security-fixes

Conversation

@dpgiakatos
Copy link
Copy Markdown
Member

Description

This PR fixes a critical stability issue where a single malformed query to /disco/events could exhaust all database connections and block the entire API. It also modernises the codebase by migrating all repositories to the SQLAlchemy 2.0 query API and resolving Pydantic v2 deprecation warnings.

Root cause: The endpoint had no time parameter requirement, no guard against unknown parameters, and loaded probe details for every event by default — triggering 130k individual SQL queries per request. Combined with synchronous SQLAlchemy blocking the async event loop, one bad request was enough to stall all connections.

Changes

API-wide

  • Unknown query parameters now return HTTP 400 with a list of unexpected and allowed parameters for that endpoint; handled by a new middleware in main.py.

/disco/events fixes

  • At least one time parameter is now required (starttime, starttime__gte, starttime__lte, endtime, endtime__gte, endtime__lte); requests without one return HTTP 400.
  • Probe details are no longer included by default; a new include_probe_details boolean parameter (default False) controls this; when enabled, a single joinedload replaces the previous 130k lazy-load queries.

Database connection pool

  • Configured QueuePool with pool_size=10, max_overflow=5, pool_timeout=30.
  • Added PostgreSQL-side statement_timeout=60000ms to kill runaway queries and release connections.
  • Added pool_pre_ping=True to drop stale connections before use.

For more details read: https://docs.sqlalchemy.org/en/20/core/engines.html#sqlalchemy.create_engine

SQLAlchemy 2.0 migration (all 14 repositories)

  • Replaced db.query(Model) with select(Model) + db.scalars() / db.scalar().
  • Replaced query.filter() with .where(), query.count() with db.scalar(select(func.count())...).
  • Updated aliased joins from .join(Alias, Model.rel) to .join(Model.rel.of_type(Alias)).
  • Updated contains_eager(rel, alias=Alias) to contains_eager(rel.of_type(Alias)).
  • Added .unique() on scalars() results to deduplicate ORM objects from joined loads.

Deprecation warnings

  • Replaced class Config: from_attributes = True with model_config = ConfigDict(from_attributes=True) across all 16 DTOs (Pydantic v2).
  • Replaced from_orm() with model_validate() (Pydantic v2).
  • Replaced declarative_base() with class-based DeclarativeBase (SQLAlchemy 2.0).

Documentation

  • Updated docs/add_new_endpoint.md with 2.0-style repository and DTO examples.

How Has This Been Tested?

Local testing.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@dpgiakatos dpgiakatos changed the title security fixes & packages upgrade API stability fixes and dependency modernisation Mar 31, 2026
@dpgiakatos dpgiakatos self-assigned this Mar 31, 2026
@dpgiakatos dpgiakatos added bug Something isn't working enhancement New feature or request labels Mar 31, 2026
@dpgiakatos
Copy link
Copy Markdown
Member Author

I need to add include_probe_details=True in disco/events endpoint in ihr-website API call.

@dpgiakatos dpgiakatos merged commit 0d49032 into main Mar 31, 2026
4 checks passed
@dpgiakatos dpgiakatos deleted the security-fixes branch March 31, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant