Orion AI Orchestration Platform

Multi-agent AI orchestration platform with real persistence, retrieval-augmented generation, distributed tracing, and JWT authentication.

---

---

Architecture

┌─────────────────────────────────────────────────────┐
│              Next.js Dashboard (Phase 5)            │
│    Login · Stats · Agent Registry · Run Inspector   │
└──────────────────────┬──────────────────────────────┘
                       │ HTTP + JWT
┌──────────────────────▼──────────────────────────────┐
│                   FastAPI (API)                     │
│   /auth  ·  /agents  ·  /runs  ·  /health          │
└──────┬──────────────┬──────────────┬────────────────┘
       │              │              │
  ┌────▼────┐   ┌─────▼─────┐  ┌───▼──────┐   ┌──────────┐
  │Postgres │   │   Redis   │  │  Worker  │   │  Jaeger  │
  │ runs    │   │  (queue)  │  │ (async)  │   │ (traces) │
  │ agents  │   └───────────┘  └──────────┘   └──────────┘
  │ docs    │
  │ tenants │
  └─────────┘

Pipeline

Each request flows through 6 explicit stages, each instrumented with OpenTelemetry:

route → plan → retrieve → generate → evaluate → guardrail

Every stage emits a RunStep with name, status, latency_ms, and metadata. The full trace is viewable in Jaeger and in the Run Inspector.

What Phase 5 adds

• pgvector — pgvector/pgvector:pg15 image, CREATE EXTENSION vector on startup, Vector(1536) column on documents table ready for real embeddings
• OpenAI + Gemini — production-ready provider adapters selectable via LLM_PROVIDER env var; mock provider still default
• OpenTelemetry — distributed tracing with FastAPIInstrumentor, per-step manual spans, trace_id stored on every run
• Jaeger — all-in-one container at localhost:16686; Run Inspector links directly to each trace
• JWT auth — POST /api/v1/auth/login → Bearer token; all routes protected; tokens carry tenant claim
• Multi-tenant — tenants + users tables; runs scoped per tenant; seed creates acme and initech with demo users
• Run Inspector — Datadog-style page per run: waterfall timeline, per-step metadata cards, response output, Jaeger link

Quick start

cp .env.example .env
make up

Service	URL
Dashboard	http://localhost:3000
Login	admin@acme.com / acme1234
Swagger UI	http://localhost:8000/docs
Jaeger UI	http://localhost:16686
PostgreSQL	localhost:5432
Redis	localhost:6379

API

# Authenticate
TOKEN=$(curl -s -X POST http://localhost:8000/api/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email":"admin@acme.com","password":"acme1234"}' | jq -r .access_token)

# List agents
curl http://localhost:8000/api/v1/agents \
  -H "Authorization: Bearer $TOKEN"

# Create a run
curl -X POST http://localhost:8000/api/v1/runs \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"input_text": "Analyze the outage in service A"}'

# List runs
curl http://localhost:8000/api/v1/runs \
  -H "Authorization: Bearer $TOKEN"

# Inspect a run
curl http://localhost:8000/api/v1/runs/<run_id> \
  -H "Authorization: Bearer $TOKEN"

LLM providers

Set LLM_PROVIDER in .env:

Value	Model	Requires
mock	Built-in mock (default)	—
openai	gpt-4o-mini	OPENAI_API_KEY
gemini	gemini-1.5-flash	GEMINI_API_KEY

Project structure

apps/
  api/        FastAPI · SQLAlchemy · OTel · JWT · orchestration pipeline
  web/        Next.js 14 · server components · login · Run Inspector
  worker/     Redis-ready async worker (Celery/Dramatiq-ready)
packages/     Shared libs (agents, retrieval, llm, observability)
docs/         Architecture docs and ADRs

Makefile

make up       # build and start all services
make down     # stop all services
make logs     # follow container logs
make test     # run pytest

Roadmap

• Alembic migrations
• pgvector semantic search with real embeddings
• Celery/Dramatiq background execution
• Prometheus + Grafana dashboards
• RBAC (roles per tenant)
• Streaming responses