Skip to content

Release: S3 migration, PTC bash, egress proxy, compose consolidation#94

Merged
usnavy13 merged 32 commits intomainfrom
dev
May 7, 2026
Merged

Release: S3 migration, PTC bash, egress proxy, compose consolidation#94
usnavy13 merged 32 commits intomainfrom
dev

Conversation

@usnavy13
Copy link
Copy Markdown
Owner

@usnavy13 usnavy13 commented May 7, 2026

Summary

Major release consolidating all dev branch improvements since the last merge to main.

  • MinIO → S3/Garage migration: Replaced MinIO client with boto3 S3, Garage as default backend (S3_* env vars). Any S3-compatible storage works.
  • Programmatic Tool Calling (PTC) for Bash: New ptc_bash_server.py enables bash scripts to call external tools mid-execution, alongside existing Python PTC support.
  • Egress proxy & firewall: Sandboxes can now install dependencies at runtime (ENABLE_SANDBOX_NETWORK=true) through an allowlist proxy (PyPI, npm, Go modules, crates.io). Firewall prevents SSRF.
  • Linked file management: Files can now be linked across sessions with proper lifecycle tracking and orphan cleanup.
  • Auth improvements: HTTP Basic auth support (LibreChat URL-credential style), AUTH_ENABLED=false for trusted networks, MASTER_API_KEY for admin endpoints.
  • File handling: Original filename preservation, Unicode/emoji sanitization matching LibreChat's approach, updated upload limits.
  • Compose consolidation: Eliminated docker-compose.prod.yml and docker-compose.local-test.yml — single docker-compose.yml is now production-ready, override file for local dev.
  • CI/CD simplification: Removed nightly workflow, leaner PR checks, release pipeline publishes multi-arch images.
  • Docs overhaul: All docs updated for S3/Garage, removed stale MinIO references, deleted stale Reference/ directory and placeholder AGENTS.md.
  • Dependency bumps: fastapi, redis, uvicorn, pytest, pytest-cov, python-dotenv.
  • Test reorganization: New unit tests for egress proxy, PTC bash, request helpers, runner nested paths. Moved integration tests to functional suite where appropriate.

Test plan

  • Unit tests pass (pytest tests/unit/)
  • Integration tests pass (pytest tests/integration/)
  • CI pipeline (flake8, black, mypy, bandit) passes
  • docker compose up -d pulls published image and starts cleanly
  • docker compose up --build -d with override file builds from source
  • Health endpoint returns healthy for API, Redis, and S3
  • Python execution with state persistence works across sessions
  • PTC Python and Bash endpoints work
  • File upload/download works

🤖 Generated with Claude Code

dependabot Bot and others added 30 commits March 6, 2026 07:15
Bumps [python-dotenv](https://github.com/theskumar/python-dotenv) from 1.2.1 to 1.2.2.
- [Release notes](https://github.com/theskumar/python-dotenv/releases)
- [Changelog](https://github.com/theskumar/python-dotenv/blob/main/CHANGELOG.md)
- [Commits](theskumar/python-dotenv@v1.2.1...v1.2.2)

---
updated-dependencies:
- dependency-name: python-dotenv
  dependency-version: 1.2.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [uvicorn[standard]](https://github.com/Kludex/uvicorn) from 0.41.0 to 0.42.0.
- [Release notes](https://github.com/Kludex/uvicorn/releases)
- [Changelog](https://github.com/Kludex/uvicorn/blob/main/docs/release-notes.md)
- [Commits](Kludex/uvicorn@0.41.0...0.42.0)

---
updated-dependencies:
- dependency-name: uvicorn[standard]
  dependency-version: 0.42.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.129.0 to 0.135.2.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](fastapi/fastapi@0.129.0...0.135.2)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-version: 0.135.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [redis](https://github.com/redis/redis-py) from 7.2.0 to 7.4.0.
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](redis/redis-py@v7.2.0...v7.4.0)

---
updated-dependencies:
- dependency-name: redis
  dependency-version: 7.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [pytest-cov](https://github.com/pytest-dev/pytest-cov) from 4.1.0 to 7.1.0.
- [Changelog](https://github.com/pytest-dev/pytest-cov/blob/master/CHANGELOG.rst)
- [Commits](pytest-dev/pytest-cov@v4.1.0...v7.1.0)

---
updated-dependencies:
- dependency-name: pytest-cov
  dependency-version: 7.1.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Bumps [pytest](https://github.com/pytest-dev/pytest) from 9.0.2 to 9.0.3.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@9.0.2...9.0.3)

---
updated-dependencies:
- dependency-name: pytest
  dependency-version: 9.0.3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
- Added AUTH_ENABLED configuration to control API key authentication on user endpoints, with detailed documentation updates.
- Introduced a new batch file upload endpoint to support multi-file uploads, preserving subdirectory structures.
- Implemented a new PTC server for bash execution, allowing users to run bash code with the same JSON protocol as the Python server.
- Updated Dockerfile to include jq for JSON processing in the bash PTC server.
- Enhanced existing endpoints and services to accommodate the new features and ensure proper handling of authentication and file management.
- Added configuration options for enabling sandbox network access, allowing sandboxes to reach the internet through an inline allowlist proxy.
- Implemented egress proxy and firewall to enforce outbound traffic restrictions, preventing SSRF vulnerabilities.
- Updated Docker configurations to support new sandbox features, including persistent skill-deps caching.
- Enhanced API with endpoints to inspect and purge the skill-deps cache, facilitating better management of installed dependencies.
- Refactored execution services to accommodate network-enabled sandboxes, ensuring proper handling of package installations across multiple languages.
- Enhanced file upload logic to skip extension checks for agent files, allowing skill-priming uploads from the LibreChat host.
- Increased the maximum number of files per session from 300 to 1000 to accommodate larger skill bundles and prevent upload errors.
- Added normalization functions for Python and Bash tool names to ensure compatibility with SDK-generated code.
- Updated file handling in execution services to support new metadata fields, including `inherited`, `modified_from`, and `entity_id`.
- Introduced read-only file handling during uploads, allowing for better management of file permissions in sandbox environments.
- Enhanced unit tests to cover new features and ensure robust validation of file and tool name handling.
…essions

- Egress proxy tunnel test: use IP literal 127.0.0.1 instead of localhost
  to avoid IPv6 resolution mismatch in CI
- Batch upload mock: add missing is_read_only param to fake_store
- Client-replay test: allow inherited file refs in exec response (matches
  LibreChat CodeExecutor.ts contract)
- Bandit B103: suppress intentional 0o1777 chmod on shared skill-deps dir

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
feat: Auth, sandbox egress, file handling, and bash batch execution
- Introduced `original_filename` field in the FileInfo model to store pre-sanitization filenames.
- Updated file upload and batch upload functions to include the original filename in metadata.
- Enhanced file listing to return the original filename if available, improving metadata accuracy.
- Adjusted file service methods to handle the new original filename parameter for better file management.
…-pass approach

Align sanitize_filename with LibreChat#12977's sanitizeFilenameSegment:
- NFC-normalize before sanitizing (handles decomposed accents)
- Two-pass: strict ASCII [a-zA-Z0-9._-], permissive non-ASCII (only
  blocks C1 controls U+0080-U+009F)
- Preserves emoji (📊) and ZWJ sequences that \w alone would strip

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The fake_store function in TestLibreChatUploadBatch had a fixed
parameter list missing the new original_filename kwarg, causing
a TypeError when the endpoint passed it through.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Updated configuration and environment variables to transition from MinIO to S3 storage, including changes to .env.example and Docker Compose files.
- Introduced a new S3Config class for managing S3 settings and removed the MinIO configuration.
- Refactored file management and state archival services to utilize the S3 client, ensuring compatibility with S3 operations.
- Adjusted health checks and service dependencies to reflect the new S3 storage integration.
- Updated documentation and comments throughout the codebase to replace references to MinIO with S3.
- Changed S3 access and secret keys in .env.example and test configuration to new values.
- Updated Docker Compose files to reflect the new S3 access keys and added default bucket environment variable.
- Modified health check command in Docker Compose to use the new status command for better service monitoring.
- Added RPC settings in garage.toml for improved service configuration.
fix: Preserve Unicode filenames in sanitization and persist original names
- Enhanced functional tests to verify that edits to mounted files produce new outputs with unique file_ids instead of in-place overwrites.
- Updated test descriptions for clarity on expected behavior regarding modified files.
- Introduced a helper function to locate modified files based on the original file_id, ensuring accurate assertions in test cases.
…etup

- Added a temporary filesystem configuration for /tmp with size and mode settings in both Docker Compose files.
- Changed the directory for empty_proc from /tmp to /var/lib/code-interpreter in the Dockerfile and related service files.
- Updated the sandbox execution commands to reflect the new empty_proc path and incorporated dynamic tmpfs size settings.
- Updated tmpfs mount options for /tmp in Docker Compose files to include noexec, nosuid, and nodev for improved security.
- Refactored sandbox execution commands to apply the new tmpfs settings consistently across service files.
- Introduced dynamic handling of skill dependencies with updated mount options to enhance security and isolation.
Replace MinIO with S3-compatible storage (Garage)
…pipeline

- Remove nightly.yml entirely
- Slim ci.yml from 11 jobs to 3 (static, unit, integration) — PR checks only, no Docker builds
- Release.yml (unchanged) handles multi-arch image builds on merge to main/dev

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
usnavy13 and others added 2 commits May 6, 2026 22:18
chore(deps): Bump 6 Python dependencies
…tale files

The repo had unnecessary docker-compose.prod.yml and docker-compose.local-test.yml
files. docker-compose.yml is now the single production-ready base (pulls published
GHCR image by default), and docker-compose.override.example.yml handles local dev
overrides. All MinIO references across 10+ docs updated to S3/Garage to match the
migration completed in #90. Removed stale Reference/ directory and placeholder
AGENTS.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@usnavy13 usnavy13 merged commit 686c410 into main May 7, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant