Skip to content

Commit 66e4d25

Browse files
apartsinclaude
apartsin
and
claude
committed
Add local model connectors, RuntimeEnvironment metadata, and browser storage
- Add 4 local/self-hosted provider connectors (Ollama, LM Studio, vLLM, LocalAI) in both TypeScript and Python, extending OpenAICompatibleProvider - Add RuntimeEnvironment enum (node/browser/universal) as static metadata on all 42 TypeScript connector classes for build-time tree-shaking and runtime guards - Add browser storage connectors: LocalStorageStorage, SessionStorageStorage, IndexedDBStorage, and BrowserSecretStore - Add runtime detection utilities: detectRuntime() and assertRuntimeCompatible() - Extend auto-detect to discover local providers via host env vars - Add 68 new tests (23 Python + 45 TypeScript), totals: 775 Python, 370 TypeScript - Update ConnectorCatalogue, CoverageMatrix, and BrowserUsage documentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 6b2f809 commit 66e4d25

75 files changed

Lines changed: 2100 additions & 14 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/ConnectorCatalogue.md‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,52 @@ pools:
137137
| **AWS Bedrock** | `provider.aws.bedrock.v1` | Managed AI service with access to multiple foundation model providers. | Claude Sonnet 4, Claude 3.5 Haiku, Llama 3.1, Mistral Large, Amazon Nova Pro/Lite, Titan Embed, Stable Diffusion XL | No free tier; $200 new-account credit (all AWS, 6-month expiry) | [docs.aws.amazon.com/bedrock](https://docs.aws.amazon.com/bedrock) |
138138
| **Google Cloud AI APIs** | `provider.google.cloud-ai.v1` | Individual AI services for speech, vision, translation, and NLU. | Speech-to-Text (Chirp), TTS (WaveNet, Neural2), Vision, Translation, NL | 60 min/mo STT; 1M chars/mo TTS; 1,000 images/mo Vision; $300 credit | [cloud.google.com/apis](https://cloud.google.com/apis) |
139139

140+
### Local / Self-Hosted Providers
141+
142+
Local model servers expose OpenAI-compatible REST APIs, so each connector extends `OpenAICompatibleProvider` with different defaults. No API key is required — authentication is disabled by default. All local providers are **Node.js only** (`RuntimeEnvironment.NODE_ONLY`).
143+
144+
| Provider | ID | Default URL | Env Var | Default Models | Description |
145+
| --- | --- | --- | --- | --- | --- |
146+
| **Ollama** | `ollama.local.v1` | `http://localhost:11434` | `OLLAMA_HOST` | llama3, codellama, mistral, gemma2 | Run open-source LLMs locally. Simple install, model library, GPU acceleration. |
147+
| **LM Studio** | `lmstudio.local.v1` | `http://localhost:1234` | `LMSTUDIO_HOST` | *(user-loaded)* | Desktop app for running local models. GUI model browser and chat. |
148+
| **vLLM** | `vllm.local.v1` | `http://localhost:8000` | `VLLM_HOST` | *(user-loaded)* | High-throughput serving engine. PagedAttention, continuous batching. |
149+
| **LocalAI** | `localai.local.v1` | `http://localhost:8080` | `LOCALAI_HOST` | *(user-loaded)* | Drop-in OpenAI replacement. Supports LLMs, image gen, audio, embeddings. |
150+
151+
**Auto-detection:** Local providers are detected via host environment variables (e.g., `OLLAMA_HOST=http://myserver:11434`). Unlike cloud providers, no API key env var is needed — the presence of the host variable alone enables the provider.
152+
153+
**Configuration:**
154+
155+
All local providers share the same configuration parameters inherited from `OpenAICompatibleProvider`:
156+
157+
| Parameter | Type | Description |
158+
| --- | --- | --- |
159+
| `base_url` | string | Server URL. Defaults to localhost (see table above). |
160+
| `api_key` | string | API key for authentication. Default: `""` (no auth). |
161+
| `models` | list | Model catalogue. Ollama ships defaults; others are empty (user-loaded). |
162+
| `capabilities` | list | Capability paths. Default: `["generation.text-generation.chat-completion"]`. |
163+
| `timeout` | integer | Request timeout in seconds. Default: `60`. |
164+
165+
**Example configuration:**
166+
167+
```yaml
168+
providers:
169+
ollama.local.v1:
170+
connector: ollama.local.v1
171+
config:
172+
base_url: http://gpu-server:11434
173+
models:
174+
- id: deepseek-r1
175+
capabilities: [generation.text-generation.chat-completion]
176+
177+
vllm.local.v1:
178+
connector: vllm.local.v1
179+
config:
180+
base_url: http://inference-server:8000
181+
models:
182+
- id: meta-llama/Llama-3-70B-Instruct
183+
capabilities: [generation.text-generation.chat-completion]
184+
```
185+
140186
### Web API Services
141187

142188
Non-AI web services can be wrapped as provider connectors using the same interface, gaining rotation, quota management, and failover. These services are accessed through virtual model names and routed through capability pools like any other model.
@@ -189,6 +235,10 @@ Non-AI web services can be wrapped as provider connectors using the same interfa
189235
| **Perplexity** | yes | - | - | - | yes | yes | - | - | Pro only |
190236
| **AWS Bedrock** | yes | yes | - | yes | - | yes | yes | yes | AWS credits |
191237
| **Google Cloud APIs** | - | - | yes | - | - | - | - | - | generous |
238+
| **Ollama** | yes | - | - | yes | - | yes | - | - | free (local) |
239+
| **LM Studio** | yes | - | - | yes | - | yes | - | - | free (local) |
240+
| **vLLM** | yes | - | - | yes | - | yes | yes | - | free (local) |
241+
| **LocalAI** | yes | yes | yes | yes | - | yes | - | - | free (local) |
192242

193243
### CDK Base Classes for Providers
194244

@@ -199,6 +249,35 @@ Non-AI web services can be wrapped as provider connectors using the same interfa
199249

200250
Both classes expose the same provider interface and the same protected hooks for subclassing. See [cdk/BaseClasses.md](cdk/BaseClasses.html#browserbAseprovider) for details and [guides/BrowserUsage.md](guides/BrowserUsage.html) for browser setup.
201251

252+
### Runtime Environment Metadata
253+
254+
Every TypeScript connector class declares a `static readonly RUNTIME` property indicating browser/Node.js compatibility. This enables build-time tree-shaking and runtime compatibility checks.
255+
256+
```typescript
257+
import { RuntimeEnvironment } from '@modelmesh/core';
258+
259+
// Check a connector's runtime requirement
260+
console.log(OllamaProvider.RUNTIME); // 'node'
261+
console.log(LocalStorageStorage.RUNTIME); // 'browser'
262+
console.log(MemoryStorage.RUNTIME); // 'universal'
263+
```
264+
265+
| Value | Constant | Environment | Examples |
266+
| --- | --- | --- | --- |
267+
| `'node'` | `RuntimeEnvironment.NODE_ONLY` | Node.js, Bun, Deno (server) | All cloud/local providers, file storage, env/dotenv/json/encrypted/keyring secret stores |
268+
| `'browser'` | `RuntimeEnvironment.BROWSER_ONLY` | Browser (window + document) | localStorage/sessionStorage/IndexedDB storage, BrowserSecretStore |
269+
| `'universal'` | `RuntimeEnvironment.UNIVERSAL` | Any JavaScript runtime | MemoryStorage, MemorySecretStore, BrowserBaseProvider, all rotation policies, ConsoleObservability |
270+
271+
**Runtime Guard** (`detectRuntime()`, `assertRuntimeCompatible()`): Detects the current environment at runtime and throws a descriptive error if a connector is used in an incompatible environment. Available from `@modelmesh/core`:
272+
273+
```typescript
274+
import { detectRuntime, assertRuntimeCompatible } from '@modelmesh/core';
275+
276+
const runtime = detectRuntime(); // 'node' or 'browser'
277+
assertRuntimeCompatible('modelmesh.localstorage.v1', RuntimeEnvironment.BROWSER_ONLY);
278+
// Throws: "Connector modelmesh.localstorage.v1 requires browser environment but running in node"
279+
```
280+
202281
### Audio Namespace
203282

204283
ElevenLabs (`elevenlabs.tts.v1`), Azure Speech (`azure.tts.v1`), and AssemblyAI (`assemblyai.stt.v1`) are accessible through the MeshClient audio namespace, which follows the OpenAI SDK audio pattern:
@@ -255,6 +334,7 @@ Interface: [ConnectorInterfaces.md — Secret Store](ConnectorInterfaces.html#se
255334
| **`secret-store.modelmesh.memory-secrets.v1`** | Holds secrets in an in-memory dictionary. Ideal for testing, scripting, and user-provided keys. Supports runtime add/remove via SecretManagement interface. | Built-in | - |
256335
| **`secret-store.modelmesh.encrypted-file.v1`** | AES-256-GCM encrypted JSON file. Secrets are decrypted at initialization using a passphrase (PBKDF2) or raw key. Supports save/load round-trips. | Built-in | - |
257336
| **`secret-store.modelmesh.keyring.v1`** | Resolves secrets from the OS keyring (macOS Keychain, Windows Credential Locker, Linux Secret Service). | Built-in | - |
337+
| **`secret-store.modelmesh.browser-secrets.v1`** | Browser localStorage-backed secret store. Persists secrets across page reloads. TypeScript / browser only. | Built-in (TS only) | - |
258338

259339
### Connector-Specific Configuration
260340

@@ -328,6 +408,15 @@ Interface: [ConnectorInterfaces.md — Secret Store](ConnectorInterfaces.html#se
328408
| `service_name` | string | Keyring service/application name. Default: `modelmesh`. |
329409
| `fail_on_missing` | boolean | Throw on missing keys. Default: `true`. |
330410

411+
**`secret-store.modelmesh.browser-secrets.v1`:** *(TypeScript / Browser only)*
412+
413+
| Parameter | Type | Description |
414+
| --- | --- | --- |
415+
| `prefix` | string | Key namespace prefix in localStorage. Default: `modelmesh-secret:`. |
416+
| `failOnMissing` | boolean | Throw on missing keys. Default: `true`. |
417+
418+
Stores secrets in the browser's localStorage. Supports the full `SecretManagement` interface (`set`, `delete`, `list`). Secrets persist across page reloads and browser restarts. **Warning:** localStorage is not encrypted — secrets are visible in browser DevTools. Suitable for user-provided API keys in personal tools; not recommended for shared production deployments.
419+
331420
### Deployment Patterns
332421

333422
| Environment | Recommended Store | Reason |
@@ -337,6 +426,8 @@ Interface: [ConnectorInterfaces.md — Secret Store](ConnectorInterfaces.html#se
337426
| Scripts | `modelmesh.memory-secrets.v1` | pass keys from user input at runtime |
338427
| Shared config | `modelmesh.encrypted-file.v1` | keys encrypted at rest, safe to commit |
339428
| Desktop apps | `modelmesh.keyring.v1` | OS-native secure storage |
429+
| Browser apps | `modelmesh.browser-secrets.v1` | persistent, user-provided keys |
430+
| Browser testing | `modelmesh.memory-secrets.v1` | no browser APIs needed |
340431
| AWS / GCP | native secret manager | IAM integration |
341432
| Serverless | cloud secret manager | runtime injection |
342433
| Client-side | server proxy | keys never reach client |
@@ -356,6 +447,9 @@ Interface: [ConnectorInterfaces.md — Storage](ConnectorInterfaces.html#storage
356447
| **`storage.redis.redis.v1`** | Redis | atomic operations | Redis Cloud 30 MB free; self-hosted open-source | low-latency multi-instance sync | [redis.io](https://redis.io) |
357448
| **`storage.modelmesh.sqlite.v1`** | SQLite | single-process only | Built-in | structured local storage, queryable state | - |
358449
| **`storage.modelmesh.memory.v1`** | in-memory | single-process only | Built-in | testing, ephemeral workloads, no persistence | - |
450+
| **`storage.modelmesh.localstorage.v1`** | browser localStorage | single-tab | Built-in (TS only) | browser apps, small state (~5-10 MB) | - |
451+
| **`storage.modelmesh.sessionstorage.v1`** | browser sessionStorage | single-tab | Built-in (TS only) | browser apps, session-scoped state | - |
452+
| **`storage.modelmesh.indexeddb.v1`** | browser IndexedDB | single-origin | Built-in (TS only) | browser apps, large state, no size limit | - |
359453

360454
### Connector-Specific Configuration
361455

@@ -406,6 +500,43 @@ Interface: [ConnectorInterfaces.md — Storage](ConnectorInterfaces.html#storage
406500

407501
No configuration parameters. All data is held in-memory and lost on process exit. Useful for testing and ephemeral workloads.
408502

503+
**`storage.modelmesh.localstorage.v1`:** *(TypeScript / Browser only)*
504+
505+
| Parameter | Type | Description |
506+
| --- | --- | --- |
507+
| `prefix` | string | Key namespace prefix in localStorage. Default: `modelmesh:`. |
508+
509+
Data is serialized as base64-encoded JSON. Persists across page reloads and browser restarts. Subject to the browser's localStorage quota (~5-10 MB per origin). Requires `RuntimeEnvironment.BROWSER_ONLY`.
510+
511+
**`storage.modelmesh.sessionstorage.v1`:** *(TypeScript / Browser only)*
512+
513+
| Parameter | Type | Description |
514+
| --- | --- | --- |
515+
| `prefix` | string | Key namespace prefix in sessionStorage. Default: `modelmesh:`. |
516+
517+
Identical to localStorage storage, but data is cleared when the browser tab closes. Useful for ephemeral browser-session state.
518+
519+
**`storage.modelmesh.indexeddb.v1`:** *(TypeScript / Browser only)*
520+
521+
| Parameter | Type | Description |
522+
| --- | --- | --- |
523+
| `dbName` | string | IndexedDB database name. Default: `modelmesh`. |
524+
| `storeName` | string | Object store name. Default: `storage`. |
525+
| `version` | number | Database schema version. Default: `1`. |
526+
527+
Natively async storage using the browser's IndexedDB API. Stores binary data directly as `Uint8Array` (no base64 overhead). No practical size limit. Recommended for browser apps with large state. Connection is lazily initialized on first use; call `close()` to release the connection.
528+
529+
---
530+
531+
### Browser Storage Deployment Patterns
532+
533+
| Scenario | Recommended Store | Reason |
534+
| --- | --- | --- |
535+
| Browser SPA, small state | `modelmesh.localstorage.v1` | Persistent, simple, synchronous-feel API |
536+
| Browser SPA, large state | `modelmesh.indexeddb.v1` | No size limit, native binary support |
537+
| Tab-scoped state | `modelmesh.sessionstorage.v1` | Auto-cleared on tab close |
538+
| Browser testing | `modelmesh.memory.v1` | No browser APIs needed |
539+
409540
---
410541

411542
## Observability Connectors

‎docs/CoverageMatrix.md‎

Lines changed: 28 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ title: "Test Coverage Matrix"
55

66
# Test Coverage Matrix
77

8-
Correlates documented features with test coverage. The project includes 640 Python tests across 14 test files and 168 TypeScript tests across 10 test files, for a total of 808 tests.
8+
Correlates documented features with test coverage. The project includes 775 Python tests across 14 test files and 370 TypeScript tests across 13 test files, for a total of 1,145 tests.
99

1010
---
1111

@@ -23,8 +23,10 @@ Correlates documented features with test coverage. The project includes 640 Pyth
2323
| CDK Base Classes | 50 | `test_cdk.py` | Covered |
2424
| Observability Stack | 26 | `test_observability.py` | Covered |
2525
| Pre-shipped Connectors | 32 | `test_connectors.py` | Covered |
26+
| New Connectors (local providers, browser) | 93 | `test_new_connectors.py` | Covered |
2627
| CDK Specialized + Mixins + Helpers | 97 | `test_specialized.py` | Covered |
27-
| **Total** | **640** | **14 files** | |
28+
| Local Provider Connectors | 28 | `test_providers.py` | Covered |
29+
| **Total** | **775** | **14 files** | |
2830

2931
### TypeScript Test Suite
3032

@@ -37,10 +39,12 @@ Correlates documented features with test coverage. The project includes 640 Pyth
3739
| CapabilityPool | 15 | `pool.test.ts` | Covered |
3840
| ModelMesh facade | 16 | `mesh.test.ts` | Covered |
3941
| Router | 5 | `router.test.ts` | Covered |
40-
| Pre-shipped Connectors | 47 | `connectors.test.ts` | Covered |
42+
| Pre-shipped Connectors + Local Providers + RuntimeEnvironment | 102 | `connectors.test.ts` | Covered |
4143
| MeshConfig + Auto-detect | 22 | `config.test.ts` | Covered |
4244
| MeshClient (OpenAI compat) | 16 | `client.test.ts` | Covered |
43-
| **Total** | **168** | **10 files** | |
45+
| Secret Stores (env, dotenv, json, memory, encrypted, keyring) | 55 | `secret-stores.test.ts` | Covered |
46+
| CORS Proxy | 12 | `proxy.test.ts` | Covered |
47+
| **Total** | **370** | **13 files** | |
4448

4549
---
4650

@@ -172,7 +176,7 @@ Correlates documented features with test coverage. The project includes 640 Pyth
172176

173177
| Feature | Doc Reference | Test(s) | Notes |
174178
| --- | --- | --- | --- |
175-
| CONNECTOR_REGISTRY (8 entries) | `ConnectorCatalogue.md` | `test_has_8_connectors`, `test_all_have_connector_id` | Registry completeness |
179+
| CONNECTOR_REGISTRY (38 Python / 42 TS entries) | `ConnectorCatalogue.md` | `test_has_expected_connectors`, `test_all_have_connector_id` | Registry completeness |
176180
| OpenAI Provider | `connectors/openai-llm.md` | `TestOpenAIProvider` (7 tests) | ID, URL, headers, models, endpoint |
177181
| Anthropic Provider | `connectors/anthropic-llm.md` | `TestAnthropicProvider` (10 tests) | ID, headers, payload, response parsing |
178182
| Env Secret Store | `connectors/modelmesh-env.md` | `TestEnvSecretStoreComprehensive` (7 tests) | ID, resolve, prefix, missing, interface |
@@ -182,6 +186,11 @@ Correlates documented features with test coverage. The project includes 640 Pyth
182186
| Encrypted File Store | `guides/SecretStores.md` | `TestEncryptedFileSecretStore` (12 tests) | Save/load, passphrase, hex key, round-trip, plaintext check |
183187
| Keyring Secret Store | `ConnectorCatalogue.md` | `TestKeyringSecretStoreComprehensive` (6 tests) | ID, service name, availability, fallback |
184188
| Azure Speech TTS | `ConnectorCatalogue.md` | `TestAzureSpeechProvider` (14 tests) | ID, region, URL, headers, SSML, models, XML escape |
189+
| Ollama Provider | `ConnectorCatalogue.md` | `TestOllamaProvider` (8 tests) | ID, URL, empty API key, 4 default models, capabilities, endpoint, runtime |
190+
| LM Studio Provider | `ConnectorCatalogue.md` | `TestLMStudioProvider` (8 tests) | ID, URL, empty API key, empty models, capabilities, endpoint, runtime |
191+
| vLLM Provider | `ConnectorCatalogue.md` | `TestVLLMProvider` (8 tests) | ID, URL, empty API key, empty models, capabilities, endpoint, runtime |
192+
| LocalAI Provider | `ConnectorCatalogue.md` | `TestLocalAIProvider` (8 tests) | ID, URL, empty API key, empty models, capabilities, endpoint, runtime |
193+
| RuntimeEnvironment metadata | `ConnectorCatalogue.md` | `TestRuntimeEnvironment` (6 tests) | Enum values, classification of BaseProvider, MemoryStorage, MemorySecretStore |
185194
| Stick-Until-Failure | `connectors/modelmesh-stick-until-failure.md` | (via `TestBaseRotation`) | Tested through base rotation tests |
186195
| Local File Storage | `connectors/modelmesh-local-file.md` | (via `TestBaseStorage`) | Tested through base storage tests |
187196

@@ -195,7 +204,18 @@ Correlates documented features with test coverage. The project includes 640 Pyth
195204
| CORS proxy URL prefixing | `BrowserUsage.md` | `test_browser_proxy_url_prefix` | proxyUrl prepended to API URL |
196205
| createBrowser() convenience | `BrowserUsage.md` | `test_create_browser` | Browser-optimized create() |
197206

198-
### 12. Audio Interfaces (`docs/ConnectorInterfaces.md`)
207+
### 12. Browser Storage & Secret Stores (`docs/ConnectorCatalogue.md`, `docs/guides/BrowserUsage.md`)
208+
209+
| Feature | Doc Reference | Test(s) | Notes |
210+
| --- | --- | --- | --- |
211+
| LocalStorageStorage connector ID | `ConnectorCatalogue.md` | `connectors.test.ts` | `modelmesh.localstorage.v1` |
212+
| SessionStorageStorage connector ID | `ConnectorCatalogue.md` | `connectors.test.ts` | `modelmesh.sessionstorage.v1` |
213+
| IndexedDBStorage connector ID | `ConnectorCatalogue.md` | `connectors.test.ts` | `modelmesh.indexeddb.v1` |
214+
| BrowserSecretStore connector ID | `ConnectorCatalogue.md` | `connectors.test.ts` | `modelmesh.browser-secrets.v1` |
215+
| Browser storage RuntimeEnvironment | `ConnectorCatalogue.md` | `connectors.test.ts` | BROWSER_ONLY classification |
216+
| Browser exports (browser.ts) | `BrowserUsage.md` | `browser-provider.test.ts` | All browser connectors exported |
217+
218+
### 13. Audio Interfaces (`docs/ConnectorInterfaces.md`)
199219

200220
| Feature | Doc Reference | Test(s) | Notes |
201221
| --- | --- | --- | --- |
@@ -204,7 +224,7 @@ Correlates documented features with test coverage. The project includes 640 Pyth
204224
| client.audio.speech.create() | `ConnectorCatalogue.md` | `test_audio_speech_create` | TTS routing through pool |
205225
| client.audio.transcriptions.create() | `ConnectorCatalogue.md` | `test_audio_transcriptions_create` | STT routing through pool |
206226

207-
### 13. CDK Specialized Classes (`docs/cdk/BaseClasses.md`)
227+
### 14. CDK Specialized Classes (`docs/cdk/BaseClasses.md`)
208228

209229
| Feature | Doc Reference | Test(s) | Notes |
210230
| --- | --- | --- | --- |
@@ -237,7 +257,7 @@ Correlates documented features with test coverage. The project includes 640 Pyth
237257
| Document | Features Documented | Tests Covering | Coverage |
238258
| --- | --- | --- | --- |
239259
| `SystemConcept.md` | Architecture overview | All tests collectively | Indirect |
240-
| `ConnectorCatalogue.md` | 8 connectors, IDs, configs | `test_connectors.py` (32 tests) | Direct |
260+
| `ConnectorCatalogue.md` | 42 connectors, IDs, configs | `test_connectors.py` + `test_new_connectors.py` + `test_providers.py` + `connectors.test.ts` | Direct |
241261
| `ConnectorInterfaces.md` | 6 interface ABCs | `test_interfaces.py` (22 tests) | Direct |
242262
| `SystemConfiguration.md` | MeshConfig, auto-detect | `test_config.py` (19 tests) | Direct |
243263
| `ModelCapabilities.md` | Capability tree hierarchy | `TestCapabilityTree` (9 tests) | Direct |

0 commit comments

Comments
 (0)