Skip to content

Commit 98703d7

Browse files
danciaclaradanciaclara
authored
Plane AI for Commercial Edition (#219)
* Plane AI for Commercial Edition * formatting fixes * formatting fixes
1 parent 7633762 commit 98703d7

4 files changed

Lines changed: 544 additions & 0 deletions

File tree

‎docs/.vitepress/config.mts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -273,6 +273,12 @@ export default withMermaid(
273273
},
274274
{ text: "DNS for Intake Email", link: "/self-hosting/govern/configure-dns-email-service" },
275275
{ text: "OpenSearch for search", link: "/self-hosting/govern/advanced-search" },
276+
{
277+
text: "Plane AI",
278+
link: "/self-hosting/govern/plane-ai",
279+
collapsed: true,
280+
items: [{ text: "AWS OpenSearch embedding", link: "/self-hosting/govern/aws-opensearch-embedding" }],
281+
},
276282
{ text: "External secrets", link: "/self-hosting/govern/external-secrets" },
277283
{ text: "External reverse proxy", link: "/self-hosting/govern/reverse-proxy" },
278284
{ text: "Private storage buckets", link: "/self-hosting/govern/private-bucket" },

‎docs/self-hosting/govern/aws-opensearch-embedding.md‎

Lines changed: 233 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,233 @@
1+
---
2+
title: AWS OpenSearch embedding
3+
description: Step-by-step guide to deploying a Cohere embedding model on AWS OpenSearch for use with Plane AI semantic search.
4+
keywords: aws opensearch, embedding model, cohere, plane ai, semantic search, ml commons, opensearch connector
5+
---
6+
7+
# Deploy an embedding model on AWS OpenSearch
8+
9+
This guide walks you through deploying a Cohere embedding model on AWS OpenSearch (managed) for Plane AI semantic search.
10+
11+
For other connector blueprints and embedding model configurations, see the [OpenSearch ML Commons remote inference blueprints](https://github.com/opensearch-project/ml-commons/tree/2.x/docs/remote_inference_blueprints).
12+
13+
## Before you begin
14+
15+
Make sure you have:
16+
17+
- An AWS OpenSearch domain with **fine-grained access control** enabled.
18+
- Admin access to OpenSearch Dashboards.
19+
- AWS CLI configured locally.
20+
- An IAM user with permissions to create roles, policies, and access Secrets Manager.
21+
- A Cohere API key.
22+
23+
## Create an IAM policy
24+
25+
1. Go to **IAM → Policies → Create Policy**.
26+
2. Select **JSON** and paste the following:
27+
28+
```json
29+
{
30+
"Version": "2012-10-17",
31+
"Statement": [
32+
{
33+
"Sid": "PassRoleAccess",
34+
"Effect": "Allow",
35+
"Action": "iam:PassRole",
36+
"Resource": "arn:aws:iam::<aws-account-number>:role/plane-opensearch-access-role"
37+
},
38+
{
39+
"Sid": "SecretManagerAccess",
40+
"Effect": "Allow",
41+
"Action": ["secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret", "secretsmanager:ListSecrets"],
42+
"Resource": "*"
43+
}
44+
]
45+
}
46+
```
47+
48+
3. Click **Next**, name the policy `plane-opensearch-access-policy`, and click **Create Policy**.
49+
50+
## Create an IAM role
51+
52+
Create an IAM role that OpenSearch can assume to access Secrets Manager.
53+
54+
1. Go to **IAM → Roles → Create Role**.
55+
2. Name the role `plane-opensearch-access-role`.
56+
3. Set this **Trust Relationship**:
57+
58+
```json
59+
{
60+
"Version": "2012-10-17",
61+
"Statement": [
62+
{
63+
"Effect": "Allow",
64+
"Principal": {
65+
"Service": "ec2.amazonaws.com"
66+
},
67+
"Action": "sts:AssumeRole"
68+
},
69+
{
70+
"Effect": "Allow",
71+
"Principal": {
72+
"AWS": "arn:aws:iam::<aws-account-number>:user/<opensearch-user>"
73+
},
74+
"Action": "sts:AssumeRole"
75+
}
76+
]
77+
}
78+
```
79+
80+
4. Attach the `plane-opensearch-access-policy` you created in Step 1.
81+
5. Click **Create Role** and note the role ARN.
82+
83+
## Grant ML permissions to the IAM role
84+
85+
1. Open **OpenSearch Dashboards**.
86+
2. Go to **Security → Roles → `ml_full_access`**.
87+
3. Open the **Mapped users** tab and click **Map users**.
88+
4. Under **Backend roles**, add the role ARN:
89+
90+
```
91+
arn:aws:iam::<aws-account-number>:role/plane-opensearch-access-role
92+
```
93+
94+
## Assume the role locally
95+
96+
Run this command to get temporary credentials:
97+
98+
```bash
99+
aws sts assume-role \
100+
--role-arn arn:aws:iam::<aws-account-number>:role/plane-opensearch-access-role \
101+
--role-session-name session
102+
```
103+
104+
Export the credentials from the response:
105+
106+
```bash
107+
export AWS_ACCESS_KEY_ID=<AccessKeyId>
108+
export AWS_SECRET_ACCESS_KEY=<SecretAccessKey>
109+
export AWS_SESSION_TOKEN=<SessionToken>
110+
```
111+
112+
## Store the Cohere API key in Secrets Manager
113+
114+
1. Go to **Secrets Manager → Store a new secret**.
115+
2. Select **Other type of secret**.
116+
3. Set the key to `azure_ai_foundry_key_cohere` and the value to your Cohere API key.
117+
4. Click **Next**, name the secret `plane-ai/cohere`, and click **Store**.
118+
5. Note the **Secret ARN** — you'll need it in the next step.
119+
120+
## Create a Cohere connector in OpenSearch
121+
122+
Using the temporary credentials from Step 4, send a `POST` request to your OpenSearch cluster.
123+
124+
**Endpoint:** `POST https://<opensearch-domain>/_plugins/_ml/connectors/_create`
125+
126+
**Request body:**
127+
128+
```json
129+
{
130+
"name": "Cohere",
131+
"description": "Cohere embedding connector",
132+
"version": "1",
133+
"protocol": "http",
134+
"parameters": {
135+
"endpoint": "https://azureaitrials-resource.services.ai.azure.com/models",
136+
"model": "embed-v-4-0",
137+
"api_version": "2024-05-01-preview",
138+
"input_type": "search_document",
139+
"truncate": "END"
140+
},
141+
"credential": {
142+
"secretArn": "<cohere-secret-arn>",
143+
"roleArn": "arn:aws:iam::<aws-account-number>:role/plane-opensearch-access-role"
144+
},
145+
"actions": [
146+
{
147+
"action_type": "predict",
148+
"method": "POST",
149+
"url": "${parameters.endpoint}/embeddings?api-version=${parameters.api_version}",
150+
"headers": {
151+
"api-key": "${credential.secretArn.azure_ai_foundry_key_cohere}",
152+
"x-ms-model-mesh-model-name": "embed-v-4-0"
153+
},
154+
"request_body": "{ \"texts\": ${parameters.texts}, \"truncate\": \"${parameters.truncate}\", \"model\": \"${parameters.model}\", \"input_type\": \"${parameters.input_type}\" }",
155+
"pre_process_function": "connector.pre_process.cohere.embedding",
156+
"post_process_function": "connector.post_process.cohere.embedding"
157+
}
158+
]
159+
}
160+
```
161+
162+
Save the `connector_id` from the response.
163+
164+
## Configure the OpenSearch cluster
165+
166+
Run these commands in **Dev Tools** in OpenSearch Dashboards.
167+
168+
### Allow the connector's external endpoints
169+
170+
```json
171+
PUT /_cluster/settings
172+
{
173+
"persistent": {
174+
"plugins.ml_commons.trusted_connector_endpoints_regex": [
175+
"^https://api\\.cohere\\.ai(/.*)?$",
176+
"^https://azureaitrials-resource\\.services\\.ai\\.azure\\.com(/.*)?$"
177+
]
178+
}
179+
}
180+
```
181+
182+
### Register the embedding model
183+
184+
```json
185+
POST /_plugins/_ml/models/_register
186+
{
187+
"name": "cohere_4_0_embed",
188+
"function_name": "remote",
189+
"connector_id": "<connector-id>",
190+
"description": "Cohere Embedding Model"
191+
}
192+
```
193+
194+
Save the `model_id` from the response.
195+
196+
### Deploy the model
197+
198+
```
199+
POST /_plugins/_ml/models/<model_id>/_deploy
200+
```
201+
202+
### Verify deployment status
203+
204+
```
205+
GET /_plugins/_ml/models/<model_id>
206+
```
207+
208+
Wait until the response shows:
209+
210+
```json
211+
"model_state": "DEPLOYED"
212+
```
213+
214+
### Test inference (optional)
215+
216+
```json
217+
POST /_plugins/_ml/models/<model_id>/_predict
218+
{
219+
"parameters": {
220+
"inputs": ["hello world"]
221+
}
222+
}
223+
```
224+
225+
## Configure Plane
226+
227+
Add the deployed model ID to `/opt/plane/plane.env`:
228+
229+
```bash
230+
EMBEDDING_MODEL_ID=<model_id>
231+
```
232+
233+
Restart Plane and complete the remaining steps in [Enable Plane AI](/self-hosting/govern/plane-ai#configure-an-embedding-model).

‎docs/self-hosting/govern/environment-variables.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,62 @@ This is where you'll make all configuration changes. Remember to restart the ins
152152
| `OPENSEARCH_PASSWORD` | Authentication password | `your-secure-password` |
153153
| `OPENSEARCH_INDEX_PREFIX` | Prefix for all index names (useful for multi-tenant setups) | (empty) |
154154

155+
### Plane AI
156+
157+
#### Plane AI replicas
158+
159+
To start Plane AI services, set each replica count to `1`:
160+
161+
| Variable | Description | Required |
162+
| ---------------------- | ---------------------------------- | -------- |
163+
| `PI_API_REPLICAS` | Plane AI API replica count | Yes |
164+
| `PI_BEAT_REPLICAS` | Plane AI Beat Worker replica count | Yes |
165+
| `PI_WORKER_REPLICAS` | Plane AI Worker replica count | Yes |
166+
| `PI_MIGRATOR_REPLICAS` | Plane AI Migrator replica count | Yes |
167+
168+
#### LLM provider API keys
169+
170+
Plane AI supports multiple LLM providers. Configure one or more by adding their API keys.
171+
172+
| Variable | Description | Required |
173+
| ------------------------ | --------------------------------------------------------------- | -------- |
174+
| `OPENAI_API_KEY` | API key for OpenAI models | Optional |
175+
| `CLAUDE_API_KEY` | API key for Anthropic models | Optional |
176+
| `GROQ_API_KEY` | API key for speech-to-text features | Optional |
177+
| `CUSTOM_LLM_ENABLED` | Set to `true` to use a custom LLM with an OpenAI-compatible API | Optional |
178+
| `CUSTOM_LLM_MODEL_KEY` | Identifier key for the custom model | Optional |
179+
| `CUSTOM_LLM_BASE_URL` | Base URL of the custom model's OpenAI-compatible endpoint | Optional |
180+
| `CUSTOM_LLM_API_KEY` | API key for the custom endpoint | Optional |
181+
| `CUSTOM_LLM_NAME` | Display name for the custom model | Optional |
182+
| `CUSTOM_LLM_DESCRIPTION` | Description of the custom model | Optional |
183+
| `CUSTOM_LLM_MAX_TOKENS` | Maximum token limit for the custom model | Optional |
184+
185+
#### Provider base URLs
186+
187+
Use these when routing requests through self-hosted gateways, proxies, or compatible third-party endpoints.
188+
189+
| Variable | Description | Default |
190+
| ----------------- | ------------------------------------------ | --------- |
191+
| `OPENAI_BASE_URL` | Custom base URL for OpenAI-compatible APIs | OpenAI |
192+
| `CLAUDE_BASE_URL` | Custom base URL for Claude-compatible APIs | Anthropic |
193+
| `COHERE_BASE_URL` | Custom base URL for Cohere APIs | Cohere |
194+
| `GROQ_BASE_URL` | Custom base URL for Groq APIs | Groq |
195+
196+
#### Embedding model configuration
197+
198+
These settings are required for semantic search and Plane AI Chat. Configure one of the following options.
199+
200+
| Variable | Description | Required |
201+
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------- |
202+
| `EMBEDDING_MODEL_ID` | ID of an existing embedding model deployed in OpenSearch (works with both self-hosted and AWS OpenSearch) | Conditional |
203+
| `EMBEDDING_MODEL` | Embedding model to automatically deploy (e.g., `cohere/embed-v4.0`, `openai/text-embedding-3-small`, `bedrock/amazon.titan-embed-text-v1`). Self-hosted OpenSearch only. | Conditional |
204+
| `COHERE_API_KEY` | API key for Cohere embedding models | Conditional |
205+
| `BR_AWS_ACCESS_KEY_ID` | AWS access key ID for Bedrock Titan embedding | Conditional |
206+
| `BR_AWS_SECRET_ACCESS_KEY` | AWS secret access key for Bedrock Titan embedding | Conditional |
207+
| `BR_AWS_REGION` | AWS region for Bedrock Titan embedding | Conditional |
208+
209+
For setup instructions, supported models, and IAM permissions, see [Enable Plane AI](/self-hosting/govern/plane-ai).
210+
155211
### API settings
156212

157213
| Variable | Description | Default Value |

0 commit comments

Comments
 (0)