-
Notifications
You must be signed in to change notification settings - Fork 19
chore: PoC wrapping official OpenAI SDK #822
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
ZhongpinWang
wants to merge
2
commits into
main
Choose a base branch
from
poc-openai-sdk-wrapper
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,120 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
| <parent> | ||
| <groupId>com.sap.ai.sdk</groupId> | ||
| <artifactId>sdk-parent</artifactId> | ||
| <version>1.18.0-SNAPSHOT</version> | ||
| <relativePath>../../pom.xml</relativePath> | ||
| </parent> | ||
| <artifactId>openai-wrapper</artifactId> | ||
| <name>SAP AI SDK - OpenAI Wrapper</name> | ||
| <description>SAP AI SDK wrapper around the official OpenAI Java SDK for use with SAP AI Core.</description> | ||
|
|
||
| <url>https://github.com/SAP/ai-sdk-java?tab=readme-ov-file#documentation</url> | ||
| <organization> | ||
| <name>SAP SE</name> | ||
| <url>https://www.sap.com</url> | ||
| </organization> | ||
| <licenses> | ||
| <license> | ||
| <name>The Apache Software License, Version 2.0</name> | ||
| <url>https://www.apache.org/licenses/LICENSE-2.0.txt</url> | ||
| </license> | ||
| </licenses> | ||
| <developers> | ||
| <developer> | ||
| <name>SAP</name> | ||
| <email>cloudsdk@sap.com</email> | ||
| <organization>SAP SE</organization> | ||
| <organizationUrl>https://www.sap.com</organizationUrl> | ||
| </developer> | ||
| </developers> | ||
| <properties> | ||
| <project.rootdir>${project.basedir}/../../</project.rootdir> | ||
| <coverage.instruction>0%</coverage.instruction> | ||
| <coverage.branch>0%</coverage.branch> | ||
| <coverage.line>0%</coverage.line> | ||
| <coverage.complexity>0%</coverage.complexity> | ||
| <coverage.method>0%</coverage.method> | ||
| <coverage.class>0%</coverage.class> | ||
| </properties> | ||
|
|
||
| <dependencies> | ||
| <!-- scope "compile" --> | ||
| <dependency> | ||
| <groupId>com.sap.ai.sdk</groupId> | ||
| <artifactId>core</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>com.sap.cloud.sdk.cloudplatform</groupId> | ||
| <artifactId>cloudplatform-connectivity</artifactId> | ||
| </dependency> | ||
| <!-- Official OpenAI Java SDK --> | ||
| <dependency> | ||
| <groupId>com.openai</groupId> | ||
| <artifactId>openai-java</artifactId> | ||
| <version>4.31.0</version> | ||
| <exclusions> | ||
| <exclusion> | ||
| <groupId>org.springframework</groupId> | ||
| <artifactId>spring-core</artifactId> | ||
| </exclusion> | ||
| </exclusions> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>com.google.code.findbugs</groupId> | ||
| <artifactId>jsr305</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.slf4j</groupId> | ||
| <artifactId>slf4j-api</artifactId> | ||
| </dependency> | ||
| <!-- scope "provided" --> | ||
| <dependency> | ||
| <groupId>org.projectlombok</groupId> | ||
| <artifactId>lombok</artifactId> | ||
| <scope>provided</scope> | ||
| </dependency> | ||
| <!-- scope "test" --> | ||
| <dependency> | ||
| <groupId>org.junit.jupiter</groupId> | ||
| <artifactId>junit-jupiter-api</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.assertj</groupId> | ||
| <artifactId>assertj-core</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.mockito</groupId> | ||
| <artifactId>mockito-core</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| </dependencies> | ||
|
|
||
| <build> | ||
| <plugins> | ||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-dependency-plugin</artifactId> | ||
| <configuration> | ||
| <ignoredUsedUndeclaredDependencies> | ||
| <!-- openai-java-core is a transitive dep of the declared openai-java umbrella --> | ||
| <ignoredUsedUndeclaredDependency>com.openai:openai-java-core</ignoredUsedUndeclaredDependency> | ||
| <ignoredUsedUndeclaredDependency>com.openai:openai-java-client-okhttp</ignoredUsedUndeclaredDependency> | ||
| <!-- connectivity-apache-httpclient5 is a transitive dep brought by core --> | ||
| <ignoredUsedUndeclaredDependency>com.sap.cloud.sdk.cloudplatform:connectivity-apache-httpclient5</ignoredUsedUndeclaredDependency> | ||
| </ignoredUsedUndeclaredDependencies> | ||
| <ignoredUnusedDeclaredDependencies> | ||
| <!-- openai-java is the umbrella dependency that brings openai-java-core and openai-java-client-okhttp --> | ||
| <ignoredUnusedDeclaredDependency>com.openai:openai-java</ignoredUnusedDeclaredDependency> | ||
| <!-- cloudplatform-connectivity provides the Destination API used by this module --> | ||
| <ignoredUnusedDeclaredDependency>com.sap.cloud.sdk.cloudplatform:cloudplatform-connectivity</ignoredUnusedDeclaredDependency> | ||
| </ignoredUnusedDeclaredDependencies> | ||
| </configuration> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
| </project> |
148 changes: 148 additions & 0 deletions
148
...per/src/main/java/com/sap/ai/sdk/foundationmodels/openaiwrapper/OpenAiClientProvider.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,148 @@ | ||
| package com.sap.ai.sdk.foundationmodels.openaiwrapper; | ||
|
|
||
| import com.openai.client.OpenAIClient; | ||
| import com.openai.client.okhttp.OpenAIOkHttpClient; | ||
| import com.openai.credential.BearerTokenCredential; | ||
| import com.sap.ai.sdk.core.AiCoreService; | ||
| import com.sap.ai.sdk.core.AiModel; | ||
| import com.sap.cloud.sdk.cloudplatform.connectivity.DefaultHttpDestination; | ||
| import com.sap.cloud.sdk.cloudplatform.connectivity.Header; | ||
| import com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination; | ||
| import java.util.Collection; | ||
| import java.util.Locale; | ||
| import javax.annotation.Nonnull; | ||
| import lombok.extern.slf4j.Slf4j; | ||
|
|
||
| /** | ||
| * Provider for the official OpenAI Java SDK client, configured to work with SAP AI Core. | ||
| * | ||
| * <p>This is a thin wrapper around the <a href="https://github.com/openai/openai-java">official | ||
| * OpenAI Java SDK</a> that handles: | ||
| * | ||
| * <ul> | ||
| * <li><b>Base URL construction</b>: Resolves the correct deployment URL from SAP AI Core using | ||
| * the model name (and optionally resource group). | ||
| * <li><b>Authentication</b>: Injects the SAP AI Core OAuth bearer token into each request via a | ||
| * dynamic {@link BearerTokenCredential}. | ||
| * </ul> | ||
| * | ||
| * <p>The returned {@link OpenAIClient} is the official SDK client and can be used with the full | ||
| * OpenAI API surface (chat completions, embeddings, streaming, function calling, etc.) as | ||
| * documented in the <a href="https://platform.openai.com/docs">OpenAI API documentation</a>. | ||
| * | ||
| * <p><b>No additional abstraction layers are added.</b> Request and response types are the official | ||
| * SDK types. | ||
| * | ||
| * <p>Example usage: | ||
| * | ||
| * <pre>{@code | ||
| * import com.openai.models.chat.completions.ChatCompletionCreateParams; | ||
| * import com.openai.models.ChatModel; | ||
| * import com.openai.models.chat.completions.ChatCompletion; | ||
| * | ||
| * OpenAIClient client = OpenAiClientProvider.forModel(OpenAiModel.GPT_4O); | ||
| * | ||
| * ChatCompletionCreateParams params = ChatCompletionCreateParams.builder() | ||
| * .addUserMessage("Hello, world!") | ||
| * .model(ChatModel.GPT_4O) | ||
| * .build(); | ||
| * | ||
| * ChatCompletion completion = client.chat().completions().create(params); | ||
| * }</pre> | ||
| * | ||
| * @see <a href="https://github.com/openai/openai-java">OpenAI Java SDK</a> | ||
| * @see <a href="https://api.sap.com/api/AI_CORE_API/overview">SAP AI Core</a> | ||
| */ | ||
| @Slf4j | ||
| public final class OpenAiClientProvider { | ||
|
|
||
| private OpenAiClientProvider() {} | ||
|
|
||
| /** | ||
| * Creates an {@link OpenAIClient} for the given model using the default {@link AiCoreService}. | ||
| * | ||
| * <p>This resolves the deployment URL for the given model from SAP AI Core and configures the | ||
| * official OpenAI client with the correct base URL and authentication. | ||
| * | ||
| * @param model The {@link AiModel} to use (e.g. an {@code OpenAiModel} constant). | ||
| * @return A configured {@link OpenAIClient} ready to make requests via SAP AI Core. | ||
| */ | ||
| @Nonnull | ||
| public static OpenAIClient forModel(@Nonnull final AiModel model) { | ||
| return forModel(new AiCoreService(), model); | ||
| } | ||
|
|
||
| /** | ||
| * Creates an {@link OpenAIClient} for the given model using the provided {@link AiCoreService}. | ||
| * | ||
| * @param aiCoreService The {@link AiCoreService} to use for deployment resolution. | ||
| * @param model The {@link AiModel} to use. | ||
| * @return A configured {@link OpenAIClient} ready to make requests via SAP AI Core. | ||
| */ | ||
| @Nonnull | ||
| public static OpenAIClient forModel( | ||
| @Nonnull final AiCoreService aiCoreService, @Nonnull final AiModel model) { | ||
| final HttpDestination destination = aiCoreService.getInferenceDestination().forModel(model); | ||
| return buildClient(destination); | ||
| } | ||
|
|
||
| /** | ||
| * Creates an {@link OpenAIClient} using a custom {@link HttpDestination}. | ||
| * | ||
| * <p>This is useful for advanced scenarios where you need full control over the destination | ||
| * configuration. | ||
| * | ||
| * @param destination The destination to use. | ||
| * @return A configured {@link OpenAIClient}. | ||
| */ | ||
| @Nonnull | ||
| public static OpenAIClient withCustomDestination(@Nonnull final HttpDestination destination) { | ||
| return buildClient(destination); | ||
| } | ||
|
|
||
| @Nonnull | ||
| private static OpenAIClient buildClient(@Nonnull final HttpDestination destination) { | ||
| final DefaultHttpDestination defaultDest = (DefaultHttpDestination) destination; | ||
| final String baseUrl = defaultDest.getUri().toString() + "v1/"; | ||
| log.debug("Building OpenAI client with base URL: {}", baseUrl); | ||
|
|
||
| return OpenAIOkHttpClient.builder() | ||
| .baseUrl(baseUrl) | ||
| .credential(BearerTokenCredential.create(() -> extractBearerToken(defaultDest))) | ||
| .putHeader("AI-Resource-Group", getResourceGroupHeader(defaultDest)) | ||
| .build(); | ||
| } | ||
|
|
||
| /** | ||
| * Extracts the bearer token from the destination's authorization headers. | ||
| * | ||
| * <p>The SAP Cloud SDK destination resolves the OAuth token from the service binding. This method | ||
| * extracts it so it can be passed to the OpenAI SDK's credential mechanism. | ||
| */ | ||
| @Nonnull | ||
| static String extractBearerToken(@Nonnull final DefaultHttpDestination destination) { | ||
| final Collection<Header> headers = destination.getHeaders(destination.getUri()); | ||
| return headers.stream() | ||
| .filter(h -> "Authorization".equalsIgnoreCase(h.getName())) | ||
| .map(Header::getValue) | ||
| .filter(v -> v.toLowerCase(Locale.ROOT).startsWith("bearer ")) | ||
| .map(v -> v.substring("Bearer ".length()).trim()) | ||
| .findFirst() | ||
| .orElseThrow( | ||
| () -> | ||
| new IllegalStateException( | ||
| "No Bearer token found in destination headers. " | ||
| + "Ensure the destination is configured with OAuth authentication.")); | ||
| } | ||
|
|
||
| /** Extracts the AI-Resource-Group header value from the destination, defaulting to "default". */ | ||
| @Nonnull | ||
| static String getResourceGroupHeader(@Nonnull final DefaultHttpDestination destination) { | ||
| final Collection<Header> headers = destination.getHeaders(destination.getUri()); | ||
| return headers.stream() | ||
| .filter(h -> "AI-Resource-Group".equalsIgnoreCase(h.getName())) | ||
| .map(Header::getValue) | ||
| .findFirst() | ||
| .orElse("default"); | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(Major)
Afaik, OpenAI SDK do not handle token refresh. So effectively, the obtained instance of
OpenAIOkHttpClientalso "expires" which was the concern I had when I started out here.So, I add an adapter (custom
com.openai.core.http.HttpClient) in my PR as we have a familiar Apache client that handles destination logic (incl token refresh)(#794).