Skip to content

Commit 38dfa34

Browse files
TillK17TillK17
authored
PR feedback
1 parent 76a7356 commit 38dfa34

1 file changed

Lines changed: 36 additions & 64 deletions

File tree

‎tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md‎

Lines changed: 36 additions & 64 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ author_profile: https://github.com/I321506
3535

3636
[OPTION END]
3737

38-
[OPTION BEGIN [Gen AI Hub SDK]]
38+
[OPTION BEGIN [Python SDK]]
3939

4040
• Configure proxy modules by setting up environment variables for AI Core credentials.
4141

@@ -48,7 +48,7 @@ author_profile: https://github.com/I321506
4848

4949
[OPTION END]
5050

51-
[OPTION BEGIN [SAP Cloud SDK]]
51+
[OPTION BEGIN [JavaScript SDK]]
5252

5353
• Download the service key for the AI Core service instance.
5454

@@ -83,7 +83,7 @@ console.log(process.env.AICORE_SERVICE_KEY);
8383

8484
[OPTION END]
8585

86-
[OPTION BEGIN [SAP Cloud SDK for Java]]
86+
[OPTION BEGIN [Java SDK]]
8787

8888
• [Create a service key](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-service-key) for your AI Core instance and copy the JSON object.
8989

@@ -163,7 +163,7 @@ Go to the Configuration section within your chosen Resource Group.
163163

164164
[OPTION END]
165165

166-
[OPTION BEGIN [Gen AI SDK]]
166+
[OPTION BEGIN [Python SDK]]
167167

168168
• Create a folder named orchestration, then navigate to this folder using VS Code.
169169

@@ -188,7 +188,7 @@ config = ai_core_client.configuration.create(
188188
    name=config_name
189189
)
190190
print(f"Configuration created successfully with ID: {config.id} and Name: {config_name}")
191-
191+
```
192192

193193
[OPTION END]
194194

@@ -255,7 +255,7 @@ Go to the Configuration section within your chosen Resource Group.
255255

256256
[OPTION END]
257257

258-
[OPTION BEGIN [Gen AI SDK]]
258+
[OPTION BEGIN [Python SDK]]
259259

260260
• Create a folder named orchestration, then navigate to this folder using VS Code.
261261

@@ -292,7 +292,7 @@ print(f"Configuration created successfully with ID: {config.id} and Name: {confi
292292

293293
[OPTION END]
294294

295-
[OPTION BEGIN [SAP Cloud SDK ]]
295+
[OPTION BEGIN [JavaScript SDK]]
296296

297297
In this step, we define a function to create an orchestration configuration using the ConfigurationApi from the SAP AI SDK. This configuration integrates various parameters needed for orchestration, such as the executable ID and scenario ID.
298298

@@ -347,7 +347,7 @@ orchestrationConfig;
347347

348348
[OPTION END]
349349

350-
[OPTION BEGIN [SAP Cloud SDK for Java]]
350+
[OPTION BEGIN [Java SDK]]
351351

352352
In this step, we will create an orchestration configuration using the core module of the SAP Cloud SDK for Java in a maven project. This configuration integrates various parameters needed for orchestration, such as the executable ID and scenario ID.
353353

@@ -394,14 +394,6 @@ var configuration = new ConfigurationApi().create(RESOURCE_GROUP, configurationD
394394
// Print the configuration response message
395395
System.out.println(configuration.getMessage());
396396
```
397-
• If not done automaticaly by your IDE, add the following imports:
398-
399-
```java
400-
import com.sap.ai.sdk.core.client.ConfigurationApi;
401-
import com.sap.ai.sdk.core.model.AiArtifactArgumentBinding;
402-
import com.sap.ai.sdk.core.model.AiConfigurationBaseData;
403-
import com.sap.ai.sdk.core.model.AiParameterArgumentBinding;
404-
```
405397

406398
**Note**:
407399

@@ -447,7 +439,7 @@ Once the deployment begins, continue to the status page. Verify that the Deploym
447439

448440
[OPTION END]
449441

450-
[OPTION BEGIN [Gen AI SDK]]
442+
[OPTION BEGIN [Python SDK]]
451443

452444
With the configuration ID, you can proceed to deploy the orchestration and monitor its progress.
453445

@@ -505,7 +497,7 @@ Result: The code will display a loading spinner until the deployment status upda
505497

506498
[OPTION END]
507499

508-
[OPTION BEGIN [SAP Cloud SDK ]]
500+
[OPTION BEGIN [JavaScript SDK]]
509501

510502
This step involves creating a deployment using the specified configuration and resource group. The deployment is handled via the DeploymentApi, which streamlines the process of activating the orchestration setup.
511503

@@ -570,7 +562,7 @@ export async function deployOrchestration(
570562

571563
[OPTION END]
572564

573-
[OPTION BEGIN [SAP Cloud SDK for Java]]
565+
[OPTION BEGIN [Java SDK]]
574566

575567
In this step, we will create a deployment from the configuration created in the last step using the core module of the SAP Cloud SDK for Java.
576568

@@ -587,12 +579,6 @@ var deployment = new DeploymentApi().create(RESOURCE_GROUP, deploymentCreationRe
587579
// Print the deployment response message
588580
System.out.println(deployment.getMessage());
589581
```
590-
• If not done automaticaly by your IDE, add the following imports:
591-
592-
```java
593-
import com.sap.ai.sdk.core.client.DeploymentApi;
594-
import com.sap.ai.sdk.core.model.AiDeploymentCreationRequest;
595-
```
596582

597583
[OPTION END]
598584

@@ -774,7 +760,7 @@ Data masking and content filtering are available to enhance data privacy and saf
774760

775761
[OPTION END]
776762

777-
[OPTION BEGIN [Gen AI SDK]]
763+
[OPTION BEGIN [Python SDK]]
778764

779765
To begin the consumption process for the orchestration you’ve deployed, follow the process below:
780766

@@ -910,7 +896,7 @@ Data masking and content filtering are available to enhance data privacy and saf
910896

911897
[OPTION END]
912898

913-
[OPTION BEGIN [SAP Cloud SDK ]]
899+
[OPTION BEGIN [JavaScript SDK]]
914900

915901
To begin the consumption process for the orchestration you’ve deployed, follow the process below:
916902

@@ -1071,7 +1057,7 @@ Data masking and content filtering are available to enhance data privacy and saf
10711057

10721058
[OPTION END]
10731059

1074-
[OPTION BEGIN [SAP Cloud SDK for Java]]
1060+
[OPTION BEGIN [Java SDK]]
10751061

10761062
In this step, we will consume an LLM through the orchestration service with the created deployment, using the core and orchestration module of the SAP Cloud SDK for Java.
10771063

@@ -1107,42 +1093,40 @@ The next step involves creating the prompt for the LLM including both `SystemMes
11071093

11081094
```java
11091095
// Define system and user messages for prompt
1110-
var systemMessage = new SystemMessage(
1096+
var systemMessage = Message.system(
11111097
"""
11121098
You are an AI assistant designed to screen resumes for HR purposes.
11131099
Please assess the candidate qualifications based on the provided resume.
11141100
"""
11151101
);
1116-
var userMessage = new UserMessage("Candidate Resume: \n" + cvContent);
1102+
var userMessage = Message.user("Candidate Resume: \n" + cvContent);
11171103

11181104
// Define the prompt for resume screening
11191105
var prompt = new OrchestrationPrompt(systemMessage, userMessage);
11201106

11211107
```
11221108

1123-
We can define multiple models for the use case. Only use those models that are already deployed in your instances. For this example, we have selected the following three models:
1109+
1110+
We can define model parameters and a list of models to use. Only use those models that are already deployed in your instances. For this example, we have selected the following parameters and models:
11241111

11251112
```java
1126-
// List of models to iterate through
1127-
var models = List.of("gpt-4o", "mistralai--mistral-large-instruct", "anthropic--claude-3.5-sonnet");
1128-
```
11291113

1130-
With the following function we create an `OrchestrationModuleConfig` containing information about the `LLMModule`. This can be extended to contain information regarding templating, masking, filtering and grounding, if desired to use these functionality of orchestration.
1114+
// Map of model parameters, can be adapted if desired
1115+
Map<String, Object> modelParams = Map.of(
1116+
"max_tokens", 1000,
1117+
"temperature", 0.6
1118+
);
1119+
1120+
// List of models to iterate through, can be adapted if desired
1121+
var models = List.of(
1122+
OrchestrationAiModel.GPT_4O.withParams(modelParams),
1123+
OrchestrationAiModel.MISTRAL_LARGE_INSTRUCT.withParams(modelParams),
1124+
OrchestrationAiModel.CLAUDE_3_5_SONNET.withParams(modelParams)
1125+
);
11311126

1132-
```java
1133-
// Function to create orchestration module configuration
1134-
OrchestrationModuleConfig createModuleConfig(String modelName) {
1135-
var config = LLMModuleConfig.create()
1136-
.modelName(modelName)
1137-
.modelParams(Map.of( // add model parameters as needed
1138-
"max_tokens", 1000,
1139-
"temperature", 0.6
1140-
));
1141-
1142-
return new OrchestrationModuleConfig().withLlmConfig(config);
1143-
}
11441127
```
11451128

1129+
11461130
The following function writes the responses from different models, stored in a list, to a file:
11471131

11481132
```java
@@ -1151,12 +1135,12 @@ void createFileFromResponses (ArrayList<Map> responses) {
11511135
// Format model responses
11521136
var formattedResponses = responses.stream().
11531137
map(response -> "Response from model " + response.get("model") +
1154-
": \n\n" + response.get("response") + "\n" + "-".repeat(120));
1138+
": \n\n" + response.get("response"));
11551139

11561140
// Write model responses to provided file path
11571141
try {
11581142
Files.writeString(Path.of("src/main/resources/static/responses.txt"),
1159-
String.join("\n", formattedResponses.toList()));
1143+
String.join("\n\n" + "-".repeat(120) + "\n\n", formattedResponses.toList()));
11601144
} catch (IOException e) {
11611145
throw new RuntimeException(e);
11621146
}
@@ -1177,35 +1161,23 @@ var responses = new ArrayList<Map>();
11771161

11781162
// Iterate through the list of models
11791163
for (var model: models) {
1180-
System.out.println("\n=== Responses for model: %s ===\n".formatted(model));
1164+
System.out.println("\n=== Responses for model: %s ===\n"formatted(model.getName()));
11811165

11821166
// Create orchestration module configuration for current model
1183-
var moduleConfig = createModuleConfig(model);
1167+
var moduleConfig = new OrchestrationModuleConfig().withLlmConfig(model);
11841168

11851169
// Prompt model with orchestration module configuration
11861170
var response = client.chatCompletion(prompt, moduleConfig);
11871171

11881172
// Add response to list of all model responses
1189-
responses.add(Map.of("model", model, "response", response.getContent()));
1173+
responses.add(Map.of("model", model.getName(), "response", response.getContent()));
11901174

11911175
System.out.println(response.getContent());
11921176
}
11931177

11941178
// Write all responses to a file
11951179
createFileFromResponses(responses);
11961180
```
1197-
• If not done automaticaly by your IDE, add the following imports:
1198-
1199-
```java
1200-
import com.sap.ai.sdk.core.AiCoreService;
1201-
1202-
import com.sap.ai.sdk.orchestration.OrchestrationClient;
1203-
import com.sap.ai.sdk.orchestration.OrchestrationModuleConfig;
1204-
import com.sap.ai.sdk.orchestration.OrchestrationPrompt;
1205-
import com.sap.ai.sdk.orchestration.SystemMessage;
1206-
import com.sap.ai.sdk.orchestration.UserMessage;
1207-
import com.sap.ai.sdk.orchestration.model.LLMModuleConfig;
1208-
```
12091181

12101182

12111183
**Important Note**

0 commit comments

Comments
 (0)