From 7b82bebf185cf0f623bceccdc8390cc9a7819e84 Mon Sep 17 00:00:00 2001 From: Till Kurek <132505079+TillK17@users.noreply.github.com> Date: Thu, 27 Feb 2025 17:24:02 +0100 Subject: [PATCH 01/10] first 2 steps of advanced orchestration tutorial --- .../ai-core-orchestration-consumption-opt.md | 85 +++++++++++++++++-- 1 file changed, 80 insertions(+), 5 deletions(-) diff --git a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md index 2f607ecb53..b2fa42273c 100644 --- a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md +++ b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md @@ -43,7 +43,7 @@ Familiarity with the orchestration workflow is recommended [OPTION END] -[OPTION BEGIN [Gen AI Hub SDK]] +[OPTION BEGIN [Python SDK]] - **In this tutorial**, we will build upon the orchestration framework introduced in [Tutorial](https://developers.sap.com/tutorials/ai-core-orchestration-consumption.html). The focus will shift from basic orchestration to leveraging optional advanced modules to enhance data privacy and refine response quality. These enhancements include: @@ -73,7 +73,7 @@ print(cv_content) [OPTION END] -[OPTION BEGIN [SAP Cloud SDK]] +[OPTION BEGIN [JavaScript SDK]] - **In this tutorial**, we will build upon the orchestration framework introduced in [Tutorial](https://developers.sap.com/tutorials/ai-core-orchestration-consumption.html). The focus will shift from basic orchestration to leveraging optional advanced modules to enhance data privacy and refine response quality. These enhancements include: @@ -114,6 +114,43 @@ console.log(txtContent); [OPTION END] +[OPTION BEGIN [Java SDK]] + +- **In this tutorial**, we will build upon the orchestration framework introduced in [Tutorial](https://developers.sap.com/tutorials/ai-core-orchestration-consumption.html). The focus will shift from basic orchestration to leveraging optional advanced modules to enhance data privacy and refine response quality. These enhancements include: + + - **Data Masking**: Hiding sensitive information like phone numbers, organizational details, or personal identifiers. + + - **Content Filtering**: Screening for categories such as hate speech, self-harm, explicit content, and violence to ensure safe and relevant responses. + +- Here, we extend the use case introduced in Previous Tutorial, where orchestration was executed without incorporating data masking or content filtering. Here, we will include these advanced modules to improve data privacy, security, and response quality. + +**NOTE** : If you are continuing with the same project from the previous tutorial, skip steps 1 and 2. Otherwise, create a new Java Maven project using the already deployed orchestration URL to access the Harmonized API. Please find detailed information on orchestration configuration and deployment in the previous tutorial or our [GitHub repository](https://github.com/SAP/ai-sdk-java). + + + +- The [cv.txt](img/cv.txt) file, containing the resume content, must be added to the working directory. Use the following code to load the file content: + + +```java +// Adapt filepath to the location you stored the file +var filePath = "path/to/cv.txt"; + +// Read file into string +String cvContent; +try { + cvContent = new String(Files.readAllBytes(Paths.get(filePath))); +} catch (IOException e) { + throw new RuntimeException(e); +} + +// Print file content +System.out.println(cvContent); + +``` + +[OPTION END] + + [OPTION BEGIN [Bruno]] - **In this tutorial**, we will build upon the orchestration framework introduced in [Tutorial](https://developers.sap.com/tutorials/ai-core-orchestration-consumption.html). The focus will shift from basic orchestration to leveraging optional advanced modules to enhance data privacy and refine response quality. These enhancements include: @@ -226,7 +263,7 @@ You are a helpful AI assistant for HR. Summarize the following CV in 10 sentence [OPTION END] -[OPTION BEGIN [Gen AI SDK]] +[OPTION BEGIN [Python SDK]] - To define how the AI should process the resume, we need a template comprising **SystemMessage** and **UserMessage** components: @@ -271,7 +308,7 @@ models = [ ``` [OPTION END] -[OPTION BEGIN [SAP Cloud SDK ]] +[OPTION BEGIN [JavaScript SDK ]] - To define how the AI should process the resume, we need a template comprising **SystemMessage** and **UserMessage** components: @@ -316,6 +353,44 @@ const models = [ [OPTION END] +[OPTION BEGIN [Java SDK]] + +The next step involves creating the prompt for the LLM including both `SystemMessage` and `UserMessage` components. + +• `SystemMessage`: Defines the AI assistant's role and instructions. + +• `UserMessage`: Represents the user's input (i.e., the CV content) to be processed by the LLM. + +```java +// Define system and user messages for prompt +var systemMessage = Message.system( + """ + You are an AI assistant designed to screen resumes for HR purposes. + Please assess the candidate qualifications based on the provided resume. + """ +); +var userMessage = Message.user("Candidate Resume: \n" + cvContent); + +// Define the prompt for resume screening +var prompt = new OrchestrationPrompt(systemMessage, userMessage); + +``` + + +We can define model parameters and a list of models to use. Only use those models that are already deployed in your instances. For this example, we have selected the following parameters and models: + +```java +// List of models with parameters to iterate through, can be adapted if desired +var models = Stream.of( + OrchestrationAiModel.GPT_4O, + OrchestrationAiModel.MISTRAL_LARGE_INSTRUCT, + OrchestrationAiModel.CLAUDE_3_5_SONNET + ).map(model -> model.withParam(MAX_TOKENS, 1000).withParam(TEMPERATURE, 0.6)).toList(); + +``` + +[OPTION END] + ### Setting Up Data Masking Parameters [OPTION BEGIN [AI Launchpad]] @@ -337,7 +412,7 @@ const models = [ [OPTION END] -[OPTION BEGIN [Gen AI SDK]] +[OPTION BEGIN [Python SDK]] - The **Data Masking** Module ensures data privacy by anonymizing or pseudonymizing sensitive information before it is processed. From 9aa1e107f1d0cb2d423d3135b44826fb8dcdc3e9 Mon Sep 17 00:00:00 2001 From: Till Kurek <132505079+TillK17@users.noreply.github.com> Date: Wed, 5 Mar 2025 10:28:23 +0100 Subject: [PATCH 02/10] Reintegration of java content in first orchestration tutorial --- .../ai-core-orchestration-consumption.md | 247 +++++++++++++++++- 1 file changed, 234 insertions(+), 13 deletions(-) diff --git a/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md b/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md index 7b0ec49079..ba6e8ae0e2 100644 --- a/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md +++ b/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md @@ -35,7 +35,7 @@ author_profile: https://github.com/I321506 [OPTION END] -[OPTION BEGIN [Gen AI Hub SDK]] +[OPTION BEGIN [Python SDK]] • Configure proxy modules by setting up environment variables for AI Core credentials. @@ -48,7 +48,7 @@ author_profile: https://github.com/I321506 [OPTION END] -[OPTION BEGIN [SAP Cloud SDK]] +[OPTION BEGIN [JavaScript SDK]] • Download the service key for the AI Core service instance. @@ -66,9 +66,9 @@ author_profile: https://github.com/I321506 • SAP Cloud SDK for AI: Uses the `dotenv` library to load environment variables. If you encounter issues with the dotenv library, ensure it is installed correctly by running: - ```javascript - npm install dotenv - ``` + ```javascript + npm install dotenv + ``` ```javascript import dotenv from 'dotenv'; @@ -83,6 +83,40 @@ console.log(process.env.AICORE_SERVICE_KEY); [OPTION END] +[OPTION BEGIN [Java SDK]] + +• [Create a service key](https://help.sap.com/docs/sap-ai-core/sap-ai-core-service-guide/create-service-key) for your AI Core instance and copy the JSON object. + +• Create a `.env` file in the **working directory from which you run the code**. Add the following line using the copied JSON: + +```dotenv +AICORE_SERVICE_KEY={"clientid": "...", "clientsecret": "...", "url": "...", "serviceurls": { "AI_API_URL": "..." } } +``` + +• **IMPORTANT:** The value of `AICORE_SERVICE_KEY` must be a single line, so remove any line breaks from the service key JSON. + +• This tutorial is designed for a Java [maven project](https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html). Add the following dependencies to your project `pom.xml` file: + +```xml + + com.sap.ai.sdk + core + + ${ai-sdk.version} + + + + com.sap.ai.sdk + orchestration + + ${ai-sdk.version} + +``` + +• For other options of access configuration and detailed information on installation and usage of the **SAP Cloud SDK for AI (for Java)**, visit the official [GitHub repository](https://github.com/SAP/ai-sdk-java). This page provides comprehensive steps to set up and integrate the SDK effectively in your projects. + +[OPTION END] + [OPTION BEGIN [Bruno]] #### Download and Import the Bruno Collection - Download the [bruno_collections](img/Bruno_Collection.json) file @@ -148,7 +182,7 @@ Go to the Configuration section within your chosen Resource Group. [OPTION END] -[OPTION BEGIN [Gen AI SDK]] +[OPTION BEGIN [Python SDK]] • Create a folder named orchestration, then navigate to this folder using VS Code. @@ -206,7 +240,7 @@ print(f"Configuration created successfully with ID: {config.id} and Name: {confi [OPTION END] -[OPTION BEGIN [SAP Cloud SDK ]] +[OPTION BEGIN [JavaScript SDK]] In this step, we define a function to create an orchestration configuration using the ConfigurationApi from the SAP AI SDK. This configuration integrates various parameters needed for orchestration, such as the executable ID and scenario ID. @@ -259,6 +293,45 @@ orchestrationConfig; • config_name: Choose a unique name for the configuration (e.g., "config-new-orchestration") +[OPTION END] + +[OPTION BEGIN [Java SDK]] + +In this step, we will create an orchestration configuration using the core module of the [SAP Cloud SDK for Java](https://github.com/SAP/cloud-sdk-java). This configuration integrates various parameters needed for orchestration, such as the executable ID and scenario ID. + +• Add the following code to your project to create an orchestration configuration: + +```java +// Define the resource group, change this to your resource group name +var RESOURCE_GROUP = "yourResourceGroup"; + +// Define parameter and input artifact bindings you may need for orchestration +var modelFilterList = AiParameterArgumentBinding.create() + .key("modelFilterList").value("null"); +var modelFilterListType = AiParameterArgumentBinding.create() + .key("modelFilterListType").value("allow"); + +// Create a configuration data object for your configuration +var configurationData = AiConfigurationBaseData.create() + .name("orchestration-config") // Choose a meaningful name + .executableId("orchestration") // Orchestration executable ID + .scenarioId("orchestration") // Orchestration scenario ID + .addParameterBindingsItem(modelFilterList) + .addParameterBindingsItem(modelFilterListType); + +// Create the configuration with your individual resource group +var configuration = new ConfigurationApi().create(RESOURCE_GROUP, configurationData); + +// Print the configuration response message +System.out.println(configuration.getMessage()); +``` + +**Note**: + +• `scenarioId` and `executableId`: Both are set to "orchestration" for this tutorial. + +• `name`: Choose a unique name for the configuration (e.g., "config-new-orchestration") + [OPTION END] [OPTION BEGIN [Bruno]] @@ -299,7 +372,7 @@ Once the deployment begins, continue to the status page. Verify that the Deploym [OPTION END] -[OPTION BEGIN [Gen AI SDK]] +[OPTION BEGIN [Python SDK]] With the configuration ID, you can proceed to deploy the orchestration and monitor its progress. @@ -357,7 +430,7 @@ Result: The code will display a loading spinner until the deployment status upda [OPTION END] -[OPTION BEGIN [SAP Cloud SDK ]] +[OPTION BEGIN [JavaScript SDK]] This step involves creating a deployment using the specified configuration and resource group. The deployment is handled via the DeploymentApi, which streamlines the process of activating the orchestration setup. @@ -422,6 +495,26 @@ export async function deployOrchestration( [OPTION END] +[OPTION BEGIN [Java SDK]] + +In this step, we will create a deployment from the configuration created in the previous step using the core module of the [SAP Cloud SDK for Java](https://github.com/SAP/cloud-sdk-java). + +• Add the following code to your project to create an orchestration deployment: + +```java +// Create a deployment creation request with the ID of the created configuration +var deploymentCreationRequest = + AiDeploymentCreationRequest.create().configurationId(configuration.getId()); + +// Create the deployment with the deployment creation request +var deployment = new DeploymentApi().create(RESOURCE_GROUP, deploymentCreationRequest); + +// Print the deployment response message +System.out.println(deployment.getMessage()); +``` + +[OPTION END] + [OPTION BEGIN [Bruno]] #### Update Configuration ID and Create Deployment - Navigate to the create_deployment request. @@ -602,7 +695,7 @@ Data masking and content filtering are available to enhance data privacy and saf [OPTION END] -[OPTION BEGIN [Gen AI SDK]] +[OPTION BEGIN [Python SDK]] To begin the consumption process for the orchestration you’ve deployed, follow the process below: @@ -734,7 +827,7 @@ Data masking and content filtering are available to enhance data privacy and saf [OPTION END] -[OPTION BEGIN [SAP Cloud SDK ]] +[OPTION BEGIN [JavaScript SDK]] To begin the consumption process for the orchestration you’ve deployed, follow the process below: @@ -775,7 +868,7 @@ const templateConfig = { template: [ { role: 'system', - content: 'You are an AI assistant designed to screen resumes for HR purposes. Please assess the candidate qualifications based on the provided resume.', + content: 'You are a helpful AI assistant for HR. Summarize the following CV in 10 sentences, focusing on key qualifications, work experience, and achievements. Include personal contact information, organizational history, and personal interests.', }, { role: 'user', @@ -895,6 +988,134 @@ Data masking and content filtering are available to enhance data privacy and saf [OPTION END] +[OPTION BEGIN [Java SDK]] + +In this step, we will consume an LLM through the orchestration service with the created deployment, using the core and orchestration module of the [SAP Cloud SDK for Java](https://github.com/SAP/cloud-sdk-java). + +To begin the consumption process, follow the steps below: + + +**Prepare the CV File** + +• Download the [cv.txt](img/cv.txt) file, which contains the CV used this tutorial. Add it to your project. + +• Read the CV file from the correct path using the following code: + +```java +// Adapt filepath to the location you stored the file +var filePath = "path/to/cv.txt"; + +// Read file into string +String cvContent; +try { + cvContent = new String(Files.readAllBytes(Paths.get(filePath))); +} catch (IOException e) { + throw new RuntimeException(e); +} + +// Print file content +System.out.println(cvContent); + +``` + +The next step involves creating the prompt for the LLM including both `SystemMessage` and `UserMessage` components. + +• `SystemMessage`: Defines the AI assistant's role and instructions. + +• `UserMessage`: Represents the user's input (i.e., the CV content) to be processed by the LLM. + +```java +// Define system and user messages for prompt +var systemMessage = Message.system( + """ + You are a helpful AI assistant for HR. Summarize the following CV in 10 sentences, + using on key qualifications, work experience, and achievements. Include personal contact information, + organizational history, and personal interests. + """ +); +var userMessage = Message.user("Candidate Resume: \n" + cvContent); + +// Define the prompt for resume screening +var prompt = new OrchestrationPrompt(systemMessage, userMessage); + +``` + + +We can define model parameters and a list of models to use. Only use those models that are already deployed in your instances. For this example, we have selected the following parameters and models: + +```java +// List of models with parameters to iterate through, can be adapted if desired +var models = Stream.of( + OrchestrationAiModel.GPT_4O, + OrchestrationAiModel.MISTRAL_LARGE_INSTRUCT, + OrchestrationAiModel.CLAUDE_3_5_SONNET + ).map(model -> model.withParam(MAX_TOKENS, 1000).withParam(TEMPERATURE, 0.6)).toList(); + +``` + + +The following function writes the responses from different models, stored in a list, to a file: + +```java +// Function writitng responses to a file +void createFileFromResponses (ArrayList responses) { + // Format model responses + var formattedResponses = responses.stream(). + map(response -> "Response from model " + response.get("model") + + ": \n\n" + response.get("response")); + + // Write model responses to provided file path + try { + Files.writeString(Path.of("provided/path/to/responses.txt"), + String.join("\n\n" + "-".repeat(120) + "\n\n", formattedResponses.toList())); + } catch (IOException e) { + throw new RuntimeException(e); + } +} +``` + +**Generate Responses for Multiple Models** + +This step outlines the process of generating responses for a set of queries using different models. We iterate through the list of models created earlier and query the model with the created prompt using an `OrchestrationClient`. + +```java +// Create the client used for interaction with orchestration service +var client = new OrchestrationClient(new AiCoreService() + .getInferenceDestination(RESOURCE_GROUP).forScenario("orchestration")); + +// A list to store all responses from the different models +var responses = new ArrayList(); + +// Iterate through the list of models +for (var model: models) { + System.out.println("\n=== Responses for model: %s ===\n"formatted(model.getName())); + + // Create orchestration module configuration for current model + var moduleConfig = new OrchestrationModuleConfig().withLlmConfig(model); + + // Prompt model with orchestration module configuration + var response = client.chatCompletion(prompt, moduleConfig); + + // Add response to list of all model responses + responses.add(Map.of("model", model.getName(), "response", response.getContent())); + + System.out.println(response.getContent()); +} + +// Write all responses to a file +createFileFromResponses(responses); +``` + +**Important Note** + +Ensure at least one orchestration deployment is ready to be consumed during this process. + +**Optional Advanced Modules** + +Together with document grounding and templating, data masking and content filtering are available to enhance data privacy and safety. Data masking hides sensitive information like phone numbers or organization names, while content filtering can screen for categories such as hate self-harm, sexual content, and violence. In this tutorial, the response generated by the LLM models may carry sensitive information, such as names and phone numbers. For further enhancement, refer to the next tutorial on implementing these modules. + +[OPTION END] + [OPTION BEGIN [Bruno]] - Go to the 08_consume_model section in the collection. @@ -946,4 +1167,4 @@ Data masking and content filtering are available to enhance data privacy and saf Data masking and content filtering are available to enhance data privacy and safety. Data masking hides sensitive information like phone numbers or organization names, while content filtering can screen for categories such as hate self-harm, sexual content, and violence. In this tutorial, the response generated by the LLM models may carry sensitive information, such as names and phone numbers etc.. For further enhancement, refer to the next tutorial on implementing these modules. -[OPTION END] \ No newline at end of file +[OPTION END] From 9b8e188cb4c3c115187474c5a3a849b6deedc2bd Mon Sep 17 00:00:00 2001 From: Till Kurek <132505079+TillK17@users.noreply.github.com> Date: Wed, 5 Mar 2025 12:53:54 +0100 Subject: [PATCH 03/10] all other Java steps for orchestration 2 tutorial + additional small fixes --- .../ai-core-orchestration-consumption-opt.md | 182 ++++++++++++++---- 1 file changed, 142 insertions(+), 40 deletions(-) diff --git a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md index b2fa42273c..160f38d79b 100644 --- a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md +++ b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md @@ -60,7 +60,7 @@ Familiarity with the orchestration workflow is recommended - The [cv.txt](img/cv.txt) file, containing the resume content, must be added to the working directory. Use the following code to load the file content: -```CODE +```python from gen_ai_hub.orchestration.utils import load_text_file # Load the CV file content @@ -90,7 +90,7 @@ print(cv_content) - The [cv.txt](img/cv.txt) file, containing the resume content, must be added to the working directory. Use the following code to load the file content: -```CODE +```javascript // Define the file path const filePath = './cv.txt'; @@ -273,7 +273,7 @@ You are a helpful AI assistant for HR. Summarize the following CV in 10 sentence Use the following code to create the template: -```CODE +```python from gen_ai_hub.orchestration.models.message import SystemMessage, UserMessage from gen_ai_hub.orchestration.models.template import Template, TemplateValue @@ -295,7 +295,7 @@ template = Template( ``` - Select the models to be used for this orchestration: -```CODE +```python from gen_ai_hub.orchestration.models.llm import LLM # List of models to use @@ -318,7 +318,7 @@ models = [ Use the following code to create the template: -```CODE +```javascript / Define the template for resume screening const templateConfig = { @@ -326,7 +326,7 @@ const templateConfig = { template: [ { role: 'system', - content: 'You are an AI assistant designed to screen resumes for HR purposes. Please assess the candidate qualifications based on the provided resume.', + content: 'You are a helpful AI assistant for HR. Summarize the following CV in 10 sentences, focusing on key qualifications, work experience, and achievements. Include personal contact information, organizational history, and personal interests.', }, { role: 'user', @@ -340,7 +340,7 @@ console.log('Resume screening template configuration defined successfully.'); ``` - Select the models to be used for this orchestration: -```CODE +```javascript // List of models to iterate through const models = [ @@ -364,10 +364,11 @@ The next step involves creating the prompt for the LLM including both `SystemMes ```java // Define system and user messages for prompt var systemMessage = Message.system( - """ - You are an AI assistant designed to screen resumes for HR purposes. - Please assess the candidate qualifications based on the provided resume. - """ + """ + You are a helpful AI assistant for HR. Summarize the following CV in 10 sentences, + using on key qualifications, work experience, and achievements. Include personal contact information, + organizational history, and personal interests. + """ ); var userMessage = Message.user("Candidate Resume: \n" + cvContent); @@ -422,7 +423,7 @@ var models = Stream.of( For this tutorial, we use anonymization: -```CODE +```python from gen_ai_hub.orchestration.models.data_masking import DataMasking from gen_ai_hub.orchestration.models.sap_data_privacy_integration import SAPDataPrivacyIntegration, MaskingMethod, ProfileEntity @@ -446,12 +447,9 @@ data_masking = DataMasking( **NOTE** : Here, we mask email, phone, person, organization, and location data. - - - [OPTION END] -[OPTION BEGIN [SAP Cloud SDK ]] +[OPTION BEGIN [JavaScript SDK]] - The **Data Masking** Module ensures data privacy by anonymizing or pseudonymizing sensitive information before it is processed. @@ -461,7 +459,7 @@ data_masking = DataMasking( For this tutorial, we use anonymization: -```CODE +```javascript // Define the data masking configuration const dataMaskingConfig = { @@ -487,8 +485,27 @@ console.log('Data Masking configuration defined successfully.'); **NOTE** : Here, we mask email, phone, person, organization, and location data. +[OPTION END] +[OPTION BEGIN [Java SDK]] +- The **Data Masking** Module ensures data privacy by anonymizing or pseudonymizing sensitive information before it is processed. + + - **Anonymization**: Irreversibly replaces personal identifiers with placeholders (e.g., MASKED_ENTITY). + + - **Pseudonymization**: Substitutes identifiers with reversible tokens (e.g., MASKED_ENTITY_ID). + +For this tutorial, we use anonymization: + +```java +// Define the data masking configuration +var dataMasking = DpiMasking.anonymization().withEntities(EMAIL, PERSON, PHONE, ORG, LOCATION); + +System.out.println("Data Masking defined successfully."); + +``` + +**NOTE** : Here, we mask email, phone, person, organization, and location data. [OPTION END] @@ -629,11 +646,11 @@ Navigate to the **Input Filtering** section. [OPTION END] -[OPTION BEGIN [Gen AI SDK]] +[OPTION BEGIN [Python SDK]] - The **Content Filtering** Module allows screening of both input and output content to remove inappropriate or unwanted elements. This is achieved through configurable thresholds: -```CODE +```python from gen_ai_hub.orchestration.models.azure_content_filter import AzureContentFilter # Configure input and output content filters @@ -642,16 +659,12 @@ output_filter = AzureContentFilter(hate=6, sexual=4, self_harm=0, violence=4) ``` - - **NOTE** : Adjust thresholds for hate, sexual, self-harm, and violence categories based on your use case. - - - Then Combine the template, models, and modules into orchestration configurations: -```CODE +```python from gen_ai_hub.orchestration.models.config import OrchestrationConfig # Create configurations for each model @@ -670,20 +683,15 @@ for model in models: ``` - - **NOTE** : Ensure that your orchestration deployment is in Running Status and ready to be consumed during this process. - - - [OPTION END] -[OPTION BEGIN [SAP Cloud SDK ]] +[OPTION BEGIN [JavaScript SDK]] - The **Content Filtering** Module allows screening of both input and output content to remove inappropriate or unwanted elements. This is achieved through configurable thresholds: -```CODE +```javascript const filteringModuleConfig = { input: { @@ -717,16 +725,12 @@ console.log('Content Filtering configuration defined successfully.'); ``` - - **NOTE** : Adjust thresholds for hate, sexual, self-harm, and violence categories based on your use case. - - - Then Combine the template, models, and modules into orchestration configurations: -```CODE +```javascript // Function to create configuration for each model const createModelConfig = (modelName) => ({ @@ -751,6 +755,33 @@ const deploymentConfig = { [OPTION END] +[OPTION BEGIN [Java SDK]] + +- The **Content Filtering** Module allows screening of both input and output content to remove inappropriate or unwanted elements. This is achieved through configurable thresholds: + +```java +// Define an input content filter, adjust thresholds for your needs +var inputFilter = new AzureContentFilter() + .hate(ALLOW_ALL) + .selfHarm(ALLOW_SAFE) + .sexual(ALLOW_SAFE_LOW_MEDIUM) + .violence(ALLOW_SAFE_LOW_MEDIUM); + +// Define an output content filter, adjust thresholds for your needs +var outputFilter = new AzureContentFilter() + .hate(ALLOW_ALL) + .selfHarm(ALLOW_SAFE) + .sexual(ALLOW_SAFE_LOW_MEDIUM) + .violence(ALLOW_SAFE_LOW_MEDIUM); + +System.out.println("Content Filtering defined successfully."); + +``` + +**NOTE** : Adjust thresholds for hate, sexual, self-harm, and violence categories based on your use case. + +[OPTION END] + [OPTION BEGIN [Bruno]] The **Content Filtering** Module allows screening of both input and output content to remove inappropriate or unwanted elements. This is achieved through configurable thresholds: @@ -868,11 +899,11 @@ The **Content Filtering** Module allows screening of both input and output conte [OPTION END] -[OPTION BEGIN [Gen AI SDK]] +[OPTION BEGIN [Python SDK]] - Finally, execute the orchestration and collect the results: -```CODE +```python from gen_ai_hub.orchestration.service import OrchestrationService # Initialize an empty list to store the responses @@ -912,11 +943,11 @@ with open("model_responses.txt", "w") as file: [OPTION END] -[OPTION BEGIN [SAP Cloud SDK ]] +[OPTION BEGIN [JavaScript SDK]] - Finally, execute the orchestration and collect the results: -```CODE +```javascript import {OrchestrationClient,buildAzureContentFilter} from '@sap-ai-sdk/orchestration'; import { writeFileStrSync } from "https://deno.land/std@0.52.0/fs/mod.ts"; @@ -972,6 +1003,77 @@ async function generateResponsesForModels(txtContent) { By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. +[OPTION END] + +[OPTION BEGIN [Java SDK]] + +The following function writes the responses from different models, stored in a list, to a file (same as in previous tutorial): + +```java +// Function writing responses to a file +void createFileFromResponses (ArrayList responses) { + // Format model responses + var formattedResponses = responses.stream(). + map(response -> "Response from model " + response.get("model") + + ": \n\n" + response.get("response")); + + // Write model responses to provided file path + try { + Files.writeString(Path.of("provided/path/to/responses.txt"), + String.join("\n\n" + "-".repeat(120) + "\n\n", formattedResponses.toList())); + } catch (IOException e) { + throw new RuntimeException(e); + } +} +``` + +**Generate Responses for Multiple Models** + +This step outlines the process of generating responses for a set of queries using different models. We iterate through the list of models created earlier and query the model with the created prompt using an `OrchestrationClient`. + +**NOTE** : Ensure that your orchestration deployment is in Running Status and ready to be consumed during this process.** + +```java +// Create the client used for interaction with orchestration service +var client = new OrchestrationClient(new AiCoreService() + .getInferenceDestination(RESOURCE_GROUP).forScenario("orchestration")); + +// A list to store all responses from the different models +var responses = new ArrayList(); + +// Iterate through the list of models +for (var model: models) { + System.out.println("\n=== Responses for model: %s ===\n".formatted(model.getName())); + + // Create orchestration module configuration for current model + var moduleConfig = new OrchestrationModuleConfig() + .withLlmConfig(model) + .withMaskingConfig(dataMasking) + .withInputFiltering(inputFilter) + .withOutputFiltering(outputFilter); + + // Prompt model with orchestration module configuration + var response = client.chatCompletion(prompt, moduleConfig); + + // Add response to list of all model responses + responses.add(Map.of("model", model.getName(), "response", response.getContent())); + + System.out.println(response.getContent()); +} + +// Write all responses to a file +createFileFromResponses(responses); + +``` + +- A **model_responses.txt** file will be generated, containing outputs from all the models used. + + **Conclusion** : + Once the orchestration completes, you can observe that the output is now more refined, with sensitive information masked and inappropriate content filtered. This demonstrates the power of advanced modules like data masking and content filtering to enhance privacy and ensure response quality. + + While this tutorial used a resume screening use case, the same principles can be applied to other use cases. You can customize the Data Masking and Content Filtering settings based on your specific requirements to handle sensitive or categorized data effectively. + + By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. [OPTION END] From 3f14c3664150b1892afe62d8c99e0be619180e09 Mon Sep 17 00:00:00 2001 From: Till Kurek <132505079+TillK17@users.noreply.github.com> Date: Wed, 5 Mar 2025 12:58:37 +0100 Subject: [PATCH 04/10] small fixes --- .../ai-core-orchestration-consumption.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md b/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md index ba6e8ae0e2..078b00e17e 100644 --- a/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md +++ b/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md @@ -1057,7 +1057,7 @@ var models = Stream.of( The following function writes the responses from different models, stored in a list, to a file: ```java -// Function writitng responses to a file +// Function writing responses to a file void createFileFromResponses (ArrayList responses) { // Format model responses var formattedResponses = responses.stream(). @@ -1088,7 +1088,7 @@ var responses = new ArrayList(); // Iterate through the list of models for (var model: models) { - System.out.println("\n=== Responses for model: %s ===\n"formatted(model.getName())); + System.out.println("\n=== Responses for model: %s ===\n".formatted(model.getName())); // Create orchestration module configuration for current model var moduleConfig = new OrchestrationModuleConfig().withLlmConfig(model); From 4e98d4526d713541d1d0d50fa236eb09fe61a6e1 Mon Sep 17 00:00:00 2001 From: Till Kurek <132505079+TillK17@users.noreply.github.com> Date: Wed, 5 Mar 2025 13:07:24 +0100 Subject: [PATCH 05/10] small fix --- .../ai-core-orchestration-consumption-opt.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md index 160f38d79b..e354efe096 100644 --- a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md +++ b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md @@ -1034,6 +1034,9 @@ This step outlines the process of generating responses for a set of queries usin **NOTE** : Ensure that your orchestration deployment is in Running Status and ready to be consumed during this process.** ```java +// Define the resource group, change this to your resource group name +var RESOURCE_GROUP = "yourResourceGroup"; + // Create the client used for interaction with orchestration service var client = new OrchestrationClient(new AiCoreService() .getInferenceDestination(RESOURCE_GROUP).forScenario("orchestration")); From e962280ad5a7cb1f3a4f9e5604d0b500a0ca798a Mon Sep 17 00:00:00 2001 From: Till Kurek <132505079+TillK17@users.noreply.github.com> Date: Wed, 5 Mar 2025 13:09:49 +0100 Subject: [PATCH 06/10] small fix --- .../ai-core-orchestration-consumption-opt.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md index e354efe096..7fe18f22cf 100644 --- a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md +++ b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md @@ -990,18 +990,19 @@ async function generateResponsesForModels(txtContent) { 'utf-8' ); } - // Example usage with resume content - generateResponsesForModels(txtContent); +// Example usage with resume content +generateResponsesForModels(txtContent); ``` + - A **model_responses.txt** file will be generated, containing outputs from all the models used. - **Conclusion** : - Once the orchestration completes, you can observe that the output is now more refined, with sensitive information masked and inappropriate content filtered. This demonstrates the power of advanced modules like data masking and content filtering to enhance privacy and ensure response quality. +**Conclusion** : +Once the orchestration completes, you can observe that the output is now more refined, with sensitive information masked and inappropriate content filtered. This demonstrates the power of advanced modules like data masking and content filtering to enhance privacy and ensure response quality. - While this tutorial used a resume screening use case, the same principles can be applied to other use cases. You can customize the Data Masking and Content Filtering settings based on your specific requirements to handle sensitive or categorized data effectively. +While this tutorial used a resume screening use case, the same principles can be applied to other use cases. You can customize the Data Masking and Content Filtering settings based on your specific requirements to handle sensitive or categorized data effectively. - By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. +By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. [OPTION END] From b8818cadf01e4ba0fa0f2cac90a884edad95c04e Mon Sep 17 00:00:00 2001 From: Till Kurek <132505079+TillK17@users.noreply.github.com> Date: Wed, 5 Mar 2025 13:12:20 +0100 Subject: [PATCH 07/10] follow-up fixes --- .../ai-core-orchestration-consumption-opt.md | 36 +++++++++---------- 1 file changed, 17 insertions(+), 19 deletions(-) diff --git a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md index 7fe18f22cf..3fdba29c01 100644 --- a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md +++ b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md @@ -889,13 +889,12 @@ The **Content Filtering** Module allows screening of both input and output conte ![img](img/image023.png) - **Conclusion** : - Once the orchestration completes, you can observe that the output is now more refined, with sensitive information masked and inappropriate content filtered. This demonstrates the power of advanced modules like data masking and content filtering to enhance privacy and ensure response quality. - - While this tutorial used a resume screening use case, the same principles can be applied to other use cases. You can customize the Data Masking and Content Filtering settings based on your specific requirements to handle sensitive or categorized data effectively. +**Conclusion** : +Once the orchestration completes, you can observe that the output is now more refined, with sensitive information masked and inappropriate content filtered. This demonstrates the power of advanced modules like data masking and content filtering to enhance privacy and ensure response quality. - By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. +While this tutorial used a resume screening use case, the same principles can be applied to other use cases. You can customize the Data Masking and Content Filtering settings based on your specific requirements to handle sensitive or categorized data effectively. +By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. [OPTION END] @@ -931,15 +930,15 @@ with open("model_responses.txt", "w") as file:         file.write("-" * 80 + "\n")  # Add a separator between model responses ``` -- A **model_responses.txt** file will be generated, containing outputs from all the models used. - **Conclusion** : - Once the orchestration completes, you can observe that the output is now more refined, with sensitive information masked and inappropriate content filtered. This demonstrates the power of advanced modules like data masking and content filtering to enhance privacy and ensure response quality. +- A **model_responses.txt** file will be generated, containing outputs from all the models used. - While this tutorial used a resume screening use case, the same principles can be applied to other use cases. You can customize the Data Masking and Content Filtering settings based on your specific requirements to handle sensitive or categorized data effectively. +**Conclusion** : +Once the orchestration completes, you can observe that the output is now more refined, with sensitive information masked and inappropriate content filtered. This demonstrates the power of advanced modules like data masking and content filtering to enhance privacy and ensure response quality. - By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. +While this tutorial used a resume screening use case, the same principles can be applied to other use cases. You can customize the Data Masking and Content Filtering settings based on your specific requirements to handle sensitive or categorized data effectively. +By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. [OPTION END] @@ -1072,12 +1071,12 @@ createFileFromResponses(responses); - A **model_responses.txt** file will be generated, containing outputs from all the models used. - **Conclusion** : - Once the orchestration completes, you can observe that the output is now more refined, with sensitive information masked and inappropriate content filtered. This demonstrates the power of advanced modules like data masking and content filtering to enhance privacy and ensure response quality. +**Conclusion** : +Once the orchestration completes, you can observe that the output is now more refined, with sensitive information masked and inappropriate content filtered. This demonstrates the power of advanced modules like data masking and content filtering to enhance privacy and ensure response quality. - While this tutorial used a resume screening use case, the same principles can be applied to other use cases. You can customize the Data Masking and Content Filtering settings based on your specific requirements to handle sensitive or categorized data effectively. +While this tutorial used a resume screening use case, the same principles can be applied to other use cases. You can customize the Data Masking and Content Filtering settings based on your specific requirements to handle sensitive or categorized data effectively. - By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. +By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. [OPTION END] @@ -1089,12 +1088,11 @@ createFileFromResponses(responses); By following these steps, you can successfully mask sensitive data and apply content filtering while consuming the deployed model. ![img](img/image.png) - **Conclusion** : - Once the orchestration completes, you can observe that the output is now more refined, with sensitive information masked and inappropriate content filtered. This demonstrates the power of advanced modules like data masking and content filtering to enhance privacy and ensure response quality. +**Conclusion** : +Once the orchestration completes, you can observe that the output is now more refined, with sensitive information masked and inappropriate content filtered. This demonstrates the power of advanced modules like data masking and content filtering to enhance privacy and ensure response quality. - While this tutorial used a resume screening use case, the same principles can be applied to other use cases. You can customize the Data Masking and Content Filterin settings based on your specific requirements to handle sensitive or categorized data effectively. +While this tutorial used a resume screening use case, the same principles can be applied to other use cases. You can customize the Data Masking and Content Filterin settings based on your specific requirements to handle sensitive or categorized data effectively. - By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. +By incorporating these optional modules, you can tailor your Response to meet organizational data security policies and ensure safe, reliable responses for diverse scenarios. [OPTION END] - From 1cc74131d5f79cab262aaa22426bfe617e76a596 Mon Sep 17 00:00:00 2001 From: Till Kurek <132505079+TillK17@users.noreply.github.com> Date: Wed, 5 Mar 2025 13:13:36 +0100 Subject: [PATCH 08/10] Update ai-core-orchestration-consumption-opt.md --- .../ai-core-orchestration-consumption-opt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md index 3fdba29c01..f500bca897 100644 --- a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md +++ b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md @@ -1019,7 +1019,7 @@ void createFileFromResponses (ArrayList responses) { // Write model responses to provided file path try { - Files.writeString(Path.of("provided/path/to/responses.txt"), + Files.writeString(Path.of("provided/path/to/model_responses.txt"), String.join("\n\n" + "-".repeat(120) + "\n\n", formattedResponses.toList())); } catch (IOException e) { throw new RuntimeException(e); From ccf984591c6d4772ab18b568fc269c9d9e751ea4 Mon Sep 17 00:00:00 2001 From: Till Kurek <132505079+TillK17@users.noreply.github.com> Date: Wed, 5 Mar 2025 14:43:31 +0100 Subject: [PATCH 09/10] small code improvement --- .../ai-core-orchestration-consumption-opt.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md index f500bca897..cf2e5b69cf 100644 --- a/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md +++ b/tutorials/ai-core-orchestration-consumption-opt/ai-core-orchestration-consumption-opt.md @@ -1041,6 +1041,12 @@ var RESOURCE_GROUP = "yourResourceGroup"; var client = new OrchestrationClient(new AiCoreService() .getInferenceDestination(RESOURCE_GROUP).forScenario("orchestration")); +// Create orchestration module configuration with masking and filtering +var moduleConfig = new OrchestrationModuleConfig() +.withMaskingConfig(dataMasking) +.withInputFiltering(inputFilter) +.withOutputFiltering(outputFilter); + // A list to store all responses from the different models var responses = new ArrayList(); @@ -1048,15 +1054,8 @@ var responses = new ArrayList(); for (var model: models) { System.out.println("\n=== Responses for model: %s ===\n".formatted(model.getName())); - // Create orchestration module configuration for current model - var moduleConfig = new OrchestrationModuleConfig() - .withLlmConfig(model) - .withMaskingConfig(dataMasking) - .withInputFiltering(inputFilter) - .withOutputFiltering(outputFilter); - - // Prompt model with orchestration module configuration - var response = client.chatCompletion(prompt, moduleConfig); + // Prompt LLM with specific LLM config for model + var response = client.chatCompletion(prompt, moduleConfig.withLlmConfig(model)); // Add response to list of all model responses responses.add(Map.of("model", model.getName(), "response", response.getContent())); From 871fdeecf321669914a2b4e19d38cf842edea2bc Mon Sep 17 00:00:00 2001 From: Till Kurek <132505079+TillK17@users.noreply.github.com> Date: Wed, 5 Mar 2025 14:45:38 +0100 Subject: [PATCH 10/10] small code improvement --- .../ai-core-orchestration-consumption.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md b/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md index 078b00e17e..5d366b0950 100644 --- a/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md +++ b/tutorials/ai-core-orchestration-consumption/ai-core-orchestration-consumption.md @@ -1083,6 +1083,9 @@ This step outlines the process of generating responses for a set of queries usin var client = new OrchestrationClient(new AiCoreService() .getInferenceDestination(RESOURCE_GROUP).forScenario("orchestration")); +// Create orchestration module configuration +var moduleConfig = new OrchestrationModuleConfig(); + // A list to store all responses from the different models var responses = new ArrayList(); @@ -1090,11 +1093,8 @@ var responses = new ArrayList(); for (var model: models) { System.out.println("\n=== Responses for model: %s ===\n".formatted(model.getName())); - // Create orchestration module configuration for current model - var moduleConfig = new OrchestrationModuleConfig().withLlmConfig(model); - - // Prompt model with orchestration module configuration - var response = client.chatCompletion(prompt, moduleConfig); + // Prompt LLM with specific LLM config for model + var response = client.chatCompletion(prompt, moduleConfig.withLlmConfig(model)); // Add response to list of all model responses responses.add(Map.of("model", model.getName(), "response", response.getContent()));