You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: tutorials/ai-core-custom-slm/ai-core-custom-slm.md
+16-16Lines changed: 16 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,29 +8,29 @@ author_name: Dhrubajyoti Paul
8
8
author_profile: https://github.com/dhrubpaul
9
9
---
10
10
# Using small language models on SAP AI Core
11
-
<!-- description --> In this tutorial we are going to learn on how to deploy a custom LLM on AI core using ollama for the example we would be taking Gemma as a model from hugging face and deploy it on SAP AI core.
11
+
<!-- description --> In this tutorial we are going to learn on how to deploy a custom LLM on AI core using Ollama for the example we would be taking Gemma as a model from hugging face and deploy it on SAP AI core.
12
12
13
13
## You will learn
14
-
- How to Deploy ollama on AI core
15
-
- Add models to ollama and inference models
14
+
- How to Deploy Ollama on AI core
15
+
- Add models to Ollama and inference models
16
16
17
17
## Prerequisites
18
18
Ai core setup and basic knowledge: [Link to documentation](https://developers.sap.com/tutorials/ai-core-setup.html)
19
19
Ai core Instance with Standard Plan or Extended Plan
20
20
Docker Desktop Setup [Download and Install](https://www.docker.com/products/docker-desktop)
21
-
Github Account
21
+
GitHub Account
22
22
23
23
### Architecture Overview
24
-
In this tutorial we are deploying ollama an open-source project that serves as a powerful and user-friendly platform for running LLMs on on SAP AI core. which acts as a bridge between the complexities of LLM technology and the desire for an accessible and customizable AI experience.
24
+
In this tutorial we are deploying Ollama an open-source project that serves as a powerful and user-friendly platform for running LLMs on on SAP AI core. which acts as a bridge between the complexities of LLM technology and the desire for an accessible and customizable AI experience.
25
25
26
26

27
27
28
-
We can pick any model from the above model hubs and connect it to AI core for the example we are going to deploy ollama on AI core and enable Gemma and inference the same.
28
+
We can pick any model from the above model hubs and connect it to AI core for the example we are going to deploy Ollama on AI core and enable Gemma and inference the same.
29
29
30
30
### Adding workflow file to github
31
31
Workflows for SAP AI Core are created using YAML or JSON files that are compatible with the SAP AI Core schema. Let’s start with adding a Argo Workflow file to manage: `ollama`.
32
32
33
-
In your Github Create a new repository, click **Add file** > **Create new file**.
33
+
In your GitHub Create a new repository, click **Add file** > **Create new file**.
34
34
35
35

36
36
@@ -113,10 +113,10 @@ RUN apt-get update && \
113
113
apt-get clean && \
114
114
rm -rf /var/lib/apt/lists/*
115
115
116
-
# Install ollama
116
+
# Install Ollama
117
117
RUN curl -fsSL https://ollama.com/install.sh | sh
118
118
119
-
# Expose port and set environment variables for ollama
119
+
# Expose port and set environment variables for Ollama
@@ -153,7 +153,7 @@ RUN mkdir -p /nonexistent/.ollama && \
153
153
chmod -R 770 /nonexistent
154
154
# chmod -R 777 /nonexistent/.ollama
155
155
156
-
# Start nginx and ollama service
156
+
# Start nginx and Ollama service
157
157
CMD service nginx start && /usr/local/bin/ollama serve
158
158
159
159
```
@@ -192,7 +192,7 @@ A Pop up will appear on screen and add the following Json with the details to yo
192
192
}
193
193
```
194
194
195
-
### Onboarding Github and application on AI core
195
+
### Onboarding GitHub and application on AI core
196
196
197
197
Select on your SAP AI Core connection under **Workspaces app** in the SAP AI Launchpad.
198
198
@@ -216,7 +216,7 @@ Use the following information as reference:
216
216
217
217
- **Password:** Paste your GitHub Personal Access Token, generated in the previous step.
218
218
219
-
> Note: Password does not gets validated at time of Adding Github Repository its just meant to save Github Creds to AI core. Passwords gets validated at time of creating Application or when Application refreshes connection to AI core.
219
+
> Note: Password does not gets validated at time of Adding GitHub Repository its just meant to save GitHub credentials to AI core. Passwords gets validated at time of creating Application or when Application refreshes connection to AI core.
220
220
221
221
You will see your GitHub onboarding completed in a few seconds. As a next steps we will enable an application on AI core.
222
222
@@ -280,15 +280,15 @@ Once you create the deployment, wait for the current status to be set to RUNNING
280
280
281
281

282
282
283
-
Once the deployment is running, you can access the LLM’s using ollama.
283
+
Once the deployment is running, you can access the LLM’s using Ollama.
284
284
285
285
### Pulling llava-phi3 and Performing Inference
286
286
287
-
Now we need to import llava-phi3 to our ollama pod before we can inference the model so here we would be using SAP AI API to call pull model call in Ollama.
287
+
Now we need to import llava-phi3 to our Ollama pod before we can inference the model so here we would be using SAP AI API to call pull model call in Ollama.
288
288
289
289
[OPTION BEGIN [Postman]]
290
290
291
-
- Setting up AI core Auth Creds
291
+
- Setting up AI core Auth Credentials
292
292

293
293
294
294
- adding Resource groups to headers
@@ -302,7 +302,7 @@ Now we need to import llava-phi3 to our ollama pod before we can inference the m
302
302
```
303
303
For your reference, please see the screenshots below.
304
304

305
-
- Once the model is pulled to AI core we can check the list of models deployed under ollama deployment via the following.
305
+
- Once the model is pulled to AI core we can check the list of models deployed under Ollama deployment via the following.
0 commit comments