sap-tutorials · jmmargo · Mar 1, 2024 · Feb 29, 2024
diff --git a/tutorials/ai-core-tensorflow-byod/ai-core-tensorflow-byod.md b/tutorials/ai-core-tensorflow-byod/ai-core-tensorflow-byod.md
@@ -140,7 +140,7 @@ response = ai_core_client.object_store_secrets.create(
     type = "S3",
     name = "my-s3-secret1",
     path_prefix = "movie-clf",
-    endpoint = "s3-eu-central-1.amazonaws.com", # Change this
+    endpoint = "s3.eu-central-1.amazonaws.com", # Change this
     bucket = "asd-11111111-2222-3333-4444-55555555555", # Change this
     region = "eu-central-1", # Change this
     data = {
@@ -159,6 +159,71 @@ You should see the following response:
 > Note that depending on your region, your  AWS endpoint syntax may differ from the example above. In the event of an error, try this step again with alternative syntax. For available syntaxes, please see the [AWS documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteEndpoints.html)
 
 
+### Create workflow to serve your model
+
+
+Save the following executable file in your local system:
+
+| Filename | Download link |
+| -------- | ------------- |
+| `serving_executable.yaml` | [LINK](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/ai-core-tensorflow-byod/files/workflow/serving_executable.yaml) |
+
+In the executable, ensure the following.
+
+<!-- border -->![img](img/acs/6_0.png)
+
+1. Ensure that your `resourcePlan` is set to `infer.s`. This will enable the GPU node in deployment. Find all the available resource plans [here](https://help.sap.com/viewer/2d6c5984063c40a59eda62f4a9135bee/LATEST/en-US/57f4f19d9b3b46208ee1d72017d0eab6.html).
+
+2. Replace `docker-registry-secret` with the name of your docker registry secret. You can create and use multiple docker secrets in SAP AI Core. [See how to create docker registry secret](https://help.sap.com/viewer/2d6c5984063c40a59eda62f4a9135bee/LATEST/en-US/b29c7437a54f46f39c911052b05aabb1.html).
+
+3. Set your docker image URL.
+
+Save your executable.
+
+
+### Sync workflow with SAP AI Core
+
+
+You will create a folder in your GitHub repository connected SAP AI Core, where you will store the workflow (executable). You will then register this folder as an **Application** in SAP AI Core to enable syncing of the workflow as an executable.
+
+> You can create multiple **Applications** in SAP AI Core for syncing multiple folders. This helps you organize separate folders for storing workflows YAML files for separate use cases.
+
+1. Create a folder named `tutorial-tf-text-clf` in your GitHub repository connected to SAP AI Core. Place the following workflows inside it:
+
+    <!-- border -->![img](img/acs/6_1.png)
+
+2. Edit and execute the code below to create an **Application** and sync the folder `tutorial-tf-text-clf`.
+
+    ```PYTHON[4]
+    response = ai_core_client.applications.create(
+    application_name = "tf-clf-app",
+    revision = "HEAD",
+    repository_url = "https://github.com/YOUR_GITHUB_USERNAME/YOUR_REPO_NAME", # Change this
+    path = "tutorial-tf-text-clf"
+    )
+
+    print(response.__dict__)
+    ```
+You should then see:
+
+    <!-- border -->![img](img/acs/6_2.png)
+
+3. Verify your workflow sync status, using the following code:
+
+    ```PYTHON
+    response = ai_core_client.applications.get_status(application_name = 'tf-clf-app')
+
+    print(response.__dict__)
+    print('*'*80)
+    print(response.sync_ressources_status[0].__dict__)
+    ```
+You should then see:
+
+    <!-- border -->![img](img/acs/6_3.png)
+
+After yourGia workflows are synced, your **Scenario** will be automatically created in SAP AI Core. The name and ID of the scenario will be same as the one mentioned in your workflows. After The syncing, your workflow will be recognized as an executable.
+
+
 ### Register model as artifact
 
 
@@ -261,73 +326,6 @@ Follow the steps to upload the files downloaded in step two as a docker image.
     <!-- border -->![img](img/docker-push.png)
 
 
-
-### Create workflow to serve your model
-
-
-Save the following executable file in your local system:
-
-| Filename | Download link |
-| -------- | ------------- |
-| `serving_executable.yaml` | [LINK](https://raw.githubusercontent.com/sap-tutorials/Tutorials/master/tutorials/ai-core-tensorflow-byod/files/workflow/serving_executable.yaml) |
-
-In the executable, ensure the following.
-
-<!-- border -->![img](img/acs/6_0.png)
-
-1. Ensure that your `resourcePlan` is set to `infer.s`. This will enable the GPU node in deployment. Find all the available resource plans0 [here](https://help.sap.com/viewer/2d6c5984063c40a59eda62f4a9135bee/LATEST/en-US/57f4f19d9b3b46208ee1d72017d0eab6.html).
-
-2. Replace `docker-registry-secret` with the name of your docker registry secret. You can create and use multiple docker secrets in SAP AI Core. [See how to create docker registry secret](https://help.sap.com/viewer/2d6c5984063c40a59eda62f4a9135bee/LATEST/en-US/b29c7437a54f46f39c911052b05aabb1.html).
-
-3. Set your docker image URL.
-
-Save your executable.
-
-
-### Sync workflow with SAP AI Core
-
-
-You will create a folder in your GitHub repository connected SAP AI Core, where you will store the workflow (executable). You will then register this folder as an **Application** in SAP AI Core to enable syncing of the workflow as an executable.
-
-> You can create multiple **Applications** in SAP AI Core for syncing multiple folders. This helps you organize separate folders for storing workflows YAML files for separate use cases.
-
-1. Create a folder named `tutorial-tf-text-clf` in your GitHub repository connected to SAP AI Core. Place the following workflows inside it:
-
-    <!-- border -->![img](img/acs/6_1.png)
-
-2. Edit and execute the code below to create an **Application** and sync the folder `tutorial-tf-text-clf`.
-
-    ```PYTHON[4]
-    response = ai_core_client.applications.create(
-    application_name = "tf-clf-app",
-    revision = "HEAD",
-    repository_url = "https://github.com/YOUR_GITHUB_USERNAME/YOUR_REPO_NAME", # Change this
-    path = "tutorial-tf-text-clf"
-    )
-
-    print(response.__dict__)
-    ```
-You should then see:
-
-    <!-- border -->![img](img/acs/6_2.png)
-
-3. Verify your workflow sync status, using the following code:
-
-    ```PYTHON
-    response = ai_core_client.applications.get_status(application_name = 'tf-clf-app')
-
-    print(response.__dict__)
-    print('*'*80)
-    print(response.sync_ressources_status[0].__dict__)
-    ```
-You should then see:
-
-    <!-- border -->![img](img/acs/6_3.png)
-
-After you workflows are synced, your **Scenario** will be automatically created in SAP AI Core. The name and ID of the scenario will be same as the one mentioned in your workflows. After The syncing, your workflow will be recognized as an executable.
-
-
-
 ### Create configuration for deployment
 
 

diff --git a/tutorials/ai-core-tensorflow-byod/files/infer/Dockerfile b/tutorials/ai-core-tensorflow-byod/files/infer/Dockerfile
@@ -1,9 +1,9 @@
-FROM tensorflow/tensorflow:latest-gpu
+FROM tensorflow/tensorflow:2.10.0-gpu-jupyter
 
 ENV LANG C.UTF-8
 
 COPY requirements.txt ./requirements.txt
-RUN pip3 install -r requirements.txt
+RUN pip3 install --ignore-installed -r requirements.txt
 
 ENV SERVE_FILES_PATH=/mnt/models
 

diff --git a/tutorials/ai-core-tensorflow-byod/files/infer/requirements.txt b/tutorials/ai-core-tensorflow-byod/files/infer/requirements.txt
@@ -1,4 +1,3 @@
-scikit-learn==0.24.2
-joblib==1.2.0
-Flask==2.3.2
-gunicorn==20.1.0
+scikit-learn
+Flask
+gunicorn
diff --git a/tutorials/ai-core-tensorflow-byod/files/infer/serve.py b/tutorials/ai-core-tensorflow-byod/files/infer/serve.py
@@ -1,77 +1,68 @@
-# -*- coding: utf-8 -*-
-"""
-Inference script that extends from the base infer interface
-"""
-import os
-from os.path import exists
-from joblib import load
 import logging
 
 from flask import Flask
 from flask import request as call_request
 
-from tf_template import Model, TextProcess
+from tf_template import Model, TextProcess, AVAILABLE_GPUS
 
 app = Flask(__name__)
+app.logger.setLevel(logging.INFO)
+# app.logger.addHandler(logging.FileHandler('server.log'))  # Uncomment to save logs to file `server.log`
 
-text_process = None
-model = None
-
-@app.before_first_request
-def init():
-    """
-    Load the model if it is available locally
-    """
-    import tensorflow as tf
-    import logging
-    logging.info(f"Num GPUs Available: {len(tf.config.list_physical_devices('GPU'))}")
-
-    global text_process, model
-    #
-    # Load text pre and post processor
-    text_process = TextProcess(os.environ['SERVE_FILES_PATH'])
-    text_process.max_pad_len
-    #
-    # load model
-    model = Model(
-        os.environ['SERVE_FILES_PATH']
-    )
+app_has_run_before: bool = False
+
+
+@app.before_request
+def first_run():
+    global app_has_run_before
+    if not app_has_run_before:
+        app.logger.info(f"Num GPUs Available: {len(AVAILABLE_GPUS)}")
 
-    return None
+        app.config['text_process'] = TextProcess()
+        app.config['model'] = Model()
+
+        app_has_run_before = True
 
 
 @app.route("/v1/predict", methods=["POST"])
-def predict():
-    """
-    Perform an inference on the model created in initialize
-
-    Returns:
-        String prediction of the label for the given test data
-    """
-    global model, text_process
-    #
+def predict() -> str:
+    text_process = app.config['text_process']
+    model = app.config['model']
+
     input_data = dict(call_request.json)
     text = str(input_data['text'])
-    #
-    # Log first
-    logging.info("Requested text: " +
-        str(text)
-    )
-    #
-    # Prediction
+
+    app.logger.info(f'Requested text: {text}')
     prediction = model.predict(
-        text_process.pre_process([text]) # Important to pass as list
+        text_process.pre_process(text)
     )
-    logging.info(f"Prediction: {str(prediction)}")
-    #
-    output = text_process.post_process(prediction)
-    #
-    # Response
-    return output
+
+    app.logger.info(f"Prediction: {prediction}")
+
+    return text_process.post_process(prediction)
 
 
 if __name__ == "__main__":
-    init()
     app.run(host="0.0.0.0", debug=True, port=9001)
 
-# curl --location --request POST 'http://localhost:9001/v1/predict' --header 'Content-Type: application/json' --data-raw '{"text": "A restaurant with great ambiance"}'
+"""
+To run and debug locally:
+1. Install
+   - flask
+   - scikit-learn
+   - tensorflow==2.10.0
+
+2. Run the server from the project's root directory
+$ export SERVE_FILES_PATH=tf_files && python server/serve.py
+
+As an alternative, the server can be run like this:
+$ export SERVE_FILES_PATH=../tf_files && gunicorn --chdir server serve:app -b 0.0.0.0:9001
+
+3. Query the endpoint
+$ curl --location --request POST 'http://localhost:9001/v1/predict' --header 'Content-Type: application/json' --data-raw '{"text": "A restaurant with great ambiance"}'
+
+4. Result should be
+{
+  "negative": 0.5039926171302795
+}
+"""