from Algorithmia import ADK
# API calls will begin at the apply() method, with the request body passed as 'input'
# For more details, see algorithmia.com/developers/algorithm-development/languages
def apply(input):
# If your apply function uses state that's loaded into memory via load, you can pass that loaded state to your apply
# function by defining an additional "globals" parameter in your apply function; but it's optional!
return "hello {}".format(str(input))
# This turns your library code into an algorithm that can run on the platform.
# If you intend to use loading operations, remember to pass a `load` function as a second variable.
algorithm = ADK(apply)
# The 'serve()' function actually starts the algorithm, you can follow along in the source code
# to see how everything works.
algorithm.init("Algorithmia")This document will describe the following:
- What is an Algorithm Development Kit
- Changes to Algorithm development
- Example workflows you can use to create your own Algorithms.
An Algorithm Development Kit is a package that contains all of the necessary components to convert a regular application into one that can be executed and run on Algorithmia.
To do that, an ADK must be able to communicate with langserver.
To keep things simple, an ADK exposes some optional functions, along with an apply function that acts as the explicit entrypoint into your algorithm.
Along with those basics, the ADK also exposes the ability to execute your algorithm locally, without langserver; which enables better debuggability.
This kit, when implemented by an algorithm developer - enables an easy way to get started with your project, along with well defined hooks to integrate with an existing project.
Algorithm development does change with this introduction:
- Primary development file has been renamed to
src/Algorithm.pyto aide in understanding around what this file actually does / why it's important - An additional import (
from algorithm import ADK) - An optional
load()function that can be implemented- This enables a dedicated function for preparing your algorithm for runtime operations, such as model loading, configuration, etc
- A call to the handler function with your
applyand optionalloadfunctions as inputs-
algorithm = ADK(apply) algorithm.init("Algorithmia")
- Converts the project into an executable, rather than a library
- Which will interact with the
langserverservice on Algorithmia - But is debuggable via stdin/stdout when executed locally / outside of an Algorithm container
- When a payload is provided to
init(), that payload will be directly provided to your algorithm when executed locally, bypassing stdin parsing and simplifying debugging!
- When a payload is provided to
- This includes being able to step through your algorithm code in your IDE of choice! Just execute your
src/Algorithm.pyscript and try stepping through your code with your favorite IDE
- Which will interact with the
-
Check out these examples to help you get started:
from Algorithmia import ADK
# API calls will begin at the apply() method, with the request body passed as 'input'
# For more details, see algorithmia.com/developers/algorithm-development/languages
def apply(input):
# If your apply function uses state that's loaded into memory via load, you can pass that loaded state to your apply
# function by defining an additional "globals" parameter in your apply function; but it's optional!
return "hello {}".format(str(input))
# This turns your library code into an algorithm that can run on the platform.
# If you intend to use loading operations, remember to pass a `load` function as a second variable.
algorithm = ADK(apply)
# The 'serve()' function actually starts the algorithm, you can follow along in the source code
# to see how everything works.
algorithm.init("Algorithmia")from Algorithmia import ADK
import Algorithmia
# API calls will begin at the apply() method, with the request body passed as 'input'
# For more details, see algorithmia.com/developers/algorithm-development/languages
def apply(input, globals):
# If your apply function uses state that's loaded into memory via load, you can pass that loaded state to your apply
# function by defining an additional "globals" parameter in your apply function.
return "hello {} {}".format(str(input), str(globals['payload']))
def load():
# Here you can optionally define a function that will be called when the algorithm is loaded.
# The return object from this function can be passed directly as input to your apply function.
# A great example would be any model files that need to be available to this algorithm
# during runtime.
# Any variables returned here, will be passed as the secondary argument to your 'algorithm' function
globals = {}
globals['payload'] = "Loading has been completed."
return globals
def handle_exceptions(exp):
# Additionally, if you plan to ship failure or metric information critical to your business to an aggregation
# service using Algorithmia insights or a third party solution; it's recommended to implement this function. This
# function takes 2 inputs parameters; the exception and and returns nothing. If any exceptions are detected; you
# can process them here. This reduces the need to use built-in try/catch logic, which reduces the chances for
# catching the wrong error!
client = Algorithmia.client()
client.report_insights({"exception": str(exp)})
# This turns your library code into an algorithm that can run on the platform.
# If you intend to use loading operations, remember to pass a `load` function as a second variable.
algorithm = ADK(apply, load, handle_exceptions)
# The 'init()' function actually starts the algorithm, you can follow along in the source code
# to see how everything works.
algorithm.init("Algorithmia")from Algorithmia import ADK
import Algorithmia
from Algorithmia.errors import AlgorithmException
import torch
from PIL import Image
import json
from torchvision import models, transforms
client = Algorithmia.client()
def load_labels(label_path, client):
local_path = client.file(label_path).getFile().name
with open(local_path) as f:
labels = json.load(f)
labels = [labels[str(k)][1] for k in range(len(labels))]
return labels
def load_model(model_paths, client):
model = models.squeezenet1_1()
local_file = client.file(model_paths["filepath"]).getFile().name
weights = torch.load(local_file)
model.load_state_dict(weights)
return model.float().eval()
def get_image(image_url, smid_algo, client):
input = {"image": image_url, "resize": {"width": 224, "height": 224}}
result = client.algo(smid_algo).pipe(input).result["savePath"][0]
local_path = client.file(result).getFile().name
img_data = Image.open(local_path)
return img_data
def infer_image(image_url, n, globals):
model = globals["model"]
labels = globals["labels"]
image_data = get_image(image_url, globals["SMID_ALGO"], globals["CLIENT"])
transformed = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])])
img_tensor = transformed(image_data).unsqueeze(dim=0)
infered = model.forward(img_tensor)
preds, indicies = torch.sort(torch.softmax(infered.squeeze(), dim=0), descending=True)
predicted_values = preds.detach().numpy()
indicies = indicies.detach().numpy()
result = []
for i in range(n):
label = labels[indicies[i]].lower().replace("_", " ")
confidence = float(predicted_values[i])
result.append({"label": label, "confidence": confidence})
return result
def load(manifest):
globals = {}
globals["SMID_ALGO"] = "algo://util/SmartImageDownloader/0.2.x"
globals["model"] = load_model(manifest["squeezenet"], client)
globals["labels"] = load_labels(manifest["label_file"], client)
return globals
def handle_exceptions(exp):
if isinstance(exp, AlgorithmException):
client.report_insights({"AlgorithmException": str(exp)})
else:
client.report_insights({"CriticalException": str(exp)})
def apply(input, globals):
if isinstance(input, dict):
if "n" in input:
n = input["n"]
else:
n = 3
if "data" in input:
if isinstance(input["data"], str):
output = infer_image(input["data"], n, globals)
elif isinstance(input["data"], list):
for row in input["data"]:
row["predictions"] = infer_image(row["image_url"], n, globals)
output = input["data"]
else:
raise Exception("\"data\" must be a image url or a list of image urls (with labels)")
return output
else:
raise Exception("\"data\" must be defined")
else:
raise Exception("input must be a json object")
algorithm = ADK(apply_func=apply, load_func=load, exception_func=handle_exceptions)
algorithm.init({"data": "https://i.imgur.com/bXdORXl.jpeg"})To compile the template readme, please check out embedme utility and run the following:
npm install -g npx
npx embedme --stdout README_template.md > README.md
Publishing should be automatic on new releases, but if you wish to publish manually this is the process first make sure to update the version in setup.py Then go through the following Then, install these python dependencies
pip install wheel==0.33
pip install setuptools==41.6
pip install twine==1.15
Setup your ~/.pypirc file:
index-servers =
pypi
pypitest
[pypi]
repository: https://upload.pypi.org/legacy/
username: algorithmia
password: {{...}}
[pypitest]
repository: https://test.pypi.org/legacy/
username: algorithmia
password: {{...}}
The passwords (and the pypirc file itself) can be found in our devtools service Make sure to update your setup.py with the new version before compiling. Also make sure that this is created on Linux and not any other platform. Compile via setup.py:
python setup.py sdist bdist_wheel --universal
python -m twine upload -r pypitest dist/*
Verify that it works on pytest, then:
python -m twine upload -r pypi dist/*
and you're done :)
