|
| 1 | +--- |
| 2 | +layout: doc-page |
| 3 | +title: Basic Python Usage |
| 4 | +--- |
| 5 | + |
| 6 | +TypeChat is currently a small library, so we can get a solid understanding |
| 7 | +just by going through the following example: |
| 8 | + |
| 9 | +```py |
| 10 | +import asyncio |
| 11 | +import sys |
| 12 | + |
| 13 | +from dotenv import dotenv_values |
| 14 | +from typechat import (Failure, TypeChatJsonTranslator, TypeChatValidator, |
| 15 | + create_language_model, process_requests) |
| 16 | + |
| 17 | +import schema as sentiment # See below for what's in schema.py. |
| 18 | + |
| 19 | +async def main(): |
| 20 | + env_vals = dotenv_values() |
| 21 | + model = create_language_model(env_vals) |
| 22 | + validator = TypeChatValidator(sentiment.Sentiment) |
| 23 | + translator = TypeChatJsonTranslator(model, validator, sentiment.Sentiment) |
| 24 | + |
| 25 | + async def request_handler(message: str): |
| 26 | + result = await translator.translate(message) |
| 27 | + if isinstance(result, Failure): |
| 28 | + print(result.message) |
| 29 | + else: |
| 30 | + result = result.value |
| 31 | + print(f"The sentiment is {result['sentiment']}") |
| 32 | + |
| 33 | + filename = sys.argv[1] if len(sys.argv) == 2 else None |
| 34 | + await process_requests("😀> ", filename, request_handler) |
| 35 | + |
| 36 | +asyncio.run(main()) |
| 37 | +``` |
| 38 | + |
| 39 | +Let's break it down step-by-step. |
| 40 | + |
| 41 | +## Providing a Model |
| 42 | + |
| 43 | +TypeChat can be used with any language model. |
| 44 | +As long as you have a class with the following shape... |
| 45 | + |
| 46 | +```py |
| 47 | +class TypeChatLanguageModel(Protocol): |
| 48 | + |
| 49 | + async def complete(self, prompt: str | list[PromptSection]) -> Result[str]: |
| 50 | + """ |
| 51 | + Represents a AI language model that can complete prompts. |
| 52 | +
|
| 53 | + TypeChat uses an implementation of this protocol to communicate |
| 54 | + with an AI service that can translate natural language requests to JSON |
| 55 | + instances according to a provided schema. |
| 56 | + The `create_language_model` function can create an instance. |
| 57 | + """ |
| 58 | + ... |
| 59 | +``` |
| 60 | + |
| 61 | +then you should be able to try TypeChat out with such a model. |
| 62 | + |
| 63 | +The key thing here is providing a `complete` method. |
| 64 | +`complete` is just a function that takes a `string` and eventually returns a |
| 65 | +string (wrapped in a `Result`) if all goes well. |
| 66 | + |
| 67 | +For convenience, TypeChat provides two functions out of the box to connect to |
| 68 | +the OpenAI API and Azure's OpenAI Services. |
| 69 | +You can call these directly. |
| 70 | + |
| 71 | +```py |
| 72 | +def create_openai_language_model( |
| 73 | + api_key: str, |
| 74 | + model: str, |
| 75 | + endpoint: str = "https://api.openai.com/v1/chat/completions", |
| 76 | + org: str = "" |
| 77 | +): |
| 78 | + ... |
| 79 | + |
| 80 | +def create_azure_openai_language_model(api_key: str, endpoint: str): ... |
| 81 | +``` |
| 82 | + |
| 83 | +For even more convenience, TypeChat also provides a function to infer whether |
| 84 | +you're using OpenAI or Azure OpenAI. |
| 85 | + |
| 86 | +```ts |
| 87 | +def create_language_model( |
| 88 | + vals: dict[str, str | None] |
| 89 | +) -> TypeChatLanguageModel: ... |
| 90 | +``` |
| 91 | + |
| 92 | +With `create_language_model`, you can populate your environment variables and |
| 93 | +pass them in. |
| 94 | +Based on whether `OPENAI_API_KEY` or `AZURE_OPENAI_API_KEY` is set, you'll get |
| 95 | +a model of the appropriate type. |
| 96 | + |
| 97 | +The `TypeChatLanguageModel` returned by these functions has a few writable |
| 98 | +attributes you might find useful: |
| 99 | + |
| 100 | +- `max_retry_attempts` |
| 101 | +- `retry_pause_seconds` |
| 102 | +- `timeout_seconds` |
| 103 | + |
| 104 | +Though note that these are unstable. |
| 105 | + |
| 106 | +Regardless of how you decide to construct your model, it is important to avoid committing credentials directly in source. |
| 107 | +One way to make this work between production and development environments is to use a `.env` file in development, and specify that `.env` in your `.gitignore`. |
| 108 | +You can use a library like [`python-dotenv`](https://pypi.org/project/python-dotenv/) to help load these up. |
| 109 | + |
| 110 | +```py |
| 111 | +from dotenv import load_dotenv |
| 112 | +load_dotenv() |
| 113 | + |
| 114 | +// ... |
| 115 | + |
| 116 | +import typechat |
| 117 | +model = typechat.create_language_model(os.environ) |
| 118 | +``` |
| 119 | + |
| 120 | +## Defining and Loading the Schema |
| 121 | + |
| 122 | +TypeChat describes types to language models to help guide their responses. |
| 123 | +To do so, all we have to do is define either a [`@dataclass`](https://docs.python.org/3/library/dataclasses.html) or a [`TypedDict`](https://typing.readthedocs.io/en/latest/spec/typeddict.html) class to describe the response we're expecting. |
| 124 | +Here's what our schema file `schema.py` look like: |
| 125 | + |
| 126 | +```py |
| 127 | +from dataclasses import dataclass |
| 128 | +from typing import Literal |
| 129 | + |
| 130 | +@dataclass |
| 131 | +class Sentiment: |
| 132 | + """ |
| 133 | + The following is a schema definition for determining the sentiment of a some user input. |
| 134 | + """ |
| 135 | + |
| 136 | + sentiment: Literal["negative", "neutral", "positive"] |
| 137 | +``` |
| 138 | + |
| 139 | +Here, we're saying that the `sentiment` attribute has to be one of three possible strings: `negative`, `neutral`, or `positive`. |
| 140 | +We did this with [the `typing.Literal` hint](https://docs.python.org/3/library/typing.html#typing.Literal). |
| 141 | + |
| 142 | +We defined `Sentiment` as a `@dataclass` so we could have all of the conveniences of standard Python objects - for example, to access the `sentiment` attribute, we can just write `value.sentiment`. |
| 143 | +If we declared `Sentiment` as a `TypedDict`, TypeChat would provide us with a `dict`. |
| 144 | +That would mean that to access the value of `sentiment`, we would have to write `value["sentiment"]`. |
| 145 | + |
| 146 | +Note that while we used [the built-in `typing` module](https://docs.python.org/3/library/typing.html), [`typing_extensions`](https://pypi.org/project/typing-extensions/) is supported as well. |
| 147 | +TypeChat also understands constructs like `Annotated` and `Doc` to add comments to individual attributes. |
| 148 | + |
| 149 | +## Creating a Validator |
| 150 | + |
| 151 | +A validator really has two jobs generating a textual schema for language models, and making sure any data fits a given shape. |
| 152 | +The built-in validator looks roughly like this: |
| 153 | + |
| 154 | +```py |
| 155 | +class TypeChatValidator(Generic[T]): |
| 156 | + """ |
| 157 | + Validates an object against a given Python type. |
| 158 | + """ |
| 159 | + |
| 160 | + def __init__(self, py_type: type[T]): |
| 161 | + """ |
| 162 | + Args: |
| 163 | +
|
| 164 | + py_type: The schema type to validate against. |
| 165 | + """ |
| 166 | + ... |
| 167 | + |
| 168 | + def validate_object(self, obj: object) -> Result[T]: |
| 169 | + """ |
| 170 | + Validates the given Python object according to the associated schema type. |
| 171 | +
|
| 172 | + Returns a `Success[T]` object containing the object if validation was successful. |
| 173 | + Otherwise, returns a `Failure` object with a `message` property describing the error. |
| 174 | + """ |
| 175 | + ... |
| 176 | +``` |
| 177 | + |
| 178 | +To construct a validator, we just have to pass in the type we defined: |
| 179 | + |
| 180 | +```py |
| 181 | +import schema as sentiment |
| 182 | +validator = TypeChatValidator(sentiment.Sentiment) |
| 183 | +``` |
| 184 | + |
| 185 | +## Creating a JSON Translator |
| 186 | + |
| 187 | +A `TypeChatJsonTranslator` brings all these concepts together. |
| 188 | +A translator takes a language model, a validator, and our expected type, and |
| 189 | +provides a way to translate some user input into objects following our schema. |
| 190 | +To do so, it crafts a prompt based on the schema, reaches out to the model, |
| 191 | +parses out JSON data, and attempts validation. |
| 192 | +Optionally, it will craft repair prompts and retry if validation fails. |
| 193 | + |
| 194 | +```py |
| 195 | +translator = TypeChatJsonTranslator(model, validator, sentiment.Sentiment) |
| 196 | +``` |
| 197 | + |
| 198 | +When we are ready to translate a user request, we can call the `translate` |
| 199 | +method. |
| 200 | + |
| 201 | +```ts |
| 202 | +translator.translate("Hello world! 🙂"); |
| 203 | +``` |
| 204 | + |
| 205 | +We'll come back to this. |
| 206 | + |
| 207 | +## Creating a "REPL"` |
| 208 | + |
| 209 | +TypeChat exports a `process_requests` function that makes it easy to |
| 210 | +experiment with TypeChat. |
| 211 | +Depending on its second argument, it either creates an interactive command |
| 212 | +line (if given `None`), or reads lines from the given a file path. |
| 213 | + |
| 214 | +```ts |
| 215 | +async def request_handler(message: str): |
| 216 | + ... |
| 217 | + |
| 218 | +filename = sys.argv[1] if len(sys.argv) == 2 else None |
| 219 | +await process_requests("😀> ", filename, request_handler) |
| 220 | +``` |
| 221 | + |
| 222 | +`process_requests` takes 3 things. |
| 223 | +First, there's the prompt string - this is what a user will see before their |
| 224 | +own input in interactive scenarios. |
| 225 | +You can make this playful. |
| 226 | +We like to use emoji here. 😄 |
| 227 | + |
| 228 | +Next, we take a text file name. |
| 229 | +Input strings will be read from this file one line at a time. |
| 230 | +If the file name was `None`, `process_requests` will work on standard input |
| 231 | +and provide an interactive prompt (assuming `sys.stdin.isatty()` is true). |
| 232 | +By checking `sys.argv`, our script makes our program interactive unless the |
| 233 | +person running the program provided an input file as a command line argument |
| 234 | +(e.g. `python ./example.py inputFile.txt`). |
| 235 | + |
| 236 | +Finally, there's the request handler. |
| 237 | +We'll fill that in next. |
| 238 | + |
| 239 | +## Translating Requests |
| 240 | + |
| 241 | +Our handler receives some user input (the `message` string) each time it's |
| 242 | +called. |
| 243 | +It's time to pass that string into over to our `translator` object. |
| 244 | + |
| 245 | +```ts |
| 246 | +async def request_handler(message: str): |
| 247 | + result = await translator.translate(message) |
| 248 | + if isinstance(result, Failure): |
| 249 | + print(result.message) |
| 250 | + else: |
| 251 | + print(f"The sentiment is {result.value.sentiment}") |
| 252 | +``` |
| 253 | + |
| 254 | +We're calling the `translate` method on each string and getting a response. |
| 255 | +If something goes wrong, TypeChat will retry requests up to a maximum |
| 256 | +specified by `retry_max_attempts` on our `model`. |
| 257 | +However, if the initial request as well as all retries fail, `result` will be |
| 258 | +a `typechat.Failure` and we'll be able to grab a `message` explaining what |
| 259 | +went wrong. |
| 260 | + |
| 261 | +In the ideal case, `result` will be a `typechat.Success` and we'll be able to |
| 262 | +access our well-typed `value` property! |
| 263 | +This will correspond to the type that we passed in when we created our |
| 264 | +translator object (i.e. `Sentiment`). |
| 265 | + |
| 266 | +That's it! |
| 267 | +You should now have a basic idea of TypeChat's APIs and how to get started |
| 268 | +with a new project. 🎉 |
0 commit comments