Skip to content

Split out a ModelHint type for clarity #38

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jspahrsummers merged 3 commits into from
Nov 7, 2024
Merged

Split out a ModelHint type for clarity #38

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c9d33ad
Split out a `ModelHint` type for clarity
jspahrsummers Nov 6, 2024
6aaaaf9
Remove duplicated comment
jspahrsummers Nov 6, 2024
0cc9ae7
Regenerate JSON Schema
jspahrsummers Nov 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions schema/schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1012,6 +1012,16 @@
],
"type": "object"
},
"ModelHint": {
"description": "Hints to use for model selection.\n\nKeys not declared here are currently left unspecified by the spec and are up\nto the client to interpret.",
"properties": {
"name": {
"description": "A hint for a model name.\n\nThe client SHOULD treat this as a substring of a model name; for example:\n - `claude-3-5-sonnet` should match `claude-3-5-sonnet-20241022`\n - `sonnet` should match `claude-3-5-sonnet-20241022`, `claude-3-sonnet-20240229`, etc.\n - `claude` should match any Claude model\n\nThe client MAY also map the string to a different provider's model name or a different model family, as long as it fills a similar niche; for example:\n - `gemini-1.5-flash` could match `claude-3-haiku-20240307`",
"type": "string"
}
},
"type": "object"
},
"ModelPreferences": {
"description": "The server's preferences for model selection, requested of the client during sampling.\n\nBecause LLMs can vary along multiple dimensions, choosing the \"best\" model is\nrarely straightforward. Different models excel in different areas—some are\nfaster but less capable, others are more capable but more expensive, and so\non. This interface allows servers to express their priorities across multiple\ndimensions to help clients make an appropriate selection for their use case.\n\nThese preferences are always advisory. The client MAY ignore them. It is also\nup to the client to decide how to interpret these preferences and how to\nbalance them against other considerations.",
"properties": {
Expand All @@ -1022,9 +1032,9 @@
"type": "number"
},
"hints": {
"description": "Optional string hints to use for model selection. How these hints are\ninterpreted depends on the key(s) in each record:\n\n- If the record contains a `name` key:\n - The client SHOULD treat this as a substring of a model name; for example:\n - `claude-3-5-sonnet` should match `claude-3-5-sonnet-20241022`\n - `sonnet` should match `claude-3-5-sonnet-20241022`, `claude-3-sonnet-20240229`, etc.\n - `claude` should match any Claude model\n - The client MAY also map the string to a different provider's model name or a different model family, as long as it fills a similar niche; for example:\n - `gemini-1.5-flash` could match `claude-3-haiku-20240307`\n\nAll other keys are currently left unspecified by the spec and are up to the\nclient to interpret.\n\nIf multiple hints are specified, the client MUST evaluate them in order\n(such that the first match is taken).\n\nThe client SHOULD prioritize these hints over the numeric priorities, but\nMAY still use the priorities to select from ambiguous matches.",
"description": "Optional hints to use for model selection.\n\nIf multiple hints are specified, the client MUST evaluate them in order\n(such that the first match is taken).\n\nThe client SHOULD prioritize these hints over the numeric priorities, but\nMAY still use the priorities to select from ambiguous matches.",
"items": {
"$ref": "#/definitions/Record<string,string>"
"$ref": "#/definitions/ModelHint"
},
"type": "array"
},
Expand Down Expand Up @@ -1340,9 +1350,6 @@
],
"type": "object"
},
"Record<string,string>": {
"type": "object"
},
"Request": {
"properties": {
"method": {
Expand Down
37 changes: 23 additions & 14 deletions schema/schema.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -806,27 +806,15 @@ export interface ImageContent {
*/
export interface ModelPreferences {
/**
* Optional string hints to use for model selection. How these hints are
* interpreted depends on the key(s) in each record:
*
* - If the record contains a `name` key:
* - The client SHOULD treat this as a substring of a model name; for example:
* - `claude-3-5-sonnet` should match `claude-3-5-sonnet-20241022`
* - `sonnet` should match `claude-3-5-sonnet-20241022`, `claude-3-sonnet-20240229`, etc.
* - `claude` should match any Claude model
* - The client MAY also map the string to a different provider's model name or a different model family, as long as it fills a similar niche; for example:
* - `gemini-1.5-flash` could match `claude-3-haiku-20240307`
*
* All other keys are currently left unspecified by the spec and are up to the
* client to interpret.
* Optional hints to use for model selection.
*
* If multiple hints are specified, the client MUST evaluate them in order
* (such that the first match is taken).
*
* The client SHOULD prioritize these hints over the numeric priorities, but
* MAY still use the priorities to select from ambiguous matches.
*/
hints?: Record<"name" | string, string>[];
hints?: ModelHint[];

/**
* How much to prioritize cost when selecting a model. A value of 0 means cost
Expand Down Expand Up @@ -862,6 +850,27 @@ export interface ModelPreferences {
intelligencePriority?: number;
}

/**
* Hints to use for model selection.
*
* Keys not declared here are currently left unspecified by the spec and are up
* to the client to interpret.
*/
export interface ModelHint {
/**
* A hint for a model name.
*
* The client SHOULD treat this as a substring of a model name; for example:
* - `claude-3-5-sonnet` should match `claude-3-5-sonnet-20241022`
* - `sonnet` should match `claude-3-5-sonnet-20241022`, `claude-3-sonnet-20240229`, etc.
* - `claude` should match any Claude model
*
* The client MAY also map the string to a different provider's model name or a different model family, as long as it fills a similar niche; for example:
* - `gemini-1.5-flash` could match `claude-3-haiku-20240307`
*/
name?: string;
}

/* Autocomplete */
/**
* A request from the client to the server, to ask for completion options.
Expand Down