Skip to content

Express fms model adapters as combination of keys#364

Draft
JRosenkranz wants to merge 1 commit into
mainfrom
specific_adapter_keys
Draft

Express fms model adapters as combination of keys#364
JRosenkranz wants to merge 1 commit into
mainfrom
specific_adapter_keys

Conversation

@JRosenkranz
Copy link
Copy Markdown
Collaborator

This PR addresses allowing a more specific key for fms model adapters. This could help in the case where we may want to use a specific adapter only for certain devices, or linear types, etc.

@JRosenkranz JRosenkranz requested a review from ani300 November 19, 2024 19:09
@ani300
Copy link
Copy Markdown
Collaborator

ani300 commented Nov 25, 2024

is there any reason you are using frozenset for the keys?

@ani300
Copy link
Copy Markdown
Collaborator

ani300 commented Nov 25, 2024

Looking at the PR in more details, I have the following questions:

  1. If we have 5 adapter steps for HF checkpoints, and 1 step that only needs to run on HF checkpoints on CPU, we would register 2 adapters, one for (source: "hf",) with 5 steps and one for (source: "hf", device: "cpu") with 5+1 steps?
  2. Then, we would pick on CPU the most specific "hf"+"cpu" adapter, while on GPU we would pick the "hf" one only because there is nothing more specific?
  3. Are we sure we want to use itertools.combinations? How are we ensuring we are not creating far too many combinations to the point our program slows down to a crawl? We can maybe take a look at how pytorch handles its dispatcher keys, as I think we are trying to solve the same problem here and they have done it very efficiently

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants