Skip to content

Commit 8d91a5a

Browse files
Update README.md (deepspeedai#916)
1 parent b04fedd commit 8d91a5a

1 file changed

Lines changed: 4 additions & 1 deletion

File tree

  • benchmarks/inference/deepspeedometer

benchmarks/inference/deepspeedometer/README.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# DeepSpeedometer
22

3+
NOTE: This is an experimental tool and is not currently being supported since it's not fully functional. Please use the MII benchmark which can be found here:
4+
https://github.com/microsoft/DeepSpeedExamples/tree/master/benchmarks/inference/mii
5+
36
This benchmark is designed to measure performance of LLM serving solutions. Using a different number of parallel clients sending requests to an inference server, we gather data to plot throughput-latency curves and find the saturation point of an inference server that demonstrates the maximum performance.
47

58
## Installation
@@ -82,4 +85,4 @@ The DeepSpeedometer benchmark was designed to allow easily adding support for ne
8285
3. Define a `*ClientConfig` class that inherits from the `BaseConfigModel` class. Place any configuration options (i.e., user-passed command line arguments) necessary for your defined `*Client` class in here.
8386
4. Import the newly added `*Client` and `*ClientConfig` into `clients/__init__.py` and add them to the `client_config_classes` and `client_classes` dictionaries.
8487

85-
For the simplest example of adding a new client, take a look at the `clients/dummy_client.py` file where we have defined a client that does not stand up a server and only returns a sample of the input prompt after a short sleep cycle. We use this as a light-weight class for unit testing.
88+
For the simplest example of adding a new client, take a look at the `clients/dummy_client.py` file where we have defined a client that does not stand up a server and only returns a sample of the input prompt after a short sleep cycle. We use this as a light-weight class for unit testing.

0 commit comments

Comments
 (0)