Skip to content

Commit e86d0c6

Browse files
microsoft-fevieiraawan-10santacmllekurile
authored
Refactor deepspeed-chat into a python package. (deepspeedai#731)
Co-authored-by: Ammar Ahmad Awan <ammar.awan@microsoft.com> Co-authored-by: Michael Santacroce <santacml@users.noreply.github.com> Co-authored-by: Lev Kurilenko <lekurile@microsoft.com> Co-authored-by: Lev Kurilenko <113481193+lekurile@users.noreply.github.com>
1 parent 60e412e commit e86d0c6

20 files changed

Lines changed: 212 additions & 65 deletions

File tree

‎applications/DeepSpeed-Chat/.gitignore‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
6+
# C extensions
7+
*.so
8+
9+
# Distribution / packaging
10+
.Python
11+
build/
12+
develop-eggs/
13+
dist/
14+
downloads/
15+
eggs/
16+
.eggs/
17+
lib/
18+
lib64/
19+
parts/
20+
sdist/
21+
var/
22+
wheels/
23+
pip-wheel-metadata/
24+
share/python-wheels/
25+
*.egg-info/
26+
.installed.cfg
27+
*.egg
28+
MANIFEST
29+
30+
# PyInstaller
31+
# Usually these files are written by a python script from a template
32+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
33+
*.manifest
34+
*.spec
35+
36+
# Installer logs
37+
pip-log.txt
38+
pip-delete-this-directory.txt
39+
40+
# Unit test / coverage reports
41+
htmlcov/
42+
.tox/
43+
.nox/
44+
.coverage
45+
.coverage.*
46+
.cache
47+
nosetests.xml
48+
coverage.xml
49+
*.cover
50+
*.py,cover
51+
.hypothesis/
52+
.pytest_cache/
53+
54+
# Translations
55+
*.mo
56+
*.pot
57+
58+
# Django stuff:
59+
*.log
60+
local_settings.py
61+
db.sqlite3
62+
db.sqlite3-journal
63+
64+
# Flask stuff:
65+
instance/
66+
.webassets-cache
67+
68+
# Scrapy stuff:
69+
.scrapy
70+
71+
# Sphinx documentation
72+
docs/_build/
73+
74+
# PyBuilder
75+
target/
76+
77+
# Jupyter Notebook
78+
.ipynb_checkpoints
79+
80+
# IPython
81+
profile_default/
82+
ipython_config.py
83+
84+
# pyenv
85+
.python-version
86+
87+
# pipenv
88+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
89+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
90+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
91+
# install all needed dependencies.
92+
#Pipfile.lock
93+
94+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
95+
__pypackages__/
96+
97+
# Celery stuff
98+
celerybeat-schedule
99+
celerybeat.pid
100+
101+
# SageMath parsed files
102+
*.sage.py
103+
104+
# Environments
105+
.env
106+
.venv
107+
env/
108+
venv/
109+
ENV/
110+
env.bak/
111+
venv.bak/
112+
113+
# Spyder project settings
114+
.spyderproject
115+
.spyproject
116+
117+
# Rope project settings
118+
.ropeproject
119+
120+
# mkdocs documentation
121+
/site
122+
123+
# mypy
124+
.mypy_cache/
125+
.dmypy.json
126+
dmypy.json
127+
128+
# Pyre type checker
129+
.pyre/
130+
131+
results/
132+
outputs/
133+
134+
.amltconfig
135+
.test_output
136+
*.hdf5
137+
*.h5

‎applications/DeepSpeed-Chat/README.md‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -33,20 +33,24 @@ A fast, affordable, scalable and open system framework for enabling end-to-end R
3333
<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc -->
3434
## Table of Contents
3535

36+
- [🐕DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales🐕](#deepspeed-chat-easy-fast-and-affordable-rlhf-training-of-chatgpt-like-models-at-all-scales)
37+
- [Table of Contents](#table-of-contents)
3638
- [📰 Latest News 📰](#-latest-news-)
37-
- [🚀 What is DeepSpeed Chat 🚀️](#-what-is-deepspeed-chat-)
39+
- [🚀 What is DeepSpeed Chat 🚀](#-what-is-deepspeed-chat-)
3840
- [🧨 Capabilities 🧨](#-capabilities-)
3941
- [☕ Quick Start ☕](#-quick-start-)
4042
- [🐼 Installation](#-installation)
41-
- [🐼 Single Script for Training 3-Step RLHF Pipeline](#-one-single-script-completes-all-three-stages-of-rlhf-training-and-generate-your-first-chatgpt-model)
43+
- [🐼 One Single Script Completes All Three Steps of RLHF Training and Generate Your First ChatGPT Model](#-one-single-script-completes-all-three-steps-of-rlhf-training-and-generate-your-first-chatgpt-model)
4244
- [🐼 Demonstration: Individual Step Fine-Tuning](#-demonstration-individual-step-fine-tuning)
4345
- [🕐 Step 1 - Supervised Fine-Tuning](#-step-1---supervised-fine-tuning)
4446
- [🕑 Step 2 - Reward Model](#-step-2---reward-model)
4547
- [🕒 Step 3 - Reinforcement Learning with Human Feedback](#-step-3---reinforcement-learning-with-human-feedback)
46-
- [🐼 Adding and using your own datasets in DeepSpeed-Chat](#-adding-and-using-your-own-datasets-in-deepspeed-chat)
47-
- [🐼 Customizing RLHF training pipeline via DeepSpeed-Chat’s APIs](#-customizing-your-own-rlhf-training-pipeline-using-deepspeed-chats-rlhf-apis)
48-
- [🐼 Serving Your Model: Plug-in and Test!](#-serving-plug-in-your-final-model-trained-by-deepspeed-chat-and-test-it-out)
48+
- [🐼 Adding and using your own datasets in DeepSpeed-Chat](#-adding-and-using-your-own-datasets-in-deepspeed-chat)
49+
- [🐼 Customizing your own RLHF training pipeline using DeepSpeed-Chat’s RLHF APIs](#-customizing-your-own-rlhf-training-pipeline-using-deepspeed-chats-rlhf-apis)
50+
- [🐼 Serving: Plug-in your final model trained by DeepSpeed-Chat and test it out!](#-serving-plug-in-your-final-model-trained-by-deepspeed-chat-and-test-it-out)
4951
- [🔥 Training Performance Evaluation 🔥](#-training-performance-evaluation-)
52+
- [🐲 Superior Model Scale and Low Training Cost](#-superior-model-scale-and-low-training-cost)
53+
- [🐲 Throughput and Model Size Scalability Comparisons with Existing RLHF Systems](#-throughput-and-model-size-scalability-comparisons-with-existing-rlhf-systems)
5054
- [😽 Supported Models 😽](#-supported-models-)
5155
- [🔬 Build Pipeline Status 🔬](#-build-pipeline-status-)
5256
- [⚓ Documentation and Tutorial ⚓](#-documentation-and-tutorial-)
@@ -119,6 +123,7 @@ pip install deepspeed>=0.9.0
119123
git clone https://github.com/microsoft/DeepSpeedExamples.git
120124
cd DeepSpeedExamples/applications/DeepSpeed-Chat/
121125
pip install -r requirements.txt
126+
pip install -e .
122127
```
123128

124129
### 🐼 One Single Script Completes All Three Steps of RLHF Training and Generate Your First ChatGPT Model

applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/ppo_trainer.py renamed to applications/DeepSpeed-Chat/dschat/rlhf/ppo_trainer.py

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,17 +4,12 @@
44
# DeepSpeed Team
55
import torch
66
import torch.nn.functional as F
7-
import sys
8-
import os
97
import time
108
import deepspeed
119
from deepspeed.runtime.zero.partition_parameters import ZeroParamStatus
1210
from deepspeed.accelerator import get_accelerator
1311

14-
sys.path.append(
15-
os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)))
16-
17-
from utils.utils import print_rank_0
12+
from dschat.utils.utils import print_rank_0
1813

1914

2015
def print_all_ranks(tag, value, rank):

applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/rlhf_engine.py renamed to applications/DeepSpeed-Chat/dschat/rlhf/rlhf_engine.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,10 @@
99
from deepspeed.ops.adam import DeepSpeedCPUAdam
1010
from transformers import AutoModelForCausalLM, get_scheduler
1111

12-
from utils.ds_utils import get_train_ds_config, get_eval_ds_config
13-
from utils.module.lora import convert_linear_layer_to_lora, only_optimize_lora_parameters, make_model_gradient_checkpointing_compatible
14-
from utils.model.model_utils import create_hf_model, create_critic_model
15-
from utils.utils import get_optimizer_grouped_parameters
12+
from dschat.utils.ds_utils import get_train_ds_config, get_eval_ds_config
13+
from dschat.utils.module.lora import convert_linear_layer_to_lora, only_optimize_lora_parameters, make_model_gradient_checkpointing_compatible
14+
from dschat.utils.model.model_utils import create_hf_model, create_critic_model
15+
from dschat.utils.utils import get_optimizer_grouped_parameters
1616
"""
1717
TODOs:
1818
* support HF models for critic (for debugging), must be a previously saved ckpt from step-2

applications/DeepSpeed-Chat/training/utils/data/data_utils.py renamed to applications/DeepSpeed-Chat/dschat/utils/data/data_utils.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
import os
1515
import hashlib
1616
from itertools import chain
17-
from . import raw_datasets
17+
from dschat.utils.data import raw_datasets
1818
from deepspeed.accelerator import get_accelerator
1919

2020

applications/DeepSpeed-Chat/training/utils/data/raw_datasets.py renamed to applications/DeepSpeed-Chat/dschat/utils/data/raw_datasets.py

File renamed without changes.

applications/DeepSpeed-Chat/training/utils/ds_utils.py renamed to applications/DeepSpeed-Chat/dschat/utils/ds_utils.py

File renamed without changes.

applications/DeepSpeed-Chat/training/utils/model/model_utils.py renamed to applications/DeepSpeed-Chat/dschat/utils/model/model_utils.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,8 @@
1212
from huggingface_hub import snapshot_download
1313
from transformers.deepspeed import HfDeepSpeedConfig
1414

15-
from .reward_model import RewardModel
16-
from ..utils import load_state_dict_into_model
17-
from ..utils import print_rank_0
15+
from dschat.utils.model.reward_model import RewardModel
16+
from dschat.utils.utils import load_state_dict_into_model, print_rank_0
1817

1918

2019
def configure_dropout(model_config, dropout):

applications/DeepSpeed-Chat/training/utils/model/reward_model.py renamed to applications/DeepSpeed-Chat/dschat/utils/model/reward_model.py

File renamed without changes.

applications/DeepSpeed-Chat/training/utils/module/lora.py renamed to applications/DeepSpeed-Chat/dschat/utils/module/lora.py

File renamed without changes.

0 commit comments

Comments
 (0)