fix-step3-readme (deepspeedai#286)

zhangfanTJU · yaozhewei · web-flow · commit a3e4857ab309 · 2023-04-28T13:44:22.000-07:00
* fix-step3-readme

* Update README.md

---------

Co-authored-by: Zhewei Yao &lt;zheweiy@berkeley.edu&gt;
diff --git a/applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/training_scripts/README.md b/applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/training_scripts/README.md
@@ -7,9 +7,6 @@ If you don't have step 1 and step 2 models. You may simply try
 ``` bash
 --actor_model_name_or_path facebook/opt-1.3b --critic_model_name_or_path facebook/opt-350m
 ```
-⚡⚡⚡ When you use above script, please make sure you comment out the following such that it won't load the model weight from previous paths.
-```bash
-applications/DeepSpeed-Chat/training/utils/model/model_utils.py#L60
-```
+⚡⚡⚡ When you use above script, please make sure you modify parameter `rlhf_training` to False when calling the `create_critic_model` function twice in [rlhf_engine.py](./../../step3_rlhf_finetuning/rlhf_engine.py) such that it won't load the model weight from previous paths.
 
 For the models we support, please see [our landing page](./../../../README.md#-supported-models-)
