Skip to content

Fix the inference-test.py script and update text-generation README#219

Merged
RezaYazdaniAminabadi merged 4 commits into
masterfrom
lekurile/fix_text_generation_ex
Nov 12, 2022
Merged

Fix the inference-test.py script and update text-generation README#219
RezaYazdaniAminabadi merged 4 commits into
masterfrom
lekurile/fix_text_generation_ex

Conversation

@lekurile
Copy link
Copy Markdown
Contributor

@lekurile lekurile commented Nov 12, 2022

This PR fixes several issues with the inference-test.py text-generation script and updates the README to describe the purpose of the DSPipeline utility class.

The issues resolved are:

  • No local_rank argument
  • DSPipeline class doesn't support use_cache
  • Meta tensors not working due to missing base_dir and checkpoint arguments to the DeepSpeed init_inference API call.

@RezaYazdaniAminabadi @awan-10 @mrwyattii @jeffra

@RezaYazdaniAminabadi
Copy link
Copy Markdown
Contributor

RezaYazdaniAminabadi commented Nov 12, 2022

Thanks @lekurile all LGTM :)

@RezaYazdaniAminabadi RezaYazdaniAminabadi merged commit 78ef018 into master Nov 12, 2022
hwchen2017 pushed a commit that referenced this pull request Jun 8, 2025
@lekurile
Fix the inference-test.py script and update text-generation README (#219
f8931eb 
)

* Fix inference-test.py script and update README

* change init arg name to model to match HF

* revert to model_name

* Update README to point to proper header
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jeffra jeffra Awaiting requested review from jeffra

@samyam samyam Awaiting requested review from samyam

@tjruwase tjruwase Awaiting requested review from tjruwase tjruwase is a code owner

@ShadenSmith ShadenSmith Awaiting requested review from ShadenSmith

@conglongli conglongli Awaiting requested review from conglongli

@awan-10 awan-10 Awaiting requested review from awan-10

@cli99 cli99 Awaiting requested review from cli99

@eltonzheng eltonzheng Awaiting requested review from eltonzheng

@minjiaz minjiaz Awaiting requested review from minjiaz

@duli2012 duli2012 Awaiting requested review from duli2012

@mrwyattii mrwyattii Awaiting requested review from mrwyattii

@yaozhewei yaozhewei Awaiting requested review from yaozhewei

@arashb arashb Awaiting requested review from arashb

@xiaoxiawu-microsoft xiaoxiawu-microsoft Awaiting requested review from xiaoxiawu-microsoft

1 more reviewer

@RezaYazdaniAminabadi RezaYazdaniAminabadi RezaYazdaniAminabadi approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@lekurile @RezaYazdaniAminabadi