Skip to content

Commit 902a0f6

Browse files
Update README.md to add python formatting in code examples (deepspeedai#729)
1 parent 12f78ec commit 902a0f6

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

  • inference/huggingface/zero_inference

inference/huggingface/zero_inference/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ INT4 weight quantization can be easily enabled with a few lines of configuration
115115

116116
### Quantized Initialization
117117
This is the easiest way to getting started. By providing a few lines of hints in ds_config, the model will be on-the-fly quantized during model initialization (e.g., AutoModel.from_pretrained). All candidate layers will be automatically quantized.
118-
```
118+
```python
119119
ds_config = {
120120
'weight_quantization': {
121121
'quantized_initialization': {
@@ -135,7 +135,7 @@ Currently, ZeRO-inference can quantize the weight matrix of nn.Embedding and nn.
135135
136136
### Post Initialization Quantization
137137
In this mode, model is first loaded in FP16 format and then convert into INT4. The advantage of enabling this mode is that users will have an overview of the model architecture. Thus, they will have fine-grained control over the quantization decision. For example, which layer should be quantized with which quantization configuration can be controlled. Only a few lines of code changes are needed. Note that we plan to expand this mode to accommodate more formats in the near future.
138-
```
138+
```python
139139
from deepspeed.compression.inference.quantization import _init_group_wise_weight_quantization
140140
ds_config = {
141141
'weight_quantization': {

0 commit comments

Comments
 (0)