Skip to content

Commit 2729732

Browse files
nv-kkudrynskinv-kkudrynski
authored
Update README.md
1 parent 3d497a6 commit 2729732

1 file changed

Lines changed: 0 additions & 2 deletions

File tree

‎TensorFlow/LanguageModeling/BERT/README.md‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -175,8 +175,6 @@ To enable mixed precision, you can simply add the values to the environmental va
175175
176176
#### Enabling TF32
177177
178-
This section is model specific and needs to show how to enable TF32. How is TF32 being implemented? Tweaking layers, preprocessing data, etc…
179-
180178
TensorFloat-32 (TF32) is the new math mode in [NVIDIA A100](#https://www.nvidia.com/en-us/data-center/a100/) GPUs for handling the matrix math also called tensor operations. TF32 running on Tensor Cores in A100 GPUs can provide up to 10x speedups compared to single-precision floating-point math (FP32) on Volta GPUs.
181179
182180
TF32 Tensor Cores can speed up networks using FP32, typically with no loss of accuracy. It is more robust than FP16 for models which require high dynamic range for weights or activations.

0 commit comments

Comments
 (0)