Commits

Breadcrumbs

History for

ontoken-drop

User selector

All users

All time

Commit History

Commits on Sep 3, 2021

• add pseudocode for not dropping tokens.

  awan-10

  committed
  ae0fd0b

  Copy full SHA for ae0fd0b

Commits on Sep 1, 2021

• Use mpu in zero.Init() (#1325)

  tjruwase

  authored
  e08c239

  Copy full SHA for e08c239

• Support client lr schedulers that are not subclass of torch _LRScheduler (#1337)

  tjruwase

  authored
  8e301b6

  Copy full SHA for 8e301b6

• Added drop_last to DeepSpeedDataLoader (#1321)

  Show description for c0b27fb
  

  3 peopleauthored
  c0b27fb

  Copy full SHA for c0b27fb

Commits on Aug 31, 2021

• Remove duplicate clip grad function in deepspeed (#1333)

  Show description for ddffbae
  awan-10

  authored
  ddffbae

  Copy full SHA for ddffbae

Commits on Aug 28, 2021

• Use clone to avoid checkpoint bloat (#1326)

  tjruwase

  authored
  336dd08

  Copy full SHA for 336dd08

Commits on Aug 27, 2021

• Activation checkpointing improvements (#1254)

  Show description for 85acf14
  tjruwase

  authored
  85acf14

  Copy full SHA for 85acf14

• Add more synchronizations and barriers for the multi-gpu inference case (#1309)

  Show description for 0ec11da
  RezaYazdaniAminabadi

  authored
  0ec11da

  Copy full SHA for 0ec11da

Commits on Aug 26, 2021

• Reducing the memory-overhead of creating model for multi-GPU run (#1244)

  Show description for 49b6a63
  

  RezaYazdaniAminabadi

  and

  jeffra

  authored
  49b6a63

  Copy full SHA for 49b6a63

Commits on Aug 25, 2021

• Support Callable type for client optimizer and lr_scheduler (#1316)

  Show description for 274c375
  tjruwase

  authored
  274c375

  Copy full SHA for 274c375

• [zero] restore fp16 params if no zero ckpts available (#1322)

  Show description for aa12129
  jeffra

  authored
  aa12129

  Copy full SHA for aa12129

Commits on Aug 17, 2021

• MoE read the docs update (#1312)

  jeffra

  authored
  9cb64a1

  Copy full SHA for 9cb64a1

• DeepSpeed MoE (#1310)

  Show description for f284324
  

  10 peopleauthored
  f284324

  Copy full SHA for f284324

Commits on Aug 16, 2021

• Curriculum learning (#1307)

  Show description for b2b34ae
  

  conglongli

  and

  jeffra

  authored
  b2b34ae

  Copy full SHA for b2b34ae

Commits on Aug 6, 2021

• Use correct input size for splits (#1284)

  Show description for c543a41
  tjruwase

  authored
  c543a41

  Copy full SHA for c543a41

• Align gradient partitioning order (#1274)

  tjruwase

  authored
  b1b4175

  Copy full SHA for b1b4175

Commits on Aug 5, 2021

• Update adam.py (#1278)

  Show description for 00320a9
  DT6A

  authored
  00320a9

  Copy full SHA for 00320a9

Commits on Aug 2, 2021

• ZeRO-1 empty grads fix + tests (#1273)

  Show description for adc21a4
  jeffra

  authored
  adc21a4

  Copy full SHA for adc21a4

Commits on Jul 28, 2021

• Use correct default for round robin gradients (#1258)

  Show description for 97f7ed9
  tjruwase

  authored
  97f7ed9

  Copy full SHA for 97f7ed9

• Make round robin gradient partitioning configurable (default False) (#1256)

  tjruwase

  authored
  4d420df

  Copy full SHA for 4d420df

Commits on Jul 27, 2021

• ignore overlap/contiguous_gradients if using zero 1 (#1246)

  Show description for 6ae756c
  

  jeffra

  and

  tjruwase

  authored
  6ae756c

  Copy full SHA for 6ae756c

Commits on Jul 20, 2021

• Fixing inference api for FP32 and non-masking GPT-based models (#1204)

  Show description for 6ba9628
  RezaYazdaniAminabadi

  authored
  6ba9628

  Copy full SHA for 6ba9628

• [zero2] zero_param_shapes: switch to round_robin_fp16_groups (#1240)

  Show description for 7392e45
  stas00

  authored
  7392e45

  Copy full SHA for 7392e45

Commits on Jul 16, 2021

• [see_memory_usage] fix deprecation (#1234)

  Show description for 32e85ed
  

  stas00

  and

  tjruwase

  authored
  32e85ed

  Copy full SHA for 32e85ed

Commits on Jul 14, 2021

• [zero3] fix reference counting in backward over multiple forwards (#1227)

  Show description for 3fa2420
  stas00

  authored
  3fa2420

  Copy full SHA for 3fa2420

Commits on Jul 13, 2021

• [zero3] release tmp memory when consolidating fp16 weights take2 (#1223)

  Show description for 8103946
  

  stas00

  and

  tjruwase

  authored
  8103946

  Copy full SHA for 8103946

• ZeRO2-Offload: Disable copy overlapping (#1219)

  Show description for 97207c8
  tjruwase

  authored
  97207c8

  Copy full SHA for 97207c8

Commits on Jul 12, 2021

• revert part of #1220 (#1221)

  Show description for bc451c0
  stas00

  authored
  bc451c0

  Copy full SHA for bc451c0

• [zero3] release tmp memory when consolidating fp16 weights (#1220)

  Show description for 2660cc4
  

  stas00

  and

  tjruwase

  authored
  2660cc4

  Copy full SHA for 2660cc4

• [model weights] zero_to_fp32 multiple improvements (#1181)

  Show description for 2a92106
  

  stas00

  and

  tjruwase

  authored
  2a92106

  Copy full SHA for 2a92106

• json wrong format (#1210)

  Show description for 5652072
  

  GeorgeSen

  and

  tjruwase

  authored
  5652072

  Copy full SHA for 5652072

• improve debug (#1215)

  stas00

  authored
  5127b2f

  Copy full SHA for 5127b2f

Commits on Jul 9, 2021

• [zero.Init] post_init partitining is to be run only by a child module (#1202)

  Show description for 497b741
  

  stas00

  and

  tjruwase

  authored
  497b741

  Copy full SHA for 497b741

• [zero3] params_to_reduce isn't always there (#1214)

  Show description for 91f58c0
  stas00

  authored
  91f58c0

  Copy full SHA for 91f58c0

Commits on Jul 1, 2021

• contiguous gradients should be set to True by default (#1199)

  Show description for c9fee82
  samyam

  authored
  c9fee82

  Copy full SHA for c9fee82