Commits

Breadcrumbs

History for

ondev

User selector

All users

All time

Commit History

Commits on Jul 24, 2023

• Normilizing codebert embeddings. Plot score histograms.

  joaomonteirof

  committed
  1f5d3ba

  Copy full SHA for 1f5d3ba

Commits on May 10, 2023

• Fix: Using tensor with dummy value rather than None for the temperature in the mlm case.

  joaomonteirof

  committed
  4df8d49

  Copy full SHA for 4df8d49

Commits on May 9, 2023

• Minor refactor to be compatible with newer versions of Transformers.

  joaomonteirof

  committed
  a095dbd

  Copy full SHA for a095dbd

Commits on Mar 27, 2023

• Added wandb options including whther to write grads to logs.

  joaomonteirof

  committed
  6772c76

  Copy full SHA for 6772c76

• Ensuring shuffling of dataset. Fix for toolkit.

  joaomonteirof

  committed
  9344ea3

  Copy full SHA for 9344ea3

Commits on Mar 23, 2023

• Moving to a different implementation of all_gather.

  joaomonteirof

  committed
  951b188

  Copy full SHA for 951b188

Commits on Mar 22, 2023

• Fix on logging embedding norms data. Save steps now matches logging steps.

  joaomonteirof

  committed
  1496a64

  Copy full SHA for 1496a64

• Merge pull request #1 from bigcode-project/small-changes

  Show description for e9daacb
  joaomonteirof

  authored
  e9daacb

  Copy full SHA for e9daacb

• small tweaks

  

  leandro von werra

  committed
  0cc79de

  Copy full SHA for 0cc79de

Commits on Mar 21, 2023

• Modified embeding normalization to avoid numerical issues. New initial values for the temp.

  joaomonteirof

  committed
  ed37e00

  Copy full SHA for ed37e00

Commits on Mar 20, 2023

• Logging embedding norms and temp coef. Minor refactor on tokenizer prep. Changes on evaluation notebook to include baseline.

  joaomonteirof

  committed
  c5dc5a7

  Copy full SHA for c5dc5a7

Commits on Mar 9, 2023

• Safer gather operation for pair of tensors

  joaomonteirof

  committed
  95f3dbe

  Copy full SHA for 95f3dbe

Commits on Mar 7, 2023

• Added options for bf16 and setting the padding id for encoders.

  joaomonteirof

  committed
  c881523

  Copy full SHA for c881523

• Added wandb logging and trainer optmization-scheduler rather than own. Hyperparamas now match roberta.

  joaomonteirof

  committed
  8d95224

  Copy full SHA for 8d95224

• Setting alpha=1 in exp_configs.py will now turn off the contrastive loss.

  joaomonteirof

  committed
  32f8d2c

  Copy full SHA for 32f8d2c

Commits on Mar 6, 2023

• Added hyperparam to compute contrastive loss with local batch only.

  joaomonteirof

  committed
  ad8c557

  Copy full SHA for ad8c557

• Fix: trying passing data_files arg to load dataset.

  joaomonteirof

  committed
  1b19db2

  Copy full SHA for 1b19db2

• Moved to existing implementation of data prep for MLM.

  joaomonteirof

  committed
  2806e74

  Copy full SHA for 2806e74

Commits on Mar 3, 2023

• Some cleaning up.

  joaomonteirof

  committed
  ad100c1

  Copy full SHA for ad100c1

• Option from training data, and default to bigcode/the-stack-march-sample

  joaomonteirof

  committed
  a6b7556

  Copy full SHA for a6b7556

Commits on Feb 27, 2023

• Refactor: using hf trainer now.

  joaomonteirof

  committed
  1255060

  Copy full SHA for 1255060

Commits on Feb 17, 2023

• Added evaluation task with the validation set of CodeSearchNet.

  joaomonteirof

  committed
  fbc35a8

  Copy full SHA for fbc35a8

Commits on Feb 13, 2023

• Added evaluation task and refactored accordingly. Included eval data under resources.

  joaomonteirof

  committed
  708096e

  Copy full SHA for 708096e

Commits on Feb 12, 2023

• Fix: not taping grads of embedding ops for positive pairs to avoid multi-gpu errors

  joaomonteirof

  committed
  75c2e6c

  Copy full SHA for 75c2e6c

Commits on Feb 10, 2023

• Some minor general fixes.

  joaomonteirof

  committed
  b1b2a8f

  Copy full SHA for b1b2a8f

• First commit.

  joaomonteirof

  committed
  68b087a

  Copy full SHA for 68b087a

Loading