Code

Adding grpo training#1233

Code

Open

Goekdeniz-Guelmez wants to merge 77 commits intoml-explore:mainml-explore/mlx-examples:mainfrom Goekdeniz-Guelmez:adding-GRPO-trainingGoekdeniz-Guelmez/mlx-examples:adding-GRPO-trainingCopy head branch name to clipboard

+1,215-63

Commits

Commits on Jan 28, 2025

• initial commit, gn

  Goekdeniz-Guelmez

  committed
  5e0ae83

  Copy full SHA for 5e0ae83

Commits on Jan 29, 2025

• Merge branch 'ml-explore:main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  b1e573d

  Copy full SHA for b1e573d

Commits on Jan 30, 2025

• updates ans fixing the KL div lines

  Goekdeniz-Guelmez

  committed
  93370ff

  Copy full SHA for 93370ff

Commits on Jan 31, 2025

• updates

  Goekdeniz-Guelmez

  committed
  6c58aa9

  Copy full SHA for 6c58aa9

• grpo_trainer shoudl be done

  Goekdeniz-Guelmez

  committed
  80bcf68

  Copy full SHA for 80bcf68

• update

  Goekdeniz-Guelmez

  committed
  a57d553

  Copy full SHA for a57d553

• update lora.py

  Goekdeniz-Guelmez

  committed
  243c962

  Copy full SHA for 243c962

Commits on Feb 3, 2025

• adding function for R1

  Goekdeniz-Guelmez

  committed
  d034ca3

  Copy full SHA for d034ca3

• Merge branch 'ml-explore:main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  734d6f4

  Copy full SHA for 734d6f4

• dataset wrapper done

  Goekdeniz-Guelmez

  committed
  a3ed632

  Copy full SHA for a3ed632

• Merge branch 'adding-GRPO-training' of https://github.com/Goekdeniz-Guelmez/mlx-examples into adding-GRPO-training

  Goekdeniz-Guelmez

  committed
  41ff536

  Copy full SHA for 41ff536

• starting fist training test run

  Goekdeniz-Guelmez

  committed
  23d75cd

  Copy full SHA for 23d75cd

• first working prototype, will try training out at home

  Goekdeniz-Guelmez

  committed
  1d9e480

  Copy full SHA for 1d9e480

• optims

  Goekdeniz-Guelmez

  committed
  05d921b

  Copy full SHA for 05d921b

• fixes

  Goekdeniz-Guelmez

  committed
  40bca77

  Copy full SHA for 40bca77

• print func name

  Goekdeniz-Guelmez

  committed
  06f9c29

  Copy full SHA for 06f9c29

• fix name funcs

  Goekdeniz-Guelmez

  committed
  54e295e

  Copy full SHA for 54e295e

• updates

  Goekdeniz-Guelmez

  committed
  ca32424

  Copy full SHA for ca32424

Commits on Feb 4, 2025

• first succesfull training run

  Goekdeniz-Guelmez

  committed
  7173840

  Copy full SHA for 7173840

• adding args into dataset handling

  Goekdeniz-Guelmez

  committed
  bd1a42e

  Copy full SHA for bd1a42e

• better create_dataset

  Goekdeniz-Guelmez

  committed
  7b01414

  Copy full SHA for 7b01414

Commits on Feb 5, 2025

• fix cache handling

  Goekdeniz-Guelmez

  committed
  0a09a93

  Copy full SHA for 0a09a93

• udpate

  Goekdeniz-Guelmez

  committed
  2a8e6f6

  Copy full SHA for 2a8e6f6

• fix testing

  Goekdeniz-Guelmez

  committed
  d84ad0c

  Copy full SHA for d84ad0c

• udpates

  Goekdeniz-Guelmez

  committed
  a33cad8

  Copy full SHA for a33cad8

• smoll fix

  Goekdeniz-Guelmez

  committed
  35a2d99

  Copy full SHA for 35a2d99

• updates

  Goekdeniz-Guelmez

  committed
  0a19522

  Copy full SHA for 0a19522

• updates

  Goekdeniz-Guelmez

  committed
  bcfa55d

  Copy full SHA for bcfa55d

Commits on Feb 6, 2025

• Merge branch 'ml-explore:main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  94dcd0f

  Copy full SHA for 94dcd0f

Commits on Feb 9, 2025

• fix

  Goekdeniz-Guelmez

  committed
  9ba6146

  Copy full SHA for 9ba6146

• freeze ref model

  Goekdeniz-Guelmez

  committed
  39e9469

  Copy full SHA for 39e9469

• fix

  Goekdeniz-Guelmez

  committed
  5417990

  Copy full SHA for 5417990

• fix: prevent gradients from flowing through the reference model's logits

  Goekdeniz-Guelmez

  committed
  a527cdb

  Copy full SHA for a527cdb

• rebase loss calculation

  Goekdeniz-Guelmez

  committed
  0071252

  Copy full SHA for 0071252

Commits on Feb 10, 2025

• Merge branch 'main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  0dac286

  Copy full SHA for 0dac286

• nits

  Goekdeniz-Guelmez

  committed
  d9da35f

  Copy full SHA for d9da35f

• removing helper functions

  Goekdeniz-Guelmez

  committed
  f88e897

  Copy full SHA for f88e897

• nits

  Goekdeniz-Guelmez

  committed
  e5aa2c3

  Copy full SHA for e5aa2c3

• nits

  Goekdeniz-Guelmez

  committed
  b7bc811

  Copy full SHA for b7bc811

• nits

  Goekdeniz-Guelmez

  committed
  88ca747

  Copy full SHA for 88ca747

Commits on Feb 11, 2025

• updates

  Goekdeniz-Guelmez

  committed
  e96afe9

  Copy full SHA for e96afe9

• fix

  Goekdeniz-Guelmez

  committed
  e80bf95

  Copy full SHA for e80bf95

• fix

  Goekdeniz-Guelmez

  committed
  35ecc17

  Copy full SHA for 35ecc17

• small fix

  Goekdeniz-Guelmez

  committed
  978deab

  Copy full SHA for 978deab

Commits on Feb 12, 2025

• update new iterade batches function + nits

  Goekdeniz-Guelmez

  committed
  5aeefc8

  Copy full SHA for 5aeefc8

• Merge branch 'adding-GRPO-training' of https://github.com/Goekdeniz-Guelmez/mlx-examples into adding-GRPO-training

  Goekdeniz-Guelmez

  committed
  c42e858

  Copy full SHA for c42e858

• updates

  Goekdeniz-Guelmez

  committed
  e33d9d5

  Copy full SHA for e33d9d5

• Merge branch 'ml-explore:main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  3823154

  Copy full SHA for 3823154

• small fix

  Goekdeniz-Guelmez

  committed
  a7273f6

  Copy full SHA for a7273f6

• quick prompting fix

  Goekdeniz-Guelmez

  committed
  8179b99

  Copy full SHA for 8179b99

Commits on Feb 13, 2025

• nits

  Goekdeniz-Guelmez

  committed
  65a49dd

  Copy full SHA for 65a49dd

Commits on Feb 14, 2025

• reduncancy fix + nits

  Goekdeniz-Guelmez

  committed
  baeb9f1

  Copy full SHA for baeb9f1

Commits on Feb 15, 2025

• removing comments + adding temperature + reward weighting

  Goekdeniz-Guelmez

  committed
  5ec4790

  Copy full SHA for 5ec4790

• removing print and switching some variables in the math

  Goekdeniz-Guelmez

  committed
  6a6bd53

  Copy full SHA for 6a6bd53

Commits on Feb 17, 2025

• Merge branch 'ml-explore:main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  1eea135

  Copy full SHA for 1eea135

• fix generation cutoff in evaluation

  Goekdeniz-Guelmez

  committed
  541f0be

  Copy full SHA for 541f0be

Commits on Feb 19, 2025

• Merge branch 'ml-explore:main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  11c8991

  Copy full SHA for 11c8991

Commits on Feb 21, 2025

• little faster generation + prints ot a examplke generatino in validation mode, more optimization in trianing function

  Goekdeniz-Guelmez

  committed
  2f20107

  Copy full SHA for 2f20107

• Huge speed improvement in validation mode.

  Goekdeniz-Guelmez

  committed
  6086137

  Copy full SHA for 6086137

• training mode working too got from 2 toks/sec to 30 toks/sec with raw 1.5B model

  Goekdeniz-Guelmez

  committed
  710bc14

  Copy full SHA for 710bc14

• fix

  Goekdeniz-Guelmez

  committed
  c51b0a2

  Copy full SHA for c51b0a2

Commits on Feb 22, 2025

• nits

  Goekdeniz-Guelmez

  committed
  79de353

  Copy full SHA for 79de353

• generation speed improvement in training too from 3 t/s to 15 t/s

  Goekdeniz-Guelmez

  committed
  235348c

  Copy full SHA for 235348c

• nits

  Goekdeniz-Guelmez

  committed
  d653371

  Copy full SHA for d653371

• clean up and readding temperature argument

  Goekdeniz-Guelmez

  committed
  d9c4c6e

  Copy full SHA for d9c4c6e

• fix wrong generation in train

  Goekdeniz-Guelmez

  committed
  9705ed9

  Copy full SHA for 9705ed9

• add usage in LORA.md

  Goekdeniz-Guelmez

  committed
  c0bd89a

  Copy full SHA for c0bd89a

• Merge branch 'ml-explore:main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  bd5f081

  Copy full SHA for bd5f081

Commits on Feb 24, 2025

• adding custom system message integration in dataset, more opimizations (generates now faster, while same RAM usage), fix for the identical generatrions, seperated the reward functions into a sepera…

  Show description for e4eac9c
  Goekdeniz-Guelmez

  committed
  e4eac9c

  Copy full SHA for e4eac9c

• last update, gn

  Goekdeniz-Guelmez

  committed
  53185c7

  Copy full SHA for 53185c7

Commits on Feb 25, 2025

• Merge branch 'ml-explore:main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  ef6ff92

  Copy full SHA for ef6ff92

Commits on Feb 26, 2025

• smoll fix

  Goekdeniz-Guelmez

  committed
  fab2dc2

  Copy full SHA for fab2dc2

Commits on Feb 27, 2025

• Merge branch 'ml-explore:main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  f27ed26

  Copy full SHA for f27ed26

Commits on Feb 28, 2025

• Merge branch 'ml-explore:main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  a04eb02

  Copy full SHA for a04eb02

• batching fix

  Goekdeniz-Guelmez

  committed
  15d5327

  Copy full SHA for 15d5327

• Merge branch 'main' into adding-GRPO-training

  Goekdeniz-Guelmez

  authored
  80e10a5

  Copy full SHA for 80e10a5

• updates

  Goekdeniz-Guelmez

  committed
  925e114

  Copy full SHA for 925e114