Code

Added custom model inference.#437

Code

Open

JoelNiklaus wants to merge 34 commits intohuggingface:mainhuggingface/lighteval:mainfrom JoelNiklaus:add-custom-modelJoelNiklaus/lighteval:add-custom-modelCopy head branch name to clipboard

+881

Commits

Commits on Dec 11, 2024

• Added first version of custom model.

  JoelNiklaus

  committed
  5909d4a

  Copy full SHA for 5909d4a

• Merge branch 'main' into add-custom-model

  JoelNiklaus

  committed
  a2d6b63

  Copy full SHA for a2d6b63

• Merge branch 'main' into add-custom-model

  JoelNiklaus

  authored
  2283c89

  Copy full SHA for 2283c89

Commits on Dec 12, 2024

• Merge branch 'main' into add-custom-model

  JoelNiklaus

  authored
  9563fab

  Copy full SHA for 9563fab

• Merge branch 'main' into add-custom-model

  clefourrier

  authored
  319d482

  Copy full SHA for 319d482

• Merge branch 'main' into add-custom-model

  clefourrier

  authored
  464edfe

  Copy full SHA for 464edfe

• Moved custom model config.

  JoelNiklaus

  committed
  6096042

  Copy full SHA for 6096042

• Added warning.

  JoelNiklaus

  committed
  a7e1fe5

  Copy full SHA for a7e1fe5

• Added custom model example for google translate.

  JoelNiklaus

  committed
  24b8bd3

  Copy full SHA for 24b8bd3

• Added documentation for custom model config.

  JoelNiklaus

  committed
  c177a8e

  Copy full SHA for c177a8e

• Added docs.

  JoelNiklaus

  committed
  d712cdb

  Copy full SHA for d712cdb

• Merge branch 'main' into add-custom-model

  JoelNiklaus

  authored
  7553147

  Copy full SHA for 7553147

• Fixed path error.

  JoelNiklaus

  committed
  b41949c

  Copy full SHA for b41949c

• Fixed doc error.

  JoelNiklaus

  committed
  aaaadb0

  Copy full SHA for aaaadb0

• Added requirements file for google translate.

  JoelNiklaus

  committed
  c85065f

  Copy full SHA for c85065f

• Moved model loading function to reduce merge conflicts with litellm inference.

  JoelNiklaus

  committed
  f1103da

  Copy full SHA for f1103da

• Added diskcache and get source and target language from the task name.

  JoelNiklaus

  committed
  71f871e

  Copy full SHA for 71f871e

• Fixed problem with removing languages in the context.

  JoelNiklaus

  committed
  d1af518

  Copy full SHA for d1af518

Commits on Dec 13, 2024

• Added retry logic.

  JoelNiklaus

  committed
  2511158

  Copy full SHA for 2511158

Commits on Dec 16, 2024

• Merge branch 'main' into add-custom-model

  JoelNiklaus

  authored
  7d5f76d

  Copy full SHA for 7d5f76d

• Update google-translate requirements.

  JoelNiklaus

  committed
  743a284

  Copy full SHA for 743a284

Commits on Dec 17, 2024

• Added another example for a custom model.

  JoelNiklaus

  committed
  1a37f71

  Copy full SHA for 1a37f71

• Made local mt model example more general to support madlad400 as well.

  JoelNiklaus

  committed
  2f27645

  Copy full SHA for 2f27645

• Merge branch 'main' into add-custom-model

  JoelNiklaus

  authored
  a4d4fee

  Copy full SHA for a4d4fee

Commits on Dec 18, 2024

• Merge branch 'main' into add-custom-model

  clefourrier

  authored
  bd08781

  Copy full SHA for bd08781

• Make sure generation can happen on the GPU.

  JoelNiklaus

  committed
  b7106e4

  Copy full SHA for b7106e4

Commits on Dec 19, 2024

• Fixed issue with src and tgt lang for seamless model.

  JoelNiklaus

  committed
  a7d176c

  Copy full SHA for a7d176c

• Added cleanup to free the GPU memory again.

  JoelNiklaus

  committed
  f1ba65c

  Copy full SHA for f1ba65c

Commits on Dec 22, 2024

• Fix dependency issues by switching to deep-translator.

  JoelNiklaus

  committed
  ace6e59

  Copy full SHA for ace6e59

• Made inference code more robust against empty responses.

  JoelNiklaus

  committed
  cfd7254

  Copy full SHA for cfd7254

Commits on Dec 23, 2024

• Merge branch 'main' into add-custom-model

  JoelNiklaus

  authored
  3ddc104

  Copy full SHA for 3ddc104

Commits on Jan 2, 2025

• Merge branch 'main' into add-custom-model

  clefourrier

  authored
  f6df2a3

  Copy full SHA for f6df2a3

Commits on Jan 7, 2025

• Merge branch 'main' into add-custom-model

  JoelNiklaus

  authored
  348e427

  Copy full SHA for 348e427

Commits on Jan 11, 2025

• Merge branch 'main' into add-custom-model

  JoelNiklaus

  authored
  a63f4b3

  Copy full SHA for a63f4b3