Code

Wordpiece Detokenizer Support#369

Code

Merged

apaniukov merged 1 commit intoopenvinotoolkit:masteropenvinotoolkit/openvino_tokenizers:masterfrom apaniukov:wordpiece-detokenizerapaniukov/openvino_tokenizers:wordpiece-detokenizerCopy head branch name to clipboardJan 15, 2025

+18,228-17,130

Commits

Commits on Jan 15, 2025

• Wordpiece Detokenizer Support

  Show description for d493321
  apaniukov

  committed
  d493321

  Copy full SHA for d493321