Skip to content

Commit

Permalink
anonymized
Browse files Browse the repository at this point in the history
  • Loading branch information
Jonas Zausinger committed Jan 30, 2025
1 parent 73e2aee commit 850697d
Show file tree
Hide file tree
Showing 2 changed files with 0 additions and 13 deletions.
13 changes: 0 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,6 @@

Introducing "Number Token Loss" (NTL) for language models to improve numerical reasoning by using regression-based loss functions that account for the proximity of numbers, achieving better performance on math tasks without increasing computational overhead.

## Resources
Find our paper [here](https://arxiv.org/abs/2411.02083) and the poster of the NeurIPS 2024 MathAI workshop [here](https://github.com/tum-ai/number-token-loss/blob/main/resources/neurips_mathai_poster.pdf "Poster")

## Setup

### Via Python
Expand Down Expand Up @@ -201,13 +198,3 @@ e.g for Standard T5 + **NTL-WAS**:
python src/ntl/run_language_modeling.py model_args=vanilla_t5_ntl model_args.number_token_loss_with_wasserstein=true training_args=eval model_args.model_name_or_path=<path to checkpoint file>
```
## Citation
If you use this work, please cite:
```bib
@inproceedings{zausinger24regress,
title={Regress, Don't Guess--A Regression-like Loss on Number Tokens for Language Models},
author={Zausinger, Jonas and Pennig, Lars and Chlodny, Kacper and Limbach, Vincent and Ketteler, Anna and Prein, Thorben and Singh, Vishwa Mohan and Danziger, Michael and Born, Jannis},
booktitle={The 4th Workshop on Mathematical Reasoning and AI at NeurIPS'24},
year={2024}
}
Binary file removed resources/neurips_mathai_poster.pdf
Binary file not shown.

0 comments on commit 850697d

Please sign in to comment.