Optimize byte pair merge for really big tokens (40x faster for a 2500 token word) #218
| Job | Run time |
|---|---|
| 23s | |
| 22m 21s | |
| 3m 38s | |
| 23m 15s | |
| 3m 39s | |
| 2h 22m 19s | |
| 3m 30s | |
| 23m 50s | |
| 3m 40s | |
| 22m 43s | |
| 3m 30s | |
| 3m 35s | |
| 4m 8s | |
| 3m 36s | |
| 3m 21s | |
| 3m 38s | |
| 4m 20s | |
| 3m 7s | |
| 4m 7s | |
| 2m 33s | |
| 2m 59s | |
| 4h 48m 12s |