ByteLevelBPETokenizer with Greek gives weird symbols.

[gdet]

Hello,

I have followed your steps in this article https://huggingface.co/blog/how-to-train to train a model in Greek language.All files I used is in UTF-8 encoding. When using ByteLevelBPETokenizer I get weird symbols. I read in other issues here that this is normal but there is no normal character in my file merges.txt. Also when I try to print it to see if it tokenizes a word correctly it prints this:

 result = tokenizer.encode("Γεια τι κάνεις;")
 print (result .tokens)
 ['<s>', 'Îĵεια', 'ĠÏĦι', 'ĠκάνειÏĤ', ';', '</s>']

Is this normal? Or ByteLevelBPETokenizer is not suitable for Greek characters? Also is it possible to tranform this output to readable string to check if it is correct?
Example of merges.txt:

 ĠÏĦ ο
 ÏĦ η
 ĠÎ ½
 ĠÏĦ οÏħ

Thank you

[julien-c]

Yes, this is to be expected. See @n1t0's answer in this thread for context.