I haven't selected in this project as i was new to the transformers and finetuning the language models like bert. But, here is what I got to know and i have just done half the part.
Firstly i have analysed the data of product_idtype and product length. I have applied log to product length as there are very large variations in product length like some having length 10 and other 20000. So applied log, which is also good to use for training the data with correct prediction.
Later, I have removed the stop words, special characters, and convert all to the lower case characters for training data.
The training in my laptop isn't possible due to low computation power and space. Soon, I would complete this whole project.