Why is multiprocessing forced to spawn processes rather than forking

[mbsabath]

❓ The question

Hi all, I'm working on a change to OLMo to support use of additional Pytorch Dataset Classes in our fork of OLMO, and I'm getting some OOM errors due to the use of process spawning rather than forming. I'm considering making process start method configurable, but wanted to understand more about the reasons for forcing all multiprocessing to be done with spawn before I went ahead with the change.