This week's menu:
- Lecture: pdf version, odp version
- Seminar & homework: here
- Video (both lecture and seminar)
- Alternative lecture: Parameter servers from CMU 10-605 - here
- Alternative seminar: python multiprocessing - playlist
Note: while you can do the entire thing in colab, we recommend using a node with at least 4 cpus to better "feel" the difference :)
Most modern PCs already have 4+ cores, but you can also find free cloud alternatives here. The easiest version is to use Kaggle kernels.
Note 2: The practice notebook was tested in Linux and MacOS. Running in Windows may cause problems due to inability to fork processes. When in doubt, try docker(kitematic) or run a linux VM. WSL might also do the trick.
More stuff:
- Numba parallel - a way to develop threaded parallel code in python without GIL
- joblib - a library of multiprocessing primitives similar to mp.Pool, but with some extra conveniences
- BytePS paper - https://www.usenix.org/system/files/osdi20-jiang.pdf