Use of machine learning
Pylinac uses machine learning to classify images. This works by using "supervised" learning algorithms. Essentially, you feed information to a classifier algorithm and tell it "This is a picket fence, this is a winston-lutz image,..." and so on. If you feed it enough images it can extract information about the image that defines its uniqueness (called "training"). Then, when fed new images of a certain classification it can correctly identify the image as being a certain type even though it's never seen that specific image before (called "prediction").
A classifier can be built using any number of inputs and "features"; as well, many types of classifiers exist. Some classifiers are more robust than others while some are quicker, etc. Pylinac uses a Support Vector Machine (SVM), which is fairly robust, and the resulting classifier isn't actually that big in terms of space. This allows a pylinac user to download the classifier on demand quickly. An SVM works by looking at the features between two or more datasets and fitting a line between them that maximizes the space on either side, like so:
Only the features that bound the fitted line on either side are important (bolded points in above image) and are thus called "support vectors". The good news is that all other data points are superfluous, so even if we have voluminous total data points, only the points that make up the support vectors are kept/matter. Notably, an SVM will contort the feature space (pixel data for images) to maximize the space between the classifications. This means that while images may appear similar, an SVM will find the differences and maximize those differences in ways that are not straightforward (called principal component analysis):
So pylinac uses an SVM and we kinda know how it works. How do we go from theory to practice? There are several machine learning libraries out there, but scikit-learn is one of the most popular as well as easiest to use. It allows for simple data input, training, and output.
First, the images are classified manually. This just means all images are of known types. I.e. all the picket fence images are actually picket fence images, etc. Each image type is given a label or classification. Each image is downsampled to be 100x100 pixels, and all pixel values are normalized. This allows each image to be comparable to one another.
Then, each image is fed to the classifier along with the known classification label. This is done for all the images of all the classifications. The classifier will then "train" on all the images to learn how to classify the images. At the end of the training, a test data set is used to test the accuracy and precision of the trained classifier. Once the classifier is deemed accurate enough, the classifier can simply be pickled like any other Python object and saved to file. This classifier is hosted online and downloaded by a pylinac user when the classifier is needed.