Merge pull request #4 from anras5/feature/description

Feature/description

README.md

@@ -1,8 +1,39 @@
-# Blood vessels
+# Retinal blood vessels segmentation
 
-## How to run
+Detection of retinal blood vessels using classical methods (edge detection using filters) and deep neural network methods (UNet).
+
+## Web application
 
 Windows-64
+
+1. Download current UNet release from [Releases](https://github.com/anras5/BloodVessels/releases).
+2. Place downloaded `.pth` file in `models` folder. 
+3. Execute commands:
 ```commandline
 conda create --name <env> --file .\requirements.txt
-```
+conda activate <env>
+python main.py
+```
+
+## Jupyter Notebook
+
+All core functionalities of the project are located in the `notebooks` folder.
+
+## Built with
+
+- Python 3.9
+- PyTorch
+- OpenCV
+- Flask
+
+## Web application
+
+Web application allows the user to upload their own images and see the results.
+
+![img.png](readme-images/main_page.png)
+
+![results.png](readme-images/results.png)
+
+## Contact
+
+[email protected]

notebooks/classifiers/unet.py

@@ -5,19 +5,50 @@
 
 
 def mask_parse(mask):
+    """Function used to change mask from (512, 512, 1) to (512, 512, 3)
+
+    Parameters
+    ----------
+    mask: np.ndarray
+        mask to be changed
+
+    Returns
+    -------
+    np.ndarray
+        changed mask
+    """
     mask = np.expand_dims(mask, axis=-1)  # (512, 512, 1)
     mask = np.concatenate([mask, mask, mask], axis=-1)  # (512, 512, 3)
     return mask
 
 
 class UNetClassifier:
+    """Class used to make predictions based on uploaded `UNet` model"""
 
     def __init__(self, model_path):
+        """
+        Parameters
+        ----------
+        model_path: str
+            Path to the file with `.pth` extension with model
+        """
         self.model = UNet(3)
         self.model.load_state_dict(torch.load(model_path, map_location='cuda'))
         self.model.eval()
 
     def predict(self, image: np.ndarray) -> np.ndarray:
+        """Function used to predict the image
+
+        Parameters
+        ----------
+        image: np.ndarray
+            Matrix representing the image to be made predictions upon
+
+        Returns
+        -------
+        np.ndarray
+            Predicted map with detected blood vessels
+        """
         image = cv2.resize(image, dsize=(512, 512), interpolation=cv2.INTER_CUBIC)
         img = image / 255.0
         img = np.transpose(img, (2, 0, 1))

notebooks/main.ipynb

@@ -9,6 +9,41 @@
     "collapsed": false
    }
   },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "Main notebook for the project.\n",
+    "List of contents:\n",
+    "1. Description of the problem\n",
+    "2. Classic method\n",
+    "3. Deep learning method\n",
+    "4. Conclusion"
+   ],
+   "metadata": {
+    "collapsed": false
+   }
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "# Description of the problem"
+   ],
+   "metadata": {
+    "collapsed": false
+   }
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "Having an image of the retina of the eye as an input, we have to detect which of the individual pixels are blood vessels. \\\n",
+    "Two approaches were used for this:\n",
+    "1. Classic method\n",
+    "2. Deep learning method"
+   ],
+   "metadata": {
+    "collapsed": false
+   }
+  },
   {
    "cell_type": "markdown",
    "source": [
@@ -53,7 +88,20 @@
   {
    "cell_type": "markdown",
    "source": [
-    "# Classic method"
+    "# Classic method\n",
+    "\n",
+    "The classic approach uses filters to detect the edges of objects in the image. Detailed process below:\n",
+    "\n",
+    "1. Separation data into RGB channels and extraction of the green channel for processing.\n",
+    "2. Application of a CLAHE filter.\n",
+    "3. Application of erosion and dilation (morphological filters) for each kernel from `kernel_sizes`\n",
+    "4. Application of CLAHE filter.\n",
+    "5. Application of image thresholding.\n",
+    "6. Determination of the mask.\n",
+    "7. Determination of the contours.\n",
+    "8. Re-application of the thresholding.\n",
+    "\n",
+    "All functionality is included in the `ClassicClassifier` class. Below you can see the result of using the classifier."
    ],
    "metadata": {
     "collapsed": false
@@ -125,6 +173,15 @@
     "collapsed": false
    }
   },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "As you can see, you can easily recognize the contours of the correct image, but there are a lot of distortions and incorrectly classified pixels. This is indicated by Recall and Precision."
+   ],
+   "metadata": {
+    "collapsed": false
+   }
+  },
   {
    "cell_type": "markdown",
    "source": [
@@ -137,7 +194,18 @@
   {
    "cell_type": "markdown",
    "source": [
-    "For Deep Learning we will be using U-Net"
+    "To use deep learning, we decided to use the UNet network, which is one of the best known when it comes to detecting data from medical images. Below is the structure of the network.\n",
+    "\n",
+    "![unet](../readme-images/u-net-architecture.png)"
+   ],
+   "metadata": {
+    "collapsed": false
+   }
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "Steps used in training"
    ],
    "metadata": {
     "collapsed": false
@@ -207,6 +275,15 @@
     "collapsed": false
    }
   },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "Dataset used to upload data to the model"
+   ],
+   "metadata": {
+    "collapsed": false
+   }
+  },
   {
    "cell_type": "code",
    "execution_count": 6,
@@ -240,6 +317,15 @@
     "collapsed": false
    }
   },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "Loss function: combination of Dice and Binary Cross Entropy"
+   ],
+   "metadata": {
+    "collapsed": false
+   }
+  },
   {
    "cell_type": "code",
    "execution_count": 7,
@@ -265,6 +351,15 @@
     "collapsed": false
    }
   },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "Constants used in learning"
+   ],
+   "metadata": {
+    "collapsed": false
+   }
+  },
   {
    "cell_type": "code",
    "execution_count": 9,
@@ -282,6 +377,15 @@
     "collapsed": false
    }
   },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "Main loop"
+   ],
+   "metadata": {
+    "collapsed": false
+   }
+  },
   {
    "cell_type": "code",
    "execution_count": 7,
@@ -1214,6 +1318,15 @@
     "collapsed": false
    }
   },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "Predictions made using trained neural network"
+   ],
+   "metadata": {
+    "collapsed": false
+   }
+  },
   {
    "cell_type": "code",
    "execution_count": 10,
@@ -1331,6 +1444,20 @@
    "metadata": {
     "collapsed": false
    }
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "# Conclusions\n",
+    "\n",
+    "1. The neural network is better at classifying individual pixels as blood vessels. This is indicated by higher Recall and Precision values, but also Accuracy and Specificity.\n",
+    "2. Although the classical approach gives worse results, its gives results faster and there is no learning process, which can be critical in some applications.\n",
+    "\n",
+    "The trained model is used in the web application attached to the project. For more information, check out the README of the project."
+   ],
+   "metadata": {
+    "collapsed": false
+   }
   }
  ],
  "metadata": {

notebooks/unet/parts.py

@@ -3,6 +3,13 @@
 
 
 class ConvolutionalBlock(nn.Module):
+    """Convolutional block for UNet.
+    Consists of two Convolutional layers with 3x3 kernel and 1 pixel padding.
+    Each Convolutional layer is ended with ReLU activation function.
+
+    Useful links:
+    1. https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md
+    """
     def __init__(self, in_c, out_c):
         super().__init__()
         self.double_conv = nn.Sequential(
@@ -19,6 +26,7 @@ def forward(self, x):
 
 
 class DownBlock(nn.Module):
+    """Down Block consists of `ConvolutionalBlock` + MaxPooling layer with 2x2 kernel"""
     def __init__(self, in_c, out_c):
         super().__init__()
         self.conv = ConvolutionalBlock(in_c, out_c)
@@ -30,6 +38,7 @@ def forward(self, inputs):
 
 
 class UpBlock(nn.Module):
+    """Up Block consists of `ConvTranspose` layer with 2x2 kernel and 2 pixels stride"""
     def __init__(self, in_c, out_c):
         super().__init__()
         self.up = nn.ConvTranspose2d(in_c, out_c, kernel_size=2, stride=2)
@@ -42,6 +51,9 @@ def forward(self, inputs, skipped):
 
 
 class OutBlock(nn.Module):
+    """Out Block is the final layer of the UNet.
+    It consists of one Convolutional Layer with 1x1 kernel and sigmoid activation function.
+    """
     def __init__(self, in_c, out_c):
         super(OutBlock, self).__init__()
         self.conv = nn.Conv2d(in_c, out_c, kernel_size=1)

notebooks/unet/unet.py

@@ -5,7 +5,14 @@
 
 
 class UNet(nn.Module):
+    """Class used to build actual UNet using parts"""
     def __init__(self, n_channels):
+        """
+        Parameters
+        ----------
+        n_channels: int
+            number of input channels
+        """
         super(UNet, self).__init__()
 
         self.down1 = DownBlock(n_channels, 64)

notebooks/utils/accuracy.py

@@ -4,6 +4,7 @@
 
 
 def calculate_accuracy(y_pred, y_true):
+    """Function used to compute accuracy for two pytorch tensors"""
     y_true = y_true.cpu().detach().numpy()
     y_true = y_true > 0.5
     y_true = y_true.astype(np.uint8)

notebooks/utils/confusion_matrix.py

@@ -6,6 +6,7 @@
 
 
 class ConfusionMatrix:
+    """Class used to compute confusion matrix, accuracy, recall, precision and specificity for two numpy.ndarray"""
 
     def __init__(self, true, pred):
         self.matrix = confusion_matrix(true.flatten(), pred.flatten())
@@ -27,6 +28,7 @@ def __str__(self):
                f"{'Specificity:'.ljust(13, ' ')}{self.specificity:.3f}\n"
 
     def heatmap(self, annot=True, fmt=".0f", norm=LogNorm(), cmap='crest', cbar_kws=None):
+        """Display a heatmap of confusion matrix using Seaborn"""
         if cbar_kws is None:
             cbar_kws = {'ticks': []}
         sns.heatmap(self.df_matrix, annot=annot, fmt=fmt, norm=norm, cmap=cmap, cbar_kws=cbar_kws)