Multilayer Perceptron with Julia and FluxML

Goal

Design and train a high-performing multilayer perceptron in Julia and FluxML that accurately classifies MNIST handwritten digits in 10 classes

Design

Neural Network Architecture

• Input Layer Nodes: 784
• Hidden Layers: 3
• Hidden Layer Nodes: [25, 25, 25]
• Output Layer Nodes: 10

Hyperparameters

• Learning Rate ($\alpha$): 0.1
• Momentum ($\psi$): 0.0001
• Weight Decay ($\lambda$): 0.0004
• Batch Size: 250

Training

• Epochs: 1000
• Loss Function: Cross Entropy
• Optimizer: Gradient Descent ($\alpha$, $\psi$)
• Regularizer: L2 (Weight Decay)

Training Metrics

Loss Log

At Epoch 1000

Training Loss: 0.006

Validation Loss: 0.194

Accuracy Log

At Epoch 1000

Training Accuracy: 99.98%

Validation Accuracy: 95.84

Test

Test Accuracy: 96.46%