Overfitting Amp Underfitting Ml
Last updated: 1/9/2025
Overfitting and underfitting are the most common reasons why [[machine-learning|ML]] algorithms perform poorly. Two metrics for this is the bias (the actual error rate), and the variance (how much the error rate of the training dataset and the test dataset is)
Underfitting
This is when the model cannot accurately predict the data, and the model that perfectly models the data is more complex then the trained model.
Reasons for Underfitting:
- Size of the training set is not large enough (try [[data-augmentation|data augmentation]])
- Model is too simple
- Training data is not cleaned and contains noise
Solutions:
- Increase the model complexity
- Increase the number of features
- Remove noise from the dataset
- Increase the length of the training
Note that these solutions should not just be thrown whilly nilly at the problem and hope that it works. It is important to figure out why the model is underfitting and then apply the appropriate solutions.
Overfitting
Overfitting happens when the model starts learning on the noise and does not make accurate predictions on the testing data. It has low bias but high variance.
Reasons for Overfitting:
- Too much training
- Training data is small (model memorizes data)
- Model is too complex (it can memorize the data too well)
Solutions:
- Increase the training data
- Reduce the model complexity
- try [[Early Stopping|early stopping]] during the training phase (have an eye over the loss over the training period as soon as loss begins to increase stop training)
- Try [[Ridge Regularization|ridge regularization]] and [[Lasso Regularization|lasso regularization]]
- Use dropout layers
- Use feature-engineering techniques