overfit and underfit
What does Overfit and Underfit mean? What is overfitting? What is underfitting? What is generalization?
Overfitting: when a model learns -more than it should- the data's patterns (training set), but on new unseen data (testing set) it keeps applying the previous learned patterns resulting in wrong results.
Underfitting: when a model doesn't learn well the data's patterns (training set) because of a small amount of training time, or the model is not powerful enough or it is over-regularized.
What we want is generalization: models that perform good on new unseen data.
A model trained on more complete data will naturally generalize better.
When that is no longer possible, the next best solution is to use techniques like regularization.
These place constraints on the quantity and type of information your model can store.
If a network can only afford to memorize a small number of patterns, the optimization process will force it to focus on the most prominent patterns, which have a better chance of generalizing well.
import tensorflow as tf from tensorflow.keras import layers from tensorflow.keras import regularizers print(tf.__version__) # make sure to pip install this first: # !pip install -q git+https://github.com/tensorflow/docs import tensorflow_docs as tfdocs import tensorflow_docs.modeling import tensorflow_docs.plots from IPython import display from matplotlib import pyplot as plt import numpy as np import pathlib import shutil import tempfile logdir = pathlib.Path(tempfile.mkdtemp())/"tensorboard_logs" shutil.rmtree(logdir, ignore_errors=True)