overfit and underfit

What does Overfit and Underfit mean? What is overfitting? What is underfitting? What is generalization?

Overfitting: when a model learns -more than it should- the data's patterns (training set), but on new unseen data (testing set) it keeps applying the previous learned patterns resulting in wrong results.

Underfitting: when a model doesn't learn well the data's patterns (training set) because of a small amount of training time, or the model is not powerful enough or it is over-regularized.

What we want is generalization: models that perform good on new unseen data.

A model trained on more complete data will naturally generalize better.
When that is no longer possible, the next best solution is to use techniques like regularization.

These place constraints on the quantity and type of information your model can store.
If a network can only afford to memorize a small number of patterns, the optimization process will force it to focus on the most prominent patterns, which have a better chance of generalizing well.

- tensorflow.org


import tensorflow as tf

from tensorflow.keras import layers
from tensorflow.keras import regularizers


# make sure to pip install this first:
# !pip install -q git+https://github.com/tensorflow/docs
import tensorflow_docs as tfdocs
import tensorflow_docs.modeling
import tensorflow_docs.plots

from  IPython import display
from matplotlib import pyplot as plt

import numpy as np

import pathlib
import shutil
import tempfile

logdir = pathlib.Path(tempfile.mkdtemp())/"tensorboard_logs"
shutil.rmtree(logdir, ignore_errors=True)
python tensorflow machine learning ml overfitting underfitting tf artificial intelligence ai