Hyperparameter
Settings that are configured before training begins and control how the model learns, as opposed to parameters which are learned during training. Examples include learning rate, batch size, and number of layers.
Why It Matters
Proper hyperparameter tuning can improve model performance by 10-30% without changing architecture or data. It is one of the highest-ROI activities in ML development.
Example
Setting the learning rate to 0.001, batch size to 32, and number of training epochs to 100 before starting to train a neural network.
Think of it like...
Like the settings on an oven before baking — temperature, time, and rack position are set before you start, and they dramatically affect the final result.
Related Terms
Learning Rate
A hyperparameter that controls how much the model's weights are adjusted in response to errors during each training step. It determines the size of the steps taken during gradient descent optimization.
Batch Size
The number of training examples processed together before the model updates its parameters. Batch size affects training speed, memory usage, and how smoothly the model learns.
Epoch
One complete pass through the entire training dataset during model training. Models typically require multiple epochs to learn effectively, with each pass refining the model's understanding.
Hyperparameter Tuning
The process of systematically searching for the best combination of hyperparameters for a model. Since hyperparameters are set before training, finding optimal values requires experimentation.