Machine learning: build your first mental model
Build a beginner’s mental model of machine learning: define features and labels, make a baseline, evaluate carefully, and avoid data leakage.
Learn it here, one small step at a time.
4 guided lessons · About 2h 30m including practice · No subscription
- Describe features, labels, and a baseline
- Choose a split and metric that fit the task
- Spot overfitting and data leakage
Recommended first: Python: your first practical programs, Pandas: explore real-world data. These foundations are suggestions, not a requirement to begin.
Choose a prediction and define its inputs
Machine learning finds patterns in examples. The label is what you want to predict; features are the information available to make that prediction. For example, a model might use a home’s size and age to estimate its sale price. Be precise about when each feature is available.
Begin with a baseline: a simple rule you can beat, such as predicting the training-set average price. A complicated model is not useful unless it improves a meaningful measure on data it did not learn from.
| Question | Example |
|---|---|
| Label | Sale price |
| Features | Size and age at the time of listing |
| Prediction time | Before the sale |
| Baseline | Predict the average training price |
Write a prediction question for a made-up dataset. Label the outcome, list two features, and note when each feature becomes available.
Reveal a suggested answer
Otherwise the model uses information it would not have when making a real prediction, creating an unrealistic score.
Optional further reading
Core lesson and practice are on this page. For more detail, visit the official free reference ↗.
Separate data and choose a useful metric
A training set teaches the model. A held-out test set helps estimate how it performs on examples it has not seen. Do not repeatedly tune choices using the test set; that turns it into part of the training process. Keep related records together when splitting if they belong to the same person or event.
Choose a metric based on the cost of mistakes. Accuracy can look high when one class is much more common than another. For regression, compare prediction errors in units people understand.
| Task | Possible metric | What it measures |
|---|---|---|
| Predict a numeric amount | Mean absolute error | Average size of the error |
| Flag a rare event | Precision and recall | Correct alerts and missed events |
| Balanced class prediction | Accuracy plus class details | Fraction classified correctly |
A dataset has 95 normal cases and 5 rare cases. Explain why a model that always predicts “normal” has misleading 95% accuracy.
Reveal a suggested answer
How costly different kinds of mistakes are for the people using the prediction.
Optional further reading
Core lesson and practice are on this page. For more detail, visit the official free reference ↗.
Train a model and compare it with a baseline
Training adjusts a model to fit examples. Make one change at a time, train using only the training set, and compare the result with your baseline on held-out data. Keep a record of the data split, features, model choice, and metric so the experiment can be repeated.
A model score is not the whole story. Check errors for different groups or ranges, inspect examples it gets wrong, and ask whether the input data represents the situation where the model will be used.
| Run | Change | Held-out result |
|---|---|---|
| Baseline | Predict average | Record score and units |
| Model A | Use size feature | Compare with baseline |
| Model B | Add age feature | Check improvement and errors |
Write down the baseline first. Then add one feature and predict what should improve. Compare both results on exactly the same held-out examples.
Reveal a suggested answer
It shows whether the model adds value beyond a simple rule and gives you a fair point of comparison.
Optional further reading
Core lesson and practice are on this page. For more detail, visit the official free reference ↗.
Recognize overfitting and leakage
Overfitting happens when a model learns details that work on its training examples but do not generalize. A much better training score than validation score is a warning. Data leakage occurs when training uses information unavailable at prediction time, including labels or future facts disguised as features.
Fit preprocessing steps only on the training data, then apply the fitted steps to validation and test data. After deployment, monitor whether the inputs and error patterns change; past performance is not a guarantee about the future.
| Signal | Possible problem | Next check |
|---|---|---|
| Training score far better | Overfitting | Compare against validation data |
| Future value in a feature | Leakage | Check when the value exists |
| New population performs worse | Data shift | Review recent input and outcome patterns |
Imagine predicting whether a customer will renew next month. Would “renewal date” be a safe input today? Explain your answer.
Reveal a suggested answer
The information is not available at prediction time, so the model would be tested with knowledge it cannot have in real use.
Optional further reading
Core lesson and practice are on this page. For more detail, visit the official free reference ↗.
Your final knowledge check.
Score 80% or more to pass. Sign in to save your best result.