Foundations of Regression
This course covers linear regression, logistic regression, the sigmoid, decision boundaries, and MLE (why not SSE for probabilities).
Prerequisites
- Comfort with high-school algebra (lines, slopes, exponents)
- Basic Python helps but is not required, equations and intuition come first
- No machine learning background assumed, this is the recommended starting course on the Engineering track
Prerequisites
High School Algebra & Basic Calculus
Linear equations, slope-intercept form, derivatives, chain rule
Python & NumPy
Basic programming, array operations, for loops
Lessons
Linear regression: lines, SSR & gradient descent
Find the line that minimises prediction error
Why logistic regression?
Linear outputs break classification; you need bounded probabilities
The sigmoid: secret sauce of logistic regression
Sigmoid squashes any number into (0, 1)
Decision boundaries
The boundary where P(y=1) = 0.5 is a line in feature space
Intuition behind logistic regression
SSE fails for probabilities; MLE creates a convex objective
Log-likelihood instead of squared error
Bernoulli likelihood → log-likelihood → binary cross-entropy loss
Unlocks
Deep Neural Networks
Stack nonlinearities and repeat gradient descent at scale
Multi-class classification & softmax
Extend binary ideas to K > 2 classes