Math foundation
Derivatives as "how fast, right here"
Stop thinking of it as a limit. It is a local rate, and that is all you need.
The formal definition of a derivative involves a limit, and that formality is what makes people bounce off calculus. For everything you will actually build — rendering, animation, optimisation, signal processing — a derivative is simply: if I nudge the input slightly, how much does the output move?
# The only definition you need in practice: a small nudge
def derivative(f, x, h=1e-6):
return (f(x + h) - f(x - h)) / (2 * h) # central difference: more accurate
f = lambda x: x**3
for x in [0, 1, 2]:
print(x, derivative(f, x), 3 * x**2) # they agree: the rule is 3x^2
# What you actually use it for:
# gradient descent -> step opposite the gradient to reduce error
# normal of a curve -> derivative gives the tangent, rotate for the normal
# animation -> derivative of position is velocity; again is accelerationThree rules cover the overwhelming majority of what you will meet, and each has a picture:
Chain rule — rates multiply along a composition. If `dy/du = 3` and `du/dx = 2` at the point, then `dy/dx = 6` there. That is the rule behind backpropagation.
Product rule — when two factors both change, you get two contributions: the change in one times the other, plus the change in the other times the first.
Stationary points — where the derivative is zero, a smooth function is momentarily flat. They are candidates for local extrema, but can also be stationary inflection points (as with `x³` at zero); zero derivative alone does not classify them.
What does the derivative mean practically?
Derivative
Local rate of change — nudge the input, see how the output moves.
Chain rule
Rates multiply along a composition; it is the whole of backprop.
Stationary point
Where the derivative is zero; may be an extremum or a stationary inflection.
Review cards
Derivative
Local rate of change — nudge the input, see how the output moves.
Chain rule
Rates multiply along a composition; it is the whole of backprop.
Stationary point
Where the derivative is zero; may be an extremum or a stationary inflection.
Sources for this lesson
Below are the references, editions and original links for further reading and checking.
CourseEssence of Linear Algebra / Calculus / Differential Equationsfree
3Blue1Brown (Grant Sanderson)
持续更新
「数形结合」最好的入口。先看它再看任何图形学教材，否则公式只是一堆符号。
BookDeep Learningfree
Ian Goodfellow, Yoshua Bengio, Aaron Courville
MIT Press（免费在线）
深度学习理论基础。第 6 章讲反向传播——本质就是链式法则沿复合链逐层倒着用一遍。
BookCalculus Volume 1, section 4.3: Maxima and Minimafree
OpenStax
Open textbook, online edition
States Fermat’s theorem and demonstrates that derivative-zero critical points are candidates, not necessarily extrema; x^3 has a stationary non-extremum at zero.
Lights up these nodes in the hub：f-cal