Derivatives – "Math for Non-Geeks"

The derivative ${\displaystyle f'(x)}$ is one of the central concepts within calculus. For a given function ${\displaystyle f(x)}$, the derivative ${\displaystyle f'(x)}$ is another function which specifies the rate of change of ${\displaystyle f(x)}$ in ${\displaystyle x}$. It is used in various scientific disciplines, basically everywhere, where there is a "rate of change" within a dynamical system. Knowing about derivatives means having a powerful tool at hand: it allows you to describe and predict rates of change in a huge variety of applications.

Te derivative is a mathematical object, which becomes useful in many situations. Depending on the situation, there are several intuitions which can make this abstract object come alive in your mind:

• Derivative as instantaneous rate of change: The derivative corresponds to what we intuitively understand as the rate of change of a function ${\displaystyle f(x)}$ at some instant ${\displaystyle x}$. A rate of change (${\displaystyle {\tfrac {\Delta f}{\Delta x}}}$) describes how much a quantity changes (${\displaystyle \Delta f}$) in relation to the change of some reference quantity (${\displaystyle \Delta x}$). If we let (${\displaystyle \Delta x}$) run to 0, we get the rate of change within an "infinitely small amount of time". An example are speeds: Consider a given time-dependent position ${\displaystyle s(t)}$, i.e. the function ${\displaystyle f}$ is re-labeld as ${\displaystyle s}$ and ${\displaystyle x}$ is re-labelled as ${\displaystyle t}$. The quotient ${\displaystyle {\tfrac {\Delta s}{\Delta t}}}$ of "travelled distance" ${\displaystyle \Delta x}$ and "elapsed time" ${\displaystyle \Delta t}$ just describes the "average speed". In order to get the speed ${\displaystyle v(t)}$ at some time ${\displaystyle t}$, we make the time difference ${\displaystyle \Delta t}$ smaller and smaller, such that the "average speed" ${\displaystyle {\tfrac {\Delta s}{\Delta t}}}$ goes over to an "instantaneous speed" ${\displaystyle v(t)}$ . This ${\displaystyle v(t)}$ is called first derivative and mathematicians write ${\displaystyle v(t)=s'(t)}$ .
• Derivative as tangent slope: The derivative corresponds to the slope that the tangent of the graph has at the location of the derivative. Thus the derivative solves the geometric problem of determining the tangent to a graph by a point.
• Derivative as slope of the locally best linear approximation: Any function that has a derivative a point can be well approximated by a linear function in an environment around this point. The derivative corresponds to the slope of this linear function. This is useful if the function is hard to compute: the linear approximation can be computed way easier in many cases.
• Derivative as generalised slope: How steep is a given function? At first, the concept of the "slope of a function" is only defined for linear functions. But we can use the derivative to define the "slope" also for non-linear functions.

We will discuss these intuitions in detail in the following and use them to derive a formal definition of the derivative. We will also see that derivable functions are "kink-free", which is why they are also called smooth functions (think of smoothly bending some dough or tissue).

The derivative corresponds to the rate of change of a function ${\displaystyle f}$. How can this rate of change of a function be determined or defined? Let, for example be ${\displaystyle f}$ a real-valued function, which has the following graph:

For example, ${\displaystyle f}$ may describe a physical quantity in relation to another quantity. For example, ${\displaystyle f(x)}$ could correspond to the distance covered by an object at the time ${\displaystyle x}$. ${\displaystyle f(x)}$ could also be the air pressure at the altitude ${\displaystyle x}$ or the population size of a species at the time ${\displaystyle x}$. Now let us take the argument ${\displaystyle {\tilde {x}}}$, where the function has the function value ${\displaystyle f({\tilde {x}})}$:

Let us assume that ${\displaystyle f(x)}$ is the distance travelled by a car at the time ${\displaystyle x}$. Then the current rate of change of ${\displaystyle f}$ at the position ${\displaystyle {\tilde {x}}}$ is equal to the velocity of the car at the time ${\displaystyle {\tilde {x}}}$.

It is hard to determine the velocity directly with only ${\displaystyle f(x)}$ given. But we can estimate it. We take a point in time ${\displaystyle x_{1}}$ shortly after ${\displaystyle {\tilde {x}}}$ and look at the average speed in that time ${\displaystyle v={\frac {\text{distance}}{\text{time}}}}$. The distance travelled in that time is ${\displaystyle f(x_{1})-f({\tilde {x}})}$, while the time difference is ${\displaystyle x_{1}-{\tilde {x}}}$. Thus the car has the average speed

${\displaystyle {\frac {f(x_{1})-f({\tilde {x}})}{x_{1}-{\tilde {x}}}}}$

This quotient, which indicates the average rate of change of the function ${\displaystyle f}$ in the interval ${\displaystyle [{\tilde {x}},x_{1}]}$, is called difference quotient. As its name suggests, it is a quotient of two differences. In the following figure we see that this difference quotient is equal to the slope of the secant passing through the points ${\displaystyle ({\tilde {x}},f({\tilde {x}}))}$ and ${\displaystyle (x_{1},f(x_{1}))}$:

This average speed is a good approximation of the current speed of our car at the time ${\displaystyle {\tilde {x}}}$. It is only an approximation since the movement of the car between ${\displaystyle {\tilde {x}}}$ and ${\displaystyle x_{1}}$ need not be uniform - it can accelerate or decelerate. But we should get a better result if we shorten the period for calculating the average speed. So let's look at a time ${\displaystyle x_{2}}$ which is even closer to ${\displaystyle {\tilde {x}}}$ and determine the average speed ${\displaystyle {\tfrac {f(x_{2})-f({\tilde {x}})}{x_{2}-{\tilde {x}}}}}$ for the new time interval between ${\displaystyle {\tilde {x}}}$ and ${\displaystyle x_{2}}$:

We can shorten the time difference even further by taking a sequence ${\displaystyle (x_{n})_{n\in \mathbb {N} }}$ of times which converge towards ${\displaystyle {\tilde {x}}}$. For every ${\displaystyle x_{n}}$ we calculate the average speed ${\displaystyle {\tfrac {f(x_{n})-f({\tilde {x}})}{x_{n}-{\tilde {x}}}}}$ of the car in the period from ${\displaystyle {\tilde {x}}}$ to ${\displaystyle x_{n}}$. The shorter ${\displaystyle x_{n}-{\tilde {x}}}$, the less the car should be able to accelerate or decelerate in this period of time. So the average speed should converge to the current speed of the car at time ${\displaystyle {\tilde {x}}}$:

Thus we have found a method to determine the current rate of change of ${\displaystyle f}$ at time ${\displaystyle {\tilde {x}}}$: We take any sequence of arguments ${\displaystyle (x_{n})_{n\in \mathbb {N} }}$, which are all different from ${\displaystyle {\tilde {x}}}$ and for which ${\displaystyle \lim _{n\to \infty }x_{n}={\tilde {x}}}$. For every ${\displaystyle x_{n}}$ we determine the quotient ${\displaystyle {\tfrac {f(x_{n})-f({\tilde {x}})}{x_{n}-{\tilde {x}}}}}$. The current rate of change is the limit of these quotients:

${\displaystyle {\text{Rate of change of }}f{\text{ at the argument }}{\tilde {x}}=\lim _{n\to \infty }{\frac {f(x_{n})-f({\tilde {x}})}{x_{n}-{\tilde {x}}}}}$

The derivative or ${\displaystyle f}$ at ${\displaystyle {\tilde {x}}}$ is denoted as ${\displaystyle f'({\tilde {x}})}$. So we have the mathematical definition:

${\displaystyle f'({\tilde {x}})=\lim _{n\to \infty }{\frac {f(x_{n})-f({\tilde {x}})}{x_{n}-{\tilde {x}}}}}$

The limit of the difference quotient is sometimes also called differential quotient.

What happens if we do not choose ${\displaystyle x_{n}}$ in the future, but in the past of ${\displaystyle {\tilde {x}}}$? Let us draw this situation in a picture:

The average speed in the interval from ${\displaystyle x_{n}}$ to ${\displaystyle {\tilde {x}}}$ is then equal to ${\displaystyle {\tfrac {f({\tilde {x}})-f(x_{n})}{{\tilde {x}}-x_{n}}}}$. If we extend this fraction by a factor of ${\displaystyle -1}$, we get

${\displaystyle {\frac {f({\tilde {x}})-f(x_{n})}{{\tilde {x}}-x_{n}}}={\frac {-\left(f({\tilde {x}})-f(x_{n})\right)}{-({\tilde {x}}-x_{n})}}={\frac {f(x_{n})-f({\tilde {x}})}{x_{n}-{\tilde {x}}}}}$

We get the same term as in the previous section. This gives the average speed, no matter if ${\displaystyle x_{n}<{\tilde {x}}}$ or ${\displaystyle x_{n}>{\tilde {x}}}$. Thus, in the case of a negative time interval with ${\displaystyle x_{n}<{\tilde {x}}}$ the average speed should also be close to the current speed of the car at the time ${\displaystyle {\tilde {x}}}$, if ${\displaystyle x_{n}}$ is only sufficiently close to ${\displaystyle {\tilde {x}}}$. There is

${\displaystyle f'({\tilde {x}})=\lim _{n\to \infty }{\frac {f(x_{n})-f({\tilde {x}})}{x_{n}-{\tilde {x}}}}}$

where ${\displaystyle (x_{n})_{n\in \mathbb {N} }}$ is any sequence of different from ${\displaystyle {\tilde {x}}}$ with ${\displaystyle \lim _{n\to \infty }x_{n}={\tilde {x}}}$. The sequence elements of ${\displaystyle (x_{n})_{n\in \in \mathbb {N} }}$ can sometimes be larger and sometimes smaller than ${\displaystyle {\tilde {x}}}$ depending on the index ${\displaystyle n}$:

Let now ${\displaystyle f:D\to \mathbb {R} }$ be a real-valued function and let ${\displaystyle {\tilde {x}}\in D}$. As we have seen above, there is

${\displaystyle f'({\tilde {x}})=\lim _{n\to \infty }{\frac {f(x_{n})-f({\tilde {x}})}{x_{n}-{\tilde {x}}}}}$

where ${\displaystyle \left(x_{n}\right)_{n\in \mathbb {N} }}$ is a sequence of arguments different from ${\displaystyle {\tilde {x}}}$ which converges to ${\displaystyle {\tilde {x}}}$. In order to have at least one such sequence of arguments, ${\displaystyle {\tilde {x}}}$ must be an accumulation point of the domain ${\displaystyle D}$ (an element is an accumulation point of a set exactly when there is a sequence not including that number but converging towards it). This may sound more complicated than it often is. In most cases ${\displaystyle D\subseteq \mathbb {R} }$ is an interval and then every ${\displaystyle {\tilde {x}}\in D}$ is an accumulation point of ${\displaystyle D}$. For the definition of the differential quotient it should not matter which sequence ${\displaystyle (x_{n})_{n\in \mathbb {N} }}$ we choose. Accordingly, we can define the derivative:

We can shorten this definition by using limits for functions. As a reminder: There is according to definition: ${\displaystyle \lim _{x\to c}g(x)=L}$ if and only if ${\displaystyle \lim _{n\to \infty }g(x_{n})=L}$ for all sequences ${\displaystyle (x_{n})_{n\in \mathbb {N} }}$ of arguments non-equal to ${\displaystyle c}$ with ${\displaystyle \lim _{n\to \infty }x_{n}=c}$. So:

There is an equivalent option to define the derivative. For this we go from the differential quotient ${\displaystyle \lim _{x\to {\tilde {x}}}{\tfrac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}}$ and perform the substitution ${\displaystyle x={\tilde {x}}+h}$. The new variable ${\displaystyle h}$ just describes the difference between ${\displaystyle {\tilde {x}}}$ and the point where the difference quotient is formed. For ${\displaystyle x\to {\tilde {x}}}$, equivalently goes ${\displaystyle h\to 0}$. So we can also define the derivative as follows

We have come to know the derivative as the current rate of change of a quantity. As such, it occurs frequently in science or applications. Several variables are defined as rates of change, for example:

• velocity: The velocity is the instantaneous rate of change of the distance travelled by an object.
• Acceleration: The acceleration is the instantaneous rate of change of the speed of an object.
• Pressure change: Let ${\displaystyle p(h)}$ the air pressure at altitude ${\displaystyle h}$. The derivative ${\displaystyle p'(h)}$ is the rate of change of air pressure with altitude. This example shows that the rate of change need not always be related to time. It can also be the rate of change with respect to another quantity, e.g. altitude.
• Chemical reaction rate: Let's consider a chemical reaction ${\displaystyle A\to B}$. Let ${\displaystyle d_{A}(t)}$ the concentration of the substance ${\displaystyle A}$ at time ${\displaystyle t}$. The derivative ${\displaystyle d_{A}'(t)}$ is the instantaneous rate of change of the concentration of ${\displaystyle A}$ and thus indicates how much of the substance ${\displaystyle A}$ is converted into the substance ${\displaystyle B}$. Thus ${\displaystyle d_{A}'(t)}$ indicates the chemical reaction rate for the reaction ${\displaystyle A\to B}$.
• Often the number of individuals ${\displaystyle N(t)}$ in a population is considered (for example the number of people on the planet, the number of bacteria in a Petri dish, the number of animals of a species or the number of atoms of a radioactive substance). The derivative ${\displaystyle N'(t)}$ represents the instantaneous rate of change of individuals at the time ${\displaystyle t}$.

The terms "difference quotient" and "differential quotient" are mathematically defined as follows:

${\displaystyle \underbrace {f'({\tilde {x}})} _{\text{derivative}}=\underbrace {\lim _{x\to {\tilde {x}}}\underbrace {\tfrac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}} _{\text{difference quotient}}} _{\text{differential quotient}}}$

If a function ${\displaystyle f:D\to \mathbb {R} }$ with ${\displaystyle D\subseteq \mathbb {R} }$ is differentiable at every point within its domain of definition, then ${\displaystyle f}$ has a derivative at every point in ${\displaystyle D}$. The function that assigns its derivative ${\displaystyle f'({\tilde {x}})}$ to every m argument ${\displaystyle {\tilde {x}}}$ is called derivative function of ${\displaystyle f}$:

Historically, different notations have been developed to represent the derivative of a function. In this article we have only learned about the notation ${\displaystyle f'}$ for the derivative of ${\displaystyle f}$. It goes back to the mathematician Joseph-Louis Lagrange , who introduced it in 1797. Within this notation the second derivative of ${\displaystyle f}$ is denoted ${\displaystyle f''}$ and the ${\displaystyle n}$-th derivative is denoted ${\displaystyle f^{(n)}}$ .

Isaac Newton - (the founder of differential calculus besides Leibniz) - denoted the first derivative of ${\displaystyle x}$ with ${\displaystyle {\dot {x}}}$, accordingly he denoted the second derivative by ${\displaystyle {\ddot {x}}}$. Nowadays this notation is mainly used in physics for the derivative with respect to time.

Gottfried Wilhelm Leibniz introduced for the first derivative of ${\displaystyle f}$ with respect to the variable ${\displaystyle x}$ the notation ${\displaystyle {\tfrac {\mathrm {d} f}{\mathrm {d} x}}(x)}$. This notation is read as "d f over d x of x". The second derivative is then denoted ${\displaystyle {\tfrac {\mathrm {d} ^{2}f}{\mathrm {d} x^{2}}}(x)}$ and the ${\displaystyle n}$-th derivative is written as ${\displaystyle {\tfrac {\mathrm {d} ^{n}f}{\mathrm {d} x^{n}}}(x)}$.

The notation of Leibniz is mathematically speaking not a fraction! The symbols ${\displaystyle \mathrm {d} f}$ and ${\displaystyle \mathrm {d} x}$ are called differentials, but in modern calculus (apart from the theory of so-called "differential forms") they have only a symbolic meaning. They are only allowed in this notation as formal differential quotients. Now there are applications of derivatives (like the "chain rule" or "integration by substitution"), in which the differentials ${\displaystyle \mathrm {d} f}$ or ${\displaystyle \mathrm {d} x}$ can be handled as if they were ordinary variables and in which one can come to correct solutions. But since there are no differentials in modern calculus, such calculations are not mathematically correct.

The notation ${\displaystyle Df}$ or ${\displaystyle D_{x}f(x)}$ for the first derivative of ${\displaystyle f}$ dates back to Leonhard Euler. In this notation, the second derivative is written as ${\displaystyle D^{2}f}$ or ${\displaystyle D_{x}^{2}f(x)}$ and the ${\displaystyle n}$-th derivative as ${\displaystyle D^{n}f}$ or ${\displaystyle D_{x}^{n}f(x)}$.

Notation of the …	1st derivative	2nd derivative	${\displaystyle n}$-th derivative
Lagrange	${\displaystyle f'}$	${\displaystyle f''}$	${\displaystyle f^{(n)}}$
Newton	${\displaystyle {\dot {f}}}$	${\displaystyle {\ddot {f}}}$	${\displaystyle {\overset {\,n}{\dot {f}}}}$
Leibniz	${\displaystyle {\tfrac {\mathrm {d} f}{\mathrm {d} x}}}$	${\displaystyle {\tfrac {\mathrm {d} ^{2}f}{\mathrm {d} x^{2}}}}$	${\displaystyle {\tfrac {\mathrm {d} ^{n}f}{\mathrm {d} x^{n}}}}$
Euler	${\displaystyle Df}$	${\displaystyle D^{2}f}$	${\displaystyle D^{n}f}$

The derivative ${\displaystyle f'({\tilde {x}})}$ corresponds to the limit value ${\displaystyle \lim _{x\to {\tilde {x}}}{\tfrac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}}$. The difference quotient ${\displaystyle {\tfrac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}}$ is the slope of the secant between the points ${\displaystyle ({\tilde {x}},f({\tilde {x}}))}$ and ${\displaystyle (x,f(x))}$. In the case of the boundary value formation ${\displaystyle x\to {\tilde {x}}}$, this secant merges into the tangent that touches the graph of ${\displaystyle f}$ at the point ${\displaystyle ({\tilde {x}},f({\tilde {x}}))}$:

Damit ist die derivative ${\displaystyle f'({\tilde {x}})}$ gleich der Steigung der Tangente am Graphen durch den Punkt ${\displaystyle ({\tilde {x}},f({\tilde {x}}))}$. Die derivative kann also genutzt werden, um die Tangente an einem Graphen zu bestimmen. Somit löst sie auch ein geometrisches Problem. Mit ${\displaystyle f'({\tilde {x}})}$ kennen wir die Steigung der Tangente and with ${\displaystyle ({\tilde {x}},f({\tilde {x}}))}$ einen Punkt auf der Tangente. Damit können wir die functionsgleichung dieser Tangente bestimmen.

Thus the derivative ${\displaystyle f'({\tilde {x}})}$ is equal to the slope of the tangent to the graph through the point ${\displaystyle ({\tilde {x}},f({\tilde {x}}))}$. we may also use the derivative to compute the tangent to a graph. With ${\displaystyle f'({\tilde {x}})}$ we know the slope of the tangent. The offset can be determined using that ${\displaystyle ({\tilde {x}},f({\tilde {x}}))}$ is a point on the tangent. The following question illustrates how this works:

The derivative can be used to approximate a function. One may even define the derivative as the "best linear approximation" to a function. To find this approximation we start with the definition of the derivative as a limit:

${\displaystyle f'({\tilde {x}})=\lim _{x\to {\tilde {x}}}{\frac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}}$

The difference quotient ${\displaystyle {\tfrac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}}$ gets arbitrarily close to the derivative ${\displaystyle f'({\tilde {x}})}$, if ${\displaystyle x}$ gets sufficiently close to ${\displaystyle {\tilde {x}}}$. For ${\displaystyle x\approx {\tilde {x}}}$ we can write:

${\displaystyle f'({\tilde {x}})\approx {\frac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}}$

In the following we assume, that the expression ${\displaystyle x\approx {\tilde {x}}}$ for "${\displaystyle x}$ is approximately as large as ${\displaystyle {\tilde {x}}}$" is well defined and obeys the common arithmetic laws for equations. So we can change this equation to

${\displaystyle {\begin{aligned}&&f'({\tilde {x}})&\approx {\frac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}\\[0.3em]&\implies {}&f'({\tilde {x}})\cdot (x-{\tilde {x}})&\approx f(x)-f({\tilde {x}})\\[0.3em]&\implies {}&f({\tilde {x}})+f'({\tilde {x}})\cdot (x-{\tilde {x}})&\approx f(x)\\[0.3em]&\implies {}&f(x)&\approx f({\tilde {x}})+f'({\tilde {x}})\cdot (x-{\tilde {x}})\end{aligned}}}$

If ${\displaystyle x}$ is sufficiently close to ${\displaystyle {\tilde {x}}}$, then ${\displaystyle f(x)}$ is approximately equal to ${\displaystyle f({\tilde {x}})+f'({\tilde {x}})\cdot (x-{\tilde {x}})}$. This value can thus be used as an approximation of ${\displaystyle f(x)}$ near the derivative position. The function with the assignment rule ${\displaystyle x\mapsto f({\tilde {x}})+f'({\tilde {x}})\cdot (x-{\tilde {x}})}$ is a linear function, since ${\displaystyle {\tilde {x}}}$ is an arbitrary but fixed point.

The assignment rule ${\displaystyle t(x)=f({\tilde {x}})+f'({\tilde {x}})\cdot (x-{\tilde {x}})}$ describes the tangent, which touches the graph of the function at the position where the derivative is taken. Thus, the tangent near the point of contact is a good approximation of the graph. This is also shown in the following diagram. If one zooms in close enough at a point in a differential function, the graph looks approximately like a straight line:

This line is described by the assignment rule ${\displaystyle t(x)=f({\tilde {x}})+f'({\tilde {x}})\cdot (x-{\tilde {x}})}$ and corresponds to the tangent of the graph at this position.

Let's take a look at the above mentioned example. For this we consider the sine function ${\displaystyle \sin(x)}$. Its graph is

As we shall see, the derivative of the sine is the cosine and thus

${\displaystyle \sin '(0)=\cos(0)=1}$

the linear approximation of the sine is hence

${\displaystyle \sin(x)\approx \sin(0)+\sin '(0)\cdot (x-0)=0+1\cdot (x-0)=x}$

In the vicinity of zero, there is ${\displaystyle \sin(x)\approx x}$. This is the so called small-angle approximation. Thus, ${\displaystyle \sin \left({\tfrac {1}{4}}\right)}$ can be approximated by ${\displaystyle {\tfrac {1}{4}}}$. With ${\displaystyle \sin \left({\tfrac {1}{4}}\right)=0{.}2474\ldots }$ this approximation is also quite good. The following diagram shows that near zero, the sine function can be described approximately by a line ${\displaystyle \sin(x)\approx x}$:

The diagram also shows that this approximation is only good near the derivative point. For values ${\displaystyle x}$ far away from zero, ${\displaystyle \sin(x)}$ differs greatly from ${\displaystyle x}$. The approximation ${\displaystyle \sin(x)\approx x}$ is therefore only meaningful for small arguments!

How good is the approximation ${\displaystyle f(x)\approx f({\tilde {x}})+f'({\tilde {x}})\cdot (x-{\tilde {x}})}$? To answer this, let ${\displaystyle \epsilon (x)}$ be the value with

${\displaystyle {\frac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}=f'({\tilde {x}})+\epsilon (x)}$

The value ${\displaystyle \epsilon (x)}$ is therefore the difference between the difference quotient ${\displaystyle {\tfrac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}}$ and the derivative ${\displaystyle f'({\tilde {x}})}$. This difference disappears in the limit ${\displaystyle x\to {\tilde {x}}}$, because for this limit the difference quotient turns into a differential quotient, i.e. the derivative ${\displaystyle f'({\tilde {x}})}$. There is also ${\displaystyle \lim _{x\to {\tilde {x}}}\epsilon (x)=0}$. Now we can rearrange the above equation and get

${\displaystyle {\begin{aligned}&&{\frac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}&=f'({\tilde {x}})+\epsilon (x)\\[0.3em]&\implies {}&f(x)-f({\tilde {x}})&=f'({\tilde {x}})\cdot (x-{\tilde {x}})+\epsilon (x)\cdot (x-{\tilde {x}})\\[0.3em]&\implies {}&f(x)&=f({\tilde {x}})+f'({\tilde {x}})\cdot (x-{\tilde {x}})+\underbrace {\epsilon (x)\cdot (x-{\tilde {x}})} _{:=\ \delta (x)}\\[0.3em]&\implies {}&f(x)&=f({\tilde {x}})+f'({\tilde {x}})\cdot (x-{\tilde {x}})+\delta (x)\end{aligned}}}$

The error between ${\displaystyle f(x)}$ and ${\displaystyle f({\tilde {x}})+f'({\tilde {x}})\cdot (x-{\tilde {x}})}$ is thus equal to the term ${\displaystyle \delta (x)=\epsilon (x)\cdot (x-{\tilde {x}})}$. Because of ${\displaystyle \lim _{x\to {\tilde {x}}}\epsilon (x)=0}$ there is

${\displaystyle \lim _{x\to {\tilde {x}}}\delta (x)=\lim _{x\to {\tilde {x}}}\underbrace {\epsilon (x)} _{\to 0}\cdot \underbrace {(x-{\tilde {x}})} _{\to 0}=0}$

So the error ${\displaystyle \delta (x)}$ disappears for ${\displaystyle x\to {\tilde {x}}}$. But we can say even more: ${\displaystyle \delta (x)}$ decreases faster than a linear term towards zero. Even if we divide ${\displaystyle \delta (x)}$ by ${\displaystyle x-{\tilde {x}}}$ and thus greatly increase this term near ${\displaystyle {\tilde {x}}}$, then ${\displaystyle {\tfrac {\delta (x)}{x-{\tilde {x}}}}}$ disappears for ${\displaystyle x\to {\tilde {x}}}$. There is

${\displaystyle \lim _{x\to {\tilde {x}}}{\frac {\delta (x)}{x-{\tilde {x}}}}=\lim _{x\to {\tilde {x}}}{\frac {\epsilon (x)(x-{\tilde {x}})}{x-{\tilde {x}}}}=\lim _{x\to {\tilde {x}}}\epsilon (x)=0}$

The error ${\displaystyle \delta (x)}$ in the approximation ${\displaystyle f(x)\approx f({\tilde {x}})+f'({\tilde {x}})\cdot (x-{\tilde {x}})}$ thus falls off to zero faster than linear for ${\displaystyle x\to {\tilde {x}}}$. Let us summarize the previous argumentation in one theorem:

The fact that differentiable functions can be approximated by linear functions characterises the derivative. Every function ${\displaystyle f}$ is differentiable at the position ${\displaystyle {\tilde {x}}}$, if a real number ${\displaystyle c\in \mathbb {R} }$ (best approximation parameter) as well as a function ${\displaystyle \delta }$ exist, such that that ${\displaystyle f(x)=f({\tilde {x}})+c\cdot (x-{\tilde {x}})+\delta (x)}$ and ${\displaystyle \lim _{x\to {\tilde {x}}}{\tfrac {\delta (x)}{x-{\tilde {x}}}}=0}$ apply. Its derivative is then ${\displaystyle f'({\tilde {x}})=c}$. There is

${\displaystyle {\begin{aligned}f'({\tilde {x}})&=\lim _{x\to {\tilde {x}}}{\frac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}\\[0.3em]&=\lim _{x\to {\tilde {x}}}{\frac {f({\tilde {x}})+c\cdot (x-{\tilde {x}})+\delta (x)-f({\tilde {x}})}{x-{\tilde {x}}}}\\[0.3em]&=\lim _{x\to {\tilde {x}}}{\frac {c\cdot (x-{\tilde {x}})+\delta (x)}{x-{\tilde {x}}}}\\[0.3em]&=\lim _{x\to {\tilde {x}}}c+\underbrace {\frac {\delta (x)}{x-{\tilde {x}}}} _{\to 0}\\[0.3em]&=c\end{aligned}}}$

So we can also define the derivative as follows:

There is a further characterisation of derivative. We start with the formula

${\displaystyle {\frac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}=f'({\tilde {x}})+\epsilon (x)}$

Where ${\displaystyle \epsilon (x)}$ is the difference between the difference quotient and the derivative (which disappears for ${\displaystyle x\to {\tilde {x}}}$). If we rearrange this formula we get:

${\displaystyle {\begin{aligned}&&{\frac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}&=f'({\tilde {x}})+\epsilon (x)\\[0.3em]&\implies {}&f(x)-f({\tilde {x}})&=f'({\tilde {x}})\cdot (x-{\tilde {x}})+\epsilon (x)\cdot (x-{\tilde {x}})\\[0.3em]&\implies {}&f(x)&=f({\tilde {x}})+(f'({\tilde {x}})+\epsilon (x))\cdot (x-{\tilde {x}})\\[0.3em]&&&\ {\color {Gray}\left\downarrow \ \varphi (x)=f'({\tilde {x}})+\epsilon (x)\right.}\\[0.3em]&\implies {}&f(x)&=f({\tilde {x}})+\varphi (x)\cdot (x-{\tilde {x}})\end{aligned}}}$

The function ${\displaystyle \varphi (x)}$ for ${\displaystyle x\to {\tilde {x}}}$ has the property

${\displaystyle \lim _{x\to {\tilde {x}}}\varphi (x)=\lim _{x\to {\tilde {x}}}(f'({\tilde {x}})+\underbrace {\epsilon (x)} _{\to 0})=f'({\tilde {x}})}$

Thus ${\displaystyle \varphi (x)}$ can be extended to a function which is continuous at the position ${\displaystyle {\tilde {x}}}$, whereby the function value is set ${\displaystyle \varphi ({\tilde {x}})=f'({\tilde {x}})}$. This representation of a differentiable function allows a further characterisation of continuous functions:

The slope is initially only defined for linear functions ${\displaystyle g}$ with the assignment rule ${\displaystyle g(x)=mx+b}$ where ${\displaystyle m,b\in \mathbb {R} }$. For such functions the slope is equal to the value ${\displaystyle m}$ and can be calculated using the difference quotient. For two different arguments ${\displaystyle x}$ and ${\displaystyle {\tilde {x}}}$ from the domain of definition ${\displaystyle g}$ there is:

${\displaystyle {\begin{aligned}{\frac {g(x)-g({\tilde {x}})}{x-{\tilde {x}}}}&={\frac {(mx+b)-(m{\tilde {x}}+b)}{x-{\tilde {x}}}}\\[0.3em]&={\frac {mx-m{\tilde {x}}}{x-{\tilde {x}}}}\\[0.3em]&={\frac {m(x-{\tilde {x}})}{x-{\tilde {x}}}}\\[0.3em]&=m\end{aligned}}}$

Now ${\displaystyle m}$ is also the derivative of ${\displaystyle g}$ at every accumulation point ${\displaystyle {\tilde {x}}}$ of the domain of definition:

${\displaystyle {\begin{aligned}g'({\tilde {x}})&=\lim _{x\to {\tilde {x}}}{\frac {g(x)-g({\tilde {x}})}{x-{\tilde {x}}}}\\[0.3em]&{\color {OliveGreen}\left\downarrow \ {\text{see calculation above}}\right.}\\[0.3em]&=\lim _{x\to {\tilde {x}}}m=m\end{aligned}}}$

The derivative of a linear function is therefore always equal to its slope. But the derivative is more general: it is defined for all differentiable functions. (Remember: A term ${\displaystyle A}$ is a generalisation of another term ${\displaystyle B}$, if ${\displaystyle A}$ is the same as ${\displaystyle B}$ in all cases where ${\displaystyle B}$ is defined and ${\displaystyle A}$ can be applied to other cases.)

So we can consider the derivative as the slope of a function at a point. The transition slope ${\displaystyle \rightarrow }$ derivative thus changes from a global property (the slope for linear functions is defined for the whole function), to a local property (the derivative is the instantaneous rate of change of a function).

The derivative of a function ${\displaystyle f:D\to \mathbb {R} }$ is the limit of the difference quotient ${\displaystyle {\tfrac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}}$ for ${\displaystyle x\to {\tilde {x}}}$. The difference quotient can be understood as a function ${\displaystyle D\setminus \{{\tilde {x}}\}\to \mathbb {R} }$, which is defined for all ${\displaystyle x\in D}$ except for ${\displaystyle x={\tilde {x}}}$. So ${\displaystyle \lim _{x\to {\tilde {x}}}{\tfrac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}}$ is actually the limit value of a function.

The terms "Left-hand and right-hand derivative" can also be considered for the difference quotient. Thus we obtain the terms "left-hand" and "right-hand" derivative. For the left-hand derivative, only secants to the left of the considered point are evaluated. So only difference quotients ${\displaystyle {\tfrac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}}$ are considered, where ${\displaystyle xis<{\tilde {x}}}$. Then it is checked whether the difference quotient converges to a number in the limit ${\displaystyle x\to {\tilde {x}}}$ converge against a number. If the answer is yes, then this number is the left-hand derivative at that point:

${\displaystyle {f_{-}}'({\tilde {x}})=\lim _{x\uparrow {\tilde {x}}}{\frac {f(x)-f({\tilde {x}})}{x-{\tilde {x}}}}}$

Here ${\displaystyle {f_{-}}'({\tilde {x}})}$ is the notation for the left-hand derivative of ${\displaystyle f}$ at the position ${\displaystyle {\tilde {x}}}$. For this limit to make sense, there must be at least one sequence ${\displaystyle (x_{n})_{n\in \mathbb {N} }}$ of arguments that converges from the left towards ${\displaystyle {\tilde {x}}}$. So ${\displaystyle {\tilde {x}}}$ has to be an accumulation point of the set ${\displaystyle D\cap (-\infty ,{\tilde {x}})=\{x\in D:x<{\tilde {x}}\}}$.

Analogously, the right-hand derivative can be defined as follows:

functions only have a limit value at one position in their domain of definition if both the left-hand and the right-hand limit value exist at this position and both limit values match. We can apply this theorem directly to derivative functions:

We have already shown that the absolute value function ${\displaystyle f:\mathbb {R} \to \mathbb {R} :x\mapsto |x|}$ is not differentiable at ${\displaystyle {\tilde {x}}=0}$. However, we can still show that the right-hand derivative exists at this position and is equal to ${\displaystyle 1}$:

${\displaystyle {\begin{aligned}{f_{+}}'(0)&=\lim _{x\downarrow 0}{\frac {f(x)-f(0)}{x-0}}=\lim _{x\downarrow 0}{\frac {|x|-|0|}{x}}\\[0.3em]&\ {\color {OliveGreen}\left\downarrow \ x>0\implies |x|=x\right.}\\[0.3em]&=\lim _{x\downarrow 0}{\frac {x-0}{x}}=\lim _{x\downarrow 0}1=1\end{aligned}}}$

Analogously, we can show that the left-hand derivative is equal to ${\displaystyle -1}$ at this position:

${\displaystyle {\begin{aligned}{f_{-}}'(0)&=\lim _{x\uparrow 0}{\frac {f(x)-f(0)}{x-0}}=\lim _{x\uparrow 0}{\frac {|x|-|0|}{x}}\\[0.3em]&\ {\color {OliveGreen}\left\downarrow \ x<0\implies |x|=-x\right.}\\[0.3em]&=\lim _{x\uparrow 0}{\frac {-x-0}{x}}=\lim _{x\uparrow 0}-1=-1\end{aligned}}}$

Since the right-hand and left-hand derivatives do not coincide, the absolute value function cannot be differentiated at ${\displaystyle {\tilde {x}}=0}$. At this point, it has left-hand and right-hand derivatives, but no general derivative.

Weil die rechtsseitige and die linksseitige derivative nicht übereinstimmen, ist die Betragsfunktion an der Stelle ${\displaystyle {\tilde {x}}=0}$ nicht ableitbar. Sie besitzt dort zwar links- and rechtsseitige derivativeen, aber keine derivative.

In the above example we have seen that the absolute value function is not differentiable. This is because the absolute value function "has a kink" at the position ${\displaystyle \xi =0}$, so that the left-hand and right-hand derivative are different. If we go to ${\displaystyle \xi =0}$ from the left-hand side, the derivative is equal to ${\displaystyle -1}$, while the derivative from the right-hand side is equal to ${\displaystyle 1}$. The kink in the absolute value function thus prevents differentiability.

So if a function has a kink, it is not differentiable at this point. In other words: differentiable functions are kink-free. Therefore they are also called smooth functions (actually, smooth means "infinitely many times differentiable"). This does not mean, however, that kink-free functions are automatically differentiable. As an example, let us consider the sign function ${\displaystyle \operatorname {sgn}(x)}$ with the definition

${\displaystyle \operatorname {sgn}(x)={\begin{cases}1&x>0\\0&x=0\\-1&x<0\end{cases}}}$

Its graph is

This function is not differentiable at the zero point ${\displaystyle {\tilde {x}}=0}$, because near the the "jump" of the function, the difference quotient converges towards infinity. For the right-hand derivative there is for example:

${\displaystyle \lim _{h\downarrow 0}{\frac {\operatorname {sgn}(h)-\operatorname {sgn}(0)}{h}}=\lim _{h\downarrow 0}{\frac {1-0}{h}}=\lim _{h\downarrow 0}{\frac {1}{h}}=\infty }$

The sign function has no kink at the zero point. Instead, it makes a "jump" there.

At the example of the sign function we see that being "free of kinks" and "differentiable" cannot be the same. However, freedom from kinks is a prerequisite for differentiability. So differentiable functions are free of kinks.

Continuous differentiability of a function ${\displaystyle f}$ implies its differentiability, which in turn implies its continuity. The converse statements do not hold, as we will see in the course of this section:

${\displaystyle {\begin{aligned}&{\text{continuously differentiability}}\\\implies {}&{\text{differentiability}}\\\implies {}&{\text{continuity}}\end{aligned}}}$

The first implication follows directly from the definition: A function ${\displaystyle f}$ is called continuously differentiable if it is differentiable and the derivative function ${\displaystyle f'}$ is continuous. Thus, continuously differentiable functions are also differentiable. The second implication needs some more work:

We now show that every at one point differentiable function is also continuous at this point. Thus, differentiability is a stronger condition for a function than continuity:

From the previous section we know that every differentiable function is continuous:

${\displaystyle {\text{differentiability}}\implies {\text{continuity}}}$

Applying the principle of contraposition to this implication, we also get:

${\displaystyle {\text{non-continuity}}\implies {\text{non-differentiability}}}$

Take, as an for example the sign function

${\displaystyle \operatorname {sgn} :\mathbb {R} \to \mathbb {R} :x\mapsto f(x)={\begin{cases}1&x>0\\0&x=0\\-1&x<0\end{cases}}}$

It is not continuous at ${\displaystyle {\tilde {x}}=0}$. So it is also not differentiable there. We can prove non-continuity by taking a sequence ${\displaystyle x_{n}={\tfrac {1}{n}}}$. This sequence converges towards zero. If the sign function was differentiable, then the limit value ${\displaystyle \lim _{n\rightarrow \infty }{\tfrac {f(x_{n})-f(0)}{x_{n}-0}}}$ would have to exist. However

${\displaystyle \lim _{n\rightarrow \infty }{\frac {f\left({\frac {1}{n}}\right)-f(0)}{{\frac {1}{n}}-0}}=\lim _{n\rightarrow \infty }{\frac {1-0}{{\frac {1}{n}}-0}}=\lim _{n\rightarrow \infty }{n}=\infty }$

The limit value does not exist in ${\displaystyle \mathbb {R} }$. Therefore the sign function is - as expected - not differentiable at ${\displaystyle {\tilde {x}}=0}$.

In the following example, we already use some derivatives rules, which will be discussed in more detail in the next chapter. Perhaps you already know them from school. If not, they are a useful insight to what will follow.