calculus.tex

\documentclass{article}
\usepackage[left=3cm,right=3cm,top=0cm,bottom=2cm]{geometry}
\usepackage{amsmath}
\usepackage{mathtools}

\begin{document}

\title{What I think about when I think about calculus}
\author{Keith Goodman}
\maketitle

\subsection*{Introduction}

I haven't done calculus in decades. Well, that's no longer true because I
recently took a single-variable calculus course. The 13-week online class was
taught by Robert Ghrist. He's the calculus teacher you wish you had.

This document contains some of the ideas that we covered in class and some of
the ideas that occured to me while taking the class.

\subsection*{Finding partial sums using descrete calculus}

After helping my daughter with arithmetic and geometric sequences and series in
Algebra 2 and Precalculus I didn't think I could get excited about finding
partial sums. Yet that's exactly what happened when we covered the subject in
class.

Our motivation in this section is to simplify the partial sum

\[
    S(k)=\sum\limits_{n=0}^k n^3-4n^2.
\]

\noindent Notice that the series is neither arithmetic nor geometric---finally
something new.

The partial sum is analogous to the definite integral $I(k)$ in continuous
calculus:

\[
I(k)=\int\limits_{n=0}^k f(n)dn=\int\limits_{n=0}^k\frac{dF(n)}{dn}dn=F(k)-F(0)
\]

\noindent where $dn$ corresponds to $\Delta n=1$ in the discrete case and
$F(n)$ is an antiderivative of $f(n)$.

We will proceed in an analogous fashion for the discrete case:

\begin{align*}
    S(k) &= \sum\limits_{n=0}^k a_n = \sum\limits_{n=0}^k (\Delta b)_n \\
         &= (b_1 - b_0)+(b_2 - b_1)+ \ldots + (b_k - b_{k-1}) + (b_{k+1} - b_k) \\
         &= b_{k+1} - b_0
\end{align*}

\noindent where $(\Delta b)_n = b_{n+1} - b_n$, the forward difference, is the
discrete derivative and $b$ is a discrete antiderivative of $a$.

The problem of simplifying $S(k)$ has now been reduced to finding a discrete
antiderivative of $n^3-4n^2$. In other words we need to find a sequence $b_n$
such that $(\Delta b)_n = n^3-4n^2$. It is easy to find a recursion relation:
$b_{n+1} = n^3-3n^2 + b_n$. However, to fully simplify $S(k)$, we need an
analytic solution for $b$, which we will obtain by using falling powers.

Falling powers are defined as

\[
    n^{\underline{k}} = n(n-1)(n-2)\ldots(n-k+1)
\]

\noindent where $k$ is an integer and, by a second definition,
$n^{\underline{0}}=1$. For example

\[
    n^{\underline{3}} = n(n-1)(n-2) = n^3-3n^2+2n.
\]

\noindent Solving for $n^3$ gives $n^3 = n^{\underline{3}} + 3n^2-2n$.
Similarly $n^2 = n^{\underline{2}} + n$. After a little bit of algebra we get

\[
(\Delta b)_n=n^3-4n^2=n^{\underline{3}}-n^{\underline{2}}-3n^{\underline{1}}.
\]

The whole point of using falling powers is to (gently) coerce the problem into
a form solvable by the familiar mechanics of continuous calculus.  After three
lines of algebra, the discrete derivative of $n^{\underline{k}}$ is given by
the discrete power rule: $\Delta n^{\underline{k}}= kn^{\underline{k-1}}$:

\begin{align*}
\Delta n^{\underline{k}} &= (n+1)^{\underline{k}} - n^{\underline{k}} \\
             &= (n+1)n^{\underline{k-1}} - n^{\underline{k-1}}(n-k+1) \\
             &= kn^{\underline{k-1}}
\end{align*}

\noindent Therefore $b$ is quite simply given by

\[
b_n=\frac{1}{4}n^{\underline{4}}-\frac{1}{3}n^{\underline{3}}-\frac{3}{2}n^{\underline{2}}+C
\]

\noindent where $C$ is a constant. (To see that $b_n$ is correct we would
differentiate it to obtain
$n^{\underline{3}}-n^{\underline{2}}-3n^{\underline{1}}.$)

Putting it all together:

\begin{align*}
    S(k) &= \sum\limits_{n=0}^k n^3-4n^2 \\
         &= \sum\limits_{n=0}^k \Delta (\frac{1}{4}n^{\underline{4}}-\frac{1}{3}n^{\underline{3}}-\frac{3}{2}n^{\underline{2}}+C) \\
         &= \frac{1}{4}(k+1)^{\underline{4}}-\frac{1}{3}(k+1)^{\underline{3}}-\frac{3}{2}(k+1)^{\underline{2}} \\
         &= \frac{1}{12}k(k+1)(3k^2-13k-8)
\end{align*}

\noindent where, if we wish to use the last line, $k > 2$ lest we run into
troubles such as $3^{\underline{4}}=0$. For $2 \geq k \geq 0$, the partial sum
$S(k)$ is easily calculated by hand.

Let's try a simpler example:

\[
    \sum\limits_{n=0}^k n^2 \\
    =\sum\limits_{n=0}^k n^{\underline{2}}+n^{\underline{1}} \\
    =\sum\limits_{n=0}^k \Delta \left(\frac{n^{\underline{3}}}{3}+\frac{n^{\underline{2}}}{2}\right) \\
    =\frac{(k+1)^{\underline{3}}}{3}+\frac{(k+1)^{\underline{2}}}{2} \\
    =\frac{k(k+1)(2k+1)}{6}
\]

\noindent where, if we wish to use the last part, $k > 1$.

Without much effort we adapted widely known results from continuous calculus to
find partial sums of discrete series. The techniques in this section allow us
to find partial sums of sequences $a_n$ that can be written as polynomials in
$n$ and some that can be written as rational functions of $n$.


\subsection*{Taylor series of composite functions}

Let's say the bottleneck of a computer program is the calculation of $\cos(\sin
x)$ and $\sin(\cos x)$. And that we are only interested  in values of $x$ near
zero. Our job is to speed things up. We decide to replace both composite
functions with the first few terms of their Taylor series. There are two ways
we could proceed: (1) a messy brute-force calculation of derivatives; or (2)
making a composition from Taylor series we already know, followed by messy
algebra. We opt for the second choice.

The first few terms of the Taylor series of $\cos x$ and $\sin x$ centered at
$x=0$ are given by

\begin{align*}
    \cos x &= 1 - \frac{x^2}{2} + \frac{x^4}{24} + O(x^6) \\
    \sin x &= x - \frac{x^3}{6} + O(x^5)
\end{align*}

\noindent Plugging the Taylor series of $\sin x$ into that of $\cos x$ and
keeping terms up to order $x^4$ gives:

\begin{align*}
    \cos(\sin x) &= 1 - \frac{\sin^2 x}{2} + \frac{\sin^4 x}{24} + O(x^6) \\
                 &= 1 - \frac{1}{2}(x^2 - \frac{x^4}{3} + O(x^6)) + \frac{x^4}{24} + O(x^6) \\
                 &= 1 - \frac{x^2}{2} + \frac{5x^4}{24} + O(x^6)
\end{align*}

\noindent Similarly, plugging the Taylor series of $\cos x$ into that of $\sin
x$ and keeping terms up to order $x^4$ gives:

\begin{align*}
    \sin(\cos x) &= \cos x - \frac{\cos^3 x}{6} + O(x^5) \\
                 &= 1 - \frac{x^2}{4} + \frac{x^4}{24} + O(x^6) - \frac{1}{6}(1-\frac{3x^2}{2}+\frac{7x^4}{8}+O(x^6)) + O(x^5)\\
                 &= \frac{5}{6} - \frac{x^2}{4} - \frac{5x^4}{48} + O(x^5)
\end{align*}

Plugging these series into the program makes it run 100 times faster! The only
problem is that the program now returns values that are way off even though the
input values of $x$ are near zero. Further checking reveals that the expansion
of $\cos(\sin x)$ is accurate but the expansion of $\sin(\cos x)$ is way off.

What went wrong?

We expanded both $\cos x$ and $\sin x$ about $x=0$. In the case of $\cos(\sin
x)$, $\sin x$ is near zero when $x$ is near zero. Therefore it was appropriate
to expand $\cos x$ about zero as well.  However in the case of $\sin(\cos x)$,
$\cos x$ is near 1 when $x$ is near zero. So for this case we should have
expanded $\sin x$ about $x=1$.

Being aware of this issue before setting out to solve the problem would have
been nice (and is the whole point of this section), but it shouldn't be
necessary. When we expanded $\sin(\cos x)$ we got terms that contain $(1 -
x^2+O(x^4))^{2n+1}$, where $n$ is a non-negative integer. No matter how large a
value of $n$ we choose, the leading order term (for small $x$) will be
$-(2n+1)x^2$. Therefore to find the coefficent of the $x^2$ term we would
have needed to evaluate an infinite series.

The problem goes away if we exapand $\sin x$ about $x=1$:

\[
    \sin(\cos x)= \sin(1)-\frac{\cos(1)}{2}x^2+\frac{\cos(1)- 3\sin(1)}{24}x^4+O(x^6)
\]



\subsection*{Rising powers}

Although we studied falling powers in class it never occured to me that there
were rising powers. After seeing rising powers mentioned on the class forum, a
web search lead to me to the following unanswered question on Mathematics Stack
Exchange:

(1) Show that
$$
n^{\underline{p}}=(-1)^p(-n)^{\overline{p}}
$$
\indent (2) Evaluate the sum
$$
\sum\limits_{a\leq n < b} n^{\overline{p}}
$$

What better way to learn about rising powers than to answer the question?
Let's begin by considering the case where $p$ is even:

\begin{align*}
    n^{\underline{p}}&=n(n-1)(n-2)\cdots(n-p+1) && \text{Definition of falling powers}\\
&=(-n)(-n+1)(-n+2)\cdots(-n+p-1) && \text{Multiply each term by } -1\\
    &=(-n)^{\overline{p}}  && \text{Definition of rising powers}
\end{align*}

\noindent In the case where $p$ is odd, multiplying each term by $-1$ changes
the sign of the product, therefore $n^{\underline{p}}=-(-n)^{\overline{p}}$. In
general

$$
n^{\underline{p}}=(-1)^p(-n)^{\overline{p}}
$$

\noindent Equivalently, substituting $m=-n$ and rearranging gives

$$
m^{\overline{p}}=(-1)^p(-m)^{\underline{p}}
$$

Let's now find the partial sum:

\begin{align*}
\sum\limits_{n=a}^{b-1} n^{\overline{p}} &= \sum\limits_{n=a}^{b-1}(-1)^p(-n)^{\underline{p}} \\
&=\sum\limits_{n=a}^{b-1}\Delta\left[-\frac{(-1)^p}{p+1}(-n)^{\underline{p+1}}\right]\\
&=\left.\frac{(-1)^{p+1}}{p+1}(-n)^{\underline{p+1}}\right|_{n=a}^b \\
&= \frac{(-1)^{p+1}}{p+1}\left[(-b)^{\underline{p+1}}-(-a)^{\underline{p+1}}\right] \\
&= \frac{(-1)^{p+1}}{p+1}\left[(-1)^{p+1}b^{\overline{p+1}}-(-1)^{p+1}a^{\overline{p+1}}\right] \\
&= \frac{b^{\overline{p+1}}-a^{\overline{p+1}}}{p+1} \\
\end{align*}

\noindent where I used the discrete version of the Fundamental Theorem of
Integral Calculus, the discrete derivative, and $(\Delta a)_n=a_{n+1}-a_n$ is
the forward difference. These results were derived in the section on Partial
Sums.

An alternative method to calculate the sum could have started with

\begin{align*}
    n^{\overline{p}}&=n(n+1)(n+2)\cdots(n+p-1) && \text{Definition of rising powers}\\
    &=(n+p-1)^{\underline{p}}  && \text{Definition of falling powers}
\end{align*}



\subsection*{Using Taylor series to find derivatives}

We often find the Taylor series of a function $f(x)$ about $x=a$ by a
brute-force calculation of derivatives $f^{(n)}(a)$. If, however, we already
know the Taylor series of $f(x)$ about $x=a$ then we can use it to easily find
the derivative $f^{(n)}(a)$. For example what is the second derivative of $\sin
x^2$ evaluated at $x=0$?

In class we memorized the Taylor series of $\sin x$ about $x=0$:

$$
\sin x = x - \frac{x^2}{2} + \frac{x^3}{6} + O(x^5).
$$

\noindent Therefore $\sin x^2 = x^2 + O(x^4)$. The term containing $x^2$ in any
Taylor series about $x=0$ is $f''(0)x^2/2$. Setting this equal to the $x^2$
term in the Taylor series of $\sin x^2$ gives

\begin{align*}
    \frac{f''(0)x^2}{2} &= x^2 \\
    f''(0) &=  2
\end{align*}

\noindent Similarly we would easily find that the sixth derivative of
$e^{-x^2}$ evaluated at $x=0$ was $-120$.

\end{document}