\documentstyle [11pt] {article}
%
\input defs
\parindent 0in
\topmargin=-0.60in
\hoffset=-0.75in
\textheight=8.75in \vsize\textheight
\textwidth=6.5in \hsize\textwidth
%
\def\Lhat{{\hat L}}
\def\uhat{{\hat u}}
\def\ehat{{\hat e}}
\def\fhat{{\hat f}}
\def\tauhat{{\hat \tau}}
\def\uj{u\ind{}{j}}
\def\ujp{u\ind{}{j+1}}
\def\ujph{u\ind{}{\jph}}
\def\ujm{u\ind{}{j-1}}
\def\ubar{\bar u}

\def\be{\begin{equation}}
\def\ee{\end{equation}}
\def\bea{\begin{eqnarray}}
\def\eea{\end{eqnarray}}
\def\bean{\begin{eqnarray*}}
\def\eean{\end{eqnarray*}}
%
\begin{document}
%BEGIN LARGE
{\Large
%
\begin{center}
{\bf
  PHY329~~~~Finite Difference Methods~~~~Oct 14, 1998 \\
  Expected Behaviour of Finite-Difference Approximations \\
  (Convergence and Accuracy)
}
\end{center}
%

\vspace*{0.5in}
We have already seen that the basic ``control parameter'' of a typical
FDA (finite-difference approximation) is the mesh-spacing, $h$.
Fundamentally, for sensibly constructed FDA's, we expect the error 
in the approximation to go to 0 as $h$  goes to 0.

\vspace*{0.5in}
\begin{center}
{\bf Notation and Definitions} 
\end{center}

Let\\
\be
	L \, u = f
\ee
\\
denote a general {\em differential\/} system.  For simplicity and 
concreteness, you can think of $u = u(x)$ as a single function of one 
variable, but the following discussion also applies to cases in 
multiple independent variables ($u(x,t), u(x,y,t), \,\cdots$ etc.),
as well as multiple {\em dependent\/} variables ($u = {\bf u} = 
[ u_1, u_2, \, \cdots, u_n]$).

~\\ 
~\\
In (1), $L$ is some differential operator ($L = d^2/dx^2$ in the BVP
example we've studied previously), and $f$ is some specified function.
We will generically denote an FDA of (1) by\\[0.05in]
\be
	\Lhat \uhat = \fhat \nonumber
\ee
\\[0.05in]
where $\uhat$ is the discrete solution, $\fhat$ is the specified 
function evaluated on the finite-difference mesh, and $\Lhat$ is 
the finite-difference approximation to $L$.

\newpage
{\bf Truncation Error:} The truncation error, $\tauhat$, of an FDA is 
defined by
\\
\be
	\tauhat \equiv \Lhat u - \fhat
\ee
\\
where $u$ satisfies the continuum system~(1).  We note that the {\em form\/}
of the truncation error can always be computed (typically using Taylor series 
expansions) from the finite difference approximation and the 
differential equations---we will see an example shortly.

~\\
~\\
{\bf Order of a Finite-Difference Approximation:} Assuming that the FDA
is characterized by a {\em single\/} discretization scale, $h$, we 
say that the FDA is {\em $p$-th order accurate} or simply 
{\em $p$-th order} if 
\\
\be
	\lim_{h \to 0} \tauhat = O(h^p) \quad\quad
   \hbox{\rm for some integer~} p
\ee

~\\
~\\
{\bf Solution Error:} The solution error, $\ehat$, associated with an 
FDA is defined by
\\
\be
	\ehat \equiv u - \uhat
\ee

~\\
\begin{center}
{\bf Relation Between Truncation Error and Solution Error} 
\end{center}

~\\
Is is common to tacitly assume that 
\\
\[
	\tauhat = O(h^p) \quad\quad\hbox{\rm implies}\quad\quad \ehat = O(h^p)
\]
\\
This assumption is often warranted but it is extremely instructive 
to consider {\em why\/} it is warranted and to investigate in some 
detail the {\em nature\/} of the solution error, $\ehat$, for a simple 
yet representative example.

\newpage
\begin{center}
{\bf Example} 
\end{center}

~\\
Consider one of the simplest possible differential systems:
\\[0.10in]
\be
	L \, u(x) \equiv \lb \fr{d}{dx} - 1 \rb \, u(x) = 0
	\quad \hbox{\rm~on~~} 0 \le x \le 1 \quad \hbox{\rm~with~~} u(0) = 1
\ee
\\[0.10in]
Clearly,  the unique solution of this DE is 
\\[0.05in]
\[
	u(x) = e^x.
\]
\\[0.10in]
We introduce a uniform finite-difference mesh, $x_j$,
\\[0.10in]
\[
	x_j = j h, \quad j = 0, 1, \, \cdots \, n-1 
   \quad \quad \hbox{\rm where~~}h = \fr{1}{n-1}
\]
\\[0.10in]
and consider the following difference {\em operators:}
\\[0.10in]
\be
\dop x+ \uj \equiv h^{-1} \lb \ujp - \uj \rb                 
\ee
\\
\be
\dav x+ \uj \equiv \half \lb \ujp + \uj \rb                 
\ee
\\[0.10in]
Now consider Taylor series expansions of $\uj$ and $\ujp$ about 
\\[0.10in]
\[
\ujph \equiv u(x\ind{}{\jph}) \equiv 
u\lb\half\lb x\ind{}j + x\ind{}\jpo \rb\rb \equiv \ubar
\]
\\

\newpage
We have
\\[0.10in]
\bean
\ujp & = & \ubar + \lb \half h \rb \ubar' + \half \lb \half h \rb^2 \ubar''
 + \fr{1}{6} \lb \half h \rb^3 \ubar''' 
 + \fr{1}{24} \lb \half h \rb^4 \ubar'''' + O(h^5) \\[0.20in]
\uj  & = & \ubar - \lb \half h \rb \ubar' + \half \lb \half h \rb^2 \ubar''
 - \fr{1}{6} \lb \half h \rb^3 \ubar''' 
 + \fr{1}{24} \lb \half h \rb^4 \ubar'''' + O(h^5) 
\eean
\\[0.10in]
so 
\\[0.10in]
\bea
\dop x+ \uj & \equiv & h^{-1} \lb \ujp - \uj \rb =
	\ubar' + \fr{1}{24}h^2\ubar''' + O(h^4)   \\[0.20in]
\dav x+ \uj & \equiv & \half \lb \ujp + \uj \rb =
   \ubar + \fr{1}{8}h^2\ubar'' + O(h^4)       
\eea
\\[0.10in]
Thus, we can identify the {\em difference operators}, $\dop x+$ and 
$\dav x+$, with (formal) power series (in $h$) of {\em differential 
operators}:
\\[0.10in]
\bea
\dop x+ & \equiv & \fr{d}{dx} + \fr{1}{24}h^2\fr{d^3}{dx^3} + O(h^4) \\[0.20in]
\dav x+ & \equiv & 1 + \fr{1}{8}h^2\fr{d^2}{dx^2} + O(h^4)           
\eea

\newpage
We now write down our FDA of (6):
\\[0.10in]
\be
\lb \fr{d}{dx} - 1 \rb \, u(x) = 0 \quad\longrightarrow\quad
	\lb \dop x+ - \dav x+ \rb \, \uj = 0 
\ee
\\[0.10in]
(i.e. $\Lhat = \dop x+ - \dav x+$). Explicitly, we have
\\[0.10in]
\be
\fr{\ujp - \uj}{h} = \fr{\ujp + \uj}{2} \quad\quad
 j = 0, 1, \, \cdots \, n -2 \quad\quad \hbox{\rm with~~} u_0 = u(0) = 1
\ee
\\[0.20in]
Let us first consider the {\em truncation error\/}  associated with 
(13,14):
\\[0.10in]
\bean
	\tauhat  \equiv \Lhat u - \fhat = \Lhat u 
      & = & \lb \dop x+ - \dav x+ \rb \, u \\[0.20in]
		& = & \lb \fr{d}{dx} + \fr{1}{24}h^2\fr{d^3}{dx^3} -  
              1 - \fr{1}{8}h^2\fr{d^2}{dx^2} + O(h^4) \rb \, \ubar \\[0.20in]
	 	& = & \lb \fr{d}{dx} - 1 \rb \, \ubar + 
          h^2 \lb \fr{1}{24} \fr{d^3}{dx^3} - \fr{1}{8}\fr{d^2}{dx^2} \rb \,
           \ubar + O(h^4) = O(h^2) 
\eean
\\[0.10in]
where we have used the original differential equation (6)  in the 
form  
\\[0.20in]
\[
\lb \fr{d}{dx} - 1 \rb \, \ubar = 0.
\]

\newpage
Thus, since $\tauhat = O(h^2)$, our FDA is {\em second order}. Note, however,
that we had to be careful about choosing the $x$ value about which 
we performed our Taylor series expansions---in general we must choose 
the expansion point which gives the {\em highest-order\/} truncation
error or, intuitively, the point about which the scheme is naturally 
``centred''.\\[0.20in]

The FDA~(14) is sufficiently simple (and linear) that we can solve 
it explicitly:
\\[0.10in]
\[
\fr{\ujp - \uj}{h}  = \fr{\ujp + \uj}{2} \quad\longrightarrow\quad
	\ujp  =  \lb \fr{h^{-1}+\half}{h^{-1}-\half}\rb \, \uj 
    = \lb \fr{1+\fr{h}{2}}{1-\fr{h}{2}} \rb \, \uj \equiv \rho \, \uj
\]
\\[0.10in]
where
\\[0.10in]
\[
   \rho \equiv \fr{1+\fr{h}{2}}{1-\fr{h}{2}}
\]
\\[0.10in]
Thus we have
\\[0.10in]
\be
u_j = \rho^j \, u_0 = \rho^j
\ee
\\[0.10in]
where $\rho^j$ denotes the $j$-th power of $\rho$ and we have used the 
initial condition $u_0 = 1$. Now, we can write $\rho^j$ as 
follows:
\\[0.10in]
\[
	\rho^j = \exp\lb j \ln \rho \rb = 
				\exp \lb j \lb \ln \lb 1 + \fr{h}{2} \rb -          
                           \ln \lb 1 - \fr{h}{2} \rb \rb \rb.
\]

\newpage
Then, using the Taylor series expansion 
\\[0.10in]
\[
	\ln(1 + x) = x - \fr{1}{2}x^2 + \fr{1}{3}x^3 + O(x^4)
\]
\\[0.10in]
we have
\\[0.10in]
\[
\ln \lb 1 + \fr{h}{2} \rb - \ln \lb 1 - \fr{h}{2} \rb =
	h + \fr{1}{12}h^3 + O(h^5)
\]
so that 
\\[0.10in]
\bean
	\rho^j = \exp\lb jh + \lb jh \rb \fr{1}{12}h^2 + O(h^4) \rb & = &
   \exp\lb jh \rb \exp \lb jh \fr{h^2}{12} + O(h^4) \rb  \\[0.20in]
   & = &
   e^{x_j}\lb 1 + \fr{h^2}{12}\,x_j + O(h^4) \rb \\[0.20in]
	& = & e^{x_j} + \fr{1}{12}h^2\,{x_j}\,e^{x_j} + O(h^4)
\eean
\\[0.10in]
Thus, the {\em solution error}, $\ehat$ is given by
\\[0.10in]
\be
\ehat = u - \uhat = -\fr{1}{12}h^2 x e^x + O(h^4) = O(h^2)
\ee
\\[0.10in]
Note in particular, the {\em form\/} of the solution error; to leading
order it is 
\\[0.10in]
\[
h^2 \times \,\, -\fr{1}{12}xe^x = -h^2 \times e_2(x)
\]
\\[0.10in]
where $e_2(x)$ is a {\em function} ($\fr{1}{12}xe^x$) with smoothness 
(i.e. magnitude of derivatives) comparable to the continuum solution,
$u(x) = e^x$. 

\newpage
There is another way to derive~(16) which clearly illustrates the 
fundamental lesson concerning the errors in finite difference 
approximations.\\[0.20in]

Let us {\em assume\/} that $u$ and $\uhat$ are related by the 
asymptotic ($h\to0$) expansion,
\\[0.10in]
\be
	\uhat = u + h^2\,e_2 + h^4\,e_4 + \, \cdots
\ee
\\[0.10in]
which we call a {\em Richardson expansion} (after L.F.~Richardson, who
studied such matters in the early part of this century).\\[0.20in]

We start with the FDA
\\[0.10in]
\[
\Lhat \, \uhat = 0 \quad\longrightarrow\quad
	\lb \dop x+ - \dav x+ \rb \, \uhat = 0
\]
\\[0.10in]
and replace both $\Lhat$ and $\uhat$ by their ``continuum expansions''
(11,12,17):
\\[0.10in]
\[
\lb \fr{d}{dx} - 1 + h^2 \lb \fr{1}{24}\fr{d^3}{dx^3} - 
		\fr{1}{8}\fr{d^2}{dx^2}\rb + O(h^4)\rb  
\lb u + h^2 \, e_2 + O(h^4) \rb = 0
\]
\\[0.10in]
We now demand that this equation vanish, {\em order by order,} in $h$.
For the $O(1)$ and $O(h^2)$ cases we have 
\\[0.10in]
\bean
	O(1):~&~&~\lb\fr{d}{dx} - 1\rb \, u = 0 \quad\longrightarrow\quad
	u = e^x \quad\quad\hbox{\rm ({\em consistency\/} of FDA)} \\[0.20in]
	O(h^2):~&~&~\lb\fr{d}{dx} - 1\rb \,e_2 = 
		\lb\fr{1}{8}\fr{d^2}{dx^2} - \fr{1}{24}\fr{d^3}{dx^3}\rb \, u 
\eean

\newpage
Thus, we see that the error function, $e_2$, {\em itself\/} satisfies a 
differential equation which is very similar in form to the original 
DE.  Moreover, given that we know $u(x) = e^x$, we can solve this
DE for $e_2$.  Specifically, we have 
\\[0.10in]
\be
	\lb \fr{d}{dx} - 1 \rb \, e_2 = \fr{1}{12}\,e^x \quad\longrightarrow\quad
	e_2 = \fr{1}{12} x e^x 
\ee
\\[0.10in]
and since our original {\em assumption}~(17) was 
\\[0.10in]
\[
	\uhat = u + h^2\,e_2 + O(h^4) = u + \fr{1}{12}h^2 x e^x + O(h^4)
\]
\\[0.10in]
we get 
\\[0.10in]
\[
	\ehat \equiv u - \uhat = -\fr{1}{12}h^2 x e^x + O(h^4)
\]
\\[0.10in]
as previously.

\newpage
\begin{center}
{\bf Comments}
\end{center}

Even in cases where we {\em don't\/} know the continuum solution (this 
clearly includes most of the interesting instances where we are likely to 
apply finite-difference techniques), it is still very useful to 
think of {\em difference operators\/} and {\em difference solutions\/}
in terms of asymptotic expansions in powers of the mesh spacing, $h$:
\\[0.10in]
\bean
\Lhat & = & L + h^2\,L_2 + h^4\,L_4 + O(h^6) \\[0.20in]
\uhat & = & u + h^2\,e_2 + h^4\,e_4 + O(h^6) 
\eean
\\[0.10in]
so that
\\[0.10in]
\[
\Lhat \, \uhat = \fhat 
\]
\\[0.10in]
actually represents an (infinite) hierarchy of differential systems 
\\[0.10in]
\bean
L \, u & = & f \\[0.20in]
L \, e_2 & = & -L_2 \, u  \\
&\cdot& \\
&\cdot& \\
&\cdot& 
\eean
\\[0.10in]
In other words, in FDA's of continuum systems, the error is not 
``random''---rather, in principle, it is no less computable than the 
fundamental solution itself.

%END LARGE
} 
\end{document}
