Derivatives and Differences.

We saw in class that that the “right difference approximation”

f’(x) ≃ r(h) = ((f(x+h) - f(x))/h)

should have error proportional to h, while the “central difference approximation”

f’(x) ≃ c(h) = ((f(x+h) - f(x-h))/(2h))

should have error proportional to (h^2). 

Let’s see this in practice in some examples:

LeftDifference[h_,f_,x_] := ((f[x+h] - f[x])/h);
CentralDifference[h_,f_,x_] := ((f[x+h] - f[x-h])/(2h));

Example Function.

We’ll start with the function (Sqrt[x]) at x = 4.

D[(Sqrt[x]),x] /. x-> 4

Exact arithmetic.

We’re first going to evaluate in exact arithmetic so as to remove the effect of roundoff error from the computation. 

Neat trick. If you want to see a power law, evaluate at values of h whose logarithms are equally spaced, and make a log-log plot. If you have a function 

error ≃ b (h^n)

then 

log (error) ≃ log b + n log h

and the data will fit to a line with slope n.

Here’s our data table:

LeftData = Table[
{(10^(-n)),Abs[LeftDifference[(10^(-n)),Sqrt,4] - 1/4]},
{n,1,16,1}];

CentralData =
Block[{$MaxExtraPrecision=100},
Table[
{(10^(-n)),Abs[CentralDifference[(10^(-n)),Sqrt,4] - 1/4]},
{n,1,16,1}]];

Here’s a log-log plot:

ListLogLogPlot[
Reverse /@ {LeftData,CentralData},
PlotLegends->{"Left Difference","Central Difference"},
PlotMarkers->{Graphics[{PointSize[Large],Point[{0,0}]}]}
]

So now let’s fit these to lines:

LinearModelFit[N[Log10[LeftData],16],n,n]["Function"][n]

LinearModelFit[N[Log10[CentralData],16],n,n]["Function"][n]

Again, this is pretty impressive when you think about it! Our Taylor series argument correctly predicted the error behavior of these two approximations to the derivative.

Once more, with roundoff.

We now repeat exactly the same computations, but insist that Mathematica does them in machine arithmetic. We’re going to get more data, if only because the results are so incredibly weird.

LeftMachineData = Table[
{(10.0^(-n)),Abs[LeftDifference[(10.0^(-n)),Sqrt,4.0] - 0.25]},
{n,1.0,16.0,0.1}];

CentralMachineData =
Table[
{(10.0^(-n)),Abs[CentralDifference[(10.0^(-n)),Sqrt,4.0] - 0.25]},
{n,1.0,16.0,0.1}];

ListLogLogPlot[{LeftMachineData,CentralMachineData},
PlotLegends->{"Left Difference","Central Difference"},
PlotMarkers->{Graphics[{PointSize[Large],Point[{0,0}]}]}]

This is so surprising that we’d like to see the two plots together!

ListLogLogPlot[{LeftMachineData,LeftData, CentralMachineData,CentralData},
PlotLegends->{"Left Difference (Machine)","Left Difference (Exact)","Central Difference (Machine)","Central Difference (Exact)"},
PlotMarkers->{Graphics[{PointSize[Large],Point[{0,0}]}]}]

Richardson Extrapolation.

Now that we’ve seen the basic two approximations, it’s irresistible not to try to make the same plots with Richardson extrapolation. The first Richardson formula, which should have error O((h^4)) is:

ϕ[h_,f_,x_] := ((f[x+h] - f[x-h])/(2h));
Φ[h_,f_,x_] := ϕ[(h/2),f,x] + (1/3)(ϕ[(h/2),f,x] - ϕ[h,f,x]);

RichardsonData = Block[{$MaxExtraPrecision=100},
Table[
{(10^(-n)),Abs[Φ[(10^(-n)),Sqrt,4] - 1/4]},
{n,1,16,1}]
];

RichardsonMachineData =
Table[
{(10.0^(-n)),Abs[Φ[(10.0^(-n)),Sqrt,4.0] - 0.25]},
{n,1.0,16.0,0.1}];

ListLogLogPlot[
Reverse /@ (Select[#,#[[1]] > (10^(-10))&]& /@ {LeftMachineData,LeftData,
CentralMachineData,CentralData,
RichardsonMachineData,RichardsonData}),
PlotLegends->{"Left Difference (Machine)","Left Difference (Exact)","Central Difference (Machine)","Central Difference (Exact)",
"Richardson Extrapolation (Machine)", "Richardson Extrapolation (Exact)"},PlotMarkers->{Graphics[{PointSize[Large],Point[{0,0}]}]}]