{VERSION 4 0 "IBM INTEL NT" "4.0" }
{USTYLETAB {CSTYLE "Maple Input" -1 0 "Courier" 0 1 255 0 0 1 0 1 0 0 
1 0 0 0 0 1 }{CSTYLE "" -1 256 "Helvetica" 1 14 128 0 0 1 0 0 2 0 0 0 
0 0 0 1 }{CSTYLE "" -1 257 "" 0 24 0 0 255 1 0 0 0 0 0 0 0 0 0 1 }
{CSTYLE "" -1 258 "Helvetica" 0 1 128 0 0 1 0 0 0 0 0 0 0 0 0 1 }
{CSTYLE "" -1 259 "" 1 14 0 0 0 0 0 1 0 0 0 0 0 0 0 0 }{CSTYLE "" -1 
260 "" 1 14 0 0 0 0 0 1 0 0 0 0 0 0 0 0 }{CSTYLE "" -1 261 "" 1 14 0 
0 0 0 0 1 0 0 0 0 0 0 0 0 }{CSTYLE "" -1 262 "" 1 14 0 0 0 0 0 1 0 0 
0 0 0 0 0 0 }{CSTYLE "" -1 263 "" 1 14 0 0 0 0 0 1 0 0 0 0 0 0 0 0 }
{CSTYLE "" -1 264 "" 1 14 0 0 0 0 0 1 0 0 0 0 0 0 0 0 }{CSTYLE "" -1 
265 "" 1 14 0 0 0 0 0 1 0 0 0 0 0 0 0 0 }{CSTYLE "" -1 266 "" 1 14 0 
0 0 0 0 1 0 0 0 0 0 0 0 0 }{CSTYLE "" -1 267 "" 1 14 0 0 0 0 0 1 0 0 
0 0 0 0 0 0 }{PSTYLE "Normal" -1 0 1 {CSTYLE "" -1 -1 "Times" 1 12 0 
0 0 1 2 2 2 2 2 2 1 1 1 1 }1 1 0 0 0 0 1 0 1 0 2 2 0 1 }{PSTYLE "Title
" -1 256 1 {CSTYLE "" -1 -1 "Times" 1 18 0 0 0 1 2 1 1 2 2 2 1 1 1 1 }
1 1 0 0 12 12 1 0 1 0 2 2 19 1 }}
{SECT 0 {EXCHG {PARA 256 "" 0 "" {TEXT 257 23 "Method of Least Squares
" }}{PARA 0 "" 0 "" {TEXT 256 27 "Mth 351 Oct 10 2001 Maple 6" }}
{PARA 0 "" 0 "" {TEXT 258 16 "Bent E. Petersen" }{TEXT -1 0 "" }}
{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 161 "We can s
olve linear least squares problems by using either of Maple's linear a
lgebra packages, but the most convenient approach is to use the stats[
fit] package." }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "> " 
0 "" {MPLTEXT 1 0 8 "restart;" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 
0 61 "with(stats[fit,describe]):with(stats[statplots]):with(plots):" }
}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 38 "
Suppose we have some experimental data" }}{PARA 0 "" 0 "" {TEXT -1 0 "
" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 36 "Xdata:=[1,2,3,4,5,6,7,8
,9,10,11,12];" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 33 "Ydata:=[3,
2,2,3,1,4,5,6,7,5,6,8];" }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }}
{PARA 0 "" 0 "" {TEXT -1 210 "and we suspect a functional relationship
 of the form  y = f(x)  where  f  contains some adjustable parameters.
 The Maple leastsquare function accepts directly the equation that we \+
are trying to fit to our data." }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}
{PARA 0 "" 0 "" {TEXT -1 96 "Here's a little procedure to compute the \+
L2 norm of a vector (or 1-array) of arbitrary dimension" }}{PARA 0 "" 
0 "" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 11 "N2:=pr
oc(a)" }}{PARA 0 "> " 0 "" {MPLTEXT 1 0 10 "  local k;" }}{PARA 0 "> \+
" 0 "" {MPLTEXT 1 0 31 "sqrt(sum(a[k]^2,k=1..nops(a)));" }}{PARA 0 "> \+
" 0 "" {MPLTEXT 1 0 4 "end:" }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }
}{PARA 0 "" 0 "" {TEXT -1 74 "Alternately we could use the norm functi
on from the linear algebra package" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}
}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 35 "# N2:=v->linalg[norm](v,fro
benius);" }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" 
{TEXT -1 61 "It will also be handy to have a plot of the given data po
ints" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 53 "plt0:=scatterplot(Xdata,Ydata,color=red,thickness=3):
" }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT 259 
21 "Constant function fit" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "
" 0 "" {TEXT -1 28 "Consider a constant function" }}{PARA 0 "" 0 "" 
{TEXT -1 0 "" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 9 "f1:=x->c;" }
}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 52 "eqn1:=leastsquare[[x,y],y=
f1(x),\{c\}]([Xdata,Ydata]);" }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 73 "T
hus or our optimal functional relationship of the given type is given \+
by" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 25 "F1:=unapply(rhs(eqn1),x);" }}}{EXCHG {PARA 0 "" 0 "" 
{TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 55 "The stats[describe] packa
ge defines the function mean()" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}
{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 16 "ym:=mean(Ydata);" }}}{EXCHG 
{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 71 "We see as
 expected the best least squares fit is the mean of the Ydata." }}
{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 93 "The devia
tion or total residual is the square root of the sum of the squares of
 the residuals" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "> \+
" 0 "" {MPLTEXT 1 0 39 "ND1:=N2(Ydata-map(F1,Xdata)); evalf(%);" }}}
{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 33 "plt1:=plot(F1,1..12,color=bl
ack):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 52 "display(plt0,plt1,
title = \"Estimation by the mean\");" }}}{EXCHG {PARA 0 "" 0 "" {TEXT 
-1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 201 "The mean is a pretty rough est
imator. However it is useful to consider since the quantity  ND1  is u
sed to scale the total residual in other cases, to obtain a dimensionl
ess measure of goodness of fit." }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 
"" }}{PARA 0 "" 0 "" {TEXT 260 26 "Linear Function (Line) Fit" }}
{PARA 0 "" 0 "" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 
0 13 "f2:=x->m*x+b;" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 54 "eqn2
:=leastsquare[[x,y],y=f2(x),\{m,b\}]([Xdata,Ydata]);" }}}{EXCHG {PARA 
0 "> " 0 "" {MPLTEXT 1 0 25 "F2:=unapply(rhs(eqn2),x);" }}}{EXCHG 
{PARA 0 "> " 0 "" {MPLTEXT 1 0 39 "ND2:=N2(Ydata-map(F2,Xdata)); evalf
(%);" }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT 
-1 69 "The dimensionless (unadjusted) measure of goodness of fit is gi
ven by" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 29 "gf2:=1-(ND2/ND1)^2; evalf(%);" }}}{EXCHG {PARA 0 "" 
0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 74 "Close to  1  is good
, close to  0  is bad. Our result here is noncommital." }}}{EXCHG 
{PARA 0 "> " 0 "" {MPLTEXT 1 0 33 "plt2:=plot(F2,1..12,color=black):" 
}}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 54 "display([plt0,plt2],title
 = \"Least squares line fit\");" }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 
"" }}{PARA 0 "" 0 "" {TEXT 261 16 "Special Line Fit" }}{PARA 0 "" 0 "
" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 207 "The   leastsqu
are function is quite versatile. For example, if we have reason to exp
ect the slope should be 1/2 in the example above then we can restrict \+
our least squares candidates to lines with slope 1/2." }}{PARA 0 "" 0 
"" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 17 "f3:=x->(
1/2)*x+b;" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 52 "eqn3:=leastsqu
are[[x,y],y=f3(x),\{b\}]([Xdata,Ydata]);" }}}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 25 "F3:=unapply(rhs(eqn3),x);" }}}{EXCHG {PARA 0 "> " 0 "
" {MPLTEXT 1 0 39 "ND3:=N2(Ydata-map(F3,Xdata)); evalf(%);" }}}{EXCHG 
{PARA 0 "> " 0 "" {MPLTEXT 1 0 29 "gf3:=1-(ND3/ND1)^2; evalf(%);" }}}
{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 96 "As
 expected, the fit is not as good as before, since we have restricted \+
the class of candidates." }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}{EXCHG 
{PARA 0 "> " 0 "" {MPLTEXT 1 0 33 "plt3:=plot(F3,1..12,color=black):" 
}}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 46 "display([plt3,plt0],title
=\"Special line fit\");" }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }}
{PARA 0 "" 0 "" {TEXT 262 13 "Quadratic Fit" }}{PARA 0 "" 0 "" {TEXT 
-1 0 "" }}}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 19 "f4:=x->a*x^2+b*x+c;" }}}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 56 "eqn4:=leastsquare[[x,y],y=f4(x),\{a,b,c\}]([Xdata,Yda
ta]);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 25 "F4:=unapply(rhs(eq
n4),x);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 39 "ND4:=N2(Ydata-ma
p(F4,Xdata)); evalf(%);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 29 "
gf4:=1-(ND4/ND1)^2; evalf(%);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 
0 33 "plt4:=plot(F4,1..12,color=black):" }}}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 43 "display([plt4,plt0],title=\"Quadratic fit\");" }}}
{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT 263 9 "Cu
bic Fit" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 25 "f5:=x->a*x^3+b*x^2+c*x+d;" }}}{EXCHG {PARA 0 "> " 0 "
" {MPLTEXT 1 0 58 "eqn5:=leastsquare[[x,y],y=f5(x),\{a,b,c,d\}]([Xdata
,Ydata]);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 25 "F5:=unapply(rh
s(eqn5),x):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 39 "ND5:=N2(Ydat
a-map(F5,Xdata)); evalf(%);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 
29 "gf5:=1-(ND5/ND1)^2; evalf(%);" }}}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 33 "plt5:=plot(F5,1..12,color=black):" }}}{EXCHG {PARA 0 
"> " 0 "" {MPLTEXT 1 0 39 "display([plt5,plt0],title=\"Cubic fit\");" 
}}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT 264 
12 "Degree 4 fit" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}{EXCHG {PARA 0 ">
 " 0 "" {MPLTEXT 1 0 31 "f6:=x->a*x^4+b*x^3+c*x^2+d*x+e;" }}}{EXCHG 
{PARA 0 "> " 0 "" {MPLTEXT 1 0 60 "eqn6:=leastsquare[[x,y],y=f6(x),\{a
,b,c,d,e\}]([Xdata,Ydata]);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 
25 "F6:=unapply(rhs(eqn6),x):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 
0 39 "ND6:=N2(Ydata-map(F6,Xdata)); evalf(%);" }}}{EXCHG {PARA 0 "> " 
0 "" {MPLTEXT 1 0 29 "gf6:=1-(ND6/ND1)^2; evalf(%);" }}}{EXCHG {PARA 
0 "> " 0 "" {MPLTEXT 1 0 33 "plt6:=plot(F6,1..12,color=black):" }}}
{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 42 "display([plt6,plt0],title=\"
Degree 4 fit\");" }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "
" 0 "" {TEXT 265 12 "Degree 5 Fit" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}
{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 37 "f7:=x->a*x^5+b*x^4+c*x^3+d*x
^2+e*x+f;" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 62 "eqn7:=leastsqu
are[[x,y],y=f7(x),\{a,b,c,d,e,f\}]([Xdata,Ydata]);" }}}{EXCHG {PARA 0 
"> " 0 "" {MPLTEXT 1 0 25 "F7:=unapply(rhs(eqn7),x):" }}}{EXCHG {PARA 
0 "> " 0 "" {MPLTEXT 1 0 39 "ND7:=N2(Ydata-map(F7,Xdata)); evalf(%);" 
}}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 29 "gf7:=1-(ND7/ND1)^2; evalf
(%);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 33 "plt7:=plot(F7,1..12
,color=black):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 42 "display([
plt7,plt0],title=\"Degree 5 fit\");" }}}{EXCHG {PARA 0 "" 0 "" {TEXT 
-1 0 "" }}{PARA 0 "" 0 "" {TEXT 266 13 "Degree 10 Fit" }}{PARA 0 "" 0 
"" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 81 "f12:=x->
a10*x^10+a9*x^9+a8*x^8+a7*x^7+a6*x^6+a5*x^5+a4*x^4+a3*x^3+a2*x^2+a1*x+
a0;" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 86 "eqn12:=leastsquare[[
x,y],y=f12(x),\{a10,a9,a8,a7,a6,a5,a4,a3,a2,a1,a0\}]([Xdata,Ydata]);" 
}}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 27 "F12:=unapply(rhs(eqn12),x
):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 41 "ND12:=N2(Ydata-map(F1
2,Xdata)); evalf(%);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 31 "gf1
2:=1-(ND12/ND1)^2; evalf(%);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 
0 35 "plt12:=plot(F12,1..12,color=black):" }}}{EXCHG {PARA 0 "> " 0 "
" {MPLTEXT 1 0 44 "display([plt12,plt0],title=\"Degree 10 fit\");" }}}
{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 0 "" }}}{EXCHG {PARA 0 "" 0 "" 
{TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 534 "This example is typical.
 Even though our goodness of fit is about 0.988 the graph intuitively \+
looks worse. The fit is of course great near the data points, but poly
nomials of high degree tend to oscillate a lot and therefore we get la
rge (unjustified?) swings between the data points. The philosophy, mor
e or less, is a large goodness of fit is meaningless unless it is achi
eved by a polynomial of low degree, or more generally, by a function w
ith few adjustable parameters. Is it meaningful in the later case? Wel
l, maybe, maybe not." }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}{EXCHG {PARA 
0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT 267 18 "An Exponential
 Fit" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" {TEXT -1 73 "M
aple can handle any kind of linear least squares fit (but not nonlinea
r)" }}{PARA 0 "" 0 "" {TEXT -1 0 "" }}}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 22 "f20:=x->a+b*exp(x/15);" }}}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 56 "eqn20:=leastsquare[[x,y],y=f20(x),\{a,b\}]([Xdata,Yda
ta]):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 27 "F20:=unapply(rhs(e
qn20),x):" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 41 "ND20:=N2(Ydata
-map(F20,Xdata)): evalf(%);" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 
31 "gf20:=1-(ND20/ND1)^2: evalf(%);" }}}{EXCHG {PARA 0 "> " 0 "" 
{MPLTEXT 1 0 35 "plt20:=plot(F20,1..12,color=black):" }}}{EXCHG {PARA 
0 "> " 0 "" {MPLTEXT 1 0 49 "display([plt20,plt0],title=\"An exponenti
al fit\");" }}}{EXCHG {PARA 0 "" 0 "" {TEXT -1 0 "" }}{PARA 0 "" 0 "" 
{TEXT -1 161 "Would you care to use any of these models to predict the
 \"future\" (larger values of  x)? What is required to make prediciton
 anything but a meaningless exercise?" }}{PARA 0 "" 0 "" {TEXT -1 0 "
" }}}{EXCHG {PARA 0 "> " 0 "" {MPLTEXT 1 0 0 "" }}}}{MARK "97" 0 }
{VIEWOPTS 1 1 0 1 1 1803 1 1 1 1 }{PAGENUMBERS 0 1 2 33 1 1 }