<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
* with significant contributions from:
Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
<HTML>
<HEAD>
<TITLE>Estimation</TITLE>
<META NAME="description" CONTENT="Estimation">
<META NAME="keywords" CONTENT="vol2">
<META NAME="resource-type" CONTENT="document">
<META NAME="distribution" CONTENT="global">
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<LINK REL="STYLESHEET" HREF="vol2.css">
<LINK REL="previous" HREF="node13.html">
<LINK REL="up" HREF="node11.html">
<LINK REL="next" HREF="node15.html">
</HEAD>
<BODY >
<!--Navigation Panel-->
<A NAME="tex2html1627"
HREF="node15.html">
<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
SRC="icons.gif/next_motif.gif"></A>
<A NAME="tex2html1624"
HREF="node11.html">
<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
SRC="icons.gif/up_motif.gif"></A>
<A NAME="tex2html1620"
HREF="node13.html">
<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
SRC="icons.gif/previous_motif.gif"></A>
<A NAME="tex2html1626"
HREF="node1.html">
<IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents"
SRC="icons.gif/contents_motif.gif"></A>
<BR>
<B> Next:</B> <A NAME="tex2html1628"
HREF="node15.html">Raw to Calibrated Data</A>
<B> Up:</B> <A NAME="tex2html1625"
HREF="node11.html">Basic Concepts</A>
<B> Previous:</B> <A NAME="tex2html1621"
HREF="node13.html">Noise distributions</A>
<BR>
<BR>
<!--End of Navigation Panel-->
<H2><A NAME="SECTION00513000000000000000"> </A>
<A NAME="estim"> </A>
<BR>
Estimation
</H2>
A number of different statistical methods are used for estimating parameters
from a data set.
The most commonly used one is the least squares method which estimates a
parameter <IMG
WIDTH="17" HEIGHT="22" ALIGN="BOTTOM" BORDER="0"
SRC="img38.gif"
ALT="$\theta$">
by minimizing the function :
<BR><P></P>
<DIV ALIGN="CENTER">
<!-- MATH: \begin{equation}
S(\theta) = \sum_i ( y_i - f(\theta;x_i) )^2
\end{equation} -->
<TABLE WIDTH="100%" ALIGN="CENTER">
<TR VALIGN="MIDDLE"><TD ALIGN="CENTER" NOWRAP><A NAME="eq:least-sqr"> </A><IMG
WIDTH="239" HEIGHT="61"
SRC="img39.gif"
ALT="\begin{displaymath}S(\theta) = \sum_i ( y_i - f(\theta;x_i) )^2
\end{displaymath}"></TD>
<TD WIDTH=10 ALIGN="RIGHT">
(2.6)</TD></TR>
</TABLE>
</DIV>
<BR CLEAR="ALL"><P></P>
where <I>y</I> is the dependent and <I>x</I> the independent variables while
<I>f</I> is a given function.
Equation <A HREF="node14.html#eq:least-sqr">2.6</A> can be expanded to more parameters if needed.
For linear functions <I>f</I> an analytic solution can be derived whereas
an iteration scheme must be applied for most non-linear cases.
Several conditions must be fulfilled for the method to give a reliable
estimate of <IMG
WIDTH="17" HEIGHT="22" ALIGN="BOTTOM" BORDER="0"
SRC="img40.gif"
ALT="$\theta$">.
The most important assumptions are that the errors in the dependent variable
are normal distributed, the variance is homogeneous, and the independent
variables have no errors and are uncorrelated.
<P>
The other main technique for parameter estimation is the maximum likelihood
method where the joint probability of the parameter <IMG
WIDTH="16" HEIGHT="22" ALIGN="BOTTOM" BORDER="0"
SRC="img41.gif"
ALT="$\theta$">
:
<BR><P></P>
<DIV ALIGN="CENTER">
<!-- MATH: \begin{equation}
l(\theta) = \prod_i P(\theta,x_i)
\end{equation} -->
<TABLE WIDTH="100%" ALIGN="CENTER">
<TR VALIGN="MIDDLE"><TD ALIGN="CENTER" NOWRAP><A NAME="eq:max-like"> </A><IMG
WIDTH="163" HEIGHT="61"
SRC="img42.gif"
ALT="\begin{displaymath}l(\theta) = \prod_i P(\theta,x_i)
\end{displaymath}"></TD>
<TD WIDTH=10 ALIGN="RIGHT">
(2.7)</TD></TR>
</TABLE>
</DIV>
<BR CLEAR="ALL"><P></P>
is maximized.
In Equation <A HREF="node14.html#eq:max-like">2.7</A>, <I>P</I> denotes the probability density of
the individual data sets.
Normally, the logarithm likelihood
<!-- MATH: $L = \log(l)$ -->
<IMG
WIDTH="101" HEIGHT="44" ALIGN="MIDDLE" BORDER="0"
SRC="img43.gif"
ALT="$L = \log(l)$">
is used to simplify
the maximization procedure.
This method can be used for any given distribution.
For a normal distribution the two methods will give the same result.
<P>
<HR>
<!--Navigation Panel-->
<A NAME="tex2html1627"
HREF="node15.html">
<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
SRC="icons.gif/next_motif.gif"></A>
<A NAME="tex2html1624"
HREF="node11.html">
<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
SRC="icons.gif/up_motif.gif"></A>
<A NAME="tex2html1620"
HREF="node13.html">
<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
SRC="icons.gif/previous_motif.gif"></A>
<A NAME="tex2html1626"
HREF="node1.html">
<IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents"
SRC="icons.gif/contents_motif.gif"></A>
<BR>
<B> Next:</B> <A NAME="tex2html1628"
HREF="node15.html">Raw to Calibrated Data</A>
<B> Up:</B> <A NAME="tex2html1625"
HREF="node11.html">Basic Concepts</A>
<B> Previous:</B> <A NAME="tex2html1621"
HREF="node13.html">Noise distributions</A>
<!--End of Navigation Panel-->
<ADDRESS>
<I>Petra Nass</I>
<BR><I>1999-06-15</I>
</ADDRESS>
</BODY>
</HTML>
