<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
* with significant contributions from:
Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
<HTML>
<HEAD>
<TITLE>Reduction procedure</TITLE>
<META NAME="description" CONTENT="Reduction procedure">
<META NAME="keywords" CONTENT="vol2">
<META NAME="resource-type" CONTENT="document">
<META NAME="distribution" CONTENT="global">
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<LINK REL="STYLESHEET" HREF="vol2.css">
<LINK REL="next" HREF="node286.html">
<LINK REL="previous" HREF="node280.html">
<LINK REL="up" HREF="node271.html">
<LINK REL="next" HREF="node286.html">
</HEAD>
<BODY >
<!--Navigation Panel-->
<A NAME="tex2html4995"
HREF="node286.html">
<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
SRC="icons.gif/next_motif.gif"></A>
<A NAME="tex2html4992"
HREF="node271.html">
<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
SRC="icons.gif/up_motif.gif"></A>
<A NAME="tex2html4986"
HREF="node284.html">
<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
SRC="icons.gif/previous_motif.gif"></A>
<A NAME="tex2html4994"
HREF="node1.html">
<IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents"
SRC="icons.gif/contents_motif.gif"></A>
<BR>
<B> Next:</B> <A NAME="tex2html4996"
HREF="node286.html">Final results</A>
<B> Up:</B> <A NAME="tex2html4993"
HREF="node271.html">Reducing the observations</A>
<B> Previous:</B> <A NAME="tex2html4987"
HREF="node284.html">Standard stars</A>
<BR>
<BR>
<!--End of Navigation Panel-->
<H2><A NAME="SECTION001856000000000000000"> </A>
<A NAME="reductionproc"> </A>
<BR>
Reduction procedure
</H2>
<P>
After subtracting dark current and sky brightness from your stellar data,
and asking you to select a gradient estimator for each band,
the reduction program converts intensities to magnitudes.
At this stage, it warns you of any observations that seem to have zero or
negative intensities (which obviously cannot be converted to magnitudes).
These are often an indication that you have confused sky and star readings.
The program then estimates starting values for the full solution.
<P>
The program treats four different categories of stars differently:
standard stars, extinction stars, program stars, and variable stars.
It asks for the category of each file of stars when the positions are
read in.
<I>Standard</I> stars are those having known standard values, which are used
only to determine transformation coefficients.
They may also be used to determine extinction.
Be cautious about using published catalog values as standards; these often
have systematic errors that will propagate errors into everything else.
<I>Extinction</I> stars are constant stars, observed over an appreciable range of
airmass, whose standard magnitudes and colors are unknown, or too poorly
known to serve as standards.
Ordinary stars taken from catalogs of photometric data are best used as
extinction stars; later, you can compare their derived standard values with the
published ones as a rough check.
<I>Program</I> stars may be re-classified as either extinction or variable stars
during the course of the extinction solution.
If they are not used as extinction stars, they are not included in the
solutions, but are treated as variable stars at the end.
<I>Variable</I> stars are excluded from the extinction and transformation
solutions.
Their individual observations will be corrected for extinction and
transformation after the necessary parameters have been evaluated.
<P>
You will do best to maintain separate star files for each category; however,
you can intermix extinction and variable stars in a ``program star'' file, and
PEPSYS will do the best it can to sort them out, if you ask it to use program
stars for extinction (see below).
Remember that only star files need to be separated this way; a data file
normally has all the observations for a given night, regardless of the type of
object.
<P>
You can also group related files in a MIDAS ``catalog'' file.
Use the MIDAS command <TT>CREATE/TCAT</TT>
to refer to several similar *.tbl files as a catalog file.
Note that (a) you must enter the ``.cat'' suffix explicitly when giving PEPSYS a
catalog; and (b) a catalog <I>must</I> contain only tables of the same kind --
e.g., only standard stars, or only program stars.
Catalogs can be used for both star files and data files.
<P>
When all the star files have been read, the program asks for the name of the
table file that describes the instrument.
As usual, you can omit the ``.tbl'' suffix and it will be supplied.
<P>
Then the program asks for the data files.
Remember to add the ``.cat'' suffix if you use a catalog.
While reading data, it may ask you for help in supplying cross-identifications
if a star name in the data did not occur in a star table.
If all your data files have been read, but it is still asking for the name of a
data file, reply <TT>NONE</TT> and it will go on.
(This same trick works for star files too.)
<P>
The program
will display a plot of airmass vs. time for the standard and extinction
stars on each night, so you can judge whether it makes sense to try to solve for
time-dependent extinction later on.
If the airmass range is small, it will warn you that you may not be able to
determine extinction.
<P>
If the data are well distributed, and there are numerous standard stars,
it will obtain starting values of extinction coefficients from the standard
values; otherwise, it assumes reasonable extinction coefficients and estimates
starting values of the magnitudes from them.
(These starting values are simple linear fits, somewhat in the style of SNOPY,
except that robust lines are used instead of least squares.)
From the preliminary values of magnitudes, it tries to determine transformation
coefficients, if standards are available.
<P>
This whole process is iterated a few times to obtain a self-consistent set of
starting values for all the parameters, except for bandwidth factors.
Each time the program loops back to refine the starting values, it adds a line
like ``<TT>BEGIN CYCLE n</TT>'' to the log.
Don't confuse these iterations, which are just to get good starting values,
with the iterations performed later in the full solution.
<P>
One problem in this preliminary estimation is that faint stars with a lot of
photon noise might just add noise to the extinction coefficients.
The program tries to determine where stars become too faint and noisy to be
useful for estimating extinction.
The rough extinction and transformation
coefficients will be displayed as they are determined; if any fall outside a
reasonable range of values, the program will tell you and give you a chance to
try more reasonable values.
<P>
When reasonable starting values have been found for the parameters that will be
estimated in the full solution, the program still needs to decide how to do
the solution.
Should the standard values of the standard stars be used in determining
extinction (which requires estimating transformation coefficients
simultaneously), or should extinction be determined from the observations
alone, and the transformations found afterward?
The program will ask your advice.
<P>
If you have designated no extinction stars,
the reduction program will ask you whether you want to treat program stars as
extinction stars.
If you believe most of them are constant in light, you can try using all of
them as extinction stars.
If some turn out to be variable, you will have a chance to label them as such
later on.
If many of the program stars are faint, they may be too noisy to contribute
anything useful to the extinction solution; then you could leave them out and
speed up the solution.
On the other hand, values obtained from multiple observations in the general
solution will be a little more accurate than values obtained by averaging
individual data at the end.
<P>
When the program is ready to begin iterating, it will ask how much output you
want.
When you first use PEPSYS, you may find it useful to choose option 2 (display
of iteration number and variance).
After you get used to how long a given amount of data is likely to take to
reduce, you can just choose option 1 (no information about iterations).
The detailed output from the other options is quite long, and should
only be requested if the iterations are not converging properly and you want
to look at the full details.
<P>
If you ask for iteration output, you will always see the value of the weighted
sum of squares of the residuals (called WVAR in the output), and the value of
the typical residual for an observation of unit weight (called SCALE), which
should be near 0.01 magnitude.
The weights are chosen so that the error corresponding to unit weight is
intended to be 0.01 mag.
When SCALE changes, there can be considerable changes in WVAR -- in
particular, don't be alarmed if it occasionally increases.
A flag called MODE is also displayed, which is reset to zero when SCALE is
adjusted.
During normal iterations, MODE = 4; ordinarily, most of the iterations are
in mode 4, with occasional reversions to mode 1 when SCALE is adjusted.
More detailed output contains the values of all the parameters at each
iteration, and other internal values used in
the solution; these are mainly useful for debugging, and can be ignored by the
average user.
<P>
Typically 20 or 30 iterations are needed for convergence; if the data are
poorly distributed, so that some parameters are highly correlated,
more iterations will be needed.
If convergence is not reached in 99 iterations, the program will give up and
check the values of the parameters (see next paragraph).
This usually indicates that you are trying to determine parameters that are not
well constrained by the data.
It will also check the values of the partial derivatives used in the problem;
if there is an error here larger than a few parts in 10<SUP>6</SUP>, you have found a
bug in the program.
<P>
At the end of a solution, the program prints the typical error, and then
examines the parameters obtained.
If extinction or transformation coefficients or bandwidth parameters seem
<I>very</I> unreasonable, the program will simply fix them at reasonable values
and try again.
If they seem marginally unreasonable, the program will ask for your advice.
<P>
When reasonable values are obtained for all the parameters, the program will
check the errors in the transformation equations, if standard values have been
used in the extinction solution.
Then, if necessary, it will readjust the weights assigned to the standard-star
data, and repeat the solution.
Usually 3 or 4 such cycles are required to achieve complete convergence.
Thus, including the transformation parameters
in the extinction solution means the program
may take longer to reach full convergence.
<P>
Having reached a solution in which the weights are consistent with the
residuals, the program examines the stellar data again.
If you have ``program'' stars that might be used to strengthen the extinction
solution, it will ask if you want to use all, some, or none of them for
extinction.
If you reply SOME, it will go through the list of stars one by one and offer
those that look promising; you can then accept or reject each individual star
as an extinction star.
Only stars whose observations cover a significant interval of time and/or
airmass will be offered as extinction candidates.
<P>
In examining the individual stars, the program may find some that show signs of
variability.
For those that have several observations, a ``light-curve'' of residuals will
be displayed.
Pay close attention to the numbers on the vertical scale of this plot!
Each star's residuals are scaled to fill the screen.
If there are only a few data for a star, only the calculated RMS variation is
shown.
<P>
A star may show more variation than expected, but if this is under 0.01
magnitude, it may still be a useful extinction star -- indeed, the star may
not really be variable, but may have had its apparent variation enhanced by one
or two anomalous data points.
Another problem that can occur, if you have only a few extinction stars, or
only one or two nights of data, is a variation in the extinction with time.
This can produce a drift in the residuals of a standard or extinction star
that may look like some kind of slow variation.
Watch out for repeated clumps of anomalously dim measurements that occur about
the same time for one star after another; this often indicates the passage of a
patch of cirrus.
<P>
The program will show you several doubtful cases for every star that really
turns out to be variable, so be cautious in deciding to treat them as variable
stars.
If you decide that a star really looks variable, you can change its category to
``variable'', and it will be excluded from all further extinction solutions.
<P>
If you change the category of any star, the program goes back and re-does the
whole extinction solution again from the very beginning.
When you get through a consistent solution without changing any star's
category, the program announces that it has done all it can do, displays
some residual plots, and prints the final results.
<P>
All reductions are done in the instrumental system, even if
standard-star values (and hence transformations) are included as part of the
extinction solution.
It may be helpful to look at a schematic picture of the reduction process
(see Figure <A HREF="node285.html#flowchart">13.2</A> on the next page).
<P>
<BR>
<DIV ALIGN="CENTER"><A NAME="flowchart"> </A><A NAME="12532"> </A>
<TABLE WIDTH="50%">
<CAPTION><STRONG>Figure:</STRONG>
Schematic flowchart for reductions</CAPTION>
<TR><TD>
<DIV ALIGN="CENTER">
<IMG
WIDTH="718" HEIGHT="1005" ALIGN="BOTTOM" BORDER="0"
SRC="img572.gif"
ALT="\begin{picture}(120,180)
\par\put(0,170){\makebox{Pre-process data:}}
\put(60,1...
...no}}
\put(95,5){\vector(1,0){20}}
\put(105,77){\oval(34,144)[r]}
\end{picture}">
</DIV></TD></TR>
</TABLE>
</DIV>
<BR>
<P>
<HR>
<!--Navigation Panel-->
<A NAME="tex2html4995"
HREF="node286.html">
<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
SRC="icons.gif/next_motif.gif"></A>
<A NAME="tex2html4992"
HREF="node271.html">
<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
SRC="icons.gif/up_motif.gif"></A>
<A NAME="tex2html4986"
HREF="node284.html">
<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
SRC="icons.gif/previous_motif.gif"></A>
<A NAME="tex2html4994"
HREF="node1.html">
<IMG WIDTH="65" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="contents"
SRC="icons.gif/contents_motif.gif"></A>
<BR>
<B> Next:</B> <A NAME="tex2html4996"
HREF="node286.html">Final results</A>
<B> Up:</B> <A NAME="tex2html4993"
HREF="node271.html">Reducing the observations</A>
<B> Previous:</B> <A NAME="tex2html4987"
HREF="node284.html">Standard stars</A>
<!--End of Navigation Panel-->
<ADDRESS>
<I>Petra Nass</I>
<BR><I>1999-06-15</I>
</ADDRESS>
</BODY>
</HTML>
