<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="generator" content="hevea 2.16">
<style type="text/css">
.li-itemize{margin:1ex 0ex;}
.li-enumerate{margin:1ex 0ex;}
.dd-description{margin:0ex 0ex 1ex 4ex;}
.dt-description{margin:0ex;}
.toc{list-style:none;}
.footnotetext{margin:0ex; padding:0ex;}
div.footnotetext P{margin:0px; text-indent:1em;}
.thefootnotes{text-align:left;margin:0ex;}
.dt-thefootnotes{margin:0em;}
.dd-thefootnotes{margin:0em 0em 0em 2em;}
.footnoterule{margin:1em auto 1em 0px;width:50%;}
.caption{padding-left:2ex; padding-right:2ex; margin-left:auto; margin-right:auto}
.title{margin:2ex auto;text-align:center}
.titlemain{margin:1ex 2ex 2ex 1ex;}
.titlerest{margin:0ex 2ex;}
.center{text-align:center;margin-left:auto;margin-right:auto;}
.flushleft{text-align:left;margin-left:0ex;margin-right:auto;}
.flushright{text-align:right;margin-left:auto;margin-right:0ex;}
div table{margin-left:inherit;margin-right:inherit;margin-bottom:2px;margin-top:2px}
td table{margin:auto;}
table{border-collapse:collapse;}
td{padding:0;}
.cellpadding0 tr td{padding:0;}
.cellpadding1 tr td{padding:1px;}
pre{text-align:left;margin-left:0ex;margin-right:auto;}
blockquote{margin-left:4ex;margin-right:4ex;text-align:left;}
td p{margin:0px;}
.boxed{border:1px solid black}
.textboxed{border:1px solid black}
.vbar{border:none;width:2px;background-color:black;}
.hbar{border:none;height:2px;width:100%;background-color:black;}
.hfill{border:none;height:1px;width:200%;background-color:black;}
.vdisplay{border-collapse:separate;border-spacing:2px;width:auto; empty-cells:show; border:2px solid red;}
.vdcell{white-space:nowrap;padding:0px; border:2px solid green;}
.display{border-collapse:separate;border-spacing:2px;width:auto; border:none;}
.dcell{white-space:nowrap;padding:0px; border:none;}
.dcenter{margin:0ex auto;}
.vdcenter{border:solid #FF8000 2px; margin:0ex auto;}
.minipage{text-align:left; margin-left:0em; margin-right:auto;}
.marginpar{border:solid thin black; width:20%; text-align:left;}
.marginparleft{float:left; margin-left:0ex; margin-right:1ex;}
.marginparright{float:right; margin-left:1ex; margin-right:0ex;}
.theorem{text-align:left;margin:1ex auto 1ex 0ex;}
.part{margin:2ex auto;text-align:center}
</style>
<link rel="stylesheet" type="text/css" href="qtest-vh.css">
<link rel="stylesheet" type="text/css" href="http://fonts.googleapis.com/css?family=Ubuntu:regular,italic,bold,bolditalic">
<title>qtest
--- Quick Unit Tests for OCaml ---
version 2.0
</title>
</head>
<body >
<!--HEVEA command line is: hevea -moreentities -fix qtest.tex -->
<!--CUT STYLE article--><!--CUT DEF subsection 1 --><table class="title"><tr><td style="padding:1ex"><h1 class="titlemain"><span style="font-size:xx-large"><span style="font-size:150%">q</span></span><span style="font-size:xx-large"><span style="font-size:150%"><span style="font-style:italic">test</span></span></span><br>
— Quick Unit Tests for <span style="font-style:italic">OCaml</span> — <br>
version 2.0<sup><a id="text1" href="#note1">*</a></sup>
</h1><h3 class="titlerest">Vincent <span style="font-variant:small-caps">Hugot</span> & the <a href="http://batteries.forge.ocamlcore.org">Batteries</a> team</h3></td></tr>
</table><!--TOC section id="sec1" Contents-->
<h2 id="sec1" class="section">Contents</h2><!--SEC END --><ul class="toc"><li class="li-toc">
<a href="#sec2">1 Introduction</a>
</li><li class="li-toc"><a href="#sec3">2 Using q<span style="font-style:italic">test</span> : a Quick, Simple Example</a>
</li><li class="li-toc"><a href="#sec4">3 More q<span style="font-style:italic">test</span> Pragmas</a>
<ul class="toc"><li class="li-toc">
<a href="#sec5">3.1 Different Kinds of Tests</a>
<ul class="toc"><li class="li-toc">
<a href="#sec6">3.1.1 Simple Tests: <TTL>T</TTL> for "Test"</a>
</li><li class="li-toc"><a href="#sec7">3.1.2 Equality Tests: <TTL>=</TTL></a>
</li><li class="li-toc"><a href="#sec8">3.1.3 Randomized Tests: <TTL>Q</TTL> for "Quickcheck"</a>
</li><li class="li-toc"><a href="#sec9">3.1.4 Raw oUnit Tests: <TTL>R</TTL> for "Raw"</a>
</li><li class="li-toc"><a href="#sec10">3.1.5 Exception-Throwing Tests: <TTL>E</TTL> for "Exception"</a>
</li></ul>
</li><li class="li-toc"><a href="#sec11">3.2 Manipulation Pragmas</a>
<ul class="toc"><li class="li-toc">
<a href="#sec12">3.2.1 Opening Modules: <em>open</em> Pragma <TTL><...></TTL> and <TTL>--preamble</TTL> Option</a>
</li><li class="li-toc"><a href="#sec13">3.2.2 Code Injection Pragma: <TTL>{...}</TTL></a>
</li></ul>
</li></ul>
</li><li class="li-toc"><a href="#sec14">4 Technical Considerations and Other Details</a>
<ul class="toc"><li class="li-toc">
<a href="#sec15">4.1 Function Coverage</a>
</li><li class="li-toc"><a href="#sec16">4.2 Headers and Metaheaders</a>
<ul class="toc"><li class="li-toc">
<a href="#sec17">4.2.1 Aliases</a>
</li><li class="li-toc"><a href="#sec18">4.2.2 Mutually Tested Functions</a>
</li><li class="li-toc"><a href="#sec19">4.2.3 Testing Functions by the Dozen</a>
</li><li class="li-toc"><a href="#sec20">4.2.4 Metaheaders Unleashed</a>
</li><li class="li-toc"><a href="#sec21">4.2.5 Header Parameters Injection</a>
</li></ul>
</li><li class="li-toc"><a href="#sec22">4.3 Warnings and Exceptions Thrown by q<span style="font-style:italic">test</span></a>
</li><li class="li-toc"><a href="#sec23">4.4 q<span style="font-style:italic">test</span> Command-Line Options</a>
</li></ul>
</li><li class="li-toc"><a href="#sec24">5 Editor Support</a>
</li></ul>
<!--TOC section id="sec2" Introduction-->
<h2 id="sec2" class="section">1 Introduction</h2><!--SEC END --><p>
In a nutshell, q<span style="font-style:italic">test</span> is a small program which reads <span style="font-family:monospace">.ml</span> and <span style="font-family:monospace">.mli</span> source files and
extracts inline <a href="http://ounit.forge.ocamlcore.org">oUnit</a> unit tests from them. It is used internally by the <span style="font-style:italic">OCaml</span>
<a href="http://batteries.forge.ocamlcore.org">Batteries</a> project, and is shipped with it as of version 2.0, but it does not depend on
it and can be compiled and used independently.</p><p>Browse its code in the <a href="http://batteries.forge.ocamlcore.org">Batteries</a> Github
<a href="https://github.com/ocaml-batteries-team/batteries-included/tree/master/qtest">repository</a>.</p><p><em>This document is available as either </em><a href="qtest.html"><em>one big page</em></a><em> or
</em><a href="index.html"><em>several smaller pages</em></a><em>. Contents are the same.</em></p>
<!--TOC section id="sec3" Using q<span style="font-style:italic">test</span> : a Quick, Simple Example-->
<h2 id="sec3" class="section">2 Using q<span style="font-style:italic">test</span> : a Quick, Simple Example</h2><!--SEC END --><p>
Say that you have a file <span style="font-family:monospace">foo.ml</span>, which contains the implementation of your new, shiny
function <span style="font-family:monospace">foo</span>.</p><pre class="verbatim">let rec foo x0 f = function
[] -> 0 | x::xs -> f x (foo x0 f xs)
</pre><p>Maybe you don’t feel confident about that code; or maybe you do, but you know that the
function might be re-implemented less trivially in the future and want to prevent
potential regressions. Or maybe you simply think unit tests are good practice anyway. In
either case, you feel that building a separate test suite for this would be overkill.
Using q<span style="font-style:italic">test</span>, you can immediately put simple unit tests in comments near <span style="font-family:monospace">foo</span>, for
instance:</p><pre class="verbatim">(*$T foo
foo 0 ( + ) [1;2;3] = 6
foo 0 ( * ) [1;2;3] = 0
foo 1 ( * ) [4;5] = 20
foo 12 ( + ) [] = 12
*)
</pre><p>the syntax is simple: <span style="font-family:monospace">(*$</span> introduces a q<span style="font-style:italic">test</span> "pragma", such as <span style="font-family:monospace">T</span> in this case. <span style="font-family:monospace">T</span>
is by far the most common and represents a "simple" unit test. <span style="font-family:monospace">T</span> expects a "header",
which is most of the time simply the name of the function under test, here <span style="font-family:monospace">foo</span>.
Following that, each line is a "statement", which must evaluate to <span style="font-family:monospace">true</span> for the test to
pass. Furthermore, <span style="font-family:monospace">foo</span> must appear in each statement.</p><p>Now, in order to execute those tests, you need to extract them; this is done with the
q<span style="font-style:italic">test</span> executable. The command</p><pre class="console">$ qtest -o footest.ml extract foo.ml
Target file: `footest.ml'. Extraction : `foo.ml' Done.
</pre><p>will create a file <span style="font-family:monospace">footest.ml</span>; it’s not terribly human-readable, but you can see that it
contains your tests as well as some <a href="http://ounit.forge.ocamlcore.org">oUnit</a> boilerplate. Now you need to compile the
tests, for instance with <span style="font-family:monospace">ocamlbuild</span>, and assuming <a href="http://ounit.forge.ocamlcore.org">oUnit</a> was installed for <span style="font-family:monospace">ocamlfind</span>.</p><pre class="console">$ ocamlbuild -cflags -warn-error,+26 -use-ocamlfind -package oUnit \
footest.native
Finished, 10 targets (1 cached) in 00:00:00.
</pre><p>Note that the <span style="font-family:monospace">-cflags -warn-error,+26</span> is not indispensable but strongly recommended. Its
function will be explained in more detail in the more technical sections of this
documentation, but roughly it makes sure that if you write a test for <span style="font-family:monospace">foo</span>, via
<span style="font-family:monospace">(*$T foo</span> for instance, then <span style="font-family:monospace">foo</span> is <em>actually</em> tested by each statement
– the tests won’t compile if not.</p><p><span style="font-weight:bold">Important note:</span> in order for this to work, <span style="font-family:monospace">ocamlbuild</span> must know where to find
<span style="font-family:monospace">foo.ml</span>; if <span style="font-family:monospace">footest.ml</span> is not in the same directory, you must make provisions to that
effect. If <span style="font-family:monospace">foo.ml</span> needs some specific flags in order to compile, they must also be
passed.</p><p>Now there only remains to run the tests:</p><pre class="console">$ ./footest.native
..FF
==============================================================================
Failure: qtest:0:foo:3:foo.ml:10
OUnit: foo.ml:10::> foo 12 ( + ) [] = 12
------------------------------------------------------------------------------
==============================================================================
Failure: qtest:0:foo:2:foo.ml:9
OUnit: foo.ml:9::> foo 1 ( * ) [4;5] = 20
------------------------------------------------------------------------------
Ran: 4 tests in: 0.00 seconds.
FAILED: Cases: 4 Tried: 4 Errors: 0 Failures: 2 Skip:0 Todo:0
</pre><p>Oops, something’s wrong... either the tests are incorrect or <span style="font-family:monospace">foo</span> is. Finding and fixing
the problem is left as an exercise for the reader. When this is done, you get the expected</p><pre class="console">$ ./footest.native
....
Ran: 4 tests in: 0.00 seconds.
</pre><p><span style="font-weight:bold">Tip:</span> those steps are easy to automate, for instance with a small shell script:</p><pre class="console">set -e # stop on first error
qtest -o footest.ml extract foo.ml
ocamlbuild -cflags -warn-error,+26 -use-ocamlfind -package oUnit footest.native
./footest.native
</pre>
<!--TOC section id="sec4" More q<span style="font-style:italic">test</span> Pragmas-->
<h2 id="sec4" class="section">3 More q<span style="font-style:italic">test</span> Pragmas</h2><!--SEC END -->
<!--TOC subsection id="sec5" Different Kinds of Tests-->
<h3 id="sec5" class="subsection">3.1 Different Kinds of Tests</h3><!--SEC END -->
<!--TOC subsubsection id="sec6" Simple Tests: <TTL>T</TTL> for "Test"-->
<h4 id="sec6" class="subsubsection">3.1.1 Simple Tests: <TTL>T</TTL> for "Test"</h4><!--SEC END --><p>
The most common kind of tests is the simple test, an example of which is given above. It
is of the form</p><pre class="verbatim">(*$T <header>
<statement>
...
*)
</pre><p>where each <em>statement</em> must be a boolean <span style="font-style:italic">OCaml</span> expression involving the function (or
functions, as we will see when we study headers) referenced in the <em>header</em>.
The overall test is considered successful if each <em>statement</em> evaluates to <span style="font-family:monospace">true</span>. Note
that the "close comment" <span style="font-family:monospace">*)</span> must appear on a line of its own.</p><p><span style="font-weight:bold">Tip:</span> if a statement is a bit too long to fit on one line, if can be broken using a
backslash (<span style="font-family:monospace">\</span>), immediately followed by the carriage return. This also applies to
randomised tests.
</p>
<!--TOC subsubsection id="sec7" Equality Tests: <TTL>=</TTL>-->
<h4 id="sec7" class="subsubsection">3.1.2 Equality Tests: <TTL>=</TTL></h4><!--SEC END --><p>
The vast majority of test cases tend to involve the equality of
two expressions; using simple tests, one would write something like:</p><pre class="verbatim">(*$T foo
foo 1 ( * ) [4;5] = foo 3 ( * ) [1;5;2]
*)
</pre><p>While this certainly works, the failure report for such a test does not convey
any useful information besides the simple fact that the test failed. Wouldn’t it be
nice if the report also mentioned the values of the left-hand side and the
right-hand side ? Yes it would, and specialised equality tests provide such functionality,
at the cost of a little bit of boilerplate code. The bare syntax is:</p><pre class="verbatim">(*$= <header>
<lhs> <rhs>
...
*)
</pre><p>However, used bare, an equality test will not provide much more information than a simple test:
just a laconic “not equal”. In order for the values to be printed, a “value printer”
must be specified for the test. A printer is a function of type α→ <span style="font-family:sans-serif">string</span>,
where α is the type of the expressions on both side of the equality.
To pass the printer to the test, we use <em>parameter injection</em> (cf. Section <a href="#sec%3Aparam">4.2.5</a>);
equality tests have an optional argument <span style="font-family:monospace">printer</span> for this purpose.
In our example, we have α = <span style="font-family:sans-serif">int</span>, so the test becomes simply:</p><pre class="verbatim">(*$= foo & ~printer:string_of_int
(foo 1 ( * ) [4;5]) (foo 3 ( * ) [1;5;2])
*)
</pre><p>The failure report will now be more explicit, saying <span style="font-family:monospace">expected: 20 but got: 30
</span>.</p>
<!--TOC subsubsection id="sec8" Randomized Tests: <TTL>Q</TTL> for "Quickcheck"-->
<h4 id="sec8" class="subsubsection">3.1.3 Randomized Tests: <TTL>Q</TTL> for "Quickcheck"</h4><!--SEC END --><p>
Quickcheck is a small library useful for randomized unit tests. Using it is a bit more
complex, but much more rewarding than simple tests.</p><pre class="verbatim">(*$Q <header>
<generator> (fun <generated value> -> <statement>)
...
*)
</pre><p>Let us dive into an example straight-away:</p><pre class="verbatim">(*$Q foo
Q.small_int (fun i-> foo i (+) [1;2;3] = List.fold_left (+) i [1;2;3])
*)
</pre><p>The Quickcheck module is accessible simply as <em>Q</em> within inline tests; <span style="font-family:monospace">small_int</span> is a
generator, yielding a random, small integer. When the test is run, each statement will be
evaluated for a large number of random values – 100 by default. Running this test for the
above definition of foo catches the mistake easily:</p><pre class="console">law foo.ml:14::> Q.small_int (fun i-> foo i (+) [1;2;3]
= List.fold_left (+) i [1;2;3])
failed for 2
</pre><p>Note that the random value for which the test failed is provided by the error message –
here it is 2. It is also possible to generate several random values simultaneously using
tuples. For instance</p><pre class="verbatim">(Q.pair Q.small_int (Q.list Q.small_int)) \
(fun (i,l)-> foo i (+) l = List.fold_left (+) i l)
</pre><p>will generate both an integer and a list of small integers randomly. A failure will then
look like</p><pre class="console">law foo.ml:15::> (Q.pair Q.small_int (Q.list Q.small_int))
(fun (i,l)-> foo i (+) l = List.fold_left (+) i l)
failed for (727, [4; 3; 6; 1; 788; 49])
</pre><p><span style="font-weight:bold">Available Generators:</span></p><ul class="itemize"><li class="li-itemize">
<U>Simple generators:</U><br>
<span style="font-family:monospace">unit</span>, <span style="font-family:monospace">bool</span>, <span style="font-family:monospace">float</span>, <span style="font-family:monospace">pos_float</span>,
<span style="font-family:monospace">neg_float</span>, <span style="font-family:monospace">int</span>, <span style="font-family:monospace">pos_int</span>, <span style="font-family:monospace">small_int</span>,
<span style="font-family:monospace">neg_int</span>, <span style="font-family:monospace">char</span>, <span style="font-family:monospace">printable_char</span>, <span style="font-family:monospace">numeral_char</span>,
<span style="font-family:monospace">string</span>, <span style="font-family:monospace">printable_string</span>,
<span style="font-family:monospace">numeral_string</span></li><li class="li-itemize"><U>Structure generators:</U><br>
<span style="font-family:monospace">list</span> and <span style="font-family:monospace">array</span>. They take one generator as their argument. For instance
<span style="font-family:monospace">(Q.list Q.neg_int)</span> is a generator of lists of (uniformly taken) negative integers.</li><li class="li-itemize"><U>Tuples generators:</U><br>
<span style="font-family:monospace">pair</span> and <span style="font-family:monospace">triple</span> are respectively binary and ternary. See above for an example of
<span style="font-family:monospace">pair</span>.</li><li class="li-itemize"><U>Size-directed generators:</U><br>
<span style="font-family:monospace">string</span>, <span style="font-family:monospace">numeral_string</span>, <span style="font-family:monospace">printable_string</span>, <span style="font-family:monospace">list</span>
and <span style="font-family:monospace">array</span> all have <span style="font-family:monospace">*_of_size</span>
variants that take the size of the structure as their first argument.
</li></ul><p><span style="font-weight:bold">Tips:</span></p><ul class="itemize"><li class="li-itemize">
<span style="font-weight:bold">Duplicate Elements in Lists:</span>
When generating lists, avoid <span style="font-family:monospace">Q.list Q.int</span> unless you have a good reason to do so.
The reason is that, given the size of the <span style="font-family:monospace">Q.int</span> space, you are unlikely
to generate any duplicate elements. If you wish to test your function’s behaviour with
duplicates, prefer <span style="font-family:monospace">Q.list Q.small_int</span>.</li><li class="li-itemize"><span style="font-weight:bold">Changing Number of Tests:</span>
If you want a specific test to execute each of its statements a specific number of times
(deviating from the default of 100), you can specify it explicitly through
<em>parameter injection</em> (cf. Section <a href="#sec%3Aparam">4.2.5</a>) using the
<span style="font-family:monospace">count</span> : <span style="font-family:sans-serif">int</span> argument.</li><li class="li-itemize"><span style="font-weight:bold">Getting a Better Counterexample:</span>
By default, a random test stops as soon as one of its generated values
yields a failure. This first failure value is probably not the best possible
counterexample. You can <em>force</em> qtest to generate and test all <span style="font-family:monospace">count</span>
random values regardless, and to display the value which is smallest with respect
to a certain measure which you define. To this end, it suffices to use parameter injection
to pass argument <span style="font-family:monospace">small</span> : α → β,
where α is the type of
generated values and β is any totally ordered set (wrt. <). Typically
you will take β = <span style="font-family:sans-serif">int</span> or β = <span style="font-family:sans-serif">float</span>. Example:
<pre class="verbatim">let fuz x = x
let rec flu = function
| [] -> []
| x :: l -> if List.mem x l then flu l else x :: flu l
(*$Q fuz; flu & ~small:List.length
(Q.list Q.small_int) (fun x -> fuz x = flu x)
*)
</pre>The meaning of <span style="font-family:monospace"> small:List.length</span> is therefore simply:
“choose the shortest list”. For very complicated cases, you can simultaneously
increase <span style="font-family:monospace">count</span> to yield an even higher-quality counterexample.</li></ul>
<!--TOC subsubsection id="sec9" Raw oUnit Tests: <TTL>R</TTL> for "Raw"-->
<h4 id="sec9" class="subsubsection">3.1.4 Raw <a href="http://ounit.forge.ocamlcore.org">oUnit</a> Tests: <TTL>R</TTL> for "Raw"</h4><!--SEC END --><p>
When more specialised test pragmas are too restrictive, for instance if the test is too
complex to reasonably fit on one line, then one can use raw <a href="http://ounit.forge.ocamlcore.org">oUnit</a> tests.</p><pre class="verbatim">(*$R <header>
<raw oUnit test>...
...
*)
</pre><p>Here is a small example, with two tests stringed together:</p><pre class="verbatim">(*$R foo
let thing = foo 1 ( * )
and li = [4;5] in
assert_bool "something_witty" (thing li = 20);
assert_bool "something_wittier" (foo 12 ( + ) [] = 12)
*)
</pre><p>Note that if the first assertion fails, the second will not be executed; so stringing two
assertions in that mode is different in that respect from doing so under a <span style="font-family:monospace">T</span> pragma, for
instance.</p><p>That said, raw tests should only be used as a last resort; for instance you don’t
automatically get the source file and line number when the test fails. If <span style="font-family:monospace">T</span> and <span style="font-family:monospace">Q</span> do
not satisfy your needs, then it is <em>probably</em> a hint that the test is a bit complex and,
maybe, belongs in a separate test suite rather than in the middle of the source code.
</p>
<!--TOC subsubsection id="sec10" Exception-Throwing Tests: <TTL>E</TTL> for "Exception"-->
<h4 id="sec10" class="subsubsection">3.1.5 Exception-Throwing Tests: <TTL>E</TTL> for "Exception"</h4><!--SEC END --><p>
... not implemented yet...</p><p>The current usage is to use <span style="font-family:monospace">(*$T</span> and the following pattern for function <span style="font-family:monospace">foo</span> and
exception <span style="font-family:monospace">Bar</span>:</p><pre class="verbatim">try ignore (foo x); false with Bar -> true
</pre><p>If your project uses <a href="http://batteries.forge.ocamlcore.org">Batteries</a> and no pattern-matching is needed, then you can also use
the following, sexier pattern:</p><pre class="verbatim">Result.(catch foo x |> is_exn Bar)
</pre>
<!--TOC subsection id="sec11" Manipulation Pragmas-->
<h3 id="sec11" class="subsection">3.2 Manipulation Pragmas</h3><!--SEC END --><p>
Not all qtest pragmas directly translate into tests; for non-trivial projects, sometimes a
little boilerplate code is needed in order to set the tests up properly. The pragmas which
do this are collectively called "manipulation pragmas"; they are described in the next
section.
</p>
<!--TOC subsubsection id="sec12" Opening Modules: <em>open</em> Pragma <TTL><...></TTL> and <TTL>--preamble</TTL> Option-->
<h4 id="sec12" class="subsubsection">3.2.1 Opening Modules: <em>open</em> Pragma <TTL><...></TTL> and <TTL>--preamble</TTL> Option</h4><!--SEC END --><p>
The tests should have access to the same values as the code under test; however the
generated code for <span style="font-family:monospace">foo.ml</span> does not actually live inside that file. Therefore some
effort must occasionally be made to synchronise the code’s environment with the tests’.
There are three main usecases where you might want to open modules for tests:</p><ul class="itemize"><li class="li-itemize">
<span style="font-weight:bold">Project-Wide Global Open:</span><br>
It may happen that <em>every single file</em> in your project
opens a given module. This is the case for <a href="http://batteries.forge.ocamlcore.org">Batteries</a>, for instance, where every module
opens <span style="font-family:monospace">Batteries</span>. In that case simply use the <span style="font-family:monospace">--preamble</span> switch. For instance,
<pre class="verbatim">qtest --preamble "open Batteries;;" extract mod1.ml mod2.ml ... modN.ml
</pre>Note that you could insert arbitrary code using this switch.</li><li class="li-itemize"><span style="font-weight:bold">Global Open in a File:</span><br>
Now, let’s say that <span style="font-family:monospace">foo.ml</span> opens <span style="font-family:monospace">Bar</span> and <span style="font-family:monospace">Baz</span>; you want
the tests in <span style="font-family:monospace">foo.ml</span> to open them as well. Then you can use the <em>open</em> pragma in its
<em>global</em> form:
<pre class="verbatim">(*$< Bar, Baz >*)
</pre>The modules will be open for every test in the same <span style="font-family:monospace">.ml</span> file, and following the pragma.
However, in our example, you will have a duplication of code between the "open" directives
of <span style="font-family:monospace">foo.ml</span>, and the <em>open</em> pragma of q<span style="font-style:italic">test</span>, like so:
<pre class="verbatim">open Bar;; open Baz;;
(*$< Bar, Baz >*)
</pre>It might therefore be more convenient to use the <em>code injection</em> pragma (see next
section) for that purpose, so you would write instead:
<pre class="verbatim">(*${*) open Bar;; open Baz;; (*$}*)
</pre>The code between that special markup will simply be duplicated into the tests. The two
methods are equivalent, and the second one is recommended, because it reduces the chances
of an impedance mismatch between modules open for ‘<span style="font-family:monospace">foo.ml</span>‘ and its tests. Therefore, the
global form of the <em>open</em> pragma should preferentially be reserved for cases where you
<em>want</em> such a mismatch. For instance, if you have special modules useful for tests but
useless for the main code, you can easily open then for the tests alone using the pragma.</li><li class="li-itemize"><span style="font-weight:bold">Local Open for a Submodule:</span><br>
Let’s say we have the following <span style="font-family:monospace">foo.ml</span>:
<pre class="verbatim">let outer x = <something>
module Submod = struct
let inner y = 2*x
(*$T inner
inner 2 = 4
*)
end
</pre>That seems natural enough... but it won’t work, because q<span style="font-style:italic">test</span> is not actually aware that
the test is "inside" Submod (and making it aware of that would be very problematic). In
fact, so long as you use only test pragmas (ie. no manipulation pragma at all), the
positions and even the order of the tests – respective to definitions or to each other –
are unimportant, because the tests do not actually live in <span style="font-family:monospace">foo.ml</span>. So we need to open
Submod manually, using the <em>local</em> form of the <em>open</em> pragma:
<pre class="verbatim">module Submod = struct (*$< Submod *)
let inner y = 2*x
(*$T inner
inner 2 = 4
*)
end (*$>*)
</pre>Notice that the <span style="font-family:monospace"><...></span> have simply been split in two, compared to the global form.
The effect of that construct is that Submod will be open for every test between
<span style="font-family:monospace">(*$< Submod *)</span> and <span style="font-family:monospace">(*$>*)</span>.
Of course, you <em>could</em> also forgo that method entirely and do this:
<pre class="verbatim">module Submod = struct
let inner y = 2*x
(*$T &
Submod.inner 2 = 4
*)
end
</pre>... but it is impractical and you are <em>forced</em> to use an empty header because qualified
names are not acceptable as headers. The first method is therefore <em>strongly</em> recommended.
</li></ul>
<!--TOC subsubsection id="sec13" Code Injection Pragma: <TTL>{...}</TTL>-->
<h4 id="sec13" class="subsubsection">3.2.2 Code Injection Pragma: <TTL>{...}</TTL></h4><!--SEC END --><p>TODO: ocamldoc comments that define unit tests from the offered examples</p>
<!--TOC section id="sec14" Technical Considerations and Other Details-->
<h2 id="sec14" class="section">4 Technical Considerations and Other Details</h2><!--SEC END --><p>
What has been said above should suffice to cover at least 90% of use-cases for q<span style="font-style:italic">test</span>.
This section concerns itself with the remaining 10%.
</p>
<!--TOC subsection id="sec15" Function Coverage-->
<h3 id="sec15" class="subsection">4.1 Function Coverage</h3><!--SEC END --><p>
The headers of a test are not just there for decoration; three properties are enforced
when a test, say, <span style="font-family:monospace">(*$X foo</span> is compiled, where <span style="font-family:monospace">X</span> is
<span style="font-family:monospace">T</span>, <span style="font-family:monospace">R</span>, <span style="font-family:monospace">Q</span>,... :</p><ul class="itemize"><li class="li-itemize"><span style="font-family:monospace">foo</span> exists; that is to say, it is defined in the scope of the module where the testappears – though one can play with pragmas to relax this condition somewhat. At the very
least, it has to be defined <em>somewhere</em>. Failure to conform results in an <span style="font-family:monospace">Error: Unbound
value foo</span>.</li><li class="li-itemize"><span style="font-family:monospace">foo</span> is referenced in <em>each statement</em> of the test:
for <span style="font-family:monospace">T</span> and <span style="font-family:monospace">Q</span>, that means "each line". For
<span style="font-family:monospace">R</span>, that means "once somewhere in the test’s body". Failure
to conform results in a <span style="font-family:monospace">Warning 26: unused variable foo</span>,
which will be treated as an error if <span style="font-family:monospace">-warn-error +26</span> is
passed to the compiler. It goes without saying that this is warmly
recommended.</li><li class="li-itemize">the test possesses at least one statement.
</li></ul><p>Those two conditions put together offer a strong guarantee that, if a function is
referenced in a test header, then it is actually tested at least once. The list of
functions referenced in the headers of extracted tests is written by q<span style="font-style:italic">test</span> into
<span style="font-family:monospace">qtest.targets.log</span>. Each line is of the form</p><pre class="verbatim">foo.ml 42 foo
</pre><p>where <span style="font-family:monospace">foo.ml</span> is the file in which the test appears, as passed to <span style="font-family:monospace">extract</span>,
and <span style="font-family:monospace">42</span> is
the line number where the test pragma appears in <span style="font-family:monospace">foo.ml</span>. Note that a same function can be
listed several times for the same source file, if several tests involve it (say, two times
if it has both a simple test and a random one). The exact number of statements involving
<span style="font-family:monospace">foo</span> in each test is currently not taken into account in the logs.
</p>
<!--TOC subsection id="sec16" Headers and Metaheaders-->
<h3 id="sec16" class="subsection">4.2 Headers and Metaheaders</h3><!--SEC END --><p>
The informal definition of headers given in the above was actually a simplification. In
this section we explore two syntaxes available for headers.
</p>
<!--TOC subsubsection id="sec17" Aliases-->
<h4 id="sec17" class="subsubsection">4.2.1 Aliases</h4><!--SEC END --><p>
Some functions have exceedingly long names. Case in point :</p><pre class="verbatim">let rec pretentious_drivel x0 f = function [] -> x0
| x::xs -> pretentious_drivel (f x x0) f xs
</pre><pre class="verbatim">(*$T pretentious_drivel
pretentious_drivel 1 (+) [4;5] = foo 1 (+) [4;5]
... pretentious_drivel of this and that...
*)
</pre><p>The constraint that each statement must fit on one line does not play well with very long
function names. Furthermore, you <em>known</em> which function is being tested, it’s right there
is the header; no need to repeat it a dozen times. Instead, you can define an <em>alias</em>, and
write equivalently:</p><pre class="verbatim">(*$T pretentious_drivel as x
x 1 (+) [4;5] = foo 1 (+) [4;5]
... x of this and that...
*)
</pre><p>... thus saving many keystrokes, thereby contributing to the preservation of the
environment. More seriously, aliases have uses beyond just saving a few keystrokes, as we
will see in the next sections.
</p>
<!--TOC subsubsection id="sec18" Mutually Tested Functions-->
<h4 id="sec18" class="subsubsection">4.2.2 Mutually Tested Functions</h4><!--SEC END --><p>
Most of the time, a test only pertains to one function. There are times, however, when one
wishes to test two functions – or more – at the same time. For instance</p><pre class="verbatim">let rec even = function 0 -> true
| n -> odd (pred n)
and odd = function 0 -> false
| n -> even (pred n)
</pre><p>Let us say that we have the following test:</p><pre class="verbatim">(*$Q <header>
Q.small_int (fun n-> odd (abs n+3) = even (abs n))
*)
</pre><p>It involves both <span style="font-family:monospace">even</span> and <span style="font-family:monospace">odd</span>. That question is: "what is a proper header for this
test"? One could simply put "even", and thus it would be referenced as being tested in the
logs, but <span style="font-family:monospace">odd</span> would not, which is unfair. Putting "odd" is symmetrically unfair. The
solution is to put both, separated by a semi-colon:</p><pre class="verbatim">(*$Q even; odd
</pre><p>That way <em>both</em> functions are referenced in the logs:</p><pre class="verbatim"> foo.ml 37 even
foo.ml 37 odd
</pre><p>and of course the compiler enforces that both of them are actually referenced in each
statement of the test. Of course, each of them can be written under alias, in which case
the header could be <span style="font-family:monospace">even as x; odd as y</span>.
</p>
<!--TOC subsubsection id="sec19" Testing Functions by the Dozen-->
<h4 id="sec19" class="subsubsection">4.2.3 Testing Functions by the Dozen</h4><!--SEC END --><p>
Let us come back to our functions <span style="font-family:monospace">foo</span> (after correction) and <span style="font-family:monospace">pretentious_drivel</span>, as
defined above.</p><pre class="verbatim">let rec foo x0 f = function
[] -> x0 | x::xs -> f x (foo x0 f xs)
let rec pretentious_drivel x0 f = function [] -> x0
| x::xs -> pretentious_drivel (f x x0) f xs
</pre><p>You will not have failed to notice that they bear more than a passing resemblance to one
another. If you write tests for one, odds are that the same test could be useful verbatim
for the other. This is a very common case when you have closely related functions, or even
several <em>implementations</em> of the same function, for instance the old, slow, naïve,
trustworthy one and the new, fast, arcane, highly optimised version you have just written.
The typical case is sorting routines, of which there are many flavours.</p><p>For our example, recall that we have the following test for <span style="font-family:monospace">foo</span>:</p><pre class="verbatim">(*$Q foo
(Q.pair Q.small_int (Q.list Q.small_int)) \
(fun (i,l)-> foo i (+) l = List.fold_left (+) i l)
*)
</pre><p>The same test would apply to <span style="font-family:monospace">pretentious_drivel</span>; you could just copy-and-paste the test
and change the header, but it’s not terribly elegant. Instead, you can just just add the
other function to the header, separating the two by a comma, and defining an alias:</p><pre class="verbatim">(*$Q foo, pretentious_drivel as x
(Q.pair Q.small_int (Q.list Q.small_int)) \
(fun (i,l)-> x i (+) l = List.fold_left (+) i l)
*)
</pre><p>This same test will be run once for <span style="font-family:monospace">x = foo</span>, and once for <span style="font-family:monospace">x = pretentious_drivel</span>.
Actually, you need not define an alias: if the header is of the form</p><pre class="verbatim">(*$Q foo, pretentious_drivel
</pre><p>then it is equivalent to</p><pre class="verbatim">(*$Q foo, pretentious_drivel as foo
</pre><p>so you do not need to alter the body of the test if you subsequently add new functions. A
header which combines more than one "version" of a function in this way is called a
<em>metaheader</em>.
</p>
<!--TOC subsubsection id="sec20" Metaheaders Unleashed-->
<h4 id="sec20" class="subsubsection">4.2.4 Metaheaders Unleashed</h4><!--SEC END --><p>
All the constructs above can be combined without constraints: the grammar is as follows:</p><pre class="verbatim"> Metaheader ::= Binding {";" Binding}
Binding ::= Functions [ "as" ID ]
Functions ::= ID {"," ID}
ID ::= (*OCaml lower-case identifier*)
</pre>
<!--TOC subsubsection id="sec21" Header Parameters Injection-->
<h4 id="sec21" class="subsubsection">4.2.5 Header Parameters Injection</h4><!--SEC END --><p><a id="sec:param"></a>
coming soon...
</p>
<!--TOC subsection id="sec22" Warnings and Exceptions Thrown by q<span style="font-style:italic">test</span>-->
<h3 id="sec22" class="subsection">4.3 Warnings and Exceptions Thrown by q<span style="font-style:italic">test</span></h3><!--SEC END --><pre class="console">Fatal error: exception Failure("Unrecognised qtest pragma: ` T foo'")
</pre><p>You have written something like <span style="font-family:monospace">(*$ T foo</span>; there must not be any space
between <span style="font-family:monospace">(*$</span> and
the pragma.</p><pre class="console">Warning: likely qtest syntax error: `(* $T foo'. Done.
</pre><p>Self-explanatory; if <span style="font-family:monospace">$</span> is the first real character of a comment, it’s likely a mistyped
qtest pragma. This is only a warning though.</p><pre class="console">Fatal error: exception Core.Bad_header_char("M", "Mod.foo")
</pre><p>You have used a qualified name in a header, for instance <span style="font-family:monospace">(*$T Mod.foo</span>. You cannot do
that, the name must be unqualified and defined under the local scope. Furthermore, it must
be public, unless you have used pragmas to deal with private functions.</p><pre class="console">Error: Comment not terminated
Fatal error: exception Core.Unterminated_test(_, 0)
</pre><p>Most probably, you forgot the comment-closing <span style="font-family:monospace">*)</span> to close some test.</p><pre class="console">Fatal error: exception Failure("runaway test body terminator: n))*)")
</pre><p>The comment-closing <span style="font-family:monospace">*)</span> must be on a line of its own; or, put another way, every
statement must be ended by a line break.
</p>
<!--TOC subsection id="sec23" q<span style="font-style:italic">test</span> Command-Line Options-->
<h3 id="sec23" class="subsection">4.4 q<span style="font-style:italic">test</span> Command-Line Options</h3><!--SEC END --><pre class="console">$ qtest --help
** qtest (qtest)
USAGE: qtest [options] extract <file.mli?>...
OPTIONS:
--output <file.ml> (-o) def: standard output
Open or create a file for output; the resulting file will be an OCaml
source file containing all the tests.
--preamble <string> (-p) def: empty
Add code to the tests' preamble; typically this will be an instruction
of the form 'open Module;;'
--help Displays this help page and stops
</pre>
<!--TOC section id="sec24" Editor Support-->
<h2 id="sec24" class="section">5 Editor Support</h2><!--SEC END --><p>
coming soon</p><!--BEGIN NOTES document-->
<hr class="footnoterule"><dl class="thefootnotes"><dt class="dt-thefootnotes">
<a id="note1" href="#text1">*</a></dt><dd class="dd-thefootnotes"><div class="footnotetext">
This page was last updated on , ::.<br>
<br>
<br>
At the time of this document’s creation, the last commits to <a href="http://batteries.forge.ocamlcore.org">Batteries</a> concerning the documentation were:
</div></dd></dl>
<!--END NOTES-->
<!--CUT END -->
<!--HTMLFOOT-->
<!--ENDHTML-->
<!--FOOTER-->
<hr style="height:2"><blockquote class="quote"><em>This document was translated from L<sup>A</sup>T<sub>E</sub>X by
</em><a href="http://hevea.inria.fr/index.html"><em>H</em><em><span style="font-size:small"><sup>E</sup></span></em><em>V</em><em><span style="font-size:small"><sup>E</sup></span></em><em>A</em></a><em>.</em></blockquote></body>
</html>
distrib
>
Mageia
>
5
>
i586
>
media
>
core-release
>
by-pkgid
>
2726821b7228ac060f86a773c7c6078e
>
files
>
9
