<?xml version="1.0" encoding="ANSI_X3.4-1968" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=ANSI_X3.4-1968" /><title>Tear-down Races</title><meta name="generator" content="DocBook XSL Stylesheets V1.75.2" /><link rel="home" href="index.html" title="The utrace User Debugging Infrastructure" /><link rel="up" href="ch01.html" title="Chapter 1. utrace concepts" /><link rel="prev" href="ch01s03.html" title="Stopping Safely" /><link rel="next" href="ch02.html" title="Chapter 2. utrace core API" /></head><body><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Tear-down Races</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="ch01s03.html">Prev</a> </td><th width="60%" align="center">Chapter 1. utrace concepts</th><td width="20%" align="right"> <a accesskey="n" href="ch02.html">Next</a></td></tr></table><hr /></div><div class="sect1" title="Tear-down Races"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="teardown"></a>Tear-down Races</h2></div></div></div><div class="toc"><dl><dt><span class="sect2"><a href="ch01s04.html#SIGKILL">Primacy of <code class="constant">SIGKILL</code></a></span></dt><dt><span class="sect2"><a href="ch01s04.html#reap">Final callbacks</a></span></dt><dt><span class="sect2"><a href="ch01s04.html#refcount">Engine and task pointers</a></span></dt><dt><span class="sect2"><a href="ch01s04.html#reap-after-death">
Serialization of <code class="constant">DEATH</code> and <code class="constant">REAP</code>
</a></span></dt><dt><span class="sect2"><a href="ch01s04.html#interlock">Interlock with final callbacks</a></span></dt><dt><span class="sect2"><a href="ch01s04.html#barrier">Using <code class="function">utrace_barrier</code></a></span></dt></dl></div><div class="sect2" title="Primacy of SIGKILL"><div class="titlepage"><div><div><h3 class="title"><a id="SIGKILL"></a>Primacy of <code class="constant">SIGKILL</code></h3></div></div></div><p>
Ordinarily synchronization issues for tracing engines are kept fairly
straightforward by using <code class="constant">UTRACE_STOP</code>. You ask a
thread to stop, and then once it makes the
<code class="function">report_quiesce</code> callback it cannot do anything else
that would result in another callback, until you let it with a
<code class="function">utrace_control</code> call. This simple arrangement
avoids complex and error-prone code in each one of a tracing engine's
event callbacks to keep them serialized with the engine's other
operations done on that thread from another thread of control.
However, giving tracing engines complete power to keep a traced thread
stuck in place runs afoul of a more important kind of simplicity that
the kernel overall guarantees: nothing can prevent or delay
<code class="constant">SIGKILL</code> from making a thread die and release its
resources. To preserve this important property of
<code class="constant">SIGKILL</code>, it as a special case can break
<code class="constant">UTRACE_STOP</code> like nothing else normally can. This
includes both explicit <code class="constant">SIGKILL</code> signals and the
implicit <code class="constant">SIGKILL</code> sent to each other thread in the
same thread group by a thread doing an exec, or processing a fatal
signal, or making an <code class="function">exit_group</code> system call. A
tracing engine can prevent a thread from beginning the exit or exec or
dying by signal (other than <code class="constant">SIGKILL</code>) if it is
attached to that thread, but once the operation begins, no tracing
engine can prevent or delay all other threads in the same thread group
dying.
</p></div><div class="sect2" title="Final callbacks"><div class="titlepage"><div><div><h3 class="title"><a id="reap"></a>Final callbacks</h3></div></div></div><p>
The <code class="function">report_reap</code> callback is always the final event
in the life cycle of a traced thread. Tracing engines can use this as
the trigger to clean up their own data structures. The
<code class="function">report_death</code> callback is always the penultimate
event a tracing engine might see; it's seen unless the thread was
already in the midst of dying when the engine attached. Many tracing
engines will have no interest in when a parent reaps a dead process,
and nothing they want to do with a zombie thread once it dies; for
them, the <code class="function">report_death</code> callback is the natural
place to clean up data structures and detach. To facilitate writing
such engines robustly, given the asynchrony of
<code class="constant">SIGKILL</code>, and without error-prone manual
implementation of synchronization schemes, the
<span class="application">utrace</span> infrastructure provides some special
guarantees about the <code class="function">report_death</code> and
<code class="function">report_reap</code> callbacks. It still takes some care
to be sure your tracing engine is robust to tear-down races, but these
rules make it reasonably straightforward and concise to handle a lot of
corner cases correctly.
</p></div><div class="sect2" title="Engine and task pointers"><div class="titlepage"><div><div><h3 class="title"><a id="refcount"></a>Engine and task pointers</h3></div></div></div><p>
The first sort of guarantee concerns the core data structures
themselves. <span class="structname">struct utrace_engine</span> is
a reference-counted data structure. While you hold a reference, an
engine pointer will always stay valid so that you can safely pass it to
any <span class="application">utrace</span> call. Each call to
<code class="function">utrace_attach_task</code> or
<code class="function">utrace_attach_pid</code> returns an engine pointer with a
reference belonging to the caller. You own that reference until you
drop it using <code class="function">utrace_engine_put</code>. There is an
implicit reference on the engine while it is attached. So if you drop
your only reference, and then use
<code class="function">utrace_attach_task</code> without
<code class="constant">UTRACE_ATTACH_CREATE</code> to look up that same engine,
you will get the same pointer with a new reference to replace the one
you dropped, just like calling <code class="function">utrace_engine_get</code>.
When an engine has been detached, either explicitly with
<code class="constant">UTRACE_DETACH</code> or implicitly after
<code class="function">report_reap</code>, then any references you hold are all
that keep the old engine pointer alive.
</p><p>
There is nothing a kernel module can do to keep a <span class="structname">struct
task_struct</span> alive outside of
<code class="function">rcu_read_lock</code>. When the task dies and is reaped
by its parent (or itself), that structure can be freed so that any
dangling pointers you have stored become invalid.
<span class="application">utrace</span> will not prevent this, but it can
help you detect it safely. By definition, a task that has been reaped
has had all its engines detached. All
<span class="application">utrace</span> calls can be safely called on a
detached engine if the caller holds a reference on that engine pointer,
even if the task pointer passed in the call is invalid. All calls
return <code class="constant">-ESRCH</code> for a detached engine, which tells
you that the task pointer you passed could be invalid now. Since
<code class="function">utrace_control</code> and
<code class="function">utrace_set_events</code> do not block, you can call those
inside a <code class="function">rcu_read_lock</code> section and be sure after
they don't return <code class="constant">-ESRCH</code> that the task pointer is
still valid until <code class="function">rcu_read_unlock</code>. The
infrastructure never holds task references of its own. Though neither
<code class="function">rcu_read_lock</code> nor any other lock is held while
making a callback, it's always guaranteed that the <span class="structname">struct
task_struct</span> and the <span class="structname">struct
utrace_engine</span> passed as arguments remain valid
until the callback function returns.
</p><p>
The common means for safely holding task pointers that is available to
kernel modules is to use <span class="structname">struct pid</span>, which
permits <code class="function">put_pid</code> from kernel modules. When using
that, the calls <code class="function">utrace_attach_pid</code>,
<code class="function">utrace_control_pid</code>,
<code class="function">utrace_set_events_pid</code>, and
<code class="function">utrace_barrier_pid</code> are available.
</p></div><div class="sect2" title="Serialization of DEATH and REAP"><div class="titlepage"><div><div><h3 class="title"><a id="reap-after-death"></a>
Serialization of <code class="constant">DEATH</code> and <code class="constant">REAP</code>
</h3></div></div></div><p>
The second guarantee is the serialization of
<code class="constant">DEATH</code> and <code class="constant">REAP</code> event
callbacks for a given thread. The actual reaping by the parent
(<code class="function">release_task</code> call) can occur simultaneously
while the thread is still doing the final steps of dying, including
the <code class="function">report_death</code> callback. If a tracing engine
has requested both <code class="constant">DEATH</code> and
<code class="constant">REAP</code> event reports, it's guaranteed that the
<code class="function">report_reap</code> callback will not be made until
after the <code class="function">report_death</code> callback has returned.
If the <code class="function">report_death</code> callback itself detaches
from the thread, then the <code class="function">report_reap</code> callback
will never be made. Thus it is safe for a
<code class="function">report_death</code> callback to clean up data
structures and detach.
</p></div><div class="sect2" title="Interlock with final callbacks"><div class="titlepage"><div><div><h3 class="title"><a id="interlock"></a>Interlock with final callbacks</h3></div></div></div><p>
The final sort of guarantee is that a tracing engine will know for sure
whether or not the <code class="function">report_death</code> and/or
<code class="function">report_reap</code> callbacks will be made for a certain
thread. These tear-down races are disambiguated by the error return
values of <code class="function">utrace_set_events</code> and
<code class="function">utrace_control</code>. Normally
<code class="function">utrace_control</code> called with
<code class="constant">UTRACE_DETACH</code> returns zero, and this means that no
more callbacks will be made. If the thread is in the midst of dying,
it returns <code class="constant">-EALREADY</code> to indicate that the
<code class="constant">report_death</code> callback may already be in progress;
when you get this error, you know that any cleanup your
<code class="function">report_death</code> callback does is about to happen or
has just happened--note that if the <code class="function">report_death</code>
callback does not detach, the engine remains attached until the thread
gets reaped. If the thread is in the midst of being reaped,
<code class="function">utrace_control</code> returns <code class="constant">-ESRCH</code>
to indicate that the <code class="function">report_reap</code> callback may
already be in progress; this means the engine is implicitly detached
when the callback completes. This makes it possible for a tracing
engine that has decided asynchronously to detach from a thread to
safely clean up its data structures, knowing that no
<code class="function">report_death</code> or <code class="function">report_reap</code>
callback will try to do the same. <code class="constant">utrace_detach</code>
returns <code class="constant">-ESRCH</code> when the <span class="structname">struct
utrace_engine</span> has already been detached, but is
still a valid pointer because of its reference count. A tracing engine
can use this to safely synchronize its own independent multiple threads
of control with each other and with its event callbacks that detach.
</p><p>
In the same vein, <code class="function">utrace_set_events</code> normally
returns zero; if the target thread was stopped before the call, then
after a successful call, no event callbacks not requested in the new
flags will be made. It fails with <code class="constant">-EALREADY</code> if
you try to clear <code class="constant">UTRACE_EVENT(DEATH)</code> when the
<code class="function">report_death</code> callback may already have begun, if
you try to clear <code class="constant">UTRACE_EVENT(REAP)</code> when the
<code class="function">report_reap</code> callback may already have begun, or if
you try to newly set <code class="constant">UTRACE_EVENT(DEATH)</code> or
<code class="constant">UTRACE_EVENT(QUIESCE)</code> when the target is already
dead or dying. Like <code class="function">utrace_control</code>, it returns
<code class="constant">-ESRCH</code> when the thread has already been detached
(including forcible detach on reaping). This lets the tracing engine
know for sure which event callbacks it will or won't see after
<code class="function">utrace_set_events</code> has returned. By checking for
errors, it can know whether to clean up its data structures immediately
or to let its callbacks do the work.
</p></div><div class="sect2" title="Using utrace_barrier"><div class="titlepage"><div><div><h3 class="title"><a id="barrier"></a>Using <code class="function">utrace_barrier</code></h3></div></div></div><p>
When a thread is safely stopped, calling
<code class="function">utrace_control</code> with <code class="constant">UTRACE_DETACH</code>
or calling <code class="function">utrace_set_events</code> to disable some events
ensures synchronously that your engine won't get any more of the callbacks
that have been disabled (none at all when detaching). But these can also
be used while the thread is not stopped, when it might be simultaneously
making a callback to your engine. For this situation, these calls return
<code class="constant">-EINPROGRESS</code> when it's possible a callback is in
progress. If you are not prepared to have your old callbacks still run,
then you can synchronize to be sure all the old callbacks are finished,
using <code class="function">utrace_barrier</code>. This is necessary if the
kernel module containing your callback code is going to be unloaded.
</p><p>
After using <code class="constant">UTRACE_DETACH</code> once, further calls to
<code class="function">utrace_control</code> with the same engine pointer will
return <code class="constant">-ESRCH</code>. In contrast, after getting
<code class="constant">-EINPROGRESS</code> from
<code class="function">utrace_set_events</code>, you can call
<code class="function">utrace_set_events</code> again later and if it returns zero
then know the old callbacks have finished.
</p><p>
Unlike all other calls, <code class="function">utrace_barrier</code> (and
<code class="function">utrace_barrier_pid</code>) will accept any engine pointer you
hold a reference on, even if <code class="constant">UTRACE_DETACH</code> has already
been used. After any <code class="function">utrace_control</code> or
<code class="function">utrace_set_events</code> call (these do not block), you can
call <code class="function">utrace_barrier</code> to block until callbacks have
finished. This returns <code class="constant">-ESRCH</code> only if the engine is
completely detached (finished all callbacks). Otherwise it waits
until the thread is definitely not in the midst of a callback to this
engine and then returns zero, but can return
<code class="constant">-ERESTARTSYS</code> if its wait is interrupted.
</p></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="ch01s03.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="ch01.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="ch02.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">Stopping Safely </td><td width="20%" align="center"><a accesskey="h" href="index.html">Home</a></td><td width="40%" align="right" valign="top"> Chapter 2. utrace core API</td></tr></table></div></body></html>
