<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Connection Diagrams of The IPython ZMQ Cluster — IPython 2.3.0 documentation</title>
<link rel="stylesheet" href="../_static/default.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '2.3.0',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="top" title="IPython 2.3.0 documentation" href="../index.html" />
<link rel="up" title="IPython developer’s guide" href="index.html" />
<link rel="next" title="New IPython Console Lexer" href="lexer.html" />
<link rel="prev" title="Messaging for Parallel Computing" href="parallel_messages.html" />
</head>
<body>
<div style="background-color: white; text-align: left; padding: 10px 10px 15px 15px">
<a href="http://ipython.org/"><img src="../_static/logo.png" border="0" alt="IPython Documentation"/></a>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="lexer.html" title="New IPython Console Lexer"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="parallel_messages.html" title="Messaging for Parallel Computing"
accesskey="P">previous</a> |</li>
<li><a href="http://ipython.org">home</a>| </li>
<li><a href="../search.html">search</a>| </li>
<li><a href="../index.html">documentation </a> »</li>
<li><a href="index.html" accesskey="U">IPython developer’s guide</a> »</li>
</ul>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Connection Diagrams of The IPython ZMQ Cluster</a><ul>
<li><a class="reference internal" href="#all-connections">All Connections</a><ul>
<li><a class="reference internal" href="#registration">Registration</a></li>
<li><a class="reference internal" href="#heartbeat">Heartbeat</a></li>
<li><a class="reference internal" href="#schedulers">Schedulers</a></li>
<li><a class="reference internal" href="#iopub">IOPub</a></li>
<li><a class="reference internal" href="#client-connections">Client connections</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="parallel_messages.html"
title="previous chapter">Messaging for Parallel Computing</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="lexer.html"
title="next chapter">New IPython Console Lexer</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/development/parallel_connections.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="connection-diagrams-of-the-ipython-zmq-cluster">
<span id="parallel-connections"></span><h1>Connection Diagrams of The IPython ZMQ Cluster<a class="headerlink" href="#connection-diagrams-of-the-ipython-zmq-cluster" title="Permalink to this headline">¶</a></h1>
<p>This is a quick summary and illustration of the connections involved in the ZeroMQ based
IPython cluster for parallel computing.</p>
<div class="section" id="all-connections">
<h2>All Connections<a class="headerlink" href="#all-connections" title="Permalink to this headline">¶</a></h2>
<p>The IPython cluster consists of a Controller, and one or more each of clients and engines.
The goal of the Controller is to manage and monitor the connections and communications
between the clients and the engines. The Controller is no longer a single process entity,
but rather a collection of processes - specifically one Hub, and 4 (or more) Schedulers.</p>
<p>It is important for security/practicality reasons that all connections be inbound to the
controller processes. The arrows in the figures indicate the direction of the
connection.</p>
<div class="figure align-center">
<a class="reference internal image-reference" href="../_images/allconnections.png"><img alt="IPython cluster connections" src="../_images/allconnections.png" style="width: 432px;" /></a>
<p class="caption">All the connections involved in connecting one client to one engine.</p>
</div>
<p>The Controller consists of 1-5 processes. Central to the cluster is the <strong>Hub</strong>, which monitors
engine state, execution traffic, and handles registration and notification. The Hub includes a
Heartbeat Monitor for keeping track of engines that are alive. Outside the Hub are 4
<strong>Schedulers</strong>. These devices are very small pure-C MonitoredQueue processes (or optionally
threads) that relay messages very fast, but also send a copy of each message along a side socket
to the Hub. The MUX queue and Control queue are MonitoredQueue ØMQ devices which relay
explicitly addressed messages from clients to engines, and their replies back up. The Balanced
queue performs load-balancing destination-agnostic scheduling. It may be a MonitoredQueue
device, but may also be a Python Scheduler that behaves externally in an identical fashion to MQ
devices, but with additional internal logic. stdout/err are also propagated from the Engines to
the clients via a PUB/SUB MonitoredQueue.</p>
<div class="section" id="registration">
<h3>Registration<a class="headerlink" href="#registration" title="Permalink to this headline">¶</a></h3>
<div class="figure align-center">
<a class="reference internal image-reference" href="../_images/queryfade.png"><img alt="IPython Registration connections" src="../_images/queryfade.png" style="width: 432px;" /></a>
<p class="caption">Engines and Clients only need to know where the Query <tt class="docutils literal"><span class="pre">ROUTER</span></tt> is located to start
connecting.</p>
</div>
<p>Once a controller is launched, the only information needed for connecting clients and/or
engines is the IP/port of the Hub’s <tt class="docutils literal"><span class="pre">ROUTER</span></tt> socket called the Registrar. This socket
handles connections from both clients and engines, and replies with the remaining
information necessary to establish the remaining connections. Clients use this same socket for
querying the Hub for state information.</p>
</div>
<div class="section" id="heartbeat">
<h3>Heartbeat<a class="headerlink" href="#heartbeat" title="Permalink to this headline">¶</a></h3>
<div class="figure align-center">
<a class="reference internal image-reference" href="../_images/hbfade.png"><img alt="IPython Heartbeat connections" src="../_images/hbfade.png" style="width: 432px;" /></a>
<p class="caption">The heartbeat sockets.</p>
</div>
<p>The heartbeat process has been described elsewhere. To summarize: the Heartbeat Monitor
publishes a distinct message periodically via a <tt class="docutils literal"><span class="pre">PUB</span></tt> socket. Each engine has a
<tt class="docutils literal"><span class="pre">zmq.FORWARDER</span></tt> device with a <tt class="docutils literal"><span class="pre">SUB</span></tt> socket for input, and <tt class="docutils literal"><span class="pre">DEALER</span></tt> socket for output.
The <tt class="docutils literal"><span class="pre">SUB</span></tt> socket is connected to the <tt class="docutils literal"><span class="pre">PUB</span></tt> socket labeled <em>ping</em>, and the <tt class="docutils literal"><span class="pre">DEALER</span></tt> is
connected to the <tt class="docutils literal"><span class="pre">ROUTER</span></tt> labeled <em>pong</em>. This results in the same message being relayed
back to the Heartbeat Monitor with the addition of the <tt class="docutils literal"><span class="pre">DEALER</span></tt> prefix. The Heartbeat
Monitor receives all the replies via an <tt class="docutils literal"><span class="pre">ROUTER</span></tt> socket, and identifies which hearts are
still beating by the <tt class="docutils literal"><span class="pre">zmq.IDENTITY</span></tt> prefix of the <tt class="docutils literal"><span class="pre">DEALER</span></tt> sockets, which information
the Hub uses to notify clients of any changes in the available engines.</p>
</div>
<div class="section" id="schedulers">
<h3>Schedulers<a class="headerlink" href="#schedulers" title="Permalink to this headline">¶</a></h3>
<div class="figure align-center">
<a class="reference internal image-reference" href="../_images/queuefade.png"><img alt="IPython Queue connections" src="../_images/queuefade.png" style="width: 432px;" /></a>
<p class="caption">Control message scheduler on the left, execution (apply) schedulers on the right.</p>
</div>
<p>The controller has at least three Schedulers. These devices are primarily for
relaying messages between clients and engines, but the Hub needs to see those
messages for its own purposes. Since no Python code may exist between the two sockets in a
queue, all messages sent through these queues (both directions) are also sent via a
<tt class="docutils literal"><span class="pre">PUB</span></tt> socket to a monitor, which allows the Hub to monitor queue traffic without
interfering with it.</p>
<p>For tasks, the engine need not be specified. Messages sent to the <tt class="docutils literal"><span class="pre">ROUTER</span></tt> socket from the
client side are assigned to an engine via ZMQ’s <tt class="docutils literal"><span class="pre">DEALER</span></tt> round-robin load balancing.
Engine replies are directed to specific clients via the IDENTITY of the client, which is
received as a prefix at the Engine.</p>
<p>For Multiplexing, <tt class="docutils literal"><span class="pre">ROUTER</span></tt> is used for both in and output sockets in the device. Clients must
specify the destination by the <tt class="docutils literal"><span class="pre">zmq.IDENTITY</span></tt> of the <tt class="docutils literal"><span class="pre">ROUTER</span></tt> socket connected to
the downstream end of the device.</p>
<p>At the Kernel level, both of these <tt class="docutils literal"><span class="pre">ROUTER</span></tt> sockets are treated in the same way as the <tt class="docutils literal"><span class="pre">REP</span></tt>
socket in the serial version (except using ZMQStreams instead of explicit sockets).</p>
<p>Execution can be done in a load-balanced (engine-agnostic) or multiplexed (engine-specified)
manner. The sockets on the Client and Engine are the same for these two actions, but the
scheduler used determines the actual behavior. This routing is done via the <tt class="docutils literal"><span class="pre">zmq.IDENTITY</span></tt> of
the upstream sockets in each MonitoredQueue.</p>
</div>
<div class="section" id="iopub">
<h3>IOPub<a class="headerlink" href="#iopub" title="Permalink to this headline">¶</a></h3>
<div class="figure align-center">
<a class="reference internal image-reference" href="../_images/iopubfade.png"><img alt="IOPub connections" src="../_images/iopubfade.png" style="width: 432px;" /></a>
<p class="caption">stdout/err are published via a <tt class="docutils literal"><span class="pre">PUB/SUB</span></tt> MonitoredQueue</p>
</div>
<p>On the kernels, stdout/stderr are captured and published via a <tt class="docutils literal"><span class="pre">PUB</span></tt> socket. These <tt class="docutils literal"><span class="pre">PUB</span></tt>
sockets all connect to a <tt class="docutils literal"><span class="pre">SUB</span></tt> socket input of a MonitoredQueue, which subscribes to all
messages. They are then republished via another <tt class="docutils literal"><span class="pre">PUB</span></tt> socket, which can be
subscribed by the clients.</p>
</div>
<div class="section" id="client-connections">
<h3>Client connections<a class="headerlink" href="#client-connections" title="Permalink to this headline">¶</a></h3>
<div class="figure align-center">
<a class="reference internal image-reference" href="../_images/queryfade.png"><img alt="IPython client query connections" src="../_images/queryfade.png" style="width: 432px;" /></a>
<p class="caption">Clients connect to an <tt class="docutils literal"><span class="pre">ROUTER</span></tt> socket to query the hub.</p>
</div>
<p>The hub’s registrar <tt class="docutils literal"><span class="pre">ROUTER</span></tt> socket also listens for queries from clients as to queue status,
and control instructions. Clients connect to this socket via an <tt class="docutils literal"><span class="pre">DEALER</span></tt> during registration.</p>
<div class="figure align-center">
<a class="reference internal image-reference" href="../_images/notiffade.png"><img alt="IPython Registration connections" src="../_images/notiffade.png" style="width: 432px;" /></a>
<p class="caption">Engine registration events are published via a <tt class="docutils literal"><span class="pre">PUB</span></tt> socket.</p>
</div>
<p>The Hub publishes all registration/unregistration events via a <tt class="docutils literal"><span class="pre">PUB</span></tt> socket. This
allows clients to stay up to date with what engines are available by subscribing to the
feed with a <tt class="docutils literal"><span class="pre">SUB</span></tt> socket. Other processes could selectively subscribe to just
registration or unregistration events.</p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="lexer.html" title="New IPython Console Lexer"
>next</a> |</li>
<li class="right" >
<a href="parallel_messages.html" title="Messaging for Parallel Computing"
>previous</a> |</li>
<li><a href="http://ipython.org">home</a>| </li>
<li><a href="../search.html">search</a>| </li>
<li><a href="../index.html">documentation </a> »</li>
<li><a href="index.html" >IPython developer’s guide</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright The IPython Development Team.
Last updated on Sep 02, 2015.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.2.3.
</div>
</body>
</html>
distrib
>
Mageia
>
5
>
x86_64
>
media
>
core-updates
>
by-pkgid
>
c044f82ec6193fba7e13c97913613b07
>
files
>
949
