<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Installation — Buildbot 0.8.9 documentation</title>
<link rel="stylesheet" href="../_static/agogo.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '0.8.9',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="shortcut icon" href="../_static/buildbot.ico"/>
<link rel="top" title="Buildbot 0.8.9 documentation" href="../index.html" />
<link rel="up" title="Buildbot Manual" href="index.html" />
<link rel="next" title="Concepts" href="concepts.html" />
<link rel="prev" title="Introduction" href="introduction.html" />
</head>
<body>
<div class="header-wrapper">
<div class="header">
<p class="logo"><a href="../index.html">
<img class="logo" src="../_static/header-text-transparent.png" alt="Logo"/>
</a></p>
<div class="headertitle"><a
href="../index.html">Buildbot 0.8.9 documentation</a></div>
<div class="rel">
<a href="introduction.html" title="Introduction"
accesskey="P">previous</a> |
<a href="concepts.html" title="Concepts"
accesskey="N">next</a> |
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |
<a href="../genindex.html" title="General Index"
accesskey="I">index</a>
</div>
</div>
</div>
<div class="content-wrapper">
<div class="content">
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="installation">
<h1>Installation<a class="headerlink" href="#installation" title="Permalink to this headline">¶</a></h1>
<div class="section" id="buildbot-components">
<span id="id1"></span><h2>Buildbot Components<a class="headerlink" href="#buildbot-components" title="Permalink to this headline">¶</a></h2>
<p>Buildbot is shipped in two components: the <em>buildmaster</em> (called <tt class="docutils literal"><span class="pre">buildbot</span></tt>
for legacy reasons) and the <em>buildslave</em>. The buildslave component has far fewer
requirements, and is more broadly compatible than the buildmaster. You will
need to carefully pick the environment in which to run your buildmaster, but
the buildslave should be able to run just about anywhere.</p>
<p>It is possible to install the buildmaster and buildslave on the same system,
although for anything but the smallest installation this arrangement will not
be very efficient.</p>
</div>
<div class="section" id="requirements">
<span id="id2"></span><h2>Requirements<a class="headerlink" href="#requirements" title="Permalink to this headline">¶</a></h2>
<div class="section" id="common-requirements">
<span id="id3"></span><h3>Common Requirements<a class="headerlink" href="#common-requirements" title="Permalink to this headline">¶</a></h3>
<p>At a bare minimum, you'll need the following for both the buildmaster and a
buildslave:</p>
<p>Python: <a class="reference external" href="http://www.python.org">http://www.python.org</a></p>
<blockquote>
<div>Buildbot requires Python-2.5 or later on the master, although Python-2.7 is
recommended. The slave run on Python-2.4.</div></blockquote>
<p>Twisted: <a class="reference external" href="http://twistedmatrix.com">http://twistedmatrix.com</a></p>
<blockquote>
<div><p>Buildbot requires Twisted-11.0.0 or later on the master, and Twisted-8.1.0 on the slave.
In upcoming versions of Buildbot, a newer Twisted will also be required on the slave.
As always, the most recent version is recommended.</p>
<p>In some cases, Twisted is delivered as a collection of subpackages. You'll
need at least "Twisted" (the core package), and you'll also want
<a class="reference external" href="http://twistedmatrix.com/trac/wiki/TwistedMail">TwistedMail</a>, <a class="reference external" href="http://twistedmatrix.com/trac/wiki/TwistedWeb">TwistedWeb</a>, and <a class="reference external" href="http://twistedmatrix.com/trac/wiki/TwistedWords">TwistedWords</a> (for sending email,
serving a web status page, and delivering build status via IRC,
respectively). You might also want <a class="reference external" href="http://twistedmatrix.com/trac/wiki/TwistedConch">TwistedConch</a> (for the encrypted Manhole
debug port). Note that Twisted requires ZopeInterface to be installed as
well.</p>
</div></blockquote>
<p>Of course, your project's build process will impose additional
requirements on the buildslaves. These hosts must have all the tools
necessary to compile and test your project's source code.</p>
<div class="section" id="windows-support">
<h4>Windows Support<a class="headerlink" href="#windows-support" title="Permalink to this headline">¶</a></h4>
<p>Buildbot - both master and slave - runs well natively on Windows. The slave runs
well on Cygwin, but because of problems with SQLite on Cygwin, the master does
not.</p>
<p>Buildbot's windows testing is limited to the most recent Twisted and Python
versions. For best results, use the most recent available versions of these
libraries on Windows.</p>
<p>Pywin32: <a class="reference external" href="http://sourceforge.net/projects/pywin32/">http://sourceforge.net/projects/pywin32/</a></p>
<blockquote>
<div>Twisted requires PyWin32 in order to spawn processes on Windows.</div></blockquote>
</div>
</div>
<div class="section" id="buildmaster-requirements">
<span id="id4"></span><h3>Buildmaster Requirements<a class="headerlink" href="#buildmaster-requirements" title="Permalink to this headline">¶</a></h3>
<p>sqlite3: <a class="reference external" href="http://www.sqlite.org">http://www.sqlite.org</a></p>
<blockquote>
<div>Buildbot requires SQLite to store its state. Version 3.7.0 or higher is
recommended, although Buildbot will run against earlier versions -- at the
risk of "Database is locked" errors. The minimum version is 3.4.0, below
which parallel database queries and schema introspection fail.</div></blockquote>
<p>pysqlite: <a class="reference external" href="http://pypi.python.org/pypi/pysqlite">http://pypi.python.org/pypi/pysqlite</a></p>
<blockquote>
<div>The SQLite Python package is required for Python-2.5 and earlier (it is already
included in Python-2.5 and later, but the version in Python-2.5 has nasty bugs)</div></blockquote>
<p>simplejson: <a class="reference external" href="http://pypi.python.org/pypi/simplejson">http://pypi.python.org/pypi/simplejson</a></p>
<blockquote>
<div>The simplejson package is required for Python-2.5 and earlier (it is already
included as json in Python-2.6 and later)</div></blockquote>
<p>Jinja2: <a class="reference external" href="http://jinja.pocoo.org/">http://jinja.pocoo.org/</a></p>
<blockquote>
<div><p>Buildbot requires Jinja version 2.1 or higher.</p>
<p>Jinja2 is a general purpose templating language and is used by Buildbot
to generate the HTML output.</p>
</div></blockquote>
<p>SQLAlchemy: <a class="reference external" href="http://www.sqlalchemy.org/">http://www.sqlalchemy.org/</a></p>
<blockquote>
<div>Buildbot requires SQLAlchemy 0.6.0 or higher. SQLAlchemy allows Buildbot to
build database schemas and queries for a wide variety of database systems.</div></blockquote>
<p>SQLAlchemy-Migrate: <a class="reference external" href="http://code.google.com/p/sqlalchemy-migrate/">http://code.google.com/p/sqlalchemy-migrate/</a></p>
<blockquote>
<div>Buildbot requires one of the following SQLAlchemy-Migrate versions:
0.6.1, 0.7.0, and 0.7.1. Sadly, Migrate's inter-version compatibility is not
good, so other versions - newer or older - are unlikely to work correctly.
Buildbot uses SQLAlchemy-Migrate to manage schema upgrades from version to
version.</div></blockquote>
<p>Python-Dateutil: <a class="reference external" href="http://labix.org/python-dateutil">http://labix.org/python-dateutil</a></p>
<blockquote>
<div>The Nightly scheduler requires Python-Dateutil version 1.5 (the last version
to support Python-2.x). This is a small, pure-python library. Buildbot will
function properly without it if the Nightlys scheduler is not used.</div></blockquote>
</div>
</div>
<div class="section" id="installing-the-code">
<span id="id5"></span><h2>Installing the code<a class="headerlink" href="#installing-the-code" title="Permalink to this headline">¶</a></h2>
<div class="section" id="the-distribution-package">
<h3>The Distribution Package<a class="headerlink" href="#the-distribution-package" title="Permalink to this headline">¶</a></h3>
<p>Buildbot comes in two parts: <tt class="docutils literal"><span class="pre">buildbot</span></tt> (the master) and
<tt class="docutils literal"><span class="pre">buildbot-slave</span></tt> (the slave). The two can be installed individually or
together.</p>
</div>
<div class="section" id="installation-from-pypi">
<h3>Installation From PyPI<a class="headerlink" href="#installation-from-pypi" title="Permalink to this headline">¶</a></h3>
<p>The preferred way to install Buildbot is using <tt class="docutils literal"><span class="pre">pip</span></tt>. For the master:</p>
<div class="highlight-bash"><div class="highlight"><pre>pip install buildbot
</pre></div>
</div>
<p>and for the slave:</p>
<div class="highlight-bash"><div class="highlight"><pre>pip install buildbot-slave
</pre></div>
</div>
<p>When using <tt class="docutils literal"><span class="pre">pip</span></tt> to install instead of distribution specific package manangers,
e.g. via <cite>apt-get</cite> or <cite>ports</cite>, it is simpler to choose exactly which version one wants
to use. It may however be easier to install via distribution specific package mangers
but note that they may provide an earlier version than what is available via <tt class="docutils literal"><span class="pre">pip</span></tt>.</p>
</div>
<div class="section" id="installation-from-tarballs">
<h3>Installation From Tarballs<a class="headerlink" href="#installation-from-tarballs" title="Permalink to this headline">¶</a></h3>
<p>Buildbot and Buildslave are installed using the standard Python
<a class="reference external" href="http://docs.python.org/library/distutils.html">distutils</a> process. For either
component, after unpacking the tarball, the process is:</p>
<div class="highlight-bash"><div class="highlight"><pre>python setup.py build
python setup.py install
</pre></div>
</div>
<p>where the install step may need to be done as root. This will put the bulk of
the code in somewhere like <tt class="file docutils literal"><span class="pre">/usr/lib/pythonx.y/site-packages/buildbot</span></tt>. It
will also install the <strong class="command">buildbot</strong> command-line tool in
<tt class="file docutils literal"><span class="pre">/usr/bin/buildbot</span></tt>.</p>
<p>If the environment variable <tt class="docutils literal"><span class="pre">$NO_INSTALL_REQS</span></tt> is set to <tt class="docutils literal"><span class="pre">1</span></tt>, then
<tt class="file docutils literal"><span class="pre">setup.py</span></tt> will not try to install Buildbot's requirements. This is
usually only useful when building a Buildbot package.</p>
<p>To test this, shift to a different directory (like <tt class="file docutils literal"><span class="pre">/tmp</span></tt>), and run:</p>
<div class="highlight-bash"><div class="highlight"><pre>buildbot --version
<span class="c"># or</span>
buildslave --version
</pre></div>
</div>
<p>If it shows you the versions of Buildbot and Twisted, the install went
ok. If it says "no such command" or it gets an <tt class="docutils literal"><span class="pre">ImportError</span></tt>
when it tries to load the libraries, then something went wrong.
<tt class="docutils literal"><span class="pre">pydoc</span> <span class="pre">buildbot</span></tt> is another useful diagnostic tool.</p>
<p>Windows users will find these files in other places. You will need to
make sure that Python can find the libraries, and will probably find
it convenient to have <strong class="command">buildbot</strong> on your <span class="target" id="index-0"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PATH</span></tt>.</p>
</div>
<div class="section" id="installation-in-a-virtualenv">
<span id="id6"></span><h3>Installation in a Virtualenv<a class="headerlink" href="#installation-in-a-virtualenv" title="Permalink to this headline">¶</a></h3>
<p>If you cannot or do not wish to install the buildbot into a site-wide
location like <tt class="file docutils literal"><span class="pre">/usr</span></tt> or <tt class="file docutils literal"><span class="pre">/usr/local</span></tt>, you can also install
it into the account's home directory or any other location using a tool like
<a class="reference external" href="http://pypi.python.org/pypi/virtualenv">virtualenv</a>.</p>
</div>
</div>
<div class="section" id="running-buildbot-s-tests-optional">
<span id="running-buildbots-tests-optional"></span><h2>Running Buildbot's Tests (optional)<a class="headerlink" href="#running-buildbot-s-tests-optional" title="Permalink to this headline">¶</a></h2>
<p>If you wish, you can run the buildbot unit test suite. First, ensure you have
the <a class="reference external" href="http://pypi.python.org/pypi/mock">mock</a> Python module installed from
PyPi. This module is not required for ordinary Buildbot operation - only to
run the tests. Note that this is not the same as the Fedora <tt class="docutils literal"><span class="pre">mock</span></tt>
package! You can check with</p>
<div class="highlight-bash"><div class="highlight"><pre>python -mmock
</pre></div>
</div>
<p>Then, run the tests:</p>
<div class="highlight-bash"><div class="highlight"><pre><span class="nv">PYTHONPATH</span><span class="o">=</span>. trial buildbot.test
<span class="c"># or</span>
<span class="nv">PYTHONPATH</span><span class="o">=</span>. trial buildslave.test
</pre></div>
</div>
<p>Nothing should fail, although a few might be skipped.</p>
<p>If any of the tests fail for reasons other than a missing <tt class="docutils literal"><span class="pre">mock</span></tt>, you
should stop and investigate the cause before continuing the installation
process, as it will probably be easier to track down the bug early. In most
cases, the problem is incorrectly installed Python modules or a badly
configured <tt class="docutils literal"><span class="pre">PYTHONPATH</span></tt>. This may be a good time to contact the Buildbot
developers for help.</p>
</div>
<div class="section" id="creating-a-buildmaster">
<span id="id7"></span><h2>Creating a buildmaster<a class="headerlink" href="#creating-a-buildmaster" title="Permalink to this headline">¶</a></h2>
<p>As you learned earlier (<a class="reference internal" href="introduction.html#system-architecture"><em>System Architecture</em></a>), the buildmaster
runs on a central host (usually one that is publicly visible, so
everybody can check on the status of the project), and controls all
aspects of the buildbot system</p>
<p>You will probably wish to create a separate user account for the buildmaster,
perhaps named <tt class="docutils literal"><span class="pre">buildmaster</span></tt>. Do not run the buildmaster as <tt class="docutils literal"><span class="pre">root</span></tt>!</p>
<p>You need to choose a directory for the buildmaster, called the
<tt class="docutils literal"><span class="pre">basedir</span></tt>. This directory will be owned by the buildmaster. It will
contain configuration, the database, and status information - including
logfiles. On a large buildmaster this directory will see a lot of activity, so
it should be on a disk with adequate space and speed.</p>
<p>Once you've picked a directory, use the <tt class="docutils literal"><span class="pre">buildbot</span>
<span class="pre">create-master</span></tt> command to create the directory and populate it with
startup files:</p>
<div class="highlight-bash"><div class="highlight"><pre>buildbot create-master -r basedir
</pre></div>
</div>
<p>You will need to create a <a class="reference internal" href="configuration.html#configuration"><em>configuration file</em></a>
before starting the buildmaster. Most of the rest of this manual is
dedicated to explaining how to do this. A sample configuration file is
placed in the working directory, named <tt class="file docutils literal"><span class="pre">master.cfg.sample</span></tt>, which
can be copied to <tt class="file docutils literal"><span class="pre">master.cfg</span></tt> and edited to suit your purposes.</p>
<p>(Internal details: This command creates a file named
<tt class="file docutils literal"><span class="pre">buildbot.tac</span></tt> that contains all the state necessary to create
the buildmaster. Twisted has a tool called <tt class="docutils literal"><span class="pre">twistd</span></tt> which can use
this .tac file to create and launch a buildmaster instance. twistd
takes care of logging and daemonization (running the program in the
background). <tt class="file docutils literal"><span class="pre">/usr/bin/buildbot</span></tt> is a front end which runs <cite>twistd</cite>
for you.)</p>
<div class="section" id="using-a-database-server">
<h3>Using A Database Server<a class="headerlink" href="#using-a-database-server" title="Permalink to this headline">¶</a></h3>
<p>If you want to use a database server (e.g., MySQL or Postgres) as the database
backend for your Buildbot, add the <tt class="docutils literal"><span class="pre">--db</span></tt> option to the <tt class="docutils literal"><span class="pre">create-master</span></tt>
invocation to specify the <a class="reference internal" href="cfg-global.html#database-specification"><em>connection string</em></a> for
the database, and make sure that the same URL appears in the <tt class="docutils literal"><span class="pre">db_url</span></tt> of the
<a class="reference internal" href="cfg-global.html#cfg-db" title="db"><tt class="xref bb bb-cfg docutils literal"><span class="pre">db</span></tt></a> parameter in your configuration file.</p>
<div class="section" id="additional-requirements">
<h4>Additional Requirements<a class="headerlink" href="#additional-requirements" title="Permalink to this headline">¶</a></h4>
<p>Depending on the selected database, further Python packages will be required.
Consult the SQLAlchemy dialect list for a full description. The most common
choice for MySQL is</p>
<p>MySQL-python: <a class="reference external" href="http://mysql-python.sourceforge.net/">http://mysql-python.sourceforge.net/</a></p>
<blockquote>
<div>To communicate with MySQL, SQLAlchemy requires MySQL-python. Any reasonably
recent version of MySQL-python should suffice.</div></blockquote>
<p>The most common choice for Postgres is</p>
<p>Psycopg: <a class="reference external" href="http://initd.org/psycopg/">http://initd.org/psycopg/</a></p>
<blockquote>
<div>SQLAlchemy uses Psycopg to communicate with Postgres. Any reasonably
recent version should suffice.</div></blockquote>
</div>
</div>
<div class="section" id="buildmaster-options">
<h3>Buildmaster Options<a class="headerlink" href="#buildmaster-options" title="Permalink to this headline">¶</a></h3>
<p>This section lists options to the <tt class="docutils literal"><span class="pre">create-master</span></tt> command.
You can also type <tt class="docutils literal"><span class="pre">buildbot</span> <span class="pre">create-master</span> <span class="pre">--help</span></tt> for an up-to-the-moment summary.</p>
<p><tt class="docutils literal"><span class="pre">--force</span></tt></p>
<blockquote>
<div>With this option, @command{create-master} will re-use an existing master
directory.</div></blockquote>
<p><tt class="docutils literal"><span class="pre">--no-logrotate</span></tt></p>
<blockquote>
<div>This disables internal buildslave log management mechanism. With this option
buildslave does not override the default logfile name and its behaviour giving
a possibility to control those with command-line options of twistd daemon.</div></blockquote>
<p><tt class="docutils literal"><span class="pre">--relocatable</span></tt></p>
<blockquote>
<div>This creates a "relocatable" buildbot.tac, which uses relative paths instead
of absolute paths, so that the buildmaster directory can be moved about.</div></blockquote>
<p><tt class="docutils literal"><span class="pre">--config</span></tt></p>
<blockquote>
<div>The name of the configuration file to use. This configuration file need not
reside in the buildmaster directory.</div></blockquote>
<p><tt class="docutils literal"><span class="pre">--log-size</span></tt></p>
<blockquote>
<div>This is the size in bytes when to rotate the Twisted log files. The default is
10MiB.</div></blockquote>
<p><tt class="docutils literal"><span class="pre">--log-count</span></tt></p>
<blockquote>
<div>This is the number of log rotations to keep around. You can either
specify a number or <tt class="docutils literal"><span class="pre">None</span></tt> to keep all @file{twistd.log} files
around. The default is 10.</div></blockquote>
<p><tt class="docutils literal"><span class="pre">--db</span></tt></p>
<blockquote>
<div>The database that the Buildmaster should use. Note that the same value must be
added to the configuration file.</div></blockquote>
</div>
</div>
<div class="section" id="upgrading-an-existing-buildmaster">
<span id="id8"></span><h2>Upgrading an Existing Buildmaster<a class="headerlink" href="#upgrading-an-existing-buildmaster" title="Permalink to this headline">¶</a></h2>
<p>If you have just installed a new version of the Buildbot code, and you
have buildmasters that were created using an older version, you'll
need to upgrade these buildmasters before you can use them. The
upgrade process adds and modifies files in the buildmaster's base
directory to make it compatible with the new code.</p>
<div class="highlight-bash"><div class="highlight"><pre>buildbot upgrade-master basedir
</pre></div>
</div>
<p>This command will also scan your <tt class="file docutils literal"><span class="pre">master.cfg</span></tt> file for
incompatibilities (by loading it and printing any errors or deprecation
warnings that occur). Each buildbot release tries to be compatible
with configurations that worked cleanly (i.e. without deprecation
warnings) on the previous release: any functions or classes that are
to be removed will first be deprecated in a release, to give you a
chance to start using the replacement.</p>
<p>The <tt class="docutils literal"><span class="pre">upgrade-master</span></tt> command is idempotent. It is safe to run it
multiple times. After each upgrade of the buildbot code, you should
use <tt class="docutils literal"><span class="pre">upgrade-master</span></tt> on all your buildmasters.</p>
<p>In general, Buildbot slaves and masters can be upgraded independently, although
some new features will not be available, depending on the master and slave
versions.</p>
<p>Beyond this general information, read all of the sections below that apply to
versions through which you are upgrading.</p>
<div class="section" id="version-specific-notes">
<span id="buildmaster-version-specific-notes"></span><h3>Version-specific Notes<a class="headerlink" href="#version-specific-notes" title="Permalink to this headline">¶</a></h3>
<div class="section" id="upgrading-a-buildmaster-to-buildbot-0-7-6">
<h4>Upgrading a Buildmaster to Buildbot-0.7.6<a class="headerlink" href="#upgrading-a-buildmaster-to-buildbot-0-7-6" title="Permalink to this headline">¶</a></h4>
<p>The 0.7.6 release introduced the <tt class="file docutils literal"><span class="pre">public_html/</span></tt> directory, which
contains <tt class="file docutils literal"><span class="pre">index.html</span></tt> and other files served by the
<tt class="docutils literal"><span class="pre">WebStatus</span></tt> and <tt class="docutils literal"><span class="pre">Waterfall</span></tt> status displays. The
<tt class="docutils literal"><span class="pre">upgrade-master</span></tt> command will create these files if they do not
already exist. It will not modify existing copies, but it will write a
new copy in e.g. <tt class="file docutils literal"><span class="pre">index.html.new</span></tt> if the new version differs from
the version that already exists.</p>
</div>
<div class="section" id="upgrading-a-buildmaster-to-buildbot-0-8-0">
<h4>Upgrading a Buildmaster to Buildbot-0.8.0<a class="headerlink" href="#upgrading-a-buildmaster-to-buildbot-0-8-0" title="Permalink to this headline">¶</a></h4>
<p>Buildbot-0.8.0 introduces a database backend, which is SQLite by default. The
<tt class="docutils literal"><span class="pre">upgrade-master</span></tt> command will automatically create and populate this
database with the changes the buildmaster has seen. Note that, as of this
release, build history is <em>not</em> contained in the database, and is thus not
migrated.</p>
<p>The upgrade process renames the Changes pickle (<tt class="docutils literal"><span class="pre">$basedir/changes.pck</span></tt>) to
<tt class="docutils literal"><span class="pre">changes.pck.old</span></tt> once the upgrade is complete. To reverse the upgrade,
simply downgrade Buildbot and move this file back to its original name. You
may also wish to delete the state database (<tt class="docutils literal"><span class="pre">state.sqlite</span></tt>).</p>
</div>
<div class="section" id="upgrading-into-a-non-sqlite-database">
<h4>Upgrading into a non-SQLite database<a class="headerlink" href="#upgrading-into-a-non-sqlite-database" title="Permalink to this headline">¶</a></h4>
<p>If you are not using sqlite, you will need to add an entry into your
<tt class="file docutils literal"><span class="pre">master.cfg</span></tt> to reflect the database version you are using. The upgrade
process does <em>not</em> edit your <tt class="file docutils literal"><span class="pre">master.cfg</span></tt> for you. So something like:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="c"># for using mysql:</span>
<span class="n">c</span><span class="p">[</span><span class="s">'db_url'</span><span class="p">]</span> <span class="o">=</span> <span class="s">'mysql://bbuser:<password>@localhost/buildbot'</span>
</pre></div>
</div>
<p>Once the parameter has been added, invoke <tt class="docutils literal"><span class="pre">upgrade-master</span></tt>.
This will extract the DB url from your configuration file.</p>
<div class="highlight-bash"><div class="highlight"><pre>buildbot upgrade-master
</pre></div>
</div>
<p>See <a class="reference internal" href="cfg-global.html#database-specification"><em>Database Specification</em></a> for more options to specify a database.</p>
<div class="section" id="change-encoding-issues">
<h5>Change Encoding Issues<a class="headerlink" href="#change-encoding-issues" title="Permalink to this headline">¶</a></h5>
<p>The upgrade process assumes that strings in your Changes pickle are encoded in
UTF-8 (or plain ASCII). If this is not the case, and if there are non-UTF-8
characters in the pickle, the upgrade will fail with a suitable error message.
If this occurs, you have two options. If the change history is not important
to your purpose, you can simply delete <tt class="file docutils literal"><span class="pre">changes.pck</span></tt>.</p>
<p>If you would like to keep the change history, then you will need to figure out
which encoding is in use, and use
<tt class="file docutils literal"><span class="pre">contrib/fix_changes_pickle_encoding.py</span></tt> (<a class="reference internal" href="#contrib-scripts"><em>Contrib Scripts</em></a>)
to rewrite the changes pickle into Unicode before upgrading the master. A
typical invocation (with Mac-Roman encoding) might look like:</p>
<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span>python <span class="nv">$buildbot</span>/contrib/fix_changes_pickle_encoding.py changes.pck macroman
decoding bytestrings in changes.pck using macroman
converted 11392 strings
backing up changes.pck to changes.pck.old
</pre></div>
</div>
<p>If your Changes pickle uses multiple encodings, you're on your own, but the
script in contrib may provide a good starting point for the fix.</p>
</div>
</div>
<div class="section" id="upgrading-a-buildmaster-to-later-versions">
<span id="upgrading-a-buildmaster-to-later-version"></span><h4>Upgrading a Buildmaster to Later Versions<a class="headerlink" href="#upgrading-a-buildmaster-to-later-versions" title="Permalink to this headline">¶</a></h4>
<p>Up to Buildbot version 0.8.9, no further steps beyond those described
above are required.</p>
</div>
</div>
</div>
<div class="section" id="creating-a-buildslave">
<span id="id9"></span><h2>Creating a buildslave<a class="headerlink" href="#creating-a-buildslave" title="Permalink to this headline">¶</a></h2>
<p>Typically, you will be adding a buildslave to an existing buildmaster,
to provide additional architecture coverage. The buildbot
administrator will give you several pieces of information necessary to
connect to the buildmaster. You should also be somewhat familiar with
the project being tested, so you can troubleshoot build problems
locally.</p>
<p>The buildbot exists to make sure that the project's stated <tt class="docutils literal"><span class="pre">how</span> <span class="pre">to</span>
<span class="pre">build</span> <span class="pre">it</span></tt> process actually works. To this end, the buildslave should
run in an environment just like that of your regular developers.
Typically the project build process is documented somewhere
(<tt class="file docutils literal"><span class="pre">README</span></tt>, <tt class="file docutils literal"><span class="pre">INSTALL</span></tt>, etc), in a document that should
mention all library dependencies and contain a basic set of build
instructions. This document will be useful as you configure the host
and account in which the buildslave runs.</p>
<p>Here's a good checklist for setting up a buildslave:</p>
<ol class="arabic simple">
<li>Set up the account</li>
</ol>
<blockquote>
<div>It is recommended (although not mandatory) to set up a separate user
account for the buildslave. This account is frequently named
<tt class="docutils literal"><span class="pre">buildbot</span></tt> or <tt class="docutils literal"><span class="pre">buildslave</span></tt>. This serves to isolate your
personal working environment from that of the slave's, and helps to
minimize the security threat posed by letting possibly-unknown
contributors run arbitrary code on your system. The account should
have a minimum of fancy init scripts.</div></blockquote>
<ol class="arabic simple" start="2">
<li>Install the buildbot code</li>
</ol>
<blockquote>
<div>Follow the instructions given earlier (<a class="reference internal" href="#installing-the-code"><em>Installing the code</em></a>).
If you use a separate buildslave account, and you didn't install the
buildbot code to a shared location, then you will need to install it
with <tt class="docutils literal"><span class="pre">--home=~</span></tt> for each account that needs it.</div></blockquote>
<ol class="arabic simple" start="3">
<li>Set up the host</li>
</ol>
<blockquote>
<div><p>Make sure the host can actually reach the buildmaster. Usually the
buildmaster is running a status webserver on the same machine, so
simply point your web browser at it and see if you can get there.
Install whatever additional packages or libraries the project's
INSTALL document advises. (or not: if your buildslave is supposed to
make sure that building without optional libraries still works, then
don't install those libraries).</p>
<p>Again, these libraries don't necessarily have to be installed to a
site-wide shared location, but they must be available to your build
process. Accomplishing this is usually very specific to the build
process, so installing them to <tt class="file docutils literal"><span class="pre">/usr</span></tt> or <tt class="file docutils literal"><span class="pre">/usr/local</span></tt> is
usually the best approach.</p>
</div></blockquote>
<ol class="arabic simple" start="4">
<li>Test the build process</li>
</ol>
<blockquote>
<div>Follow the instructions in the <tt class="file docutils literal"><span class="pre">INSTALL</span></tt> document, in the buildslave's
account. Perform a full CVS (or whatever) checkout, configure, make,
run tests, etc. Confirm that the build works without manual fussing.
If it doesn't work when you do it by hand, it will be unlikely to work
when the buildbot attempts to do it in an automated fashion.</div></blockquote>
<ol class="arabic simple" start="5">
<li>Choose a base directory</li>
</ol>
<blockquote>
<div>This should be somewhere in the buildslave's account, typically named
after the project which is being tested. The buildslave will not touch
any file outside of this directory. Something like <tt class="file docutils literal"><span class="pre">~/Buildbot</span></tt>
or <tt class="file docutils literal"><span class="pre">~/Buildslaves/fooproject</span></tt> is appropriate.</div></blockquote>
<ol class="arabic simple" start="6">
<li>Get the buildmaster host/port, botname, and password</li>
</ol>
<blockquote>
<div><p>When the buildbot admin configures the buildmaster to accept and use
your buildslave, they will provide you with the following pieces of
information:</p>
<ul class="simple">
<li>your buildslave's name</li>
<li>the password assigned to your buildslave</li>
<li>the hostname and port number of the buildmaster, i.e. buildbot.example.org:8007</li>
</ul>
</div></blockquote>
<ol class="arabic simple" start="7">
<li>Create the buildslave</li>
</ol>
<blockquote>
<div><p>Now run the 'buildslave' command as follows:</p>
<blockquote>
<div><tt class="samp docutils literal"><span class="pre">buildslave</span> <span class="pre">create-slave</span> <em><span class="pre">BASEDIR</span></em> <em><span class="pre">MASTERHOST</span></em><span class="pre">:</span><em><span class="pre">PORT</span></em> <em><span class="pre">SLAVENAME</span></em> <em><span class="pre">PASSWORD</span></em></tt></div></blockquote>
<p>This will create the base directory and a collection of files inside,
including the <tt class="file docutils literal"><span class="pre">buildbot.tac</span></tt> file that contains all the
information you passed to the <strong class="command">buildbot</strong> command.</p>
</div></blockquote>
<ol class="arabic simple" start="8">
<li>Fill in the hostinfo files</li>
</ol>
<blockquote>
<div><p>When it first connects, the buildslave will send a few files up to the
buildmaster which describe the host that it is running on. These files
are presented on the web status display so that developers have more
information to reproduce any test failures that are witnessed by the
buildbot. There are sample files in the <tt class="file docutils literal"><span class="pre">info</span></tt> subdirectory of
the buildbot's base directory. You should edit these to correctly
describe you and your host.</p>
<p><tt class="file docutils literal"><em><span class="pre">BASEDIR</span></em><span class="pre">/info/admin</span></tt> should contain your name and email address.
This is the <tt class="docutils literal"><span class="pre">buildslave</span> <span class="pre">admin</span> <span class="pre">address</span></tt>, and will be visible from the
build status page (so you may wish to munge it a bit if
address-harvesting spambots are a concern).</p>
<p><tt class="file docutils literal"><em><span class="pre">BASEDIR</span></em><span class="pre">/info/host</span></tt> should be filled with a brief description of
the host: OS, version, memory size, CPU speed, versions of relevant
libraries installed, and finally the version of the buildbot code
which is running the buildslave.</p>
<p>The optional <tt class="file docutils literal"><em><span class="pre">BASEDIR</span></em><span class="pre">/info/access_uri</span></tt> can specify a URI which will
connect a user to the machine. Many systems accept <tt class="docutils literal"><span class="pre">ssh://hostname</span></tt> URIs
for this purpose.</p>
<p>If you run many buildslaves, you may want to create a single
<tt class="file docutils literal"><span class="pre">~buildslave/info</span></tt> file and share it among all the buildslaves
with symlinks.</p>
</div></blockquote>
<div class="section" id="buildslave-options">
<span id="id10"></span><h3>Buildslave Options<a class="headerlink" href="#buildslave-options" title="Permalink to this headline">¶</a></h3>
<p>There are a handful of options you might want to use when creating the
buildslave with the <tt class="samp docutils literal"><span class="pre">buildslave</span> <span class="pre">create-slave</span> <span class="pre"><options></span> <span class="pre">DIR</span> <span class="pre"><params></span></tt>
command. You can type <tt class="docutils literal"><span class="pre">buildslave</span> <span class="pre">create-slave</span> <span class="pre">--help</span></tt> for a summary.
To use these, just include them on the <tt class="docutils literal"><span class="pre">buildslave</span> <span class="pre">create-slave</span></tt>
command line, like this</p>
<div class="highlight-bash"><div class="highlight"><pre>buildslave create-slave --umask<span class="o">=</span>022 ~/buildslave buildmaster.example.org:42012 <span class="o">{</span>myslavename<span class="o">}</span> <span class="o">{</span>mypasswd<span class="o">}</span>
</pre></div>
</div>
<dl class="option">
<dt id="cmdoption-buildslave-create-slave--no-logrotate">
<tt class="descname">--no-logrotate</tt><tt class="descclassname"></tt><a class="headerlink" href="#cmdoption-buildslave-create-slave--no-logrotate" title="Permalink to this definition">¶</a></dt>
<dd><p>This disables internal buildslave log management mechanism. With this option
buildslave does not override the default logfile name and its behaviour giving
a possibility to control those with command-line options of twistd
daemon.</p>
</dd></dl>
<dl class="option">
<dt id="cmdoption-buildslave-create-slave--usepty">
<tt class="descname">--usepty</tt><tt class="descclassname"></tt><a class="headerlink" href="#cmdoption-buildslave-create-slave--usepty" title="Permalink to this definition">¶</a></dt>
<dd><p>This is a boolean flag that tells the buildslave whether to launch child
processes in a PTY or with regular pipes (the default) when the master does not
specify. This option is deprecated, as this particular parameter is better
specified on the master.</p>
</dd></dl>
<dl class="option">
<dt id="cmdoption-buildslave-create-slave--umask">
<tt class="descname">--umask</tt><tt class="descclassname"></tt><a class="headerlink" href="#cmdoption-buildslave-create-slave--umask" title="Permalink to this definition">¶</a></dt>
<dd><p>This is a string (generally an octal representation of an integer)
which will cause the buildslave process' <tt class="docutils literal"><span class="pre">umask</span></tt> value to be set
shortly after initialization. The <tt class="docutils literal"><span class="pre">twistd</span></tt> daemonization utility
forces the umask to 077 at startup (which means that all files created
by the buildslave or its child processes will be unreadable by any
user other than the buildslave account). If you want build products to
be readable by other accounts, you can add <tt class="docutils literal"><span class="pre">--umask=022</span></tt> to tell
the buildslave to fix the umask after twistd clobbers it. If you want
build products to be <em>writable</em> by other accounts too, use
<tt class="docutils literal"><span class="pre">--umask=000</span></tt>, but this is likely to be a security problem.</p>
</dd></dl>
<dl class="option">
<dt id="cmdoption-buildslave-create-slave--keepalive">
<tt class="descname">--keepalive</tt><tt class="descclassname"></tt><a class="headerlink" href="#cmdoption-buildslave-create-slave--keepalive" title="Permalink to this definition">¶</a></dt>
<dd><p>This is a number that indicates how frequently <tt class="docutils literal"><span class="pre">keepalive</span></tt> messages
should be sent from the buildslave to the buildmaster, expressed in
seconds. The default (600) causes a message to be sent to the
buildmaster at least once every 10 minutes. To set this to a lower
value, use e.g. <tt class="docutils literal"><span class="pre">--keepalive=120</span></tt>.</p>
<p>If the buildslave is behind a NAT box or stateful firewall, these
messages may help to keep the connection alive: some NAT boxes tend to
forget about a connection if it has not been used in a while. When
this happens, the buildmaster will think that the buildslave has
disappeared, and builds will time out. Meanwhile the buildslave will
not realize than anything is wrong.</p>
</dd></dl>
<dl class="option">
<dt id="cmdoption-buildslave-create-slave--maxdelay">
<tt class="descname">--maxdelay</tt><tt class="descclassname"></tt><a class="headerlink" href="#cmdoption-buildslave-create-slave--maxdelay" title="Permalink to this definition">¶</a></dt>
<dd><p>This is a number that indicates the maximum amount of time the
buildslave will wait between connection attempts, expressed in
seconds. The default (300) causes the buildslave to wait at most 5
minutes before trying to connect to the buildmaster again.</p>
</dd></dl>
<dl class="option">
<dt id="cmdoption-buildslave-create-slave--log-size">
<tt class="descname">--log-size</tt><tt class="descclassname"></tt><a class="headerlink" href="#cmdoption-buildslave-create-slave--log-size" title="Permalink to this definition">¶</a></dt>
<dd><p>This is the size in bytes when to rotate the Twisted log files.</p>
</dd></dl>
<dl class="option">
<dt id="cmdoption-buildslave-create-slave--log-count">
<tt class="descname">--log-count</tt><tt class="descclassname"></tt><a class="headerlink" href="#cmdoption-buildslave-create-slave--log-count" title="Permalink to this definition">¶</a></dt>
<dd><p>This is the number of log rotations to keep around. You can either
specify a number or <tt class="docutils literal"><span class="pre">None</span></tt> to keep all <tt class="file docutils literal"><span class="pre">twistd.log</span></tt> files
around. The default is 10.</p>
</dd></dl>
<dl class="option">
<dt id="cmdoption-buildslave-create-slave--allow-shutdown">
<tt class="descname">--allow-shutdown</tt><tt class="descclassname"></tt><a class="headerlink" href="#cmdoption-buildslave-create-slave--allow-shutdown" title="Permalink to this definition">¶</a></dt>
<dd><p>Can also be passed directly to the BuildSlave constructor in buildbot.tac. If
set, it allows the buildslave to initiate a graceful shutdown, meaning that it
will ask the master to shut down the slave when the current build, if any, is
complete.</p>
<p>Setting allow_shutdown to <tt class="docutils literal"><span class="pre">file</span></tt> will cause the buildslave to watch
<tt class="file docutils literal"><span class="pre">shutdown.stamp</span></tt> in basedir for updates to its mtime. When the mtime changes,
the slave will request a graceful shutdown from the master. The file does not
need to exist prior to starting the slave.</p>
<p>Setting allow_shutdown to <tt class="docutils literal"><span class="pre">signal</span></tt> will set up a SIGHUP handler to start a
graceful shutdown. When the signal is received, the slave will request a
graceful shutdown from the master.</p>
<p>The default value is <tt class="docutils literal"><span class="pre">None</span></tt>, in which case this feature will be disabled.</p>
<p>Both master and slave must be at least version 0.8.3 for this feature to work.</p>
</dd></dl>
</div>
<div class="section" id="other-buildslave-configuration">
<span id="id11"></span><h3>Other Buildslave Configuration<a class="headerlink" href="#other-buildslave-configuration" title="Permalink to this headline">¶</a></h3>
<dl class="docutils">
<dt><tt class="docutils literal"><span class="pre">unicode_encoding</span></tt></dt>
<dd><p class="first">This represents the encoding that buildbot should use when converting unicode
commandline arguments into byte strings in order to pass to the operating
system when spawning new processes.</p>
<p>The default value is what Python's <tt class="xref py py-func docutils literal"><span class="pre">sys.getfilesystemencoding</span></tt> returns, which
on Windows is 'mbcs', on Mac OSX is 'utf-8', and on Unix depends on your locale
settings.</p>
<p class="last">If you need a different encoding, this can be changed in your build slave's
<tt class="file docutils literal"><span class="pre">buildbot.tac</span></tt> file by adding a <tt class="docutils literal"><span class="pre">unicode_encoding</span></tt>
argument to the BuildSlave constructor.</p>
</dd>
</dl>
<div class="highlight-python"><div class="highlight"><pre><span class="n">s</span> <span class="o">=</span> <span class="n">BuildSlave</span><span class="p">(</span><span class="n">buildmaster_host</span><span class="p">,</span> <span class="n">port</span><span class="p">,</span> <span class="n">slavename</span><span class="p">,</span> <span class="n">passwd</span><span class="p">,</span> <span class="n">basedir</span><span class="p">,</span>
<span class="n">keepalive</span><span class="p">,</span> <span class="n">usepty</span><span class="p">,</span> <span class="n">umask</span><span class="o">=</span><span class="n">umask</span><span class="p">,</span> <span class="n">maxdelay</span><span class="o">=</span><span class="n">maxdelay</span><span class="p">,</span>
<span class="n">unicode_encoding</span><span class="o">=</span><span class="s">'utf-8'</span><span class="p">,</span> <span class="n">allow_shutdown</span><span class="o">=</span><span class="s">'signal'</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
<div class="section" id="upgrading-an-existing-buildslave">
<span id="id12"></span><h2>Upgrading an Existing Buildslave<a class="headerlink" href="#upgrading-an-existing-buildslave" title="Permalink to this headline">¶</a></h2>
<p>If you have just installed a new version of Buildbot-slave, you may need to
take some steps to upgrade it. If you are upgrading to version 0.8.2 or later,
you can run</p>
<div class="highlight-bash"><div class="highlight"><pre>buildslave upgrade-slave /path/to/buildslave/dir
</pre></div>
</div>
<div class="section" id="buildslave-version-specific-notes">
<span id="id13"></span><h3>Version-specific Notes<a class="headerlink" href="#buildslave-version-specific-notes" title="Permalink to this headline">¶</a></h3>
<div class="section" id="upgrading-a-buildslave-to-buildbot-slave-0-8-1">
<h4>Upgrading a Buildslave to Buildbot-slave-0.8.1<a class="headerlink" href="#upgrading-a-buildslave-to-buildbot-slave-0-8-1" title="Permalink to this headline">¶</a></h4>
<p>Before Buildbot version 0.8.1, the Buildbot master and slave were part of the
same distribution. As of version 0.8.1, the buildslave is a separate
distribution.</p>
<p>As of this release, you will need to install <tt class="docutils literal"><span class="pre">buildbot-slave</span></tt> to run a slave.</p>
<p>Any automatic startup scripts that had run <tt class="docutils literal"><span class="pre">buildbot</span> <span class="pre">start</span></tt> for previous versions
should be changed to run <tt class="docutils literal"><span class="pre">buildslave</span> <span class="pre">start</span></tt> instead.</p>
<p>If you are running a version later than 0.8.1, then you can skip the remainder
of this section: the <tt class="docutils literal"><span class="pre">`upgrade-slave</span></tt> command will take care of this. If
you are upgrading directly to 0.8.1, read on.</p>
<p>The existing <tt class="file docutils literal"><span class="pre">buildbot.tac</span></tt> for any buildslaves running older versions
will need to be edited or replaced. If the loss of cached buildslave state
(e.g., for Source steps in copy mode) is not problematic, the easiest solution
is to simply delete the slave directory and re-run <tt class="docutils literal"><span class="pre">buildslave</span>
<span class="pre">create-slave</span></tt>.</p>
<p>If deleting the slave directory is problematic, the change to
<tt class="file docutils literal"><span class="pre">buildbot.tac</span></tt> is simple. On line 3, replace</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildbot.slave.bot</span> <span class="kn">import</span> <span class="n">BuildSlave</span>
</pre></div>
</div>
<p>with</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">buildslave.bot</span> <span class="kn">import</span> <span class="n">BuildSlave</span>
</pre></div>
</div>
<p>After this change, the buildslave should start as usual.</p>
</div>
</div>
</div>
<div class="section" id="launching-the-daemons">
<span id="id14"></span><h2>Launching the daemons<a class="headerlink" href="#launching-the-daemons" title="Permalink to this headline">¶</a></h2>
<p>Both the buildmaster and the buildslave run as daemon programs. To
launch them, pass the working directory to the <strong class="command">buildbot</strong>
and <strong class="command">buildslave</strong> commands, as appropriate:</p>
<div class="highlight-bash"><div class="highlight"><pre><span class="c"># start a master</span>
buildbot start <span class="o">[</span> BASEDIR <span class="o">]</span>
<span class="c"># start a slave</span>
buildslave start <span class="o">[</span> SLAVE_BASEDIR <span class="o">]</span>
</pre></div>
</div>
<p>The <em>BASEDIR</em> is option and can be omitted if the current directory
contains the buildbot configuration (the <tt class="file docutils literal"><span class="pre">buildbot.tac</span></tt> file).</p>
<div class="highlight-bash"><div class="highlight"><pre>buildbot start
</pre></div>
</div>
<p>This command will start the daemon and then return, so normally it
will not produce any output. To verify that the programs are indeed
running, look for a pair of files named <tt class="file docutils literal"><span class="pre">twistd.log</span></tt> and
<tt class="file docutils literal"><span class="pre">twistd.pid</span></tt> that should be created in the working directory.
<tt class="file docutils literal"><span class="pre">twistd.pid</span></tt> contains the process ID of the newly-spawned daemon.</p>
<p>When the buildslave connects to the buildmaster, new directories will
start appearing in its base directory. The buildmaster tells the slave
to create a directory for each Builder which will be using that slave.
All build operations are performed within these directories: CVS
checkouts, compiles, and tests.</p>
<p>Once you get everything running, you will want to arrange for the
buildbot daemons to be started at boot time. One way is to use
<strong class="command">cron</strong>, by putting them in a <tt class="docutils literal"><span class="pre">@reboot</span></tt> crontab entry <a class="footnote-reference" href="#f1" id="id15">[1]</a></p>
<div class="highlight-none"><div class="highlight"><pre>@reboot buildbot start [ BASEDIR ]
</pre></div>
</div>
<p>When you run <strong class="command">crontab</strong> to set this up, remember to do it as
the buildmaster or buildslave account! If you add this to your crontab
when running as your regular account (or worse yet, root), then the
daemon will run as the wrong user, quite possibly as one with more
authority than you intended to provide.</p>
<p>It is important to remember that the environment provided to cron jobs
and init scripts can be quite different that your normal runtime.
There may be fewer environment variables specified, and the <span class="target" id="index-1"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PATH</span></tt> may
be shorter than usual. It is a good idea to test out this method of
launching the buildslave by using a cron job with a time in the near
future, with the same command, and then check <tt class="file docutils literal"><span class="pre">twistd.log</span></tt> to
make sure the slave actually started correctly. Common problems here
are for <tt class="file docutils literal"><span class="pre">/usr/local</span></tt> or <tt class="file docutils literal"><span class="pre">~/bin</span></tt> to not be on your
<span class="target" id="index-2"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PATH</span></tt>, or for <span class="target" id="index-3"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PYTHONPATH</span></tt> to not be set correctly.
Sometimes <span class="target" id="index-4"></span><tt class="xref std std-envvar docutils literal"><span class="pre">HOME</span></tt> is messed up too.</p>
<p>Some distributions may include conveniences to make starting buildbot
at boot time easy. For instance, with the default buildbot package in
Debian-based distributions, you may only need to modify
<tt class="file docutils literal"><span class="pre">/etc/default/buildbot</span></tt> (see also <tt class="file docutils literal"><span class="pre">/etc/init.d/buildbot</span></tt>, which
reads the configuration in <tt class="file docutils literal"><span class="pre">/etc/default/buildbot</span></tt>).</p>
<p>Buildbot also comes with its own init scripts that provide support for
controlling multi-slave and multi-master setups (mostly because they are based
on the init script from the Debian package). With a little modification these
scripts can be used both on Debian and RHEL-based distributions and may thus
prove helpful to package maintainers who are working on buildbot (or those that
haven't yet split buildbot into master and slave packages).</p>
<div class="highlight-bash"><div class="highlight"><pre><span class="c"># install as /etc/default/buildslave</span>
<span class="c"># or /etc/sysconfig/buildslave</span>
master/contrib/init-scripts/buildslave.default
<span class="c"># install as /etc/default/buildmaster</span>
<span class="c"># or /etc/sysconfig/buildmaster</span>
master/contrib/init-scripts/buildmaster.default
<span class="c"># install as /etc/init.d/buildslave</span>
slave/contrib/init-scripts/buildslave.init.sh
<span class="c"># install as /etc/init.d/buildmaster</span>
slave/contrib/init-scripts/buildmaster.init.sh
<span class="c"># ... and tell sysvinit about them</span>
chkconfig buildmaster reset
<span class="c"># ... or</span>
update-rc.d buildmaster defaults
</pre></div>
</div>
</div>
<div class="section" id="logfiles">
<span id="id16"></span><h2>Logfiles<a class="headerlink" href="#logfiles" title="Permalink to this headline">¶</a></h2>
<p>While a buildbot daemon runs, it emits text to a logfile, named
<tt class="file docutils literal"><span class="pre">twistd.log</span></tt>. A command like <tt class="docutils literal"><span class="pre">tail</span> <span class="pre">-f</span> <span class="pre">twistd.log</span></tt> is useful
to watch the command output as it runs.</p>
<p>The buildmaster will announce any errors with its configuration file
in the logfile, so it is a good idea to look at the log at startup
time to check for any problems. Most buildmaster activities will cause
lines to be added to the log.</p>
</div>
<div class="section" id="shutdown">
<span id="id17"></span><h2>Shutdown<a class="headerlink" href="#shutdown" title="Permalink to this headline">¶</a></h2>
<p>To stop a buildmaster or buildslave manually, use:</p>
<div class="highlight-bash"><div class="highlight"><pre>buildbot stop <span class="o">[</span> BASEDIR <span class="o">]</span>
<span class="c"># or</span>
buildslave stop <span class="o">[</span> SLAVE_BASEDIR <span class="o">]</span>
</pre></div>
</div>
<p>This simply looks for the <tt class="file docutils literal"><span class="pre">twistd.pid</span></tt> file and kills whatever
process is identified within.</p>
<p>At system shutdown, all processes are sent a <tt class="docutils literal"><span class="pre">SIGKILL</span></tt>. The
buildmaster and buildslave will respond to this by shutting down
normally.</p>
<p>The buildmaster will respond to a <tt class="docutils literal"><span class="pre">SIGHUP</span></tt> by re-reading its
config file. Of course, this only works on Unix-like systems with
signal support, and won't work on Windows. The following shortcut is
available:</p>
<div class="highlight-bash"><div class="highlight"><pre>buildbot reconfig <span class="o">[</span> BASEDIR <span class="o">]</span>
</pre></div>
</div>
<p>When you update the Buildbot code to a new release, you will need to
restart the buildmaster and/or buildslave before it can take advantage
of the new code. You can do a <tt class="samp docutils literal"><span class="pre">buildbot</span> <span class="pre">stop</span> <em><span class="pre">BASEDIR</span></em></tt> and
<tt class="samp docutils literal"><span class="pre">buildbot</span> <span class="pre">start</span> <em><span class="pre">BASEDIR</span></em></tt> in quick succession, or you can
use the <tt class="docutils literal"><span class="pre">restart</span></tt> shortcut, which does both steps for you:</p>
<div class="highlight-bash"><div class="highlight"><pre>buildbot restart <span class="o">[</span> BASEDIR <span class="o">]</span>
</pre></div>
</div>
<p>Buildslaves can similarly be restarted with:</p>
<div class="highlight-bash"><div class="highlight"><pre>buildslave restart <span class="o">[</span> BASEDIR <span class="o">]</span>
</pre></div>
</div>
<p>There are certain configuration changes that are not handled cleanly
by <tt class="docutils literal"><span class="pre">buildbot</span> <span class="pre">reconfig</span></tt>. If this occurs, <tt class="docutils literal"><span class="pre">buildbot</span> <span class="pre">restart</span></tt>
is a more robust tool to fully switch over to the new configuration.</p>
<p><tt class="docutils literal"><span class="pre">buildbot</span> <span class="pre">restart</span></tt> may also be used to start a stopped Buildbot
instance. This behaviour is useful when writing scripts that stop, start
and restart Buildbot.</p>
<p>A buildslave may also be gracefully shutdown from the
<a class="reference internal" href="cfg-statustargets.html#status-WebStatus" title="WebStatus"><tt class="xref bb bb-status docutils literal"><span class="pre">WebStatus</span></tt></a> status plugin. This is useful to shutdown a
buildslave without interrupting any current builds. The buildmaster
will wait until the buildslave is finished all its current builds, and
will then tell the buildslave to shutdown.</p>
</div>
<div class="section" id="maintenance">
<span id="id18"></span><h2>Maintenance<a class="headerlink" href="#maintenance" title="Permalink to this headline">¶</a></h2>
<p>The buildmaster can be configured to send out email notifications when a
slave has been offline for a while. Be sure to configure the buildmaster
with a contact email address for each slave so these notifications are sent
to someone who can bring it back online.</p>
<p>If you find you can no longer provide a buildslave to the project, please
let the project admins know, so they can put out a call for a
replacement.</p>
<p>The Buildbot records status and logs output continually, each time a
build is performed. The status tends to be small, but the build logs
can become quite large. Each build and log are recorded in a separate
file, arranged hierarchically under the buildmaster's base directory.
To prevent these files from growing without bound, you should
periodically delete old build logs. A simple cron job to delete
anything older than, say, two weeks should do the job. The only trick
is to leave the <tt class="file docutils literal"><span class="pre">buildbot.tac</span></tt> and other support files alone, for
which <strong class="command">find</strong>'s <tt class="docutils literal"><span class="pre">-mindepth</span></tt> argument helps skip everything in the
top directory. You can use something like the following:</p>
<div class="highlight-none"><div class="highlight"><pre>@weekly cd BASEDIR && find . -mindepth 2 i-path './public_html/*' \
-prune -o -type f -mtime +14 -exec rm {} \;
@weekly cd BASEDIR && find twistd.log* -mtime +14 -exec rm {} \;
</pre></div>
</div>
<p>Alternatively, you can configure a maximum number of old logs to be kept
using the <tt class="docutils literal"><span class="pre">--log-count</span></tt> command line option when running <tt class="docutils literal"><span class="pre">buildslave</span>
<span class="pre">create-slave</span></tt> or <tt class="docutils literal"><span class="pre">buildbot</span> <span class="pre">create-master</span></tt>.</p>
</div>
<div class="section" id="troubleshooting">
<span id="id19"></span><h2>Troubleshooting<a class="headerlink" href="#troubleshooting" title="Permalink to this headline">¶</a></h2>
<p>Here are a few hints on diagnosing common problems.</p>
<div class="section" id="starting-the-buildslave">
<span id="id20"></span><h3>Starting the buildslave<a class="headerlink" href="#starting-the-buildslave" title="Permalink to this headline">¶</a></h3>
<p>Cron jobs are typically run with a minimal shell (<tt class="file docutils literal"><span class="pre">/bin/sh</span></tt>, not
<tt class="file docutils literal"><span class="pre">/bin/bash</span></tt>), and tilde expansion is not always performed in such
commands. You may want to use explicit paths, because the <span class="target" id="index-5"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PATH</span></tt>
is usually quite short and doesn't include anything set by your
shell's startup scripts (<tt class="file docutils literal"><span class="pre">.profile</span></tt>, <tt class="file docutils literal"><span class="pre">.bashrc</span></tt>, etc). If
you've installed buildbot (or other Python libraries) to an unusual
location, you may need to add a <span class="target" id="index-6"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PYTHONPATH</span></tt> specification (note
that Python will do tilde-expansion on <span class="target" id="index-7"></span><tt class="xref std std-envvar docutils literal"><span class="pre">PYTHONPATH</span></tt> elements by
itself). Sometimes it is safer to fully-specify everything:</p>
<div class="highlight-none"><div class="highlight"><pre>@reboot PYTHONPATH=~/lib/python /usr/local/bin/buildbot \
start /usr/home/buildbot/basedir
</pre></div>
</div>
<p>Take the time to get the <tt class="docutils literal"><span class="pre">@reboot</span></tt> job set up. Otherwise, things will work
fine for a while, but the first power outage or system reboot you have will
stop the buildslave with nothing but the cries of sorrowful developers to
remind you that it has gone away.</p>
</div>
<div class="section" id="connecting-to-the-buildmaster">
<span id="id21"></span><h3>Connecting to the buildmaster<a class="headerlink" href="#connecting-to-the-buildmaster" title="Permalink to this headline">¶</a></h3>
<p>If the buildslave cannot connect to the buildmaster, the reason should
be described in the <tt class="file docutils literal"><span class="pre">twistd.log</span></tt> logfile. Some common problems
are an incorrect master hostname or port number, or a mistyped bot
name or password. If the buildslave loses the connection to the
master, it is supposed to attempt to reconnect with an
exponentially-increasing backoff. Each attempt (and the time of the
next attempt) will be logged. If you get impatient, just manually stop
and re-start the buildslave.</p>
<p>When the buildmaster is restarted, all slaves will be disconnected, and will
attempt to reconnect as usual. The reconnect time will depend upon how long the
buildmaster is offline (i.e. how far up the exponential backoff curve the
slaves have travelled). Again, <tt class="samp docutils literal"><span class="pre">buildslave</span> <span class="pre">restart</span> <em><span class="pre">BASEDIR</span></em></tt> will
speed up the process.</p>
<table class="docutils footnote" frame="void" id="f1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id15">[1]</a></td><td>This <tt class="docutils literal"><span class="pre">@reboot</span></tt> syntax is understood by Vixie cron, which is the flavor
usually provided with Linux systems. Other unices may have a cron that
doesn't understand <tt class="docutils literal"><span class="pre">@reboot</span></tt>:</td></tr>
</tbody>
</table>
</div>
<div class="section" id="contrib-scripts">
<span id="id22"></span><h3>Contrib Scripts<a class="headerlink" href="#contrib-scripts" title="Permalink to this headline">¶</a></h3>
<p>While some features of Buildbot are included in the distribution, others are
only available in <tt class="file docutils literal"><span class="pre">contrib/</span></tt> in the source directory. The latest versions
of such scripts are available at
<a class="reference external" href="http://github.com/buildbot/buildbot/tree/master/master/contrib">http://github.com/buildbot/buildbot/tree/master/master/contrib</a>.</p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sidebar">
<h3>Table Of Contents</h3>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../tutorial/index.html">Buildbot Tutorial</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="index.html">Buildbot Manual</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="introduction.html">Introduction</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="">Installation</a><ul class="simple">
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="concepts.html">Concepts</a></li>
<li class="toctree-l2"><a class="reference internal" href="configuration.html">Configuration</a></li>
<li class="toctree-l2"><a class="reference internal" href="customization.html">Customization</a></li>
<li class="toctree-l2"><a class="reference internal" href="new-style-steps.html">New-Style Build Steps</a></li>
<li class="toctree-l2"><a class="reference internal" href="cmdline.html">Command-line Tool</a></li>
<li class="toctree-l2"><a class="reference internal" href="resources.html">Resources</a></li>
<li class="toctree-l2"><a class="reference internal" href="optimization.html">Optimization</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../developer/index.html">Buildbot Development</a></li>
<li class="toctree-l1"><a class="reference internal" href="../relnotes/index.html">Release Notes for Buildbot 0.8.9</a></li>
</ul>
<h3 style="margin-top: 1.5em;">Search</h3>
<form class="search" action="../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<div class="clearer"></div>
</div>
</div>
<div class="footer-wrapper">
<div class="footer">
<div class="left">
<a href="introduction.html" title="Introduction"
>previous</a> |
<a href="concepts.html" title="Concepts"
>next</a> |
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |
<a href="../genindex.html" title="General Index"
>index</a>
<br/>
<a href="../_sources/manual/installation.txt"
rel="nofollow">Show Source</a>
</div>
<div class="right">
<div class="footer">
© Copyright Buildbot Team Members.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.2.3.
</div>
</div>
<div class="clearer"></div>
</div>
</div>
</body>
</html>
