==================
Developing Roundup
==================
:Version: $Revision$
.. note::
The intended audience of this document is the developers of the core
Roundup code. If you just wish to alter some behaviour of your Roundup
installation, see `customising roundup`_.
.. contents::
Getting Started
---------------
Anyone wishing to help in the development of Roundup must read `Roundup's
Design Document`_ and the `implementation notes`_.
All development is coordinated through two resources:
- roundup-dev mailing list at
http://lists.sourceforge.net/mailman/listinfo/roundup-devel
- Sourceforge's issue trackers at
https://sourceforge.net/tracker/?group_id=31577
Small Changes
-------------
Most small changes can be submitted through the `feature tracker`_, with
patches attached that give context diffs of the affected source.
CVS Access
----------
To get CVS access, contact richard@users.sourceforge.net.
CVS stuff:
1. to tag a release (eg. the pre-release of 0.5.0)::
cvs tag release-0-5-0-pr1
1. to make a branch (eg. branching for code freeze/release)::
cvs co -d maint-0-5 -r release-0-5-0-pr1 roundup
cd maint-0-5
cvs tag -b maint-0-5
2. to check out a branch (eg. the maintenance branch for 0.5.x)::
cvs co -d maint-0-5 -r maint-0-5
3. to merge changes from the maintenance branch to the trunk, in the
directory containing the HEAD checkout::
cvs up -j maint-0-5
though this is highly discouraged, as it generally creates a whole swag
of conflicts :(
Standard tag names:
*release-maj-min-patch[-sub]*
Release of the major.minor.patch release, possibly a beta or pre-release,
in which case *sub* will be one of "b*N*" or "pr*N*".
*maint-maj-min*
Maintenance branch for the major.minor release. Patch releases are tagged in
this branch.
Typically, release happen like this:
1. work progresses in the HEAD branch until milestones are met,
2. a series of beta releases are tagged in the HEAD until the code is
stable enough to freeze,
3. the pre-release is tagged in the HEAD, with the resultant code branched
to the maintenance branch for that release,
4. bugs in the release are patched in the maintenance branch, and the final
and patch releases are tagged there, and
5. further major work happens in the HEAD.
Project Rules
-------------
Mostly the project follows Guido's Style (though naming tends to be a little
relaxed sometimes). In short:
- 80 column width code
- 4-space indentations
- All modules must have a CVS Id line near the top
Other project rules:
- New functionality must be documented, even briefly (so at least we know
where there's missing documentation) and changes to tracker configuration
must be logged in the upgrading document.
- subscribe to roundup-checkins to receive checkin notifications from the
other developers with CVS access
- discuss any changes with the other developers on roundup-dev. If nothing
else, this makes sure there's no rude shocks
- write unit tests for changes you make (where possible), and ensure that
all unit tests run before committing changes
- run pychecker over changed code
The administrators of the project reserve the right to boot developers who
consistently check in code which is either broken or takes the codebase in
directions that have not been agreed to.
Debugging Aids
--------------
Try turning on logging of DEBUG level messages. This may be done a number
of ways, depending on what it is you're testing:
1. If you're testing the database unit tests, then set the environment
variable ``LOGGING_LEVEL=DEBUG``. This may be done like so:
LOGGING_LEVEL=DEBUG python run_tests.py
This variable replaces the older HYPERDBDEBUG environment var.
2. If you're testing a particular tracker, then set the logging level in
your tracker's ``config.ini``.
Internationalization Notes
--------------------------
How stuff works:
1. Strings that may require translation (messages in human language)
are marked in the source code. This step is discussed in
`Marking Strings for Translation`_ section.
2. These strings are all extracted into Message Template File
``locale/roundup.pot`` (_`POT` file). See `Extracting Translatable
Messages`_ below.
3. Language teams use POT file to make Message Files for national
languages (_`PO` files). All PO files for Roundup are kept in
the ``locale`` directory. Names of these files are target
locale names, usually just 2-letter language codes. `Translating
Messages`_ section of this chapter gives useful hints for
message translators.
4. Translated Message Files are compiled into binary form (_`MO` files)
and stored in ``locale`` directory (but not kept in the `Roundup
CVS`_ repository, as they may be easily made from PO files).
See `Compiling Message Catalogs`_ section.
5. Roundup installer creates runtime locale structure on the file
system, putting MO files in their appropriate places.
6. Runtime internationalization (_`I18N`) services use these MO files
to translate program messages into language selected by current
Roundup user. Roundup command line interface uses locale name
set in OS environment variable ``LANGUAGE``, ``LC_ALL``,
``LC_MESSAGES``, or ``LANG`` (in that order). Roundup Web User
Interface uses language selected by currently authenticated user.
Additional details may be found in `GNU gettext`_ and Python `gettext
module`_ documentation.
`Roundup source distribution`_ includes POT and PO files for message
translators, and also pre-built MO files to facilitate installations
from source. Roundup binary distribution includes MO files only.
.. _GNU gettext:
GNU gettext package
^^^^^^^^^^^^^^^^^^^
This chapter is full of references to GNU `gettext package`_.
GNU gettext is a "must have" for nearly all steps of internationalizing
any program, and it's manual is definetely a recommended reading
for people involved in `I18N`_.
There are GNU gettext ports to all major OS platforms.
Windows binaries are available from `GNU mirror sites`_.
Roundup does not use GNU gettext at runtime, but it's tools
are used for `extracting translatable messages`_, `compiling
message catalogs`_ and, optionally, for `translating messages`_.
Note that ``gettext`` package in some OS distributions means just
runtime tools and libraries. In such cases gettext development tools
are usually distributed in separate package named ``gettext-devel``.
Marking Strings for Translation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Strings that need translation must be marked in the source code.
Following subsections explain how this is done in different cases.
If translatable string is used as a format string, it is recommended
to always use *named* format specifiers::
_('Index of %(classname)s') % locals()
This helps translators to better understand the context of the
message and, with Python formatting, remove format specifier altogether
(which is sometimes useful, especially in singular cases of `Plural Forms`_).
When there is more than one format specifier in the translatable
format string, named format specifiers **must** be used almost always,
because translation may require different order of items.
It is better to *not* mark for translation strings that are not
locale-dependent, as this makes it more difficult to keep track
of translation completeness. For example, string ``</ol></body></html>``
(in ``index()`` method of the request handler in ``roundup_server``
script) has no human readable parts at all, and needs no translations.
Such strings are left untranslated in PO files, and are reported
as such by PO status checkers (e.g. ``msgfmt --statistics``).
Command Line Interfaces
~~~~~~~~~~~~~~~~~~~~~~~
Scripts and routines run from the command line use "static" language
defined by environment variables recognized by ``gettext`` module
from Python library (``LANGUAGE``, ``LC_ALL``, ``LC_MESSAGES``, and
``LANG``). Primarilly, these are ``roundup-admin`` script and
``admin.py`` module, but also help texts and startup error messages
in other scripts and their supporting modules.
For these interfaces, Python ``gettext`` engine must be initialized
to use Roundup message catalogs. This is normally done by including
the following line in the module imports::
from i18n import _, ngettext
Simple translations are automatically marked by calls to builtin
message translation function ``_()``::
print _("This message is translated")
Translations for messages whose grammatical depends on a number
must be done by ``ngettext()`` function::
print ngettext("Nuked %i file", "Nuked %i files", number_of_files_nuked)
Deferred Translations
~~~~~~~~~~~~~~~~~~~~~
Sometimes translatable strings appear in the source code in untranslated
form [#note_admin.py]_ and must be translated elsewhere.
Example::
for meal in ("spam", "egg", "beacon"):
print _(meal)
In such cases, strings must be marked for translation without actual
call to the translating function. To mark these strings, we use Python
feature of automatic concatenation of adjacent strings and different
types of string quotes::
strings_to_translate = (
''"This string will be translated",
""'me too',
''r"\raw string",
''"""
multiline string"""
)
.. [#note_admin.py] In current Roundup sources, this feature is
extensively used in the ``admin`` module using method docstrings
as help messages.
Web User Interface
~~~~~~~~~~~~~~~~~~
For Web User Interface, translation services are provided by Client
object. Action classes have methods ``_()`` and ``gettext()``,
delegating translation to the Client instance. In HTML templates,
translator object is available as context variable ``i18n``.
HTML templates have special markup for translatable strings.
The syntax for this markup is defined on `ZPTInternationalizationSupport`_
page. Roundup translation service currently ignores values for
``i18n:domain``, ``i18n:source`` and ``i18n:target``.
Template markup examples:
* simplest case::
<div i18n:translate="">
Say
no
more!
</div>
this will result in msgid ``"Say no more!"``, with all leading and
trailing whitespace stripped, and inner blanks replaced with single
space character.
* using variable slots::
<div i18n:translate="">
And now...<br/>
No.<span tal:replace="number" i18n:name="slideNo" /><br/>
THE LARCH
</div>
Msgid will be: ``"And now...<br /> No.${slideNo}<br /> THE LARCH"``.
Template rendering will use context variable ``number`` (you may use
any expression) to put instead of ``${slideNo}`` in translation.
* attribute translation::
<button name="btn_wink" value=" Wink " i18n:attributes="value" />
will translate the caption (and return value) for the "wink" button.
* explicit msgids. Sometimes it may be useful to specify msgid
for the element translation explicitely, like this::
<span i18n:translate="know what i mean?">this text is ignored</span>
When rendered, element contents will be replaced by translation
of the string specified in ``i18n:translate`` attribute.
* ``i18n`` in `TALES`_. You may translate strings in `TALES`_ python
expressions::
<span tal:replace="python: i18n.gettext('Oh, wicked.')" />
* plural forms. There is no markup for plural forms in `TAL`_ i18n.
You must use python expression for that::
<span tal:replace="python: i18n.ngettext(
'Oh but it\'s only %i shilling.',
'Oh but it\'s only %i shillings.',
fine) % fine"
/>
Extracting Translatable Messages
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The most common tool for message extraction is ``xgettext`` utility
from `GNU gettext package`_. Unfortunately, this utility has no means
of `Deferred Translations`_ in Python sources. There is ``xpot`` tool
from Francois Pinard free `PO utilities`_ that allows to mark strings
for deferred translations, but it does not handle `plural forms`_.
Roundup overcomes these limitations by using both of these utilities.
This means that you need both `GNU gettext`_ tools and `PO utilities`_
to build the Message Template File yourself.
Latest Message Template File is kept in `Roundup CVS`_ and distributed
with `Roundup Source`_. If you wish to rebuild the template yourself,
make sure that you have both ``xpot`` and ``xgettext`` installed and
just run ``gmake`` (or ``make``, if you are on a `GNU`_ system like
`linux`_ or `cygwin`_) in the ``locale`` directory.
For on-site i18n, Roundup provides command-line utility::
roundup-gettext <tracker_home>
extracting translatable messages from tracker's html templates.
This utility creates message template file ``messages.pot`` in
``locale`` subdirectory of the tracker home directory. Translated
messages may be put in *locale*.po files (where *locale* is selected
locale name) in the same directory, e.g.: ``locale/ru.po``.
These message catalogs are searched prior to system-wide translations
kept in the ``share`` directory.
Translating Messages
^^^^^^^^^^^^^^^^^^^^
Gettext Message File (`PO`_ file) is a plain text file, that can be created
by simple copying ``roundup.pot`` to new .po file, like this::
$ cp roundup.pot ru.po
The name of PO file is target locale name, usually just 2-letter language
code (``ru`` for Russian in the above example). Alternatively, PO file
may be initialized by ``msginit`` utility from `GNU gettext`_ tools::
$ msginit -i roundup.pot
``msginit`` will check your current locale, and initialize the header
entry, setting language name, rules for `plural forms`_ and, if available,
translator's name and email address. The name for PO file is also chosen
based on current locale.
Next, you will need to edit this file, filling all ``msgstr`` lines with
translations of the above ``msgid`` entries. PO file is a plain text
file that can be edited with any text editor. However, there are several
tools that may help you with this process:
- `poEdit`_ by Vaclav Slavik. Very nice cross-platform GUI editor.
- `KBabel`_. Being part of `KDE`_, it works in X windows only.
At the first glance looks pretty hairy, with all bells and whistles.
Haven't had much experience with it, though.
- ``po-mode`` for `emacs`_. One of `GNU gettext`_ tools. Very handy,
definitely recommended if you are comfortable with emacs. Cannot
handle `plural forms`_ per se, but allows to edit them in simple
text mode.
- `po filetype plugin`_ for `vim`_. Does not do as much as ``po-mode``,
but helps in finding untranslated and fuzzy strings, and checking
code references. Please contact `alexander smishlajev`_ if you
prefer this, as i have patched this plugin a bit. I have also
informed the original plugin author about these changes, but got
no reply so far.
Compiling Message Catalogs
^^^^^^^^^^^^^^^^^^^^^^^^^^
Message catalogs (`PO`_ files) must be compiled into binary form
(`MO`_ files) before they can be used in the application. This
compilation is handled by ``msgfmt`` utility from `GNU gettext`_
tools. ``GNUmakefile`` in the ``locale`` directory automatically
compiles all existing message catalogs after updating them from
Roundup source files. If you wish to rebuild an individual `MO`_
file without making everything else, you may, for example::
$ msgfmt --statistics -o ru.mo ru.po
This way, message translators can check their `PO`_ files without
extracting strings from source. (Note: String extraction requires
additional utility that is not part of `GNU gettext`_. See `Extracting
Translatable Messages`_.)
At run time, Roundup automatically compiles message catalogs whenever
`PO`_ file is changed.
-----------------
Back to `Table of Contents`_
.. _`Table of Contents`: index.html
.. _`Customising Roundup`: customizing.html
.. _`Roundup's Design Document`: spec.html
.. _`implementation notes`: implementation.html
.. _External hyperlink targets:
.. _alexander smishlajev:
.. _als: http://sourceforge.net/users/a1s/
.. _cygwin: http://www.cygwin.com/
.. _emacs: http://www.gnu.org/software/emacs/
.. _gettext package: http://www.gnu.org/software/gettext/
.. _gettext module: http://docs.python.org/lib/module-gettext.html
.. _GNU: http://www.gnu.org/
.. _GNU mirror sites: http://www.gnu.org/prep/ftp.html
.. _KBabel: http://i18n.kde.org/tools/kbabel/
.. _KDE: http://www.kde.org/
.. _linux: http://www.linux.org/
.. _Plural Forms:
http://www.gnu.org/software/gettext/manual/html_node/gettext_150.html
.. _po filetype plugin:
http://vim.sourceforge.net/scripts/script.php?script_id=695
.. _PO utilities: http://po-utils.progiciels-bpi.ca/
.. _poEdit: http://poedit.sourceforge.net/
.. _Roundup CVS: http://sourceforge.net/cvs/?group_id=31577
.. _Roundup Source:
.. _Roundup source distribution:
.. _Roundup binary distribution:
http://sourceforge.net/project/showfiles.php?group_id=31577
.. _TAL:
.. _Template Attribute Language:
http://dev.zope.org/Wikis/DevSite/Projects/ZPT/TAL%20Specification%201.4
.. _TALES:
.. _Template Attribute Language Expression Syntax:
http://dev.zope.org/Wikis/DevSite/Projects/ZPT/TALES%20Specification%201.3
.. _vim: http://www.vim.org/
.. _ZPTInternationalizationSupport: http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/ZPTInternationalizationSupport
.. _feature tracker: http://sourceforge.net/tracker/?group_id=31577&atid=402791
