<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Information Flow: Pipes and Filters — Botan</title> <link rel="stylesheet" href="_static/agogo.css" type="text/css" /> <link rel="stylesheet" href="_static/pygments.css" type="text/css" /> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT: './', VERSION: '1.10.14', COLLAPSE_INDEX: false, FILE_SUFFIX: '.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="_static/jquery.js"></script> <script type="text/javascript" src="_static/underscore.js"></script> <script type="text/javascript" src="_static/doctools.js"></script> <link rel="top" title="Botan" href="index.html" /> <link rel="next" title="Public Key Cryptography" href="pubkey.html" /> <link rel="prev" title="Getting Started" href="firststep.html" /> </head> <body> <div class="header-wrapper"> <div class="header"> <h1>Botan</h1> </div> </div> <div class="content-wrapper"> <div class="content"> <div class="document"> <div class="documentwrapper"> <div class="bodywrapper"> <div class="body"> <div class="section" id="information-flow-pipes-and-filters"> <h1>Information Flow: Pipes and Filters<a class="headerlink" href="#information-flow-pipes-and-filters" title="Permalink to this headline">¶</a></h1> <p>Many common uses of cryptography involve processing one or more streams of data. Botan provides services that make setting up data flows through various operations, such as compression, encryption, and base64 encoding. Each of these operations is implemented in what are called <em>filters</em> in Botan. A set of filters are created and placed into a <em>pipe</em>, and information “flows” through the pipe until it reaches the end, where the output is collected for retrieval. If you’re familiar with the Unix shell environment, this design will sound quite familiar.</p> <p>Here is an example that uses a pipe to base64 encode some strings:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">Pipe</span> <span class="nf">pipe</span><span class="p">(</span><span class="k">new</span> <span class="n">Base64_Encoder</span><span class="p">);</span> <span class="c1">// pipe owns the pointer</span> <span class="n">pipe</span><span class="p">.</span><span class="n">start_msg</span><span class="p">();</span> <span class="n">pipe</span><span class="p">.</span><span class="n">write</span><span class="p">(</span><span class="s">"message 1"</span><span class="p">);</span> <span class="n">pipe</span><span class="p">.</span><span class="n">end_msg</span><span class="p">();</span> <span class="c1">// flushes buffers, increments message number</span> <span class="c1">// process_msg(x) is start_msg() && write(x) && end_msg()</span> <span class="n">pipe</span><span class="p">.</span><span class="n">process_msg</span><span class="p">(</span><span class="s">"message2"</span><span class="p">);</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">m1</span> <span class="o">=</span> <span class="n">pipe</span><span class="p">.</span><span class="n">read_all_as_string</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span> <span class="c1">// "message1"</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">m2</span> <span class="o">=</span> <span class="n">pipe</span><span class="p">.</span><span class="n">read_all_as_string</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="c1">// "message2"</span> </pre></div> </div> <p>Bytestreams in the pipe are grouped into messages; blocks of data that are processed in an identical fashion (ie, with the same sequence of filter operations). Messages are delimited by calls to <tt class="docutils literal"><span class="pre">start_msg</span></tt> and <tt class="docutils literal"><span class="pre">end_msg</span></tt>. Each message in a pipe has its own identifier, which currently is an integer that increments up from zero.</p> <p>The <tt class="docutils literal"><span class="pre">Base64_Encoder</span></tt> was allocated using <tt class="docutils literal"><span class="pre">new</span></tt>; but where was it deallocated? When a filter object is passed to a <tt class="docutils literal"><span class="pre">Pipe</span></tt>, the pipe takes ownership of the object, and will deallocate it when it is no longer needed.</p> <p>There are two different ways to make use of messages. One is to send several messages through a <tt class="docutils literal"><span class="pre">Pipe</span></tt> without changing the <tt class="docutils literal"><span class="pre">Pipe</span></tt> configuration, so you end up with a sequence of messages; one use of this would be to send a sequence of identically encrypted UDP packets, for example (note that the <em>data</em> need not be identical; it is just that each is encrypted, encoded, signed, etc in an identical fashion). Another is to change the filters that are used in the <tt class="docutils literal"><span class="pre">Pipe</span></tt> between each message, by adding or removing filters; functions that let you do this are documented in the Pipe API section.</p> <p>Botan has about 40 filters that perform different operations on data. Here’s code that uses one of them to encrypt a string with AES:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">AutoSeeded_RNG</span> <span class="n">rng</span><span class="p">,</span> <span class="n">SymmetricKey</span> <span class="n">key</span><span class="p">(</span><span class="n">rng</span><span class="p">,</span> <span class="mi">16</span><span class="p">);</span> <span class="c1">// a random 128-bit key</span> <span class="n">InitializationVector</span> <span class="nf">iv</span><span class="p">(</span><span class="n">rng</span><span class="p">,</span> <span class="mi">16</span><span class="p">);</span> <span class="c1">// a random 128-bit IV</span> <span class="c1">// The algorithm we want is specified by a string</span> <span class="n">Pipe</span> <span class="nf">pipe</span><span class="p">(</span><span class="n">get_cipher</span><span class="p">(</span><span class="s">"AES-128/CBC"</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">iv</span><span class="p">,</span> <span class="n">ENCRYPTION</span><span class="p">));</span> <span class="n">pipe</span><span class="p">.</span><span class="n">process_msg</span><span class="p">(</span><span class="s">"secrets"</span><span class="p">);</span> <span class="n">pipe</span><span class="p">.</span><span class="n">process_msg</span><span class="p">(</span><span class="s">"more secrets"</span><span class="p">);</span> <span class="n">MemoryVector</span><span class="o"><</span><span class="n">byte</span><span class="o">></span> <span class="n">c1</span> <span class="o">=</span> <span class="n">pipe</span><span class="p">.</span><span class="n">read_all</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span> <span class="n">byte</span> <span class="n">c2</span><span class="p">[</span><span class="mi">4096</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span> <span class="mi">0</span> <span class="p">};</span> <span class="kt">size_t</span> <span class="n">got_out</span> <span class="o">=</span> <span class="n">pipe</span><span class="p">.</span><span class="n">read</span><span class="p">(</span><span class="n">c2</span><span class="p">,</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">c2</span><span class="p">),</span> <span class="mi">1</span><span class="p">);</span> <span class="c1">// use c2[0...got_out]</span> </pre></div> </div> <p>Note the use of <tt class="docutils literal"><span class="pre">AutoSeeded_RNG</span></tt>, which is a random number generator. If you want to, you can explicitly set up the random number generators and entropy sources you want to, however for 99% of cases <tt class="docutils literal"><span class="pre">AutoSeeded_RNG</span></tt> is preferable.</p> <p><tt class="docutils literal"><span class="pre">Pipe</span></tt> also has convenience methods for dealing with <tt class="docutils literal"><span class="pre">std::iostream</span></tt>. Here is an example of those, using the <tt class="docutils literal"><span class="pre">Bzip_Compression</span></tt> filter (included as a module; if you have bzlib available, check the build instructions for how to enable it) to compress a file:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">std</span><span class="o">::</span><span class="n">ifstream</span> <span class="n">in</span><span class="p">(</span><span class="s">"data.bin"</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">ios</span><span class="o">::</span><span class="n">binary</span><span class="p">)</span> <span class="n">std</span><span class="o">::</span><span class="n">ofstream</span> <span class="n">out</span><span class="p">(</span><span class="s">"data.bin.bz2"</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">ios</span><span class="o">::</span><span class="n">binary</span><span class="p">)</span> <span class="n">Pipe</span> <span class="n">pipe</span><span class="p">(</span><span class="k">new</span> <span class="n">Bzip_Compression</span><span class="p">);</span> <span class="n">pipe</span><span class="p">.</span><span class="n">start_msg</span><span class="p">();</span> <span class="n">in</span> <span class="o">>></span> <span class="n">pipe</span><span class="p">;</span> <span class="n">pipe</span><span class="p">.</span><span class="n">end_msg</span><span class="p">();</span> <span class="n">out</span> <span class="o"><<</span> <span class="n">pipe</span><span class="p">;</span> </pre></div> </div> <p>However there is a hitch to the code above; the complete contents of the compressed data will be held in memory until the entire message has been compressed, at which time the statement <tt class="docutils literal"><span class="pre">out</span> <span class="pre"><<</span> <span class="pre">pipe</span></tt> is executed, and the data is freed as it is read from the pipe and written to the file. But if the file is very large, we might not have enough physical memory (or even enough virtual memory!) for that to be practical. So instead of storing the compressed data in the pipe for reading it out later, we divert it directly to the file:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">std</span><span class="o">::</span><span class="n">ifstream</span> <span class="n">in</span><span class="p">(</span><span class="s">"data.bin"</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">ios</span><span class="o">::</span><span class="n">binary</span><span class="p">)</span> <span class="n">std</span><span class="o">::</span><span class="n">ofstream</span> <span class="n">out</span><span class="p">(</span><span class="s">"data.bin.bz2"</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">ios</span><span class="o">::</span><span class="n">binary</span><span class="p">)</span> <span class="n">Pipe</span> <span class="n">pipe</span><span class="p">(</span><span class="k">new</span> <span class="n">Bzip_Compression</span><span class="p">,</span> <span class="k">new</span> <span class="n">DataSink_Stream</span><span class="p">(</span><span class="n">out</span><span class="p">));</span> <span class="n">pipe</span><span class="p">.</span><span class="n">start_msg</span><span class="p">();</span> <span class="n">in</span> <span class="o">>></span> <span class="n">pipe</span><span class="p">;</span> <span class="n">pipe</span><span class="p">.</span><span class="n">end_msg</span><span class="p">();</span> </pre></div> </div> <p>This is the first code we’ve seen so far that uses more than one filter in a pipe. The output of the compressor is sent to the <tt class="docutils literal"><span class="pre">DataSink_Stream</span></tt>. Anything written to a <tt class="docutils literal"><span class="pre">DataSink_Stream</span></tt> is written to a file; the filter produces no output. As soon as the compression algorithm finishes up a block of data, it will send it along to the sink filter, which will immediately write it to the stream; if you were to call <tt class="docutils literal"><span class="pre">pipe.read_all()</span></tt> after <tt class="docutils literal"><span class="pre">pipe.end_msg()</span></tt>, you’d get an empty vector out. This is particularly useful for cases where you are processing a large amount of data, as it means you don’t have to store everything in memory at once.</p> <p>Here’s an example using two computational filters:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">AutoSeeded_RNG</span> <span class="n">rng</span><span class="p">,</span> <span class="n">SymmetricKey</span> <span class="n">key</span><span class="p">(</span><span class="n">rng</span><span class="p">,</span> <span class="mi">32</span><span class="p">);</span> <span class="n">InitializationVector</span> <span class="nf">iv</span><span class="p">(</span><span class="n">rng</span><span class="p">,</span> <span class="mi">16</span><span class="p">);</span> <span class="n">Pipe</span> <span class="nf">encryptor</span><span class="p">(</span><span class="n">get_cipher</span><span class="p">(</span><span class="s">"AES/CBC/PKCS7"</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">iv</span><span class="p">,</span> <span class="n">ENCRYPTION</span><span class="p">),</span> <span class="k">new</span> <span class="n">Base64_Encoder</span><span class="p">);</span> <span class="n">encryptor</span><span class="p">.</span><span class="n">start_msg</span><span class="p">();</span> <span class="n">file</span> <span class="o">>></span> <span class="n">encryptor</span><span class="p">;</span> <span class="n">encryptor</span><span class="p">.</span><span class="n">end_msg</span><span class="p">();</span> <span class="c1">// flush buffers, complete computations</span> <span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="n">encryptor</span><span class="p">;</span> </pre></div> </div> <p>You can read from a pipe while you are still writing to it, which allows you to bound the amount of memory that is in use at any one time. A common idiom for this is:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">pipe</span><span class="p">.</span><span class="n">start_msg</span><span class="p">();</span> <span class="n">SecureBuffer</span><span class="o"><</span><span class="n">byte</span><span class="p">,</span> <span class="mi">4096</span><span class="o">></span> <span class="n">buffer</span><span class="p">;</span> <span class="k">while</span><span class="p">(</span><span class="n">infile</span><span class="p">.</span><span class="n">good</span><span class="p">())</span> <span class="p">{</span> <span class="n">infile</span><span class="p">.</span><span class="n">read</span><span class="p">((</span><span class="kt">char</span><span class="o">*</span><span class="p">)</span><span class="o">&</span><span class="n">buffer</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">buffer</span><span class="p">.</span><span class="n">size</span><span class="p">());</span> <span class="k">const</span> <span class="kt">size_t</span> <span class="n">got_from_infile</span> <span class="o">=</span> <span class="n">infile</span><span class="p">.</span><span class="n">gcount</span><span class="p">();</span> <span class="n">pipe</span><span class="p">.</span><span class="n">write</span><span class="p">(</span><span class="n">buffer</span><span class="p">,</span> <span class="n">got_from_infile</span><span class="p">);</span> <span class="k">if</span><span class="p">(</span><span class="n">infile</span><span class="p">.</span><span class="n">eof</span><span class="p">())</span> <span class="n">pipe</span><span class="p">.</span><span class="n">end_msg</span><span class="p">();</span> <span class="k">while</span><span class="p">(</span><span class="n">pipe</span><span class="p">.</span><span class="n">remaining</span><span class="p">()</span> <span class="o">></span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span> <span class="k">const</span> <span class="kt">size_t</span> <span class="n">buffered</span> <span class="o">=</span> <span class="n">pipe</span><span class="p">.</span><span class="n">read</span><span class="p">(</span><span class="n">buffer</span><span class="p">,</span> <span class="n">buffer</span><span class="p">.</span><span class="n">size</span><span class="p">());</span> <span class="n">outfile</span><span class="p">.</span><span class="n">write</span><span class="p">((</span><span class="k">const</span> <span class="kt">char</span><span class="o">*</span><span class="p">)</span><span class="o">&</span><span class="n">buffer</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">buffered</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> <span class="k">if</span><span class="p">(</span><span class="n">infile</span><span class="p">.</span><span class="n">bad</span><span class="p">()</span> <span class="o">||</span> <span class="p">(</span><span class="n">infile</span><span class="p">.</span><span class="n">fail</span><span class="p">()</span> <span class="o">&&</span> <span class="o">!</span><span class="n">infile</span><span class="p">.</span><span class="n">eof</span><span class="p">()))</span> <span class="k">throw</span> <span class="n">Some_Exception</span><span class="p">();</span> </pre></div> </div> <div class="section" id="fork"> <h2>Fork<a class="headerlink" href="#fork" title="Permalink to this headline">¶</a></h2> <p>It is common that you might receive some data and want to perform more than one operation on it (ie, encrypt it with Serpent and calculate the SHA-256 hash of the plaintext at the same time). That’s where <tt class="docutils literal"><span class="pre">Fork</span></tt> comes in. <tt class="docutils literal"><span class="pre">Fork</span></tt> is a filter that takes input and passes it on to <em>one or more</em> filters that are attached to it. <tt class="docutils literal"><span class="pre">Fork</span></tt> changes the nature of the pipe system completely: instead of being a linked list, it becomes a tree or acyclic graph.</p> <p>Each filter in the fork is given its own output buffer, and thus its own message. For example, if you had previously written two messages into a pipe, then you start a new one with a fork that has three paths of filter’s inside it, you add three new messages to the pipe. The data you put into the pipe is duplicated and sent into each set of filter and the eventual output is placed into a dedicated message slot in the pipe.</p> <p>Messages in the pipe are allocated in a depth-first manner. This is only interesting if you are using more than one fork in a single pipe. As an example, consider the following:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">Pipe</span> <span class="nf">pipe</span><span class="p">(</span><span class="k">new</span> <span class="n">Fork</span><span class="p">(</span> <span class="k">new</span> <span class="n">Fork</span><span class="p">(</span> <span class="k">new</span> <span class="n">Base64_Encoder</span><span class="p">,</span> <span class="k">new</span> <span class="n">Fork</span><span class="p">(</span> <span class="nb">NULL</span><span class="p">,</span> <span class="k">new</span> <span class="n">Base64_Encoder</span> <span class="p">)</span> <span class="p">),</span> <span class="k">new</span> <span class="n">Hex_Encoder</span> <span class="p">)</span> <span class="p">);</span> </pre></div> </div> <p>In this case, message 0 will be the output of the first <tt class="docutils literal"><span class="pre">Base64_Encoder</span></tt>, message 1 will be a copy of the input (see below for how fork interprets NULL pointers), message 2 will be the output of the second <tt class="docutils literal"><span class="pre">Base64_Encoder</span></tt>, and message 3 will be the output of the <tt class="docutils literal"><span class="pre">Hex_Encoder</span></tt>. This results in message numbers being allocated in a top to bottom fashion, when looked at on the screen. However, note that there could be potential for bugs if this is not anticipated. For example, if your code is passed a filter, and you assume it is a “normal” one that only uses one message, your message offsets would be wrong, leading to some confusion during output.</p> <p>If Fork’s first argument is a null pointer, but a later argument is not, then Fork will feed a copy of its input directly through. Here’s a case where that is useful:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="c1">// have std::string ciphertext, auth_code, key, iv, mac_key;</span> <span class="n">Pipe</span> <span class="nf">pipe</span><span class="p">(</span><span class="k">new</span> <span class="n">Base64_Decoder</span><span class="p">,</span> <span class="n">get_cipher</span><span class="p">(</span><span class="s">"AES-128"</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">iv</span><span class="p">,</span> <span class="n">DECRYPTION</span><span class="p">),</span> <span class="k">new</span> <span class="n">Fork</span><span class="p">(</span> <span class="mi">0</span><span class="p">,</span> <span class="c1">// this message gets ciphertext</span> <span class="k">new</span> <span class="n">MAC_Filter</span><span class="p">(</span><span class="s">"HMAC(SHA-1)"</span><span class="p">,</span> <span class="n">mac_key</span><span class="p">)</span> <span class="p">)</span> <span class="p">);</span> <span class="n">pipe</span><span class="p">.</span><span class="n">process_msg</span><span class="p">(</span><span class="n">ciphertext</span><span class="p">);</span> <span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">plaintext</span> <span class="o">=</span> <span class="n">pipe</span><span class="p">.</span><span class="n">read_all_as_string</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span> <span class="n">SecureVector</span><span class="o"><</span><span class="n">byte</span><span class="o">></span> <span class="n">mac</span> <span class="o">=</span> <span class="n">pipe</span><span class="p">.</span><span class="n">read_all</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="k">if</span><span class="p">(</span><span class="n">mac</span> <span class="o">!=</span> <span class="n">auth_code</span><span class="p">)</span> <span class="n">error</span><span class="p">();</span> </pre></div> </div> <p>Here we wanted to not only decrypt the message, but send the decrypted text through an additional computation, in order to compute the authentication code.</p> <p>Any filters that are attached to the pipe after the fork are implicitly attached onto the first branch created by the fork. For example, let’s say you created this pipe:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">Pipe</span> <span class="nf">pipe</span><span class="p">(</span><span class="k">new</span> <span class="n">Fork</span><span class="p">(</span><span class="k">new</span> <span class="n">Hash_Filter</span><span class="p">(</span><span class="s">"SHA-256"</span><span class="p">),</span> <span class="k">new</span> <span class="n">Hash_Filter</span><span class="p">(</span><span class="s">"SHA-512"</span><span class="p">)),</span> <span class="k">new</span> <span class="n">Hex_Encoder</span><span class="p">);</span> </pre></div> </div> <p>And then called <tt class="docutils literal"><span class="pre">start_msg</span></tt>, inserted some data, then <tt class="docutils literal"><span class="pre">end_msg</span></tt>. Then <tt class="docutils literal"><span class="pre">pipe</span></tt> would contain two messages. The first one (message number 0) would contain the SHA-256 sum of the input in hex encoded form, and the other would contain the SHA-512 sum of the input in raw binary. In many situations you’ll want to perform a sequence of operations on multiple branches of the fork; in which case, use the filter described in <a class="reference internal" href="#chain"><em>Chain</em></a>.</p> </div> <div class="section" id="chain"> <span id="id1"></span><h2>Chain<a class="headerlink" href="#chain" title="Permalink to this headline">¶</a></h2> <p>A <tt class="docutils literal"><span class="pre">Chain</span></tt> filter creates a chain of filters and encapsulates them inside a single filter (itself). This allows a sequence of filters to become a single filter, to be passed into or out of a function, or to a <tt class="docutils literal"><span class="pre">Fork</span></tt> constructor.</p> <p>You can call <tt class="docutils literal"><span class="pre">Chain</span></tt>‘s constructor with up to four <tt class="docutils literal"><span class="pre">Filter</span></tt> pointers (they will be added in order), or with an array of filter pointers and a <tt class="docutils literal"><span class="pre">size_t</span></tt> that tells <tt class="docutils literal"><span class="pre">Chain</span></tt> how many filters are in the array (again, they will be attached in order). Here’s the example from the last section, using chain instead of relying on the implicit passthrough the other version used:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">Pipe</span> <span class="nf">pipe</span><span class="p">(</span><span class="k">new</span> <span class="n">Fork</span><span class="p">(</span> <span class="k">new</span> <span class="n">Chain</span><span class="p">(</span><span class="k">new</span> <span class="n">Hash_Filter</span><span class="p">(</span><span class="s">"SHA-256"</span><span class="p">),</span> <span class="k">new</span> <span class="n">Hex_Encoder</span><span class="p">),</span> <span class="k">new</span> <span class="n">Hash_Filter</span><span class="p">(</span><span class="s">"SHA-512"</span><span class="p">)</span> <span class="p">)</span> <span class="p">);</span> </pre></div> </div> </div> <div class="section" id="sources-and-sinks"> <h2>Sources and Sinks<a class="headerlink" href="#sources-and-sinks" title="Permalink to this headline">¶</a></h2> <div class="section" id="data-sources"> <h3>Data Sources<a class="headerlink" href="#data-sources" title="Permalink to this headline">¶</a></h3> <p>A <tt class="docutils literal"><span class="pre">DataSource</span></tt> is a simple abstraction for a thing that stores bytes. This type is used heavily in the areas of the API related to ASN.1 encoding/decoding. The following types are <tt class="docutils literal"><span class="pre">DataSource</span></tt>: <tt class="docutils literal"><span class="pre">Pipe</span></tt>, <tt class="docutils literal"><span class="pre">SecureQueue</span></tt>, and a couple of special purpose ones: <tt class="docutils literal"><span class="pre">DataSource_Memory</span></tt> and <tt class="docutils literal"><span class="pre">DataSource_Stream</span></tt>.</p> <p>You can create a <tt class="docutils literal"><span class="pre">DataSource_Memory</span></tt> with an array of bytes and a length field. The object will make a copy of the data, so you don’t have to worry about keeping that memory allocated. This is mostly for internal use, but if it comes in handy, feel free to use it.</p> <p>A <tt class="docutils literal"><span class="pre">DataSource_Stream</span></tt> is probably more useful than the memory based one. Its constructors take either a <tt class="docutils literal"><span class="pre">std::istream</span></tt> or a <tt class="docutils literal"><span class="pre">std::string</span></tt>. If it’s a stream, the data source will use the <tt class="docutils literal"><span class="pre">istream</span></tt> to satisfy read requests (this is particularly useful to use with <tt class="docutils literal"><span class="pre">std::cin</span></tt>). If the string version is used, it will attempt to open up a file with that name and read from it.</p> </div> <div class="section" id="data-sinks"> <h3>Data Sinks<a class="headerlink" href="#data-sinks" title="Permalink to this headline">¶</a></h3> <p>A <tt class="docutils literal"><span class="pre">DataSink</span></tt> (in <tt class="docutils literal"><span class="pre">data_snk.h</span></tt>) is a <tt class="docutils literal"><span class="pre">Filter</span></tt> that takes arbitrary amounts of input, and produces no output. This means it’s doing something with the data outside the realm of what <tt class="docutils literal"><span class="pre">Filter</span></tt>/<tt class="docutils literal"><span class="pre">Pipe</span></tt> can handle, for example, writing it to a file (which is what the <tt class="docutils literal"><span class="pre">DataSink_Stream</span></tt> does). There is no need for <tt class="docutils literal"><span class="pre">DataSink``s</span> <span class="pre">that</span> <span class="pre">write</span> <span class="pre">to</span> <span class="pre">a</span> <span class="pre">``std::string</span></tt> or memory buffer, because <tt class="docutils literal"><span class="pre">Pipe</span></tt> can handle that by itself.</p> <p>Here’s a quick example of using a <tt class="docutils literal"><span class="pre">DataSink</span></tt>, which encrypts <tt class="docutils literal"><span class="pre">in.txt</span></tt> and sends the output to <tt class="docutils literal"><span class="pre">out.txt</span></tt>. There is no explicit output operation; the writing of <tt class="docutils literal"><span class="pre">out.txt</span></tt> is implicit:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">DataSource_Stream</span> <span class="nf">in</span><span class="p">(</span><span class="s">"in.txt"</span><span class="p">);</span> <span class="n">Pipe</span> <span class="nf">pipe</span><span class="p">(</span><span class="n">get_cipher</span><span class="p">(</span><span class="s">"AES-128/CTR-BE"</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">iv</span><span class="p">),</span> <span class="k">new</span> <span class="n">DataSink_Stream</span><span class="p">(</span><span class="s">"out.txt"</span><span class="p">));</span> <span class="n">pipe</span><span class="p">.</span><span class="n">process_msg</span><span class="p">(</span><span class="n">in</span><span class="p">);</span> </pre></div> </div> <p>A real advantage of this is that even if “in.txt” is large, only as much memory is needed for internal I/O buffers will be used.</p> </div> </div> <div class="section" id="the-pipe-api"> <h2>The Pipe API<a class="headerlink" href="#the-pipe-api" title="Permalink to this headline">¶</a></h2> <div class="section" id="initializing-pipe"> <h3>Initializing Pipe<a class="headerlink" href="#initializing-pipe" title="Permalink to this headline">¶</a></h3> <p>By default, <tt class="docutils literal"><span class="pre">Pipe</span></tt> will do nothing at all; any input placed into the <tt class="docutils literal"><span class="pre">Pipe</span></tt> will be read back unchanged. Obviously, this has limited utility, and presumably you want to use one or more filters to somehow process the data. First, you can choose a set of filters to initialize the <tt class="docutils literal"><span class="pre">Pipe</span></tt> via the constructor. You can pass it either a set of up to four filter pointers, or a pre-defined array and a length:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">Pipe</span> <span class="nf">pipe1</span><span class="p">(</span><span class="k">new</span> <span class="n">Filter1</span><span class="p">(</span><span class="cm">/*args*/</span><span class="p">),</span> <span class="k">new</span> <span class="n">Filter2</span><span class="p">(</span><span class="cm">/*args*/</span><span class="p">),</span> <span class="k">new</span> <span class="n">Filter3</span><span class="p">(</span><span class="cm">/*args*/</span><span class="p">),</span> <span class="k">new</span> <span class="n">Filter4</span><span class="p">(</span><span class="cm">/*args*/</span><span class="p">));</span> <span class="n">Pipe</span> <span class="nf">pipe2</span><span class="p">(</span><span class="k">new</span> <span class="n">Filter1</span><span class="p">(</span><span class="cm">/*args*/</span><span class="p">),</span> <span class="k">new</span> <span class="n">Filter2</span><span class="p">(</span><span class="cm">/*args*/</span><span class="p">));</span> <span class="n">Filter</span><span class="o">*</span> <span class="n">filters</span><span class="p">[</span><span class="mi">5</span><span class="p">]</span> <span class="o">=</span> <span class="p">{</span> <span class="k">new</span> <span class="n">Filter1</span><span class="p">(</span><span class="cm">/*args*/</span><span class="p">),</span> <span class="k">new</span> <span class="n">Filter2</span><span class="p">(</span><span class="cm">/*args*/</span><span class="p">),</span> <span class="k">new</span> <span class="n">Filter3</span><span class="p">(</span><span class="cm">/*args*/</span><span class="p">),</span> <span class="k">new</span> <span class="n">Filter4</span><span class="p">(</span><span class="cm">/*args*/</span><span class="p">),</span> <span class="k">new</span> <span class="n">Filter5</span><span class="p">(</span><span class="cm">/*args*/</span><span class="p">)</span> <span class="cm">/* more if desired... */</span> <span class="p">};</span> <span class="n">Pipe</span> <span class="nf">pipe3</span><span class="p">(</span><span class="n">filters</span><span class="p">,</span> <span class="mi">5</span><span class="p">);</span> </pre></div> </div> <p>This is by far the most common way to initialize a <tt class="docutils literal"><span class="pre">Pipe</span></tt>. However, occasionally a more flexible initialization strategy is necessary; this is supported by 4 member functions. These functions may only be used while the pipe in question is not in use; that is, either before calling <tt class="docutils literal"><span class="pre">start_msg</span></tt>, or after <tt class="docutils literal"><span class="pre">end_msg</span></tt> has been called (and no new calls to <tt class="docutils literal"><span class="pre">start_msg</span></tt> have been made yet).</p> <dl class="function"> <dt id="Pipe::prepend__FilterP"> void <tt class="descclassname">Pipe::</tt><tt class="descname">prepend</tt><big>(</big>Filter* <em>filter</em><big>)</big><a class="headerlink" href="#Pipe::prepend__FilterP" title="Permalink to this definition">¶</a></dt> <dd><p>Calling <tt class="docutils literal"><span class="pre">prepend</span></tt> will put the passed filter first in the list of transformations. For example, if you prepend a filter implementing encryption, and the pipe already had a filter that hex encoded the input, then the next message processed would be first encrypted, and <em>then</em> hex encoded.</p> </dd></dl> <dl class="function"> <dt id="Pipe::append__FilterP"> void <tt class="descclassname">Pipe::</tt><tt class="descname">append</tt><big>(</big>Filter* <em>filter</em><big>)</big><a class="headerlink" href="#Pipe::append__FilterP" title="Permalink to this definition">¶</a></dt> <dd><p>Like <tt class="docutils literal"><span class="pre">prepend</span></tt>, but places the filter at the end of the message flow. This doesn’t always do what you expect if there is a fork.</p> </dd></dl> <dl class="function"> <dt id="Pipe::pop"> void <tt class="descclassname">Pipe::</tt><tt class="descname">pop</tt><big>(</big><big>)</big><a class="headerlink" href="#Pipe::pop" title="Permalink to this definition">¶</a></dt> <dd><p>Removes the first filter in the flow.</p> </dd></dl> <dl class="function"> <dt id="Pipe::reset"> void <tt class="descclassname">Pipe::</tt><tt class="descname">reset</tt><big>(</big><big>)</big><a class="headerlink" href="#Pipe::reset" title="Permalink to this definition">¶</a></dt> <dd><p>Removes all the filters that the pipe currently holds - it is reset to an empty/no-op state. Any data that is being retained by the pipe is retained after a <tt class="docutils literal"><span class="pre">reset</span></tt>, and <tt class="docutils literal"><span class="pre">reset</span></tt> does not affect message numbers (discussed later).</p> </dd></dl> </div> <div class="section" id="giving-data-to-a-pipe"> <h3>Giving Data to a Pipe<a class="headerlink" href="#giving-data-to-a-pipe" title="Permalink to this headline">¶</a></h3> <p>Input to a <tt class="docutils literal"><span class="pre">Pipe</span></tt> is delimited into messages, which can be read from independently (ie, you can read 5 bytes from one message, and then all of another message, without either read affecting any other messages).</p> <dl class="function"> <dt id="Pipe::start_msg"> void <tt class="descclassname">Pipe::</tt><tt class="descname">start_msg</tt><big>(</big><big>)</big><a class="headerlink" href="#Pipe::start_msg" title="Permalink to this definition">¶</a></dt> <dd><p>Starts a new message; if a message was already running, an exception is thrown. After this function returns, you can call <tt class="docutils literal"><span class="pre">write</span></tt>.</p> </dd></dl> <dl class="function"> <dt id="Pipe::write__byteCP.s"> void <tt class="descclassname">Pipe::</tt><tt class="descname">write</tt><big>(</big>const byte* <em>input</em>, size_t <em>length</em><big>)</big><a class="headerlink" href="#Pipe::write__byteCP.s" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <dl class="function"> <dt id="Pipe::write__MemoryRegion:byte:CR"> void <tt class="descclassname">Pipe::</tt><tt class="descname">write</tt><big>(</big>const MemoryRegion<byte>& <em>input</em><big>)</big><a class="headerlink" href="#Pipe::write__MemoryRegion:byte:CR" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <dl class="function"> <dt id="Pipe::write__ssCR"> void <tt class="descclassname">Pipe::</tt><tt class="descname">write</tt><big>(</big>const std::string& <em>input</em><big>)</big><a class="headerlink" href="#Pipe::write__ssCR" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <dl class="function"> <dt id="Pipe::write__DataSourceR"> void <tt class="descclassname">Pipe::</tt><tt class="descname">write</tt><big>(</big>DataSource& <em>input</em><big>)</big><a class="headerlink" href="#Pipe::write__DataSourceR" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <dl class="function"> <dt id="Pipe::write__byte"> void <tt class="descclassname">Pipe::</tt><tt class="descname">write</tt><big>(</big>byte <em>input</em><big>)</big><a class="headerlink" href="#Pipe::write__byte" title="Permalink to this definition">¶</a></dt> <dd><p>All versions of <tt class="docutils literal"><span class="pre">write</span></tt> write the input into the filter sequence. If a message is not currently active, an exception is thrown.</p> </dd></dl> <dl class="function"> <dt id="Pipe::end_msg"> void <tt class="descclassname">Pipe::</tt><tt class="descname">end_msg</tt><big>(</big><big>)</big><a class="headerlink" href="#Pipe::end_msg" title="Permalink to this definition">¶</a></dt> <dd><p>End the currently active message</p> </dd></dl> <p>Sometimes, you may want to do only a single write per message. In this case, you can use the <tt class="docutils literal"><span class="pre">process_msg</span></tt> series of functions, which start a message, write their argument into the pipe, and then end the message. In this case you would not make any explicit calls to <tt class="docutils literal"><span class="pre">start_msg</span></tt>/<tt class="docutils literal"><span class="pre">end_msg</span></tt>.</p> <p>Pipes can also be used with the <tt class="docutils literal"><span class="pre">>></span></tt> operator, and will accept a <tt class="docutils literal"><span class="pre">std::istream</span></tt>, or on Unix systems with the <tt class="docutils literal"><span class="pre">fd_unix</span></tt> module, a Unix file descriptor. In either case, the entire contents of the file will be read into the pipe.</p> </div> <div class="section" id="getting-output-from-a-pipe"> <h3>Getting Output from a Pipe<a class="headerlink" href="#getting-output-from-a-pipe" title="Permalink to this headline">¶</a></h3> <p>Retrieving the processed data from a pipe is a bit more complicated, for various reasons. The pipe will separate each message into a separate buffer, and you have to retrieve data from each message independently. Each of the reader functions has a final parameter that specifies what message to read from. If this parameter is set to <tt class="docutils literal"><span class="pre">Pipe::DEFAULT_MESSAGE</span></tt>, it will read the current default message (<tt class="docutils literal"><span class="pre">DEFAULT_MESSAGE</span></tt> is also the default value of this parameter).</p> <p>Functions in <tt class="docutils literal"><span class="pre">Pipe</span></tt> related to reading include:</p> <dl class="function"> <dt id="Pipe::read__byteP.s"> size_t <tt class="descclassname">Pipe::</tt><tt class="descname">read</tt><big>(</big>byte* <em>out</em>, size_t <em>len</em><big>)</big><a class="headerlink" href="#Pipe::read__byteP.s" title="Permalink to this definition">¶</a></dt> <dd><p>Reads up to <tt class="docutils literal"><span class="pre">len</span></tt> bytes into <tt class="docutils literal"><span class="pre">out</span></tt>, and returns the number of bytes actually read.</p> </dd></dl> <dl class="function"> <dt id="Pipe::peek__byteP.s"> size_t <tt class="descclassname">Pipe::</tt><tt class="descname">peek</tt><big>(</big>byte* <em>out</em>, size_t <em>len</em><big>)</big><a class="headerlink" href="#Pipe::peek__byteP.s" title="Permalink to this definition">¶</a></dt> <dd><p>Acts exactly like <cite>read</cite>, except the data is not actually read; the next read will return the same data.</p> </dd></dl> <dl class="function"> <dt id="Pipe::read_all"> SecureVector<byte> <tt class="descclassname">Pipe::</tt><tt class="descname">read_all</tt><big>(</big><big>)</big><a class="headerlink" href="#Pipe::read_all" title="Permalink to this definition">¶</a></dt> <dd><p>Reads the entire message into a buffer and returns it</p> </dd></dl> <dl class="function"> <dt id="Pipe::read_all_as_string"> std::string <tt class="descclassname">Pipe::</tt><tt class="descname">read_all_as_string</tt><big>(</big><big>)</big><a class="headerlink" href="#Pipe::read_all_as_string" title="Permalink to this definition">¶</a></dt> <dd><p>Like <tt class="docutils literal"><span class="pre">read_all</span></tt>, but it returns the data as a <tt class="docutils literal"><span class="pre">std::string</span></tt>. No encoding is done; if the message contains raw binary, so will the string.</p> </dd></dl> <dl class="function"> <dt id="Pipe::remaining"> size_t <tt class="descclassname">Pipe::</tt><tt class="descname">remaining</tt><big>(</big><big>)</big><a class="headerlink" href="#Pipe::remaining" title="Permalink to this definition">¶</a></dt> <dd><p>Returns how many bytes are left in the message</p> </dd></dl> <dl class="function"> <dt id="Pipe::default_msg"> Pipe::message_id <tt class="descclassname">Pipe::</tt><tt class="descname">default_msg</tt><big>(</big><big>)</big><a class="headerlink" href="#Pipe::default_msg" title="Permalink to this definition">¶</a></dt> <dd><p>Returns the current default message number</p> </dd></dl> <dl class="function"> <dt id="Pipe::message_count"> Pipe::message_id <tt class="descclassname">Pipe::</tt><tt class="descname">message_count</tt><big>(</big><big>)</big><a class="headerlink" href="#Pipe::message_count" title="Permalink to this definition">¶</a></dt> <dd><p>Returns the total number of messages currently in the pipe</p> </dd></dl> <dl class="function"> <dt id="Pipe::set_default_msg__Pipe::message_id"> <tt class="descclassname">Pipe::</tt><tt class="descname">set_default_msg</tt><big>(</big>Pipe::message_id <em>msgno</em><big>)</big><a class="headerlink" href="#Pipe::set_default_msg__Pipe::message_id" title="Permalink to this definition">¶</a></dt> <dd><p>Sets the default message number (which must be a valid message number for that pipe). The ability to set the default message number is particularly important in the case of using the file output operations (<tt class="docutils literal"><span class="pre"><<</span></tt> with a <tt class="docutils literal"><span class="pre">std::ostream</span></tt> or Unix file descriptor), because there is no way to specify the message explicitly when using the output operator.</p> </dd></dl> </div> <div class="section" id="pipe-i-o-for-unix-file-descriptors"> <h3>Pipe I/O for Unix File Descriptors<a class="headerlink" href="#pipe-i-o-for-unix-file-descriptors" title="Permalink to this headline">¶</a></h3> <p>This is a minor feature, but it comes in handy sometimes. In all installations of the library, Botan’s <tt class="docutils literal"><span class="pre">Pipe</span></tt> object overloads the <tt class="docutils literal"><span class="pre"><<</span></tt> and <tt class="docutils literal"><span class="pre">>></span></tt> operators for C++ iostream objects, which is usually more than sufficient for doing I/O.</p> <p>However, there are cases where the iostream hierarchy does not map well to local ‘file types’, so there is also the ability to do I/O directly with Unix file descriptors. This is most useful when you want to read from or write to something like a TCP or Unix-domain socket, or a pipe, since for simple file access it’s usually easier to just use C++’s file streams.</p> <p>If <tt class="docutils literal"><span class="pre">BOTAN_EXT_PIPE_UNIXFD_IO</span></tt> is defined, then you can use the overloaded I/O operators with Unix file descriptors. For an example of this, check out the <tt class="docutils literal"><span class="pre">hash_fd</span></tt> example, included in the Botan distribution.</p> </div> </div> <div class="section" id="filter-catalog"> <h2>Filter Catalog<a class="headerlink" href="#filter-catalog" title="Permalink to this headline">¶</a></h2> <p>This section documents most of the useful filters included in the library.</p> <div class="section" id="keyed-filters"> <h3>Keyed Filters<a class="headerlink" href="#keyed-filters" title="Permalink to this headline">¶</a></h3> <p>A few sections ago, it was mentioned that <tt class="docutils literal"><span class="pre">Pipe</span></tt> can process multiple messages, treating each of them the same. Well, that was a bit of a lie. There are some algorithms (in particular, block ciphers not in ECB mode, and all stream ciphers) that change their state as data is put through them.</p> <p>Naturally, you might well want to reset the keys or (in the case of block cipher modes) IVs used by such filters, so multiple messages can be processed using completely different keys, or new IVs, or new keys and IVs, or whatever. And in fact, even for a MAC or an ECB block cipher, you might well want to change the key used from message to message.</p> <p>Enter <tt class="docutils literal"><span class="pre">Keyed_Filter</span></tt>, which acts as an abstract interface for any filter that is uses keys: block cipher modes, stream ciphers, MACs, and so on. It has two functions, <tt class="docutils literal"><span class="pre">set_key</span></tt> and <tt class="docutils literal"><span class="pre">set_iv</span></tt>. Calling <tt class="docutils literal"><span class="pre">set_key</span></tt> will set (or reset) the key used by the algorithm. Setting the IV only makes sense in certain algorithms – a call to <tt class="docutils literal"><span class="pre">set_iv</span></tt> on an object that doesn’t support IVs will cause an exception. You must call <tt class="docutils literal"><span class="pre">set_key</span></tt> <em>before</em> calling <tt class="docutils literal"><span class="pre">set_iv</span></tt>.</p> <p>Here’s a example:</p> <div class="highlight-cpp"><div class="highlight"><pre><span class="n">Keyed_Filter</span> <span class="o">*</span><span class="n">aes</span><span class="p">,</span> <span class="o">*</span><span class="n">hmac</span><span class="p">;</span> <span class="n">Pipe</span> <span class="nf">pipe</span><span class="p">(</span><span class="k">new</span> <span class="n">Base64_Decoder</span><span class="p">,</span> <span class="c1">// Note the assignments to the cast and hmac variables</span> <span class="n">aes</span> <span class="o">=</span> <span class="n">get_cipher</span><span class="p">(</span><span class="s">"AES-128/CBC"</span><span class="p">,</span> <span class="n">aes_key</span><span class="p">,</span> <span class="n">iv</span><span class="p">),</span> <span class="k">new</span> <span class="n">Fork</span><span class="p">(</span> <span class="mi">0</span><span class="p">,</span> <span class="c1">// Read the section 'Fork' to understand this</span> <span class="k">new</span> <span class="n">Chain</span><span class="p">(</span> <span class="n">hmac</span> <span class="o">=</span> <span class="k">new</span> <span class="n">MAC_Filter</span><span class="p">(</span><span class="s">"HMAC(SHA-1)"</span><span class="p">,</span> <span class="n">mac_key</span><span class="p">,</span> <span class="mi">12</span><span class="p">),</span> <span class="k">new</span> <span class="n">Base64_Encoder</span> <span class="p">)</span> <span class="p">)</span> <span class="p">);</span> <span class="n">pipe</span><span class="p">.</span><span class="n">start_msg</span><span class="p">();</span> <span class="c1">// use pipe for a while, decrypt some stuff, derive new keys and IVs</span> <span class="n">pipe</span><span class="p">.</span><span class="n">end_msg</span><span class="p">();</span> <span class="n">aes</span><span class="o">-></span><span class="n">set_key</span><span class="p">(</span><span class="n">aes_key2</span><span class="p">);</span> <span class="n">aes</span><span class="o">-></span><span class="n">set_iv</span><span class="p">(</span><span class="n">iv2</span><span class="p">);</span> <span class="n">hmac</span><span class="o">-></span><span class="n">set_key</span><span class="p">(</span><span class="n">mac_key2</span><span class="p">);</span> <span class="n">pipe</span><span class="p">.</span><span class="n">start_msg</span><span class="p">();</span> <span class="c1">// use pipe for some other things</span> <span class="n">pipe</span><span class="p">.</span><span class="n">end_msg</span><span class="p">();</span> </pre></div> </div> <p>There are some requirements to using <tt class="docutils literal"><span class="pre">Keyed_Filter</span></tt> that you must follow. If you call <tt class="docutils literal"><span class="pre">set_key</span></tt> or <tt class="docutils literal"><span class="pre">set_iv</span></tt> on a filter that is owned by a <tt class="docutils literal"><span class="pre">Pipe</span></tt>, you must do so while the <tt class="docutils literal"><span class="pre">Pipe</span></tt> is “unlocked”. This refers to the times when no messages are being processed by <tt class="docutils literal"><span class="pre">Pipe</span></tt> – either before <tt class="docutils literal"><span class="pre">Pipe</span></tt>‘s <tt class="docutils literal"><span class="pre">start_msg</span></tt> is called, or after <tt class="docutils literal"><span class="pre">end_msg</span></tt> is called (and no new call to <tt class="docutils literal"><span class="pre">start_msg</span></tt> has happened yet). Doing otherwise will result in undefined behavior, probably silently getting invalid output.</p> <p>And remember: if you’re resetting both values, reset the key <em>first</em>.</p> </div> <div class="section" id="cipher-filters"> <h3>Cipher Filters<a class="headerlink" href="#cipher-filters" title="Permalink to this headline">¶</a></h3> <p>Getting a hold of a <tt class="docutils literal"><span class="pre">Filter</span></tt> implementing a cipher is very easy. Make sure you’re including the header <tt class="docutils literal"><span class="pre">lookup.h</span></tt>, and then call <tt class="docutils literal"><span class="pre">get_cipher</span></tt>. You will pass the return value directly into a <tt class="docutils literal"><span class="pre">Pipe</span></tt>. There are a couple different functions which do varying levels of initialization:</p> <dl class="function"> <dt id="get_cipher__ss.SymmetricKey.InitializationVector.Cipher_Dir"> Keyed_Filter* <tt class="descname">get_cipher</tt><big>(</big>std::string <em>cipher_spec</em>, SymmetricKey <em>key</em>, InitializationVector <em>iv</em>, Cipher_Dir <em>dir</em><big>)</big><a class="headerlink" href="#get_cipher__ss.SymmetricKey.InitializationVector.Cipher_Dir" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <dl class="function"> <dt id="get_cipher__ss.SymmetricKey.Cipher_Dir"> Keyed_Filter* <tt class="descname">get_cipher</tt><big>(</big>std::string <em>cipher_spec</em>, SymmetricKey <em>key</em>, Cipher_Dir <em>dir</em><big>)</big><a class="headerlink" href="#get_cipher__ss.SymmetricKey.Cipher_Dir" title="Permalink to this definition">¶</a></dt> <dd></dd></dl> <p>The version that doesn’t take an IV is useful for things that don’t use them, like block ciphers in ECB mode, or most stream ciphers. If you specify a cipher spec that does want a IV, and you use the version that doesn’t take one, an exception will be thrown. The <tt class="docutils literal"><span class="pre">dir</span></tt> argument can be either <tt class="docutils literal"><span class="pre">ENCRYPTION</span></tt> or <tt class="docutils literal"><span class="pre">DECRYPTION</span></tt>.</p> <p>The cipher_spec is a string that specifies what cipher is to be used. The general syntax for “cipher_spec” is “STREAM_CIPHER”, “BLOCK_CIPHER/MODE”, or “BLOCK_CIPHER/MODE/PADDING”. In the case of stream ciphers, no mode is necessary, so just the name is sufficient. A block cipher requires a mode of some sort, which can be “ECB”, “CBC”, “CFB(n)”, “OFB”, “CTR-BE”, or “EAX(n)”. The argument to CFB mode is how many bits of feedback should be used. If you just use “CFB” with no argument, it will default to using a feedback equal to the block size of the cipher. EAX mode also takes an optional bit argument, which tells EAX how large a tag size to use~–~generally this is the size of the block size of the cipher, which is the default if you don’t specify any argument.</p> <p>In the case of the ECB and CBC modes, a padding method can also be specified. If it is not supplied, ECB defaults to not padding, and CBC defaults to using PKCS #5/#7 compatible padding. The padding methods currently available are “NoPadding”, “PKCS7”, “OneAndZeros”, and “CTS”. CTS padding is currently only available for CBC mode, but the others can also be used in ECB mode.</p> <p>Some example “cipher_spec arguments are: “AES-128/CBC”, “Blowfish/CTR-BE”, “Serpent/XTS”, and “AES-256/EAX”.</p> <p>“CTR-BE” refers to counter mode where the counter is incremented as if it were a big-endian encoded integer. This is compatible with most other implementations, but it is possible some will use the incompatible little endian convention. This version would be denoted as “CTR-LE” if it were supported.</p> <p>“EAX” is a new cipher mode designed by Wagner, Rogaway, and Bellare. It is an authenticated cipher mode (that is, no separate authentication is needed), has provable security, and is free from patent entanglements. It runs about half as fast as most of the other cipher modes (like CBC, OFB, or CTR), which is not bad considering you don’t need to use an authentication code.</p> </div> <div class="section" id="hashes-and-macs"> <h3>Hashes and MACs<a class="headerlink" href="#hashes-and-macs" title="Permalink to this headline">¶</a></h3> <p>Hash functions and MACs don’t need anything special when it comes to filters. Both just take their input and produce no output until <tt class="docutils literal"><span class="pre">end_msg</span></tt> is called, at which time they complete the hash or MAC and send that as output.</p> <p>These filters take a string naming the type to be used. If for some reason you name something that doesn’t exist, an exception will be thrown.</p> <dl class="function"> <dt id="Hash_Filter::Hash_Filter__ss.s"> <tt class="descclassname">Hash_Filter::</tt><tt class="descname">Hash_Filter</tt><big>(</big>std::string <em>hash</em>, size_t <em>outlen</em><em>=0</em><big>)</big><a class="headerlink" href="#Hash_Filter::Hash_Filter__ss.s" title="Permalink to this definition">¶</a></dt> <dd><p>This constructor creates a filter that hashes its input with <tt class="docutils literal"><span class="pre">hash</span></tt>. When <tt class="docutils literal"><span class="pre">end_msg</span></tt> is called on the owning pipe, the hash is completed and the digest is sent on to the next filter in the pipeline. The parameter <tt class="docutils literal"><span class="pre">outlen</span></tt> specifies how many bytes of the hash output will be passed along to the next filter when <tt class="docutils literal"><span class="pre">end_msg</span></tt> is called. By default, it will pass the entire hash.</p> <p>Examples of names for <tt class="docutils literal"><span class="pre">Hash_Filter</span></tt> are “SHA-1” and “Whirlpool”.</p> </dd></dl> <dl class="function"> <dt id="MAC_Filter::MAC_Filter__ss.SymmetricKey.s"> <tt class="descclassname">MAC_Filter::</tt><tt class="descname">MAC_Filter</tt><big>(</big>std::string <em>mac</em>, SymmetricKey <em>key</em>, size_t <em>outlen</em><em>=0</em><big>)</big><a class="headerlink" href="#MAC_Filter::MAC_Filter__ss.SymmetricKey.s" title="Permalink to this definition">¶</a></dt> <dd><p>This constructor takes a name for a mac, such as “HMAC(SHA-1)” or “CMAC(AES-128)”, along with a key to use. The optional <tt class="docutils literal"><span class="pre">outlen</span></tt> works the same as in <tt class="docutils literal"><span class="pre">Hash_Filter</span></tt>.</p> </dd></dl> </div> <div class="section" id="pk-filters"> <h3>PK Filters<a class="headerlink" href="#pk-filters" title="Permalink to this headline">¶</a></h3> <p>There are four classes in this category, <tt class="docutils literal"><span class="pre">PK_Encryptor_Filter</span></tt>, <tt class="docutils literal"><span class="pre">PK_Decryptor_Filter</span></tt>, <tt class="docutils literal"><span class="pre">PK_Signer_Filter</span></tt>, and <tt class="docutils literal"><span class="pre">PK_Verifier_Filter</span></tt>. Each takes a pointer to an object of the appropriate type (<tt class="docutils literal"><span class="pre">PK_Encryptor</span></tt>, <tt class="docutils literal"><span class="pre">PK_Decryptor</span></tt>, etc) that is deleted by the destructor. These classes are found in <tt class="docutils literal"><span class="pre">pk_filts.h</span></tt>.</p> <p>Three of these, for encryption, decryption, and signing are much the same in terms of dataflow - ach of them buffers its input until the end of the message is marked with a call to the <tt class="docutils literal"><span class="pre">end_msg</span></tt> function. Then they encrypt, decrypt, or sign the entire input as a single blob and send the output (the ciphertext, the plaintext, or the signature) into the next filter.</p> <p>Signature verification works a little differently, because it needs to know what the signature is in order to check it. You can either pass this in along with the constructor, or call the function <tt class="docutils literal"><span class="pre">set_signature</span></tt> – with this second method, you need to keep a pointer to the filter around so you can send it this command. In either case, after <tt class="docutils literal"><span class="pre">end_msg</span></tt> is called, it will try to verify the signature (if the signature has not been set by either method, an exception will be thrown here). It will then send a single byte onto the next filter – a 1 or a 0, which specifies whether the signature verified or not (respectively).</p> <p>For more information about PK algorithms (including creating the appropriate objects to pass to the constructors), see <a class="reference internal" href="pubkey.html"><em>Public Key Cryptography</em></a>.</p> </div> <div class="section" id="encoders"> <h3>Encoders<a class="headerlink" href="#encoders" title="Permalink to this headline">¶</a></h3> <p>Often you want your data to be in some form of text (for sending over channels that aren’t 8-bit clean, printing it, etc). The filters <tt class="docutils literal"><span class="pre">Hex_Encoder</span></tt> and <tt class="docutils literal"><span class="pre">Base64_Encoder</span></tt> will convert arbitrary binary data into hex or base64 formats. Not surprisingly, you can use <tt class="docutils literal"><span class="pre">Hex_Decoder</span></tt> and <tt class="docutils literal"><span class="pre">Base64_Decoder</span></tt> to convert it back into its original form.</p> <p>Both of the encoders can take a few options about how the data should be formatted (all of which have defaults). The first is a <tt class="docutils literal"><span class="pre">bool</span></tt> which says if the encoder should insert line breaks. This defaults to false. Line breaks don’t matter either way to the decoder, but it makes the output a bit more appealing to the human eye, and a few transport mechanisms (notably some email systems) limit the maximum line length.</p> <p>The second encoder option is an integer specifying how long such lines will be (obviously this will be ignored if line-breaking isn’t being used). The default tends to be in the range of 60-80 characters, but is not specified. If you want a specific value, set it. Otherwise the default should be fine.</p> <p>Lastly, <tt class="docutils literal"><span class="pre">Hex_Encoder</span></tt> takes an argument of type <tt class="docutils literal"><span class="pre">Case</span></tt>, which can be <tt class="docutils literal"><span class="pre">Uppercase</span></tt> or <tt class="docutils literal"><span class="pre">Lowercase</span></tt> (default is <tt class="docutils literal"><span class="pre">Uppercase</span></tt>). This specifies what case the characters A-F should be output as. The base64 encoder has no such option, because it uses both upper and lower case letters for its output.</p> <p>You can find the declarations for these types in <tt class="docutils literal"><span class="pre">hex_filt.h</span></tt> and <tt class="docutils literal"><span class="pre">b64_filt.h</span></tt>.</p> </div> <div class="section" id="compressors"> <h3>Compressors<a class="headerlink" href="#compressors" title="Permalink to this headline">¶</a></h3> <p>There are two compression algorithms supported by Botan, zlib and bzip2. Only lossless compression algorithms are currently supported by Botan, because they tend to be the most useful for cryptography. However, it is very reasonable to consider supporting something like GSM speech encoding (which is lossy), for use in encrypted voice applications.</p> <p>You should always compress <em>before</em> you encrypt, because encryption seeks to hide the redundancy that compression is supposed to try to find and remove.</p> <p>To test for Bzip2, check to see if <tt class="docutils literal"><span class="pre">BOTAN_HAS_COMPRESSOR_BZIP2</span></tt> is defined. If so, you can include <tt class="docutils literal"><span class="pre">botan/bzip2.h</span></tt>, which will declare a pair of <tt class="docutils literal"><span class="pre">Filter</span></tt> objects: <tt class="docutils literal"><span class="pre">Bzip2_Compression</span></tt> and <tt class="docutils literal"><span class="pre">Bzip2_Decompression</span></tt>.</p> <p>You should be prepared to take an exception when using the decompressing filter, for if the input is not valid bzip2 data, that is what you will receive. You can specify the desired level of compression to <tt class="docutils literal"><span class="pre">Bzip2_Compression</span></tt>‘s constructor as an integer between 1 and 9, 1 meaning worst compression, and 9 meaning the best. The default is to use 9, since small values take the same amount of time, just use a little less memory.</p> <p>Zlib compression works much like Bzip2 compression. The only differences in this case are that the macro is <tt class="docutils literal"><span class="pre">BOTAN_HAS_COMPRESSOR_ZLIB</span></tt>, the header you need to include is called <tt class="docutils literal"><span class="pre">botan/zlib.h</span></tt> (remember that you shouldn’t just <tt class="docutils literal"><span class="pre">#include</span> <span class="pre"><zlib.h></span></tt>, or you’ll get the regular zlib API, which is not what you want). The Botan classes for zlib compression/decompression are called <tt class="docutils literal"><span class="pre">Zlib_Compression</span></tt> and <tt class="docutils literal"><span class="pre">Zlib_Decompression</span></tt>.</p> <p>Like Bzip2, a <tt class="docutils literal"><span class="pre">Zlib_Decompression</span></tt> object will throw an exception if invalid (in the sense of not being in the Zlib format) data is passed into it.</p> <p>While the zlib compression library uses the same compression algorithm as the gzip and zip programs, the format is different. The zlib format is defined in RFC 1950.</p> </div> </div> <div class="section" id="writing-new-filters"> <h2>Writing New Filters<a class="headerlink" href="#writing-new-filters" title="Permalink to this headline">¶</a></h2> <p>The system of filters and pipes was designed in an attempt to make it as simple as possible to write new filter types. There are four functions that need to be implemented by a class deriving from <tt class="docutils literal"><span class="pre">Filter</span></tt>:</p> <dl class="function"> <dt id="Filter::write__byteCP.s"> void <tt class="descclassname">Filter::</tt><tt class="descname">write</tt><big>(</big>const byte* <em>input</em>, size_t <em>length</em><big>)</big><a class="headerlink" href="#Filter::write__byteCP.s" title="Permalink to this definition">¶</a></dt> <dd><p>This function is what is called when a filter receives input for it to process. The filter is not required to process the data right away; many filters buffer their input before producing any output. A filter will usually have <tt class="docutils literal"><span class="pre">write</span></tt> called many times during its lifetime.</p> </dd></dl> <dl class="function"> <dt id="Filter::send__byteP.s"> void <tt class="descclassname">Filter::</tt><tt class="descname">send</tt><big>(</big>byte* <em>output</em>, size_t <em>length</em><big>)</big><a class="headerlink" href="#Filter::send__byteP.s" title="Permalink to this definition">¶</a></dt> <dd><p>Eventually, a filter will want to produce some output to send along to the next filter in the pipeline. It does so by calling <tt class="docutils literal"><span class="pre">send</span></tt> with whatever it wants to send along to the next filter. There is also a version of <tt class="docutils literal"><span class="pre">send</span></tt> taking a single byte argument, as a convenience.</p> </dd></dl> <dl class="function"> <dt id="Filter::start_msg"> void <tt class="descclassname">Filter::</tt><tt class="descname">start_msg</tt><big>(</big><big>)</big><a class="headerlink" href="#Filter::start_msg" title="Permalink to this definition">¶</a></dt> <dd><p>Implementing this function is optional. Implement it if your filter would like to do some processing or setup at the start of each message, such as allocating a data structure.</p> </dd></dl> <dl class="function"> <dt id="Filter::end_msg"> void <tt class="descclassname">Filter::</tt><tt class="descname">end_msg</tt><big>(</big><big>)</big><a class="headerlink" href="#Filter::end_msg" title="Permalink to this definition">¶</a></dt> <dd><p>Implementing this function is optional. It is called when it has been requested that filters finish up their computations. The filter should finish up with whatever computation it is working on (for example, a compressing filter would flush the compressor and <tt class="docutils literal"><span class="pre">send</span></tt> the final block), and empty any buffers in preparation for processing a fresh new set of input.</p> </dd></dl> <p>Additionally, if necessary, filters can define a constructor that takes any needed arguments, and a destructor to deal with deallocating memory, closing files, etc.</p> </div> </div> </div> </div> </div> </div> <div class="sidebar"> <h3>Table Of Contents</h3> <ul class="current"> <li class="toctree-l1"><a class="reference internal" href="index.html">Welcome</a></li> <li class="toctree-l1"><a class="reference internal" href="reading.html">Recommended Reading</a></li> <li class="toctree-l1"><a class="reference internal" href="building.html">Building The Library</a></li> <li class="toctree-l1"><a class="reference internal" href="firststep.html">Getting Started</a></li> <li class="toctree-l1 current"><a class="current reference internal" href="">Information Flow: Pipes and Filters</a><ul> <li class="toctree-l2"><a class="reference internal" href="#fork">Fork</a></li> <li class="toctree-l2"><a class="reference internal" href="#chain">Chain</a></li> <li class="toctree-l2"><a class="reference internal" href="#sources-and-sinks">Sources and Sinks</a></li> <li class="toctree-l2"><a class="reference internal" href="#the-pipe-api">The Pipe API</a></li> <li class="toctree-l2"><a class="reference internal" href="#filter-catalog">Filter Catalog</a></li> <li class="toctree-l2"><a class="reference internal" href="#writing-new-filters">Writing New Filters</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="pubkey.html">Public Key Cryptography</a></li> <li class="toctree-l1"><a class="reference internal" href="x509.html">Certificate Handling</a></li> <li class="toctree-l1"><a class="reference internal" href="ssl.html">SSL and TLS</a></li> <li class="toctree-l1"><a class="reference internal" href="bigint.html">BigInt</a></li> <li class="toctree-l1"><a class="reference internal" href="lowlevel.html">The Low-Level Interface</a></li> <li class="toctree-l1"><a class="reference internal" href="secmem.html">Secure Memory Containers</a></li> <li class="toctree-l1"><a class="reference internal" href="kdf.html">Key Derivation Functions</a></li> <li class="toctree-l1"><a class="reference internal" href="pbkdf.html">PBKDF Algorithms</a></li> <li class="toctree-l1"><a class="reference internal" href="passhash.html">Password Hashing</a></li> <li class="toctree-l1"><a class="reference internal" href="rng.html">Random Number Generators</a></li> <li class="toctree-l1"><a class="reference internal" href="fpe.html">Format Preserving Encryption</a></li> <li class="toctree-l1"><a class="reference internal" href="python.html">Python Binding</a></li> </ul> <h3 style="margin-top: 1.5em;">Search</h3> <form class="search" action="search.html" method="get"> <input type="text" name="q" /> <input type="submit" value="Go" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> <p class="searchtip" style="font-size: 90%"> Enter search terms or a module, class or function name. </p> </div> <div class="clearer"></div> </div> </div> <div class="footer-wrapper"> <div class="footer"> <div class="left"> <a href="firststep.html" title="Getting Started" accesskey="P">previous</a> | <a href="pubkey.html" title="Public Key Cryptography" accesskey="N">next</a> | <a href="genindex.html" title="General Index" accesskey="I">index</a> <br/> <a href="_sources/filters.txt" rel="nofollow">Show Source</a> </div> <div class="right"> <div class="footer"> © Copyright 2000-2011, Jack Lloyd. Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.2.3. </div> </div> <div class="clearer"></div> </div> </div> </body> </html>