1<?xml version="1.0" standalone="no"?>
2<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
3"http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"
4[
5]>
6
7<article id="index">
8  <articleinfo>
9    <title>D-Bus Test Plan</title>
10    <date>14 February 2003</date>
11    <authorgroup>
12      <author>
13	<firstname>Anders</firstname>
14	<surname>Carlsson</surname>
15	<affiliation>
16	  <orgname>CodeFactory AB</orgname>
17	  <address><email>andersca@codefactory.se</email></address>
18	</affiliation>
19      </author>
20    </authorgroup>
21  </articleinfo>
22  <sect1 id="introduction">
23    <title>Introduction</title>
24    <para>
25      This document tries to explain the details of the test plan for D-Bus
26    </para>
27    <sect2 id="importance-of-testing">
28      <title>The importance of testing</title>
29      <para>
30        As with any big library or program, testing is important. It
31        can help find bugs and regressions and make the code better
32        overall. 
33      </para>
34      <para>
35        D-Bus is a large and complex piece of software (about 25,000
36        lines of code for the client library, and 2,500 lines of code
37        for the bus daemon) and it's therefore important to try to make sure
38        that all parts of the software is functioning correctly.
39      </para>
40      <para>
41        D-Bus can be built with support for testing by passing
42        <literal>--enable-tests</literal>. to the configure script. It
43        is recommended that production systems build without testing
44        since that reduces the D-Bus client library size.
45      </para>
46    </sect2>
47  </sect1>
48  <sect1 id="client-library">
49    <title>Testing the D-Bus client library</title>
50    <para>
51      The tests for the client library consist of the dbus-test
52      program which is a unit test for all aspects of the client
53      library. Whenever a bug in the client library is found and
54      fixed, a test is added to make sure that the bug won't occur again.
55    </para>
56    <sect2 id="data-structures">
57      <title>Data Structures</title>
58      <para>
59      The D-Bus client library consists of some data structures that
60      are used internally; a linked list class, a hashtable class and
61      a string class. All aspects of those are tested by dbus-test.
62      </para>
63    </sect2>
64    <sect2 id="message-loader">
65      <title>Message loader</title>
66      <para>
67        The message loader is the part of D-Bus that takes messages in
68        raw character form and parses them, turning them into DBusMessages.
69      </para>
70      <para>
71        This is one of the parts of D-Bus that
72        <emphasis>must</emphasis> be absolutely bug-free and
73        robust. The message loader should be able to handle invalid
74        and incomplete messages without crashing. Not doing so is a
75        serious issue and can easily result in D-Bus being exploitable
76        to DoS attacks.
77      </para>
78      <para>
79        To solve these problems, there is a testing feature called the
80        Message Builder. The message builder can take a serialized
81        message in string-form and convert it into a raw character
82        string which can then be loaded by the message loader. 
83      </para>
84      <figure>
85	<title>Example of a message in string form</title>
86	<programlisting>
87          # Standard org.freedesktop.DBus.Hello message
88
89          VALID_HEADER
90          FIELD_NAME name
91          TYPE STRING
92          STRING 'org.freedesktop.DBus.Hello'
93          FIELD_NAME srvc
94          TYPE STRING
95          STRING 'org.freedesktop.DBus'
96          ALIGN 8
97          END_LENGTH Header
98          START_LENGTH Body
99          END_LENGTH Body
100        </programlisting>
101      </figure>
102      <para>
103        The file format of messages in string form is documented in
104        the D-Bus Reference Manual.
105      </para>
106      <para>
107        The message test part of dbus-test is using the message
108        builder to build different kinds of messages, both valid,
109        invalid, and invalid ones, to make sure that the loader won't
110        crash or leak memory of any of those, and that the loader
111        knows if a message is valid or not.
112      </para>
113      <para>
114        There is also a test program called
115        <literal>break-loader</literal> that loads a message in
116        string-form into raw character form using the message
117        builder. It then randomly changes the message, it can for
118        example replace single bytes of data or modify the length of
119        the message. This is to simulate network errors. The
120        break-loader program saves all the messages leading to errors
121        so it can easily be run for a long period of time.
122      </para>
123    </sect2>
124    <sect2 id="authentication">
125      <title>Authentication</title>
126      <para>
127        For testing authentication, there is a testing feature that
128        can read authentication sequences from a file and play them
129        back to a dummy server and client to make sure that
130        authentication is working according to the specification.
131      </para>
132      <figure>
133	<title>Example of an authentication script</title>
134	<programlisting>
135          ## this tests a successful auth of type EXTERNAL
136          
137          SERVER
138          SEND 'AUTH EXTERNAL USERNAME_HEX'
139          EXPECT_COMMAND OK
140          EXPECT_STATE WAITING_FOR_INPUT
141          SEND 'BEGIN'
142          EXPECT_STATE AUTHENTICATED
143        </programlisting>
144      </figure>
145    </sect2>
146  </sect1>
147  <sect1 id="daemon">
148    <title>Testing the D-Bus bus daemon</title>
149    <para>
150      Since the D-Bus bus daemon is using the D-Bus client library it
151      will benefit from all tests done on the client library, but
152      there is still the issue of testing client-server communication.
153      This is more complicated since it it may require another process
154      running.
155    </para>
156    <sect2 id="debug-transport">
157      <title>The debug transport</title>
158      <para>
159        In D-Bus, a <emphasis>transport</emphasis> is a class that
160        handles sending and receiving raw data over a certain
161        medium. The transport that is used most in D-Bus is the UNIX
162        transport with sends and recevies data over a UNIX socket. A
163        transport that tunnels data through X11 client messages is
164        also under development.
165      </para>
166      <para>
167        The D-Bus debug transport is a specialized transport that
168        works in-process. This means that a client and server that
169        exists in the same process can talk to eachother without using
170        a socket.
171      </para>
172    </sect2>
173    <sect2 id="bus-test">
174      <title>The bus-test program</title>
175      <para>
176        The bus-test program is a program that is used to test various
177        parts of the D-Bus bus daemon; robustness and that it conforms
178        to the specifications.
179      </para>
180      <para>
181        The test program has the necessary code from the bus daemon
182        linked in, and it uses the debug transport for
183        communication. This means that the bus daemon code can be
184        tested without the real bus actually running, which makes
185        testing easier.
186      </para>
187      <para>
188        The bus-test program should test all major features of the
189        bus, such as service registration, notification when things
190        occurs and message matching.
191      </para>
192    </sect2>
193  </sect1>
194  <sect1 id="other-tests">
195    <title>Other tests</title>
196
197    <sect2 id="oom-robustness">
198      <title>Out-Of-Memory robustness</title>
199      <para>
200        Since D-Bus should be able to be used in embedded devices, and
201        also as a system service, it should be able to cope with
202        low-memory situations without exiting or crashing.
203      </para>
204      <para>
205        In practice, this means that both the client and server code
206        must be able to handle dbus_malloc returning NULL. 
207      </para>
208      <para>
209        To test this, two environment variables
210        exist. <literal>DBUS_MALLOC_FAIL_NTH</literal> will make every
211        nth call to dbus_malloc return NULL, and
212        <literal>DBUS_MALLOC_FAIL_GREATER_THAN</literal> will make any
213        dbus_malloc call with a request for more than the specified
214        number of bytes fail.
215      </para>
216    </sect2>
217
218    <sect2 id="leaks-and-other-stuff">
219      <title>Memory leaks and code robustness</title> 
220      <para>
221        Naturally there are some things that tests can't be written
222        for, for example things like memory leaks and out-of-bounds
223        memory reading or writing.
224      </para>
225      <para>
226        Luckily there exists good tools for catching such errors. One
227        free good tool is <ulink url="http://devel-home.kde.org/~sewardj/">Valgrind</ulink>, which runs the program in a
228        virtual CPU which makes catching errors easy. All test programs can be run under Valgrind, 
229      </para>
230    </sect2>
231  </sect1>
232</article>
233