1Conformance Test:  Definition and Structure
2===========================================
3
4This document describes what conformance tests are, how they are created,
5and the structure they follow in CVS.
6
7Conformance Test Definition
8---------------------------
9
10Conformance tests test conformance to the POSIX* specification.  They test
11both that the header files are correct and that the behavior defined in
12the specification is implemented.
13
14Directory Structure
15-------------------
16
17The directory structure for conformance tests is as follows.
18
19All tests are located in the conformance/ directory off of posixtestsuite/.
20
21Within this directory, there are the following three subdirectories:
22
23definitions/ - This directory contains tests for each *.h file in the POSIX 
24spec.  Tests are taken from the Base Definitions document.
25
26interfaces/ - This directory contains tests for each function listed in the 
27POSIX spec.  Tests are taken from the System Interfaces document.
28
29behavior/ - This directory contains tests from any document that do not 
30directly correlate to functions (as in interfaces/) or header files 
31(as in definitions/).
32
33The breakdown of these subdirectories is as follows:
34
35definitions/
36-----------
37The definitions/ directory contains one subdirectory per *.h file.  Each
38subdirectory should have the name *_h (i.e., the *.h file name with "_"
39replacing ".").
40
41interfaces/
42----------
43The interfaces directory contains one subdirectory per function listed
44in the System Interfaces document.
45
46behavior/
47--------
48This directory is broken down by functional area in a manner similar to
49functional and stress tests (see HOWTO_<TBD> - for now, see email discussions
50on functional/stress test structure).
51
52Writing Tests
53-------------
54
55Within each lowest level directory (definitions/*_h for definitions, 
56interfaces/<function> for interfaces, and behavior/<functional area> for
57behavior), an assertions.xml file is used to describe the tests, and tests
58have the structure M-N.c or M-N.sh.  More details are below.
59
60Speculative Tests
61-----------------
62Tests which the POSIX spec uses the words "may" or "implementation-defined"
63to define are considered speculative by PTS and are created in a
64speculative/ directory off of the functional area directory.  (For
65example, mq_timedsend/speculative.)
66These tests attempt to determine which of the implementation-defined
67alternatives the current implementation implements.  PTS_PASS is always
68returned if the test is able to finish.
69
70assertions.xml
71--------------
72This file contains a list of assertions to be tested, an ID for each
73assertion, and descriptive attributes to be used in categorizing
74assertions.  For more information on creating and using these files,
75see HOWTO_Assertions.
76
77Test Cases
78----------
79Test cases are numbered M-N.c or M-N.sh where M corresponds to the
80assertion ID of the assertion being tested, and N is an ID for the specific
81tests.  (ex.  assertion 8 may have test cases 8-1.c 8-2.c and 8-3.sh)
82Generally, *.sh scripts are only used when the test case is trivial.
83
84The following documents may be helpful in test case creation:
85
86- HOWTO_DefinitionsTest - describes how to create definitions tests
87- HOWTO_ResultCodes - describes the result codes test cases should return
88- HOWTO_BoundaryTest - describes how to create boundary tests for functions
89
90* POSIX (R) is a registered trademark of the IEEE
91
92Contributors:	rusty.lynch REMOVE-THIS AT intel DOT com
93		julie.n.fleischer REMOVE-THIS AT intel DOT com
94