README.regress
1Overview.
2
3$ ./configure && make tests
4
5You'll see some progress info. A failure will cause either the make to
6abort or the driver script to report a "FATAL" failure.
7
8The test consists of 2 parts. The first is the file-based tests which is
9driven by the Makefile, and the second is a set of network or proxycommand
10based tests, which are driven by a driver script (test-exec.sh) which is
11called multiple times by the Makefile.
12
13Failures in the first part will cause the Makefile to return an error.
14Failures in the second part will print a "FATAL" message for the failed
15test and continue.
16
17OpenBSD has a system-wide regression test suite. OpenSSH Portable's test
18suite is based on OpenBSD's with modifications.
19
20
21Environment variables.
22
23SKIP_UNIT: Skip unit tests.
24SUDO: path to sudo/doas command, if desired. Note that some systems
25 (notably systems using PAM) require sudo to execute some tests.
26LTESTS: Whitespace separated list of tests (filenames without the .sh
27 extension) to run.
28OBJ: used by test scripts to access build dir.
29TEST_SHELL: shell used for running the test scripts.
30TEST_SSH_FAIL_FATAL: set to "yes" to make any failure abort the test
31 currently in progress.
32TEST_SSH_PORT: TCP port to be used for the listening tests.
33TEST_SSH_QUIET: set to "yes" to suppress non-fatal output.
34TEST_SSH_SSHD_CONFOPTS: Configuration directives to be added to sshd_config
35 before running each test.
36TEST_SSH_SSH_CONFOPTS: Configuration directives to be added to
37 ssh_config before running each test.
38TEST_SSH_TRACE: set to "yes" for verbose output from tests
39TEST_SSH_x: path to "ssh" command under test, where x is one of
40 SSH, SSHD, SSHAGENT, SSHADD, SSHKEYGEN, SSHKEYSCAN, SFTP or
41 SFTPSERVER
42USE_VALGRIND: Run the tests under valgrind memory checker.
43
44
45Individual tests.
46
47You can run an individual test from the top-level Makefile, eg:
48$ make tests LTESTS=agent-timeout
49
50If you need to manipulate the environment more you can invoke test-exec.sh
51directly if you set up the path to find the binaries under test and the
52test scripts themselves, for example:
53
54$ cd regress
55$ PATH=`pwd`/..:$PATH:. TEST_SHELL=/bin/sh sh test-exec.sh `pwd` \
56 agent-timeout.sh
57ok agent timeout test
58
59
60Files.
61
62test-exec.sh: the main test driver. Sets environment, creates config files
63and keys and runs the specified test.
64
65At the time of writing, the individual tests are:
66agent-timeout.sh: agent timeout test
67agent.sh: simple agent test
68broken-pipe.sh: broken pipe test
69connect-privsep.sh: proxy connect with privsep
70connect.sh: simple connect
71exit-status.sh: remote exit status
72forwarding.sh: local and remote forwarding
73keygen-change.sh: change passphrase for key
74keyscan.sh: keyscan
75proto-mismatch.sh: protocol version mismatch
76proto-version.sh: sshd version with different protocol combinations
77proxy-connect.sh: proxy connect
78sftp.sh: basic sftp put/get
79ssh-com-client.sh: connect with ssh.com client
80ssh-com-keygen.sh: ssh.com key import
81ssh-com-sftp.sh: basic sftp put/get with ssh.com server
82ssh-com.sh: connect to ssh.com server
83stderr-after-eof.sh: stderr data after eof
84stderr-data.sh: stderr data transfer
85transfer.sh: transfer data
86try-ciphers.sh: try ciphers
87yes-head.sh: yes pipe head
88
89
90Problems?
91
92Run the failing test with shell tracing (-x) turned on:
93$ PATH=`pwd`/..:$PATH:. sh -x test-exec.sh `pwd` agent-timeout.sh
94
95Failed tests can be difficult to diagnose. Suggestions:
96- run the individual test via ./test-exec.sh `pwd` [testname]
97- set LogLevel to VERBOSE in test-exec.sh and enable syslogging of
98 auth.debug (eg to /var/log/authlog).
99
100
101Known Issues.
102
103- Similarly, if you do not have "scp" in your system's $PATH then the
104 multiplex scp tests will fail (since the system's shell startup scripts
105 will determine where the shell started by sshd will look for scp).
106
107- Recent GNU coreutils deprecate "head -[n]": this will cause the yes-head
108 test to fail. The old behaviour can be restored by setting (and
109 exporting) _POSIX2_VERSION=199209 before running the tests.
110