README.regress
1Overview.
2
3$ ./configure && make tests
4
5You'll see some progress info. A failure will cause either the make to
6abort or the driver script to report a "FATAL" failure.
7
8The test consists of 2 parts. The first is the file-based tests which is
9driven by the Makefile, and the second is a set of network or proxycommand
10based tests, which are driven by a driver script (test-exec.sh) which is
11called multiple times by the Makefile.
12
13Failures in the first part will cause the Makefile to return an error.
14Failures in the second part will print a "FATAL" message for the failed
15test and continue.
16
17OpenBSD has a system-wide regression test suite. OpenSSH Portable's test
18suite is based on OpenBSD's with modifications.
19
20
21Environment variables.
22
23SKIP_UNIT: Skip unit tests.
24SUDO: path to sudo/doas command, if desired. Note that some systems
25 (notably systems using PAM) require sudo to execute some tests.
26LTESTS: Whitespace separated list of tests (filenames without the .sh
27 extension) to run.
28SKIP_LTESTS: Whitespace separated list of tests to skip.
29OBJ: used by test scripts to access build dir.
30TEST_SHELL: shell used for running the test scripts.
31TEST_SSH_FAIL_FATAL: set to "yes" to make any failure abort the test
32 currently in progress.
33TEST_SSH_PORT: TCP port to be used for the listening tests.
34TEST_SSH_QUIET: set to "yes" to suppress non-fatal output.
35TEST_SSH_SSHD_CONFOPTS: Configuration directives to be added to sshd_config
36 before running each test.
37TEST_SSH_SSH_CONFOPTS: Configuration directives to be added to
38 ssh_config before running each test.
39TEST_SSH_TRACE: set to "yes" for verbose output from tests
40TEST_SSH_x: path to "ssh" command under test, where x is one of
41 SSH, SSHD, SSHAGENT, SSHADD, SSHKEYGEN, SSHKEYSCAN, SFTP or
42 SFTPSERVER
43USE_VALGRIND: Run the tests under valgrind memory checker.
44
45
46Individual tests.
47
48You can run an individual test from the top-level Makefile, eg:
49$ make tests LTESTS=agent-timeout
50
51If you need to manipulate the environment more you can invoke test-exec.sh
52directly if you set up the path to find the binaries under test and the
53test scripts themselves, for example:
54
55$ cd regress
56$ PATH=`pwd`/..:$PATH:. TEST_SHELL=/bin/sh sh test-exec.sh `pwd` \
57 agent-timeout.sh
58ok agent timeout test
59
60
61Files.
62
63test-exec.sh: the main test driver. Sets environment, creates config files
64and keys and runs the specified test.
65
66At the time of writing, the individual tests are:
67connect.sh: simple connect
68proxy-connect.sh: proxy connect
69connect-privsep.sh: proxy connect with privsep
70connect-uri.sh: uri connect
71proto-version.sh: sshd version with different protocol combinations
72proto-mismatch.sh: protocol version mismatch
73exit-status.sh: remote exit status
74envpass.sh: environment passing
75transfer.sh: transfer data
76banner.sh: banner
77rekey.sh: rekey
78stderr-data.sh: stderr data transfer
79stderr-after-eof.sh: stderr data after eof
80broken-pipe.sh: broken pipe test
81try-ciphers.sh: try ciphers
82yes-head.sh: yes pipe head
83login-timeout.sh: connect after login grace timeout
84agent.sh: simple connect via agent
85agent-getpeereid.sh: disallow agent attach from other uid
86agent-timeout.sh: agent timeout test
87agent-ptrace.sh: disallow agent ptrace attach
88keyscan.sh: keyscan
89keygen-change.sh: change passphrase for key
90keygen-convert.sh: convert keys
91keygen-moduli.sh: keygen moduli
92key-options.sh: key options
93scp.sh: scp
94scp-uri.sh: scp-uri
95sftp.sh: basic sftp put/get
96sftp-chroot.sh: sftp in chroot
97sftp-cmds.sh: sftp command
98sftp-badcmds.sh: sftp invalid commands
99sftp-batch.sh: sftp batchfile
100sftp-glob.sh: sftp glob
101sftp-perm.sh: sftp permissions
102sftp-uri.sh: sftp-uri
103ssh-com-client.sh: connect with ssh.com client
104ssh-com-keygen.sh: ssh.com key import
105ssh-com-sftp.sh: basic sftp put/get with ssh.com server
106ssh-com.sh: connect to ssh.com server
107reconfigure.sh: simple connect after reconfigure
108dynamic-forward.sh: dynamic forwarding
109forwarding.sh: local and remote forwarding
110multiplex.sh: connection multiplexing
111reexec.sh: reexec tests
112brokenkeys.sh: broken keys
113sshcfgparse.sh: ssh config parse
114cfgparse.sh: sshd config parse
115cfgmatch.sh: sshd_config match
116cfgmatchlisten.sh: sshd_config matchlisten
117addrmatch.sh: address match
118localcommand.sh: localcommand
119forcecommand.sh: forced command
120portnum.sh: port number parsing
121keytype.sh: login with different key types
122kextype.sh: login with different key exchange algorithms
123cert-hostkey.sh certified host keys
124cert-userkey.sh: certified user keys
125host-expand.sh: expand %h and %n
126keys-command.sh: authorized keys from command
127forward-control.sh: sshd control of local and remote forwarding
128integrity.sh: integrity
129krl.sh: key revocation lists
130multipubkey.sh: multiple pubkey
131limit-keytype.sh: restrict pubkey type
132hostkey-agent.sh: hostkey agent
133keygen-knownhosts.sh: ssh-keygen known_hosts
134hostkey-rotate.sh: hostkey rotate
135principals-command.sh: authorized principals command
136cert-file.sh: ssh with certificates
137cfginclude.sh: config include
138allow-deny-users.sh: AllowUsers/DenyUsers
139authinfo.sh: authinfo
140
141
142Problems?
143
144Run the failing test with shell tracing (-x) turned on:
145$ PATH=`pwd`/..:$PATH:. sh -x test-exec.sh `pwd` agent-timeout.sh
146
147Failed tests can be difficult to diagnose. Suggestions:
148- run the individual test via ./test-exec.sh `pwd` [testname]
149- set LogLevel to VERBOSE in test-exec.sh and enable syslogging of
150 auth.debug (eg to /var/log/authlog).
151
152
153Known Issues.
154
155- Similarly, if you do not have "scp" in your system's $PATH then the
156 multiplex scp tests will fail (since the system's shell startup scripts
157 will determine where the shell started by sshd will look for scp).
158
159- Recent GNU coreutils deprecate "head -[n]": this will cause the yes-head
160 test to fail. The old behaviour can be restored by setting (and
161 exporting) _POSIX2_VERSION=199209 before running the tests.
162