1364468Scy# Travis Testing 2364468Scy 3364468ScyUnbound 1.10 and above leverage Travis CI to increase coverage of compilers and platforms. Compilers include Clang and GCC; while platforms include Android, iOS, Linux, and OS X on AMD64, Aarch64, PowerPC and s390x hardware. 4364468Scy 5364468ScyAndroid is tested on armv7a, aarch64, x86 and x86_64. The Android recipes build and install OpenSSL and Expat, and then builds Unbound. The testing is tailored for Android NDK-r19 and above, and includes NDK-r20 and NDK-r21. Mips and Mips64 are not tested because they are no longer supported under current NDKs. 6364468Scy 7364468ScyiOS is tested for iPhoneOS, WatchOS, AppleTVOS, iPhoneSimulator, AppleTVSimulator and WatchSimulator. The testing uses Xcode 10 on OS X 10.13. 8364468Scy 9364468ScyThe Unbound Travis configuration file `.travis.yml` does not use top-level keys like `os:` and `compiler:` so there is no matrix expansion. Instead Unbound specifies the exact job to run under the `jobs:` and `include:` keys. 10364468Scy 11364468Scy## Typical recipe 12364468Scy 13364468ScyA typical recipe tests Clang and GCC on various hardware. The hardware includes AMD64, Aarch64, PowerPC and s390x. PowerPC is a little-endian platform, and s390x is a big-endian platform. There are pairs of recipes that are similar to the following. 14364468Scy 15364468Scy``` 16364468Scy- os: linux 17364468Scy name: GCC on Linux, Aarch64 18364468Scy compiler: gcc 19364468Scy arch: arm64 20364468Scy dist: bionic 21364468Scy- os: linux 22364468Scy name: Clang on Linux, Aarch64 23364468Scy compiler: clang 24364468Scy arch: arm64 25364468Scy dist: bionic 26364468Scy``` 27364468Scy 28364468ScyOS X provides a single recipe to test Clang. GCC is not tested because GCC is an alias for Clang. 29364468Scy 30364468Scy## Sanitizer builds 31364468Scy 32364468ScyTwo sanitizer builds are tested using Clang and GCC, for a total of four builds. The first sanitizer is Undefined Behavior sanitizer (UBsan), and the second is Address sanitizer (Asan). The sanitizers are only run on AMD64 hardware. Note the environment includes `TEST_UBSAN=yes` or `TEST_ASAN=yes` for the sanitizer builds. 33364468Scy 34364468ScyThe recipes are similar to the following. 35364468Scy 36364468Scy``` 37364468Scy- os: linux 38364468Scy name: UBsan, GCC on Linux, Amd64 39364468Scy compiler: gcc 40364468Scy arch: amd64 41364468Scy dist: bionic 42364468Scy env: TEST_UBSAN=yes 43364468Scy- os: linux 44364468Scy name: UBsan, Clang on Linux, Amd64 45364468Scy compiler: clang 46364468Scy arch: amd64 47364468Scy dist: bionic 48364468Scy env: TEST_UBSAN=yes 49364468Scy``` 50364468Scy 51364468ScyWhen the Travis script encounters a sanitizer it uses different `CFLAGS` and configuration string. 52364468Scy 53364468Scy``` 54364468Scyif [ "$TEST_UBSAN" = "yes" ]; then 55364468Scy export CFLAGS="-DNDEBUG -g2 -O3 -fsanitize=undefined -fno-sanitize-recover" 56364468Scy ./configure 57364468Scy make -j 2 58364468Scy make test 59364468Scyelif [ "$TEST_ASAN" = "yes" ]; then 60364468Scy export CFLAGS="-DNDEBUG -g2 -O3 -fsanitize=address" 61364468Scy ./configure 62364468Scy make -j 2 63364468Scy make test 64364468Scy... 65364468Scy``` 66364468Scy 67364468Scy## Android builds 68364468Scy 69364468ScyTravis tests Android builds for the armv7a, aarch64, x86 and x86_64 architectures. The builds are trickier than other builds for several reasons. The testing requires installation of the Android NDK and SDK, it requires a cross-compile, and requires OpenSSL and Expat prerequisites. The Android cross-compiles also require care to set the Autotools triplet, the OpenSSL triplet, the toolchain path, the tool variables, and the sysroot. The discussion below detail the steps of the Android recipes. 70364468Scy 71364468Scy### Android job 72364468Scy 73364468ScyThe first step sets environmental variables for the cross-compile using the Travis job. A typical job with variables is shown below. 74364468Scy 75364468Scy``` 76364468Scy- os: linux 77364468Scy name: Android armv7a, Linux, Amd64 78364468Scy compiler: clang 79364468Scy arch: amd64 80364468Scy dist: bionic 81364468Scy env: 82364468Scy - TEST_ANDROID=yes 83364468Scy - AUTOTOOLS_HOST=armv7a-linux-androideabi 84364468Scy - OPENSSL_HOST=android-arm 85364468Scy - ANDROID_CPU=armv7a 86364468Scy - ANDROID_API=23 87364468Scy - ANDROID_PREFIX="$HOME/android$ANDROID_API-$ANDROID_CPU" 88364468Scy - ANDROID_SDK_ROOT="$HOME/android-sdk" 89364468Scy - ANDROID_NDK_ROOT="$HOME/android-ndk" 90364468Scy``` 91364468Scy 92364468Scy### ANDROID_NDK_ROOT 93364468Scy 94364468ScyThe second step for Android is to set the environmental variables `ANDROID_NDK_ROOT` and `ANDROID_SDK_ROOT`. This is an important step because the NDK and SDK use the variables internally to locate their own tools. Also see [Recommended NDK Directory?](https://groups.google.com/forum/#!topic/android-ndk/qZjhOaynHXc) on the android-ndk mailing list. (Many folks miss this step, or use incorrect variables like `ANDROID_NDK_HOME` or `ANDROID_SDK_HOME`). 95364468Scy 96364468ScyIf you are working from a developer machine you probably already have the necessary tools installed. You should ensure `ANDROID_NDK_ROOT` and `ANDROID_SDK_ROOT` are set properly. 97364468Scy 98364468Scy### Tool installation 99364468Scy 100364468ScyThe second step installs tools needed for OpenSSL, Expat and Unbound. This step is handled in by the script `contrib/android/install_tools.sh`. The tools include curl, tar, zip, unzip and java. 101364468Scy 102364468Scy``` 103364468Scybefore_script: 104364468Scy - | 105364468Scy if [ "$TEST_ANDROID" = "yes" ]; then 106364468Scy ./contrib/android/install_tools.sh 107364468Scy elif [ "$TEST_IOS" = "yes" ]; then 108364468Scy ./contrib/ios/install_tools.sh 109364468Scy fi 110364468Scy``` 111364468Scy 112364468Scy### NDK installation 113364468Scy 114364468ScyThe third step installs the NDK and SDK. This step is handled in by the script `contrib/android/install_ndk.sh`. The script uses `ANDROID_NDK_ROOT` and `ANDROID_SDK_ROOT` to place the NDK and SDK in the `$HOME` directory. 115364468Scy 116364468ScyIf you are working from a developer machine you probably already have a NDK and SDK installed. 117364468Scy 118364468Scy### Android environment 119364468Scy 120364468ScyThe fourth step sets the Android cross-compile environment using the script `contrib/android/setenv_android.sh`. The script is `sourced` so the variables in the script are available to the calling shell. The script sets variables like `CC`, `CXX`, `AS` and `AR`; sets `CFLAGS` and `CXXFLAGS`; sets a `sysroot` so Android headers and libraries are found; and adds the path to the toolchain to `PATH`. 121364468Scy 122364468Scy`contrib/android/setenv_android.sh` knows which toolchain and architecture to select by inspecting environmental variables set by Travis for the job. In particular, the variables `ANDROID_CPU` and `ANDROID_API` tell `contrib/android/setenv_android.sh` which tools and libraries to select. 123364468Scy 124364468ScyThe `contrib/android/setenv_android.sh` script specifies the tools in a `case` statement like the following. There is a case for each of the architectures armv7a, aarch64, x86 and x86_64. 125364468Scy 126364468Scy``` 127364468Scyarmv8a|aarch64|arm64|arm64-v8a) 128364468Scy CC="aarch64-linux-android$ANDROID_API-clang" 129364468Scy CXX="aarch64-linux-android$ANDROID_API-clang++" 130364468Scy LD="aarch64-linux-android-ld" 131364468Scy AS="aarch64-linux-android-as" 132364468Scy AR="aarch64-linux-android-ar" 133364468Scy RANLIB="aarch64-linux-android-ranlib" 134364468Scy STRIP="aarch64-linux-android-strip" 135364468Scy 136364468Scy CFLAGS="-funwind-tables -fexceptions" 137364468Scy CXXFLAGS="-funwind-tables -fexceptions -frtti" 138364468Scy``` 139364468Scy 140364468Scy### OpenSSL and Expat 141364468Scy 142364468ScyThe fifth step builds OpenSSL and Expat. OpenSSL and Expat are built for Android using the scripts `contrib/android/install_openssl.sh` and `contrib/android/install_expat.sh`. The scripts download, configure and install the latest release version of the libraries. The libraries are configured with `--prefix="$ANDROID_PREFIX"` so the headers are placed in `$ANDROID_PREFIX/include` directory, and the libraries are placed in the `$ANDROID_PREFIX/lib` directory. 143364468Scy 144364468Scy`ANDROID_PREFIX` is the value `$HOME/android$ANDROID_API-$ANDROID_CPU`. The libraries will be installed in `$HOME/android23-armv7a`, `$HOME/android23-aarch64`, etc. For Autotools projects, the appropriate `PKG_CONFIG_PATH` is exported. `PKG_CONFIG_PATH` is the userland equivalent to sysroot, and allows Autotools to find non-system headers and libraries for an architecture. Typical `PKG_CONFIG_PATH` are `$HOME/android23-armv7a/lib/pkgconfig` and `$HOME/android23-aarch64/lib/pkgconfig`. 145364468Scy 146364468ScyOpenSSL also uses a custom configuration file called `15-android.conf`. It is a copy of the OpenSSL's project file and located at `contrib/android/15-android.conf`. The Unbound version is copied to the OpenSSL source files after unpacking the OpenSSL distribution. The Unbound version has legacy NDK support removed and some other fixes, like `ANDROID_NDK_ROOT` awareness. The changes mean Unbound's `15-android.conf` will only work with Unbound, with NDK-r19 and above, and a properly set environment. 147364468Scy 148364468ScyOpenSSL is configured with `no-engine`. If you want to include OpenSSL engines then edit `contrib/android/install_openssl.sh` and remove the config option. 149364468Scy 150364468Scy### Android build 151364468Scy 152364468ScyFinally, once OpenSSL and Expat are built, then the Travis script configures and builds Unbound. The recipe looks as follows. 153364468Scy 154364468Scy``` 155364468Scyelif [ "$TEST_ANDROID" = "yes" ]; then 156364468Scy export AUTOTOOLS_BUILD="$(./config.guess)" 157364468Scy export PKG_CONFIG_PATH="$ANDROID_PREFIX/lib/pkgconfig" 158364468Scy ./contrib/android/install_ndk.sh 159364468Scy source ./contrib/android/setenv_android.sh 160364468Scy ./contrib/android/install_openssl.sh 161364468Scy ./contrib/android/install_expat.sh 162364468Scy ./configure \ 163364468Scy --build="$AUTOTOOLS_BUILD" \ 164364468Scy --host="$AUTOTOOLS_HOST" \ 165364468Scy --prefix="$ANDROID_PREFIX" \ 166364468Scy --with-ssl="$ANDROID_PREFIX" \ 167364468Scy --with-libexpat="$ANDROID_PREFIX" \ 168364468Scy --disable-gost; 169364468Scy make -j 2 170364468Scy make install 171364468Scy``` 172364468Scy 173364468ScyTravis only smoke tests an Android build using a compile, link and install. The self tests are not run. TODO: figure out how to fire up an emulator, push the tests to the device and run them. 174364468Scy 175364468Scy### Android flags 176364468Scy 177364468Scy`contrib/android/setenv_android.sh` uses specific flags for `CFLAGS` and `CXXFLAGS`. They are taken from `ndk-build`, so we consider them the official flag set. It is important to use the same flags across projects to avoid subtle problems due to mixing and matching different flags. 178364468Scy 179364468Scy`CXXFLAGS` includes `-fexceptions` and `-frtti` because exceptions and runtime type info are disabled by default. `CFLAGS` include `-funwind-tables` and `-fexceptions` to ensure C++ exceptions pass through C code, if needed. Also see `docs/CPLUSPLUS-SUPPORT.html` in the NDK docs. 180364468Scy 181364468ScyTo inspect the flags used by `ndk-build` for a platform clone ASOP's [ndk-samples](https://github.com/android/ndk-samples/tree/master/hello-jni) and build the `hello-jni` project. Use the `V=1` flag to see the full compiler output from `ndk-build`. 182364468Scy 183364468Scy## iOS builds 184364468Scy 185364468ScyTravis tests iOS builds for the armv7a, armv7s and aarch64 architectures for iPhoneOS, AppleTVOS and WatchOS. iPhoneOS is tested using both 32-bit builds (iPhones) and 64-bit builds (iPads). Travis also tests compiles against the simulators. The builds are trickier than other builds for several reasons. The testing requires a cross-compile, and requires OpenSSL and Expat prerequisites. The iOS cross-compiles also require care to set the Autotools triplet, the OpenSSL triplet, the toolchain path, the tool variables, and the sysroot. The discussion below detail the steps of the iOS recipes. 186364468Scy 187364468Scy### iOS job 188364468Scy 189364468ScyThe first step sets environmental variables for the cross-compile using the Travis job. A typical job with variables is shown below. 190364468Scy 191364468Scy``` 192364468Scy- os: osx 193364468Scy osx_image: xcode10 194364468Scy name: Apple iPhone on iOS, armv7 195364468Scy compiler: clang 196364468Scy env: 197364468Scy - TEST_IOS=yes 198364468Scy - AUTOTOOLS_HOST=armv7-apple-ios 199364468Scy - OPENSSL_HOST=ios-cross 200364468Scy - IOS_SDK=iPhoneOS 201364468Scy - IOS_CPU=armv7s 202364468Scy - IOS_PREFIX="$HOME/$IOS_SDK-$IOS_CPU" 203364468Scy``` 204364468Scy 205364468Scy### Tool installation 206364468Scy 207364468ScyThe second step installs tools needed for OpenSSL, Expat and Unbound. This step is handled in by the script `contrib/ios/install_tools.sh`. The tools include autotools, curl and perl. The installation happens at the `before_script:` stage of Travis. 208364468Scy 209364468Scy``` 210364468Scybefore_script: 211364468Scy - | 212364468Scy if [ "$TEST_ANDROID" = "yes" ]; then 213364468Scy ./contrib/android/install_tools.sh 214364468Scy elif [ "$TEST_IOS" = "yes" ]; then 215364468Scy ./contrib/ios/install_tools.sh 216364468Scy fi 217364468Scy``` 218364468Scy 219364468Scy### iOS environment 220364468Scy 221364468ScyThe third step sets the iOS cross-compile environment using the script `contrib/ios/setenv_ios.sh`. The script is `sourced` so the variables in the script are available to the calling shell. The script sets variables like `CC`, `CXX`, `AS` and `AR`; sets `CFLAGS` and `CXXFLAGS`; sets a `sysroot` so iOS headers and libraries are found; and adds the path to the toolchain to `PATH`. 222364468Scy 223364468Scy`contrib/ios/setenv_ios.sh` knows which toolchain and architecture to select by inspecting environmental variables set by Travis for the job. In particular, the variables `IOS_SDK` and `IOS_CPU` tell `contrib/ios/setenv_ios.sh` which tools and libraries to select. 224364468Scy 225364468ScyThe `contrib/ios/setenv_ios.sh` script specifies the tools to use during the cross-compile. For Apple SDKs, the tool names are the same as a desktop. There are no special prefixes for the mobile tools. 226364468Scy 227364468Scy``` 228364468ScyCPP=cpp 229364468ScyCC=clang 230364468ScyCXX=clang++ 231364468ScyLD=ld 232364468ScyAS=as 233364468ScyAR=ar 234364468ScyRANLIB=ranlib 235364468ScySTRIP=strip 236364468Scy``` 237364468Scy 238364468ScyIf you are working from a developer machine you probably already have the necessary tools installed. 239364468Scy 240364468Scy### OpenSSL and Expat 241364468Scy 242364468ScyThe fourth step builds OpenSSL and Expat. OpenSSL and Expat are built for iOS using the scripts `contrib/ios/install_openssl.sh` and `contrib/ios/install_expat.sh`. The scripts download, configure and install the latest release version of the libraries. The libraries are configured with `--prefix="$IOS_PREFIX"` so the headers are placed in `$IOS_PREFIX/include` directory, and the libraries are placed in the `$IOS_PREFIX/lib` directory. 243364468Scy 244364468Scy`IOS_PREFIX` is the value `$HOME/$IOS_SDK-$IOS_CPU`. The scheme handles both iOS SDKs and cpu architectures so the pair recieves a unique installation directory. The libraries will be installed in `$HOME/iPhoneOS-armv7s`, `$HOME/iPhoneOS-arm64`, `$HOME/iPhoneSimulator-i386`, etc. For Autotools projects, the appropriate `PKG_CONFIG_PATH` is exported. 245364468Scy 246364468Scy`PKG_CONFIG_PATH` is an important variable. It is the userland equivalent to sysroot, and allows Autotools to find non-system headers and libraries for an architecture. Typical `PKG_CONFIG_PATH` are `$HOME/iPhoneOS-armv7s/lib/pkgconfig` and `$HOME/iPhoneOS-arm64/lib/pkgconfig`. 247364468Scy 248364468ScyOpenSSL also uses a custom configuration file called `15-ios.conf`. It is a copy of the OpenSSL's project file and located at `contrib/ios/15-ios.conf`. The Unbound version is copied to the OpenSSL source files after unpacking the OpenSSL distribution. The changes mean Unbound's `15-ios.conf` will only work with Unbound and a properly set environment. 249364468Scy 250364468ScyOpenSSL is configured with `no-engine`. Engines require dynamic loading so engines are disabled permanently in `15-ios.conf`. 251364468Scy 252364468Scy### iOS build 253364468Scy 254364468ScyFinally, once OpenSSL and Expat are built, then the Travis script configures and builds Unbound. The full recipe looks as follows. 255364468Scy 256364468Scy``` 257364468Scyelif [ "$TEST_IOS" = "yes" ]; then 258364468Scy export AUTOTOOLS_BUILD="$(./config.guess)" 259364468Scy export PKG_CONFIG_PATH="$IOS_PREFIX/lib/pkgconfig" 260364468Scy source ./contrib/ios/setenv_ios.sh 261364468Scy ./contrib/ios/install_openssl.sh 262364468Scy ./contrib/ios/install_expat.sh 263364468Scy ./configure \ 264364468Scy --build="$AUTOTOOLS_BUILD" \ 265364468Scy --host="$AUTOTOOLS_HOST" \ 266364468Scy --prefix="$IOS_PREFIX" \ 267364468Scy --with-ssl="$IOS_PREFIX" \ 268364468Scy --with-libexpat="$IOS_PREFIX" \ 269364468Scy --disable-gost; 270364468Scy make -j 2 271364468Scy make install 272364468Scy``` 273364468Scy 274364468ScyTravis only smoke tests an iOS build using a compile, link and install. The self tests are not run. TODO: figure out how to fire up an simulator, push the tests to the device and run them. 275364468Scy 276364468Scy### iOS flags 277364468Scy 278364468Scy`contrib/ios/setenv_ios.sh` uses specific flags for `CFLAGS` and `CXXFLAGS`. They are taken from Xcode, so we consider them the official flag set. It is important to use the same flags across projects to avoid subtle problems due to mixing and matching different flags. 279