Import statements sorted with isort and fixed MockBuilder.py (#266)

* Fixed missing closing bracket in the MockBuilder.py

* Import statements sorted with isort

python3: Fix missed itervalues -> values

python3: Finish python3 change. python2 breaking

* Based on mmlr's 2017 work to use Python3 with
changes and adjustments for the current codebase

Introduce TEST_REQUIRES for test-only requires.

This lets recipes specify dependencies that are only needed for the test
phase. Separating them out from BUILD_[PRE]REQUIRES allows for
situations that could otherwise result in cyclic dependencies.

Note that this bumps the repository format version, causing a repository
rebuild (re-parsing of all recipes, flushing of all caches). This is
required because the dependency info files and various caches need to be
regenerated to cope with the added test requires keys.

Fixes #151.

Cleanup: Make some constructs more idiomatic.

* Change 'not x in y' to 'x not in y'.
* Use 'is' to compare None singleton.
* Use 'instanceof(x, y)' instead of 'type(x) == y'.
* Replace map builtin with list comprehension.
* Rename self argument to cls for static method.

Cleanup: Add missing raw string prefix to regex patterns.

Cleanup: Remove trailing semicolons and a unneeded pass.

Cleanup: Whitespace and line length cleanup.

PackageInfo: Create directory for HPKG info cache if needed.

PackageInfo: Simplify parsing resolvable expressions.

Use optional groups in the regular expression instead of conditional.

PackageInfo: Fix resolvable compat parsing for recipe format.

The manual parsing of the compatibility version did only take the format
used in HPKGs into account, i.e. "(compatible >= version)". It is
however also used on the expressions from recipe files, which use the
format "compat >= version". This lead to the compat version being
ignored and therefore possibly wrong dependencies to be used.

Remove the manual parsing and put it into the regular expression. Also
simplify the logic by using optional groups in the regular expression
so that the match result can be used regardless of what parts exist in
the string.

PackageInfo: Compile regular expression for version/expression.

And do it only once via class variable.

PackageInfo: Persist the HPKG info cache to a file.

This dramatically reduces the time spent on reading package infos. Using
pickle instead of JSON allows to write the cache incrementally.

PackageInfo: Make ResolvableExpression aware of 'base'.

It was previously ignored in all cases, now it is only ignored when
parsing expressions from HPKG files. Expressions coming from recipes
can now have the base modifier.

Note that apparently 'base' is never output by package list -i, which
means that 'base' was already ignored without explicitly ignoring it.

Use Unicode strings for warn() & sysExit().

Allow utf-8 in recipe files.

This is 2015, no need to stick with 1960s encodings…

Remove local check_output, we're now using Python 2.7.

Cleanup imports.

PackageInfo: implement caching for .hpkg files.

* Extracting .PackageInfo files from a package is quite slow, so cache
the result such that each package file is only parsed once per
haikuporter run.

Add support to PackageInfo for reading .DependencyInfo files.

* Similar to a .PackageInfo file, a .DependencyInfo file stores
information about a package in a format that is easily parsable (in
this case: JSON). In contrast to .PackageInfo, a .DependencyInfo file
is not constrained to the use within a hpkg and contains all
different types of dependencies (i.e. requires, build-requires and
build-prerequires).

Refactoring: Prepare PackageInfo for changes to come.

* Move parsing of .hpkg/.PackageInfo files from constructor into a
method of its own.

Refactoring: use properties where it seems fit.

* Trying to be somewhat more Pythonic.

Revert to tab spacing for Pe's sake.

Try again with whitespace.

Second attempt to fix whitespace. Options are different on OS X.

Convert tabs to spaces.

Resolvable: fix __str__()

The compatible version wasn't printed correctly.

PackageInfo: improve field extraction

Explicitly match the beginning of the line to make it less likely to
get erroneous matches in multi-line description fields. It's still
possible though.

Fix import-related warnings and errors.

Unifiy the Options and GlobalConfig somewhat

* Module GlobalConfig is replaced with new module Configuration. The
global variable globalConfiguration is gone. Instead Configuration
provides getter methods.
* For several of the (newer) command line options there are now optional
configuration file entries. If given the command line values override
the configuration file values. Configuration also provides getter
methods for these options, which should be used instead of
getOption().

PackageInfo: parse more package attributes

Namely the package name, the install path, the provides and requires.

Start with a build platform abstraction

* Add '--cross-devel-package' option to specify the path to the cross
development package. Can be used for cross-compiling.
* Add module BuildPlatform which exports a buildPlatform variable, which
is initialized to an instance of BuildPlatformHaiku or
BuildPlatformUnix as appropriate.
* Move findDirectory() to BuildPlatformHaiku. BuildPlatformUnix has a
version with some hard-coded mappings.
* Add PackageInfo class. Very simple ATM. It only extracts the version
and the architecture from a package file.
* Rework Main._initGlobalShellVariables() somewhat: Extracting the Haiku
version from the build platform's system haiku package is obviously
not correct when cross-building. Also add support for non-Haiku build
platforms.
* Remove no longer needed MachineArchitecture.getBuildTripleFor().

Cleanup imports.

PackageInfo: implement caching for .hpkg files.

* Extracting .PackageInfo files from a package is quite slow, so cache
the result such that each package file is only parsed once per
haikuporter run.

Add support to PackageInfo for reading .DependencyInfo files.

* Similar to a .PackageInfo file, a .DependencyInfo file stores
information about a package in a format that is easily parsable (in
this case: JSON). In contrast to .PackageInfo, a .DependencyInfo file
is not constrained to the use within a hpkg and contains all
different types of dependencies (i.e. requires, build-requires and
build-prerequires).

Refactoring: Prepare PackageInfo for changes to come.

* Move parsing of .hpkg/.PackageInfo files from constructor into a
method of its own.

Refactoring: use properties where it seems fit.

* Trying to be somewhat more Pythonic.

Revert to tab spacing for Pe's sake.

Try again with whitespace.

Second attempt to fix whitespace. Options are different on OS X.

Convert tabs to spaces.

Resolvable: fix __str__()

The compatible version wasn't printed correctly.

PackageInfo: improve field extraction

Explicitly match the beginning of the line to make it less likely to
get erroneous matches in multi-line description fields. It's still
possible though.

Fix import-related warnings and errors.

Unifiy the Options and GlobalConfig somewhat

* Module GlobalConfig is replaced with new module Configuration. The
global variable globalConfiguration is gone. Instead Configuration
provides getter methods.
* For several of the (newer) command line options there are now optional
configuration file entries. If given the command line values override
the configuration file values. Configuration also provides getter
methods for these options, which should be used instead of
getOption().

PackageInfo: parse more package attributes

Namely the package name, the install path, the provides and requires.

Start with a build platform abstraction

* Add '--cross-devel-package' option to specify the path to the cross
development package. Can be used for cross-compiling.
* Add module BuildPlatform which exports a buildPlatform variable, which
is initialized to an instance of BuildPlatformHaiku or
BuildPlatformUnix as appropriate.
* Move findDirectory() to BuildPlatformHaiku. BuildPlatformUnix has a
version with some hard-coded mappings.
* Add PackageInfo class. Very simple ATM. It only extracts the version
and the architecture from a package file.
* Rework Main._initGlobalShellVariables() somewhat: Extracting the Haiku
version from the build platform's system haiku package is obviously
not correct when cross-building. Also add support for non-Haiku build
platforms.
* Remove no longer needed MachineArchitecture.getBuildTripleFor().