History log of /fuchsia/zircon/system/utest/runtests-utils/runtests-utils-test.cpp
Revision Date Author Comments
# f3dd9053 27-Aug-2018 Petr Hosek <phosek@google.com>

[ulib][runtests] Format summary.json file

While this is not necessary for machines, it makes the file
significantly more redable for humans simplifying debugging.

Test: runtests-utils
Change-Id: I01f24d0a5ca4dd27e3966bfcf9987d52a6800abd


# 9f37c404 11-Jul-2018 Petr Hosek <phosek@google.com>

[runtests] Support publishing data through sinks

This enables persisting data published through sanitizer data sink and
publishing them through summary.json file. The loader service is only
changed when the output directory is set and the output directory is
used as the data sink path.

TC-27

Test: runtests-utils
Change-Id: Id76037451a79bf247d8f07046b1f7d8f6ee0c773


# 94da35d7 07-Aug-2018 Ankur Mittal <anmittal@google.com>

[runtests] Run tests as components if possible

Changed runtest to first search for cmx file for a test binary, if
found, it will use run_ctest from garnet to execute it as a component.

CP-115 #done

TEST=normal manual testing

Change-Id: Ic006d9fbb6e7047156de3dc57334f82ea104104d


# 556852df 24-Jul-2018 Gary Miguel <garymm@google.com>

[runtests] Accept a file path that specifies tests to run.

Also rename a few things for clarity.

Test: unit tests, manually tested with "runtests -f" on my linux host,
manually tested with no args on Fuchsia, which ran all tests in garnet,
and CQ will run runtests on all tests in Zircon.
Bug: IN-478 #comment
Bug: IN-527 #comment
Change-Id: I6f1485125130dadb70cefa921d80385aa0b1de33


# 5bcd7e43 29-Jun-2018 Joshua Seaton <joshuaseaton@google.com>

[runtests] Make runtests-utils-test compatible with earlier versions of macOS.

-Support for fmemopen(), although a POSIX standard, is only supported
in more recent versions of macOS.
-The WriteSummaryJSON tests now take noticeably longer, so we mark these
tests as 'medium'.

Tested: Both host and target versions of runtests-utils-test pass.

Change-Id: Ia3b00eeaab2c339106c8bddebd088b9440b2ceb7


# 7062154c 27-Jun-2018 Gary Miguel <garymm@google.com>

[runtests] Make test not incorrectly assume JSON summary order.

Test: The test failed before and passes now on my machine.
Bug: IN-496 #comment
Change-Id: I9a9a4ccefdcd2ddd295c95f319dd8c216955cbe2


# fcfd21fe 26-Jun-2018 Joshua Seaton <joshuaseaton@google.com>

[async][runtests] Specify some runtests tests as 'medium'.

There is evidence that some of the RunAllTests() host tests fire the
watchdog timer on the bots:
https://logs.chromium.org/v/?s=fuchsia%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8942983877130324176%2F%2B%2Fsteps%2Fhost_test_results%2F0%2Fsteps%2Fs___zircon_build-x64_host_tests_runtests-utils-test%2F0%2Flogs%2Fstdio%2F0

For safe measure, I mark all RunAllTests() and RunTestsInDir() here
as MEDIUM.

Tested: this change can only make *more* tests pass. We must necessarily
wait until this change is merged and the test runs in CQ before we can
verify whether host runtests-utils-test successfully runs on the bots.

Change-Id: I658521f8831b81e4f200ed11ed437168812ca9d1


# 2404ff56 22-Jun-2018 Petr Hosek <phosek@google.com>

[runtests-utils] Pass Result via pointer rather than value

We'll need to add additional data fields (e.g. list of additional data
files) to the Results struct to support use cases like code coverage,
but that means we can no longer pass Result by value since types like
fbl::Vector cannot be copied.

This change is a pure refactoring that replaces passing Result by value
with passing via unique pointer. This should be also more efficient when
handling large number of tests.

TC-27

Test: runtests-utils
Change-Id: Ib6133c34801dbaaae915de145bd502001b9a9688


# 91588818 21-Jun-2018 Joshua Seaton <joshuaseaton@google.com>

[runtests] Switch to fdio_spawn_etc

Most codepaths that create processes should use fdio_spawn rather than
launchpad directly.

Tested: runtests-utils-test passes, and there is no change in behavior
to the runtests binary.

Change-Id: Ie0f148abb3f4e25fbeaab61cba41b9f1f4b01384


# 8ebaf8a7 15-Jun-2018 Joshua Seaton <joshuaseaton@google.com>

[runtests] Create host version of runtests.

-Abstract the current runtests main() as a library method.
-Invoke with platform specific choices for
system/uapp/runtests/runtests.cpp
and
system/host/runtests/runtests.cpp

TESTED: runtests-utils-test passes as a host test; host runtests tested
by hand.

BUG: IN-358
Change-Id: Ic9bf53e4418e37d9856a5857b673f5cbd1fe2d4d


# 0760568b 15-Jun-2018 Joshua Seaton <joshuaseaton@google.com>

[runtests] Introduce scoped test directories to runtests-utils-test

Currently, the same directory is being used for file I/O across all test
cases, which could introduce unwanted dependencies. Here we introduce
ScopedTestDir to provide a directory for test scoped to each test case.

BUG: IN-478
Change-Id: If9591f6b8334b323783d5fafa84709a20be70c2b


# 59a13a9e 07-Jun-2018 Joshua Seaton <joshuaseaton@google.com>

[runtests] Define PosixRunTest() for running host tests.

Note: runtests-utils-test passes as a host test.

BUG: IN-358
Change-Id: I9f0380b4d3c38c6fea5950638e5eb0cfaa1ab3d3


# 2bc0d8c7 25-May-2018 Sean Klein <smklein@google.com>

[memfs] Split memfs into C / C++ libraries

For clients which utilize memfs to create
temporary filesystems, export a C ABI shared
library.

Internally, this C library uses a C++ library,
which is exported as source.

Change-Id: I73b95a2a2b8c4870d1b07f5f2f2525f4b0cd4e9f


# 990b5b89 24-May-2018 Joshua Seaton <joshuaseaton@google.com>

[runtests] Prepare runtests tests for cross-platform implementation.

Once we have a host implementation of 'RunTest', the existing suite of
tests will easily be made into a host test target.

BUG: IN-358

Change-Id: Icada7deb475832e3b666f654f2515d01c07a3d36


# b4404bdb 18-May-2018 Ankur Mittal <anmittal@google.com>

[runtests] Add support for reading syslogs.

IN-320 #progress

Change-Id: Icac7a0d5d8d877d444f064f0f8b5a48f12563fdc


# b0b2a718 18-May-2018 Roland McGrath <mcgrathr@google.com>

[utest][runtests-utils] Fix malloc vs delete mismatch

The right way to allocate an array is with new T[].
The right way to use unique_ptr with an array is unique_ptr<T[]>.

Change-Id: Ifcb2f9c4bcad64762dc2b034a228fb8c60f442e5


# 0986a150 18-May-2018 Ankur Mittal <anmittal@google.com>

[runtests] Format code

Using git-file-format

Change-Id: Iea897e252c6083a2b0465e2db4e0fbe48fd4095d


# e2b0adf1 15-May-2018 Gary Miguel <garymm@google.com>

[runtests] Move most code into runtests-utils and test it.

Use C++ features (rather than pure C) where possible to make
code cleaner.

Change in testing tactics:
Previously the test used its own path as the executable to be used for
testing. Switched to writing shell scripts to MemFS.
This is more flexible; it allows putting binaries in arbitrary
directories.
I think it's also more readable.

Bug: IN-223 #done
Change-Id: I66b31bf129e77be704361275a37cbdd8df9db672


# 67454463 10-May-2018 Gary Miguel <garymm@google.com>

[runtests] Make unit test not depend on external binaries.

In order to do this I changed the interface to allow the caller to
specify the entirety of argv. I think the benefit in testability is
worth the slight added interface complexity.

Bug: IN-223
Change-Id: I6d5a7f0a17d335f90eddf108b7b0acc9ae596768


# a7648ae2 08-May-2018 Gary Miguel <garymm@google.com>

[runtests] More C to C++.

Main changes:
* CamelCase instead of snake_case names, in accordance with our C++
style guide.
* Use fbl::Vector instead of list_node.
* Use fbl::MakeAutoCall and objects that clean themselves up on
destruction instead of "goto fail".
* Potential bug fix: check if get_info() return val != ZX_OK, rather
than previously checking if it was negative.
* Behavior improvement: insert a TestRun into the tests vector for all
failure modes. Previously we would only do this if we got to
launchpad_go(). There's no reason for the result to not be saved if
the failure comes earlier than that.

Bug: IN-223
Change-Id: Ib6cce2682c1ed5b2d15c13d5c193558eae2f8e88


# 299b9d4f 07-May-2018 Gary Miguel <garymm@google.com>

[runtests] Move some of runtests into a library.

Add unit tests for the library.
Unit tests are not exhaustive because AFAIK there's no existing
way to control the behavior of launchpad in unit tests.

Changes to the actual code are minimal because:
* I want to have the tests in place before making any more changes
* To keep this change conceptually focused on just a few things
(moving code + adding testing).

Future changes will make the code take advantage of more C++ features.

Bug: IN-223
Change-Id: I29a8c43e8b96db56b07f84e8f5a00cb6bd232690