Framework test lifecycle

This page covers everything that happens around your test: how a coroutine becomes a registered test, how checks record success and failure, who owns the processes it spawns, how a test ends, and what it reports when it does.

All of this lives in the reusable C++ framework rather than in the cpptb command, so an embedding application can discover a compiled catalog, select one test, run it, and consume its result directly — no particular build system or regression harness required. If you just want to run tests from the command line, Running tests is the shorter path.

The framework owns:

• compiled test registration, metadata, and exact selection;

• one TestContext and one TestResult for each invocation;

• nonfatal checks, fatal requirements, and structured warnings;

• ownership, provenance, and cleanup of processes started by a test;

• terminal test states and simulation-time timeout policy; and

• optional in-process result callbacks through ResultSink.

Process-aware diagnostic messages are provided by the separate structured logging API. Log levels and ordered log histories remain observational; this page defines the warnings and checks that change structured test results.

A harness may build simulator executables, start fresh processes, filter catalogs, enforce wall-time limits, and translate results into CI formats. Those policies are deliberately outside the lifecycle API. See Running tests for the optional reference harness.

Register and invoke tests

Register each root coroutine in the testbench translation unit:

Task<void> reset_defaults(Dut dut, TestContext& test) {
    dut.clk.set_now(0);
    test.start_clock(dut.clk, 10_ns);

    dut.rst_n.set(0);
    co_await RisingEdge{dut.clk};
    co_await ReadOnly{};
    test.expect_eq("reset count", dut.count.get(), 0u);
}

CPPTB_REGISTER_TEST(reset_defaults);

The framework exposes registered_tests<Dut>() for catalog discovery and run_registered_test(...) for exact selection. Each invocation receives fresh lifecycle state; checks, failures, warnings, and owned processes are never shared between results.

Checks and diagnostics

Checks always carry a concise user label:

test.expect("response is in range", response >= low && response <= high);
test.expect_eq("response payload", response, expected);

test.require("configuration loaded", configured);
test.require_eq("protocol version", version, kSupportedVersion);

expect() and expect_eq() are nonfatal. Independent failures accumulate in the current result. require() and require_eq() record a failure and stop the owning test cleanly, allowing structured reporting to complete.

Equality checks retain formatted actual and expected values. Formatting is only invoked after actual == expected fails, so passing checks do not build diagnostic strings.

Value type	Failure representation
Boolean	true or false
Signed and unsigned integers	Decimal value
Floating point	Round-trip decimal value
C++ enums	Symbolic name when supplied, otherwise the underlying value
Strings and string-like values	Text value
Bits<W>	Width-qualified hexadecimal, such as 12'habc
LogicBits<W>	Width-qualified binary preserving X and Z
Arrays and input ranges	Recursive list, such as [1, 2, 3]
Generated packed views	Width-qualified raw packed value

Generated HDL enums provide their symbolic names automatically. A user type can specialize DiagnosticFormatter<T> without modifying the check API:

template <>
struct cpptb::DiagnosticFormatter<Transaction> {
    static std::string format(const Transaction& value) {
        return "Transaction{address=" + std::to_string(value.address) +
               ", data=" + std::to_string(value.data) + "}";
    }
};

Warnings and process ownership

warn() records a structured warning without failing the test:

test.warn("response used the protocol fallback");

Start lifecycle-owned concurrency through the context:

Task<void> protocol_watcher(Dut dut, TestContext test) {
    while (true) {
        co_await RisingEdge{dut.response_valid};
        co_await ReadOnly{};
        test.expect_eq("response reserved bits",
                       dut.response_data.get() & 0xff00'0000u, 0u);
    }
}

Task<void> traffic_test(Dut dut, TestContext& test) {
    test.spawn_detached(protocol_watcher(dut, test));

    auto driver = test.spawn(input_driver(dut));
    co_await driver;
    test.require_eq("driver completed", driver.done(), true);
}

CPPTB_REGISTER_TEST(traffic_test);

Both forms attach the child to the current test. A detached process is detached only from a user-visible Process handle; it remains owned by the test. Normal completion, a fatal requirement, a timeout, or an uncaught child exception cancels unfinished owned processes.

The watcher takes TestContext by value because it may remain alive until the root test completes. Its checks and uncaught exceptions still carry the watcher’s process ID and spawn location. The finite driver retains a handle so the foreground sequence can await it and inspect its terminal state.

Use spawn() when work must run concurrently, needs an independent process identity, or may be cancelled through a handle. A sequential helper can be awaited directly with co_await helper(...); this avoids creating an independent process while retaining ordinary coroutine composition.

Every root and spawned process receives a stable invocation-local numeric ID. Failures, warnings, and exceptions retain that ID, the process description, and the source location of the spawn() call. An uncaught child exception is therefore attributed to its owning test and process instead of being lost or surfacing later as a generic timeout.

Lifecycle outcomes

The terminal states have explicit meanings:

Status	Meaning	Successful result?
passed	Body completed with no failures	Yes
failed	An expectation or requirement failed	No
error	Selection failed or an exception escaped	No
skipped	Metadata or the running test requested a skip	Yes
expected_failure	A test marked expected-to-fail failed normally	Yes
unexpected_pass	A test marked expected-to-fail passed	No
timed_out	The simulation-time limit expired	No

running and not_run are nonterminal states used during selection and execution. Exceptions and timeouts are never converted into expected failures.

A test can skip dynamically:

if (!dut.has_optional_feature.get()) {
    test.skip("optional feature is not present");
}

Static policy belongs in registration metadata:

CPPTB_REGISTER_TEST_WITH_OPTIONS(
    known_issue,
    (::cpptb::TestOptions{
        .tags = {"nightly", "known-issue"},
        .expected_failure = true,
        .expected_failure_reason = "tracked as RTL-241",
        .simulation_timeout = 20_us,
    }));

The timeout is measured in simulator time and cancels the test body cleanly. A harness-level wall-time timeout is a separate safeguard for a stalled simulator process.

Parameterized cases

Parameterized cases retain one typed C++ value per descriptor. Stable catalog names use test_name[case_name], and an embedding harness can inspect the case name and tags without knowing how the value is represented:

struct CounterCase {
    uint32_t pulses;
    uint32_t expected;
};

Task<void> counter_case(Dut dut, TestContext& test,
                        const CounterCase& parameter) {
    for (uint32_t i = 0; i < parameter.pulses; ++i) {
        co_await RisingEdge{dut.clk};
    }
    test.expect_eq("count", dut.count.get(), parameter.expected);
}

CPPTB_REGISTER_TEST_CASE(counter_case, empty, (CounterCase{0, 0}));
CPPTB_REGISTER_TEST_CASE_WITH_OPTIONS(
    counter_case, wraps, (CounterCase{256, 0}),
    (::cpptb::TestOptions{.tags = {"nightly", "wrap"}}));

The catalog contains counter_case[empty] and counter_case[wraps] as independently selectable tests.

Structured results

TestResult is the harness-neutral result model. It records:

• test and case names, tags, terminal status, and status reason;

• check, failure, and warning counts;

• final simulation and wall time;

• structured failure and warning records; and

• process identity, spawn location, check location, and comparison values.

ResultSink provides optional test_started, failure_recorded, warning_recorded, and test_finished callbacks. The versioned JSON serializer is another adapter over the same result model. Neither requires the public command-line harness.

CLI tag filtering, process launch policy, JUnit XML, waveform-on-failure reruns, wall-time enforcement, reproduction command rendering, and build diagnostic presentation remain harness concerns rather than framework lifecycle behavior.

Related APIs

• TestContext reference — the signatures behind every check, spawn, and registration macro on this page.

• Embedding and results — TestResult, ResultSink, and the versioned JSON contract.

• Scheduling — the wait graph a simulation-time timeout captures before cancelling the test.

• API reference — checks and registration at a glance.