# Future directions

Candidate work that is **not scheduled**. The [roadmap](roadmap.md) admits a
milestone only once a concrete project supplies the use case, semantics,
runnable example, test coverage, and performance peer. Nothing here has cleared
that bar; this page records the analysis so the reasoning survives, in the same
spirit as
`benchmarks/authoring_core/OPTIMIZATION_NOTES.md`.

Two areas are covered: capabilities the framework lacks, and the verification
corpus needed to judge whether it is actually better than the alternatives.

## Align the scheduling semantics with cocotb

:::{note}
**Largely resolved.** This section records the analysis as it stood before
`timing_backend` and `deferred_writes` shipped. Both gaps described below are
now closed by standard project configuration; see
[The write model](scheduling.md#the-write-model) for what cpptb does today.
Only item 3 at the end -- making the write model unconditional rather than a
key -- is still open. Read what follows as history, not as current behavior.
:::

cocotb is the framework most people arriving here have used, and the trigger
vocabulary already matches it deliberately: `RisingEdge`, `FallingEdge`,
`ReadOnly`, `ReadWrite`, `NextTimeStep`, with drivers, monitors and scoreboards
composing the same way. Two things then behave differently, and both are
invisible until a test produces a wrong answer.

**Writes apply immediately rather than being deferred.** In cocotb an assignment
is queued and applied at the next `ReadWrite` point, so writing straight after
`await RisingEdge(clk)` cannot affect the edge just awaited. Here `set()` applies
at once and `co_await RisingEdge{}` resumes before the design evaluates the
edge, so the same shape drives into that edge and the transaction commits a
cycle early. The symptom is a read-back returning the value just written, which
reads as a design bug.

**No project build supplies the phase waits.** `ReadWrite`, `ReadOnly` and
`NextTimeStep` are documented API, and [Performance](performance.md) benchmarks
four backends providing them, two of which pass the timing conformance suite. A
default `cpptb build` provides none of them, and a testbench using those waits
builds cleanly and fails at run time with a message naming `--vpi`. Putting
`--vpi` in `build.verilator_args` makes the waits run, and the drive point it
produces is right, but it does not give the documented contract: `cpptb build`
links Verilator's `--binary` main, which does not re-evaluate the design
between the `ReadWrite` and `ReadOnly` callbacks, so two of the five
timing-phase conformance checks fail with no diagnostic. Both contract-complete
backends need a `--cc --exe --build` link against
`src/verilator_timing_main.cpp` instead of `--binary`, which no `cpptb.toml`
key can ask for. The generated SV-DPI backends are reachable only by matching
`design.defines` against `build.cxx_flags` by hand, where defining
`CPPTB_SV_DPI_TIMING` alone silently selects the inline pump that the same
benchmarks record as invalid. [Roadmap](roadmap.md#what-the-gap-looks-like)
carries the worked examples and the observed output.

Three ways to close the gap, roughly in increasing order of commitment:

1. **Name and validate the backend selection.** A `[build] timing_backend` key
   threaded into code generation would replace a `--vpi` that does not deliver
   the contract, emit the link both contract-complete backends need, and reject
   the define combinations that currently produce wrong answers without a
   diagnostic. This is plumbing over machinery that already exists and is
   already measured.
2. **Offer deferred writes.** A queued write applied at the next settle point,
   alongside the immediate `set()`, would make a driver written the cocotb way
   correct on any backend rather than only where phases are available.
3. **Make deferred the default after an edge wait.** The most familiar, and the
   largest change: it alters what existing testbenches do and costs a scheduler
   round trip per write, so it would need the performance peer to justify it.

The first turns a flag that does not deliver the contract into a supported
choice. The second is what
actually makes a translated cocotb testbench correct. The third is the only one
that removes the difference entirely, and is the one that needs the most
evidence.

Items 1 and 2 have since shipped: `timing_backend` names the backend and
emits the link both contract-complete backends need, and
`deferred_writes = true` supplies the cocotb write model. Both are now standard
project configuration -- see
[The write model](scheduling.md#the-write-model). Item 3, making that model
unconditional rather than a per-project key, is what remains.

This has a concrete use case, a runnable example and a performance peer, so it
clears the promotion bar in [Roadmap](roadmap.md#no-priority-backlog) if it is
wanted. Found while porting Ibex's `dv/cs_registers` testbench, where the
cocotb-shaped driver was the first thing tried; see
`experiments/open_core_ports/ports/ibex_cs_registers`.

Porting Ibex's icache testbench put a size on what the convention costs, and it
is larger than "drive off the falling edge" suggests. Once drivers call each
other the rule has a second half: every task that drives has to be entered at a
drive point, return at a drive point, and not open with a wait, because a task
that opens with a wait re-anchors itself and issues its first write a clock
late. Nothing states or checks either half, so each of that port's driving
tasks had to be traced edge by edge against the SystemVerilog clocking block it
replaces before the port could be believed, and the rule survives only as a
comment at the top of `testbench.cpp`. See
`experiments/open_core_ports/ports/ibex_icache_cpptb`, whose per-item rates
agree with the UVM baseline to about 1% on `fetches/insn`, the measure that
would move first if a driver were mistimed.

## Framework capabilities

### Temporal assertions

The largest gap that no milestone currently covers. Every check today is
procedural: `expect_eq` compares a value at one instant. There is no way to say
"every request is followed by an acknowledgement within five cycles" except an
ad-hoc coroutine with hand-written bookkeeping, and nothing distinguishes a
property that passed from one that was never exercised.

This is the reason teams tolerate SystemVerilog. It is also the gap the
coroutine model is best placed to close: implication, bounded windows,
stability, and `throughout` all express naturally as awaitable expressions, and
their pass, fail, and vacuity counts belong in `TestContext` alongside existing
checks, feeding assertion coverage into `cpptb/coverage.hpp`.

It stays clear of the [deliberate non-goals](roadmap.md#deliberate-non-goals):
no factory, no phases, no objections, no sequencer. It is a missing checking
primitive, not imported architecture.

### A second simulator

Already milestone 6, and worth restating as the gate on everything else.
Verilator-only excludes encrypted vendor IP, gate-level netlists, power-aware
runs, and accurate X propagation, which is most of what an industrial user
needs. It is also the claim in the README that remains untested: transport is
described as standards-based DPI, and one backend cannot demonstrate that.

### Waveforms and transaction annotation

Milestone 7 already lists waveforms. Nothing in `include/` or the code
generator emits VCD or FST today. Debugging is where verification time is
spent, and a failure currently yields logs and nothing else.

The valuable half is not the dump itself but the annotation: transaction
recording already exists, so a scoreboard mismatch could place the offending
transaction on the waveform at the right cycle. Combined with deterministic
seeds, the deferred waveform-on-failure rerun becomes cheap and is worth
promoting out of the backlog.

### Coverage closure as a workflow

`cpptb/coverage.hpp` can merge, but merging is a primitive, not a methodology.
Closure needs merging across seeds and runs, a report ranking holes by how
reachable they are, and an answer to "what should I randomize next".

This is deliberately not
[UCIS interchange](roadmap.md#no-priority-backlog), which remains correctly
deferred: the value is the local closure loop, not a file format. Coverage as
structured data that both C++ and Python tooling can consume plays to a
strength SystemVerilog handles poorly.

### Regression orchestration

Seeds already exist in the runner. The public CLI is `build`, `list`, and
`test`. What is missing is the loop that multiplies everything else: many tests
across many seeds, run in parallel, with failure clustering, a one-line
reproduction command, and a trend over time. The backlog defers JUnit
conversion, tag filtering, and reproduction-command presentation; individually
small, together they are the difference between running tests and running a
regression.

### Simulation snapshot and restore

Long simulations pay their full cost on every run. A failure two million
cycles into a boot sequence is debugged by replaying those two million cycles
for every hypothesis, and a suite whose tests share one expensive
initialization repeats it per test. The capability wanted is a snapshot of
simulation state taken at a chosen point, and a later run that restores it
and continues from there instead of starting at time zero.

The design half is tractable. Verilator generates save/restore support under
`--savable`: `VerilatedSave` and `VerilatedRestore` round-trip the complete
model state, and a cpptb simulator is one process built around one Verilated
model, so design state, simulation time, and the generated wrapper's calendar
state all sit behind that one mechanism.

The testbench half is the real problem, and it is what shapes any honest
design. A suspended C++ coroutine is a heap frame holding arbitrary locals,
pointers, and a resumption address; there is no portable way to serialize
one, so a snapshot cannot capture "the whole test, mid-await". Two shapes
survive that constraint:

- **Declared snapshot points.** A test reaches a quiescent point — a settle
  point, where the deferred-write queue is empty by construction — and asks
  for a named snapshot alongside whatever testbench state it explicitly
  saves. A restore run reloads the design exactly and starts a *fresh*
  registered continuation coroutine there; suspended user coroutines do not
  come back. The contract is "the design resumes exactly, the testbench
  restarts deliberately", which is also what vendor `$save`-style checkpoints
  deliver to UVM in practice. Most framework-owned state already serializes
  by construction: a random stream is a seed and derivation counters,
  coverage has a schema-1 JSON model, register-model mirrors and
  `SparseMemory` would need explicit save hooks. Clocks re-register through
  the existing time-zero query path.
- **Process-level checkpointing.** `fork()` at the snapshot point captures
  everything, coroutine frames included: the parent holds the warmed state
  and forks one child per variant or hypothesis. This is cheap to prototype
  precisely because one process owns the whole simulation, and it covers the
  same-run cases — failure bisection, N seeds branching from one warmed
  reset — without any serialization design. It does not persist across runs;
  cross-run persistence via CRIU is Linux-specific and fragile against
  ASLR and library changes, and should not be the load-bearing mechanism.

The two compose: fork for exploration within a run, declared snapshots for
resuming across runs. Promotion needs a concrete workload that hurts today —
a long soak that fails late, or a directed suite replaying one boot per
test — and the acceptance gate writes itself from machinery the repository
already has: an identity gate, in the mold of `make backend-equivalence-test`,
requiring that a restored run produce results and waveform identical to the
uninterrupted run from the snapshot time onward.

### Not recommended

Further scheduler performance work. Measured ratios sit between `0.76x` and
`1.08x` against pure SystemVerilog, the optimization notes put the remaining
architectural cost near `0.7%`, and profiles attribute the rest to Verilator
itself. Speculative protocol components should also stay deferred on the
existing bar, since a component library built without a consumer is a
maintenance liability.

## A real verification corpus

The current benchmarks are framework-authored. Deciding whether cpptb genuinely
improves on the alternatives needs testbenches somebody else wrote, for designs
somebody else maintains, running workloads long enough to be meaningful.

### What UVM on Verilator can and cannot do

Verilator 5.050 supports considerably more of the verification subset than it
once did. Verified directly against the pinned version:

- classes with `rand` members and `constraint` blocks solve and randomize;
- constraint solving is delegated to an external `z3` binary, so
  `make z3-toolchain` incidentally satisfies it;
- covergroups with `coverpoint` and `bins` compile and sample.

Verilator's own contributor documentation describes its current focus as
"completing Universal Verification Methodology (UVM, IEEE 1800.2-2017)
support", so UVM is in progress rather than finished.

`experiments/uvm_comparison/` already measured what that means in practice, on
this same Verilator against Accellera UVM 1800.2-2017-1.0. UVM does run: one
test passed three times out of three. The others did not. One segfaulted on a
run, and both random tests executed while reporting scoreboard errors, so four
tests produced two clean results between them.

The consequence for comparisons follows from that record rather than from an
assumption. UVM is not yet a dependable **performance** peer on Verilator,
because the tests that exercise randomization are the ones that fail, and a
comparison is worthless if the baseline is not producing correct results. It is
a legitimate **ergonomics** peer today: port its structure and compare the
code. For performance the dependable peers on Verilator remain plain
SystemVerilog and cocotb, which the four-mode harness already supports. This is
worth re-testing periodically, since it turns on Verilator maturing rather than
on anything in this repository.

Two later ports qualify that. `experiments/open_core_ports/ports/core_ibex_uvm`
and `.../ibex_icache_uvm` run upstream UVM environments that somebody else
wrote, and the icache one passes all ten of its own tests. Both needed work to
get there, and `ibex_icache_uvm` records six Verilator defects with reduced
cases, five of them in constrained randomization: `dist` applied as an equality
against a pre-drawn sample, `std::randomize` ignoring `dist` weights, a
`solve ... before ...` disabling every `soft` constraint in the class, a `soft`
nested inside an `if` not being dropped when it conflicts, and a constrained
`randomize()` over an `inside` range not being uniform over that range. The
last of those is the whole of the stimulus difference between the icache UVM
baseline and its cpptb port. So a UVM baseline can now be made to produce
correct results on Verilator, and the stimulus it then produces still needs
checking against what its own constraints describe.

A useful side effect of Verilator's constraint support: pure-SystemVerilog
twins can now use constrained-random classes rather than procedural stimulus,
making them a more representative baseline than they were.

### Candidates, verified as live and maintained

| Project | Verification surface | Why it fits |
| --- | --- | --- |
| lowRISC/ibex | `examples/simple_system` with `ibex_simple_system.cc`, plus `dv/uvm/core_ibex` and `dv/verilator` | Its Verilator harness is already hand-written C++, so a cpptb port is a like-for-like ergonomic comparison. Runs real RISC-V software. |
| alexforencich/verilog-ethernet | around 230 cocotb testbench directories | cocotb is coroutine-based, the closest methodological peer, and already a mode in the harness. Two of its modules are vendored already. |
| openhwgroup/cva6 | `verif/` including `core-v-verif`, `env`, `regress`, `sim`, `tb`, `tests` | Application-class core with a full verification environment; the UVM half serves as an ergonomics peer. |
| pulp-platform/axi | `test/` | AXI is the protocol the component library lacks; APB is the only bus covered today. |
| lowRISC/opentitan | `hw/dv/verilator` | Chip-level with boot ROM and software. The most credible workload and the heaviest to adopt. |
| chipsalliance/caliptra-rtl | active, security root of trust | Realistic security-focused traffic; verification flow needs assessment before committing. |

### Suggested order

1. **Ibex simple system.** Smallest step with the clearest comparison, and it
   extends the existing PicoRV32 firmware workload to a maintained core whose
   Verilator flow is C++ already.
2. **One verilog-ethernet cocotb testbench.** Ported one-to-one, it measures
   cpptb against the peer methodology that most resembles it, on a design
   partly vendored already.
3. **CVA6 or pulp AXI**, depending on whether the goal is a larger workload or
   a wider protocol surface.
4. **OpenTitan**, once the earlier ports have established the porting pattern.

Vendoring should follow the precedent in
`benchmarks/framework_comparison/open_cores/`: exact pinned files, upstream
licence notices preserved in `THIRD_PARTY_NOTICES.md`, and only the RTL each
workload elaborates.

### What such a comparison must report

Performance alone would waste the exercise. Each port should record the
semantic evidence both sides produce, the wall time and its validity under the
existing environment guard, and the ergonomic measures that motivated the
framework: lines of testbench code, how much is generated rather than written,
what a failure reports before a debugger is opened, and how long a single test
takes from edit to result.