Esta página aún no está traducida; se muestra en inglés.
Testing
From a laptop with no radio to a radio on the SDR bench. The commands, flags and pass criteria are in docs/testing/README.md and tests/hil/README.md; this page says what each level is for and where a new test goes.
| Level | Runs on | Command | Catches |
|---|---|---|---|
| Unit | the host | make -C tests/host |
Wrong math, encoders, parsers, the settings format |
| Software in the loop | the host | make -C tests/host |
Whole features against simulated chips: calls, data, hotspot, CPS, storage, the UI |
| Hardware in the loop | a radio on USB | python3 -m pytest tests/hil -v -rs |
The real chips and timing; with flags, transmit and receive on air |
| Manual bench cases | the bench board and a person | tools/bench/bench_line.py HOST run bench_tx ARGS |
What can't be scripted yet (dmr-bench.md) |
| Release gate | the host | JOBS=8 tools/ci.sh |
Everything above that needs no radio, plus every board's feature matrix with zero warnings, doc links and the license checks |
tools/ci.sh must pass before every merge (Contributing); what it runs step by step is in
docs/testing/README.md section 1. The CI workflows run its pieces per push and nightly
(CI).
| Also | Command | Details |
|---|---|---|
| Coverage | tools/coverage.sh (needs gcovr) |
build/coverage/index.html; CI, Coverage |
| Sanitizers | make -C tests/host clean && make -C tests/host SANITIZE=1 |
ASan and UBSan; any report fails the test. RFOUNDRY_CI_SANITIZE=1 tools/ci.sh adds it to the host step |
| Fuzzing | RFOUNDRY_FUZZ_ITERS=1000000 make -C tests/host SANITIZE=1 |
Seeded mutation loops over the parsers of untrusted input (tests/host/fuzz.h); RFOUNDRY_FUZZ_SEED varies the seed |
Host tests
tests/host/ holds one *Test.c per area and the simulators they share:
| File | Simulates |
|---|---|
sim/dmr_sim_modem.c |
A DMR modem: plays bursts into the call stack and records what it sends |
sim/sim_flash.c |
SPI flash, for the codeplug, the settings ring and the journal |
sim/ui_sim.c, sim/ui_png.c |
The UI's display and keys, with a script language and PNG output |
sim/split_sim_modem.c, sim/splitSim.c, sim/splitPairTest.py |
Two radios' cores in lockstep virtual time, for the split duplex repeater |
cpsSim.c |
The radio's side of the CPS protocol, for tools/cps_client.py and cpsClientTest.py |
snmpPeer.c |
An SNMP manager, for the agent's end-to-end test |
fuzz.h |
The seeded mutation fuzzer the parser tests use |
stubs/ |
Stand-ins for services a test doesn't exercise |
The Makefile groups the tests into run-* targets (run-core, run-ui, run-call, run-mmdvm, run-settings,
run-cps, ...); make -C tests/host runs them all. Each test compiles the firmware's own sources with the host
compiler, so a test exercises the code that ships. Test vectors come from the standards or are recorded from real
radios and the stock codec, never from other firmware.
To add a test: write tests/host/<area>Test.c, add its rule and a run-<area> target to tests/host/Makefile, and
hook it into run (run: run-<area>). Groups are named by the area or directory they test, never by a radio or
platform. make -C tests/host check-run (part of run) fails when a test in TESTS has no ./<test> line in any
group; a program a test script drives (a simulated peer) goes in HELPERS instead. Build without warnings; ci.sh
fails on any. A parser of bytes from outside (USB, the air, the network) also gets a fuzz loop with fuzz.h.
A test of one board's own code (its calibration reader, its pin tables, its first stage) lives with the board, in
boards/<b>/tests/. Its host.mk adds the test to TESTS and to the run-boards:: target. tests/host includes
every boards/*/tests/host.mk; a fragment names its own files through HERE := $(dir $(lastword $(MAKEFILE_LIST)))
in its prerequisites, and its recipes use $< and $^, since HERE changes with the next fragment. The binaries are built in tests/host/ like the rest.
The firmware build only compiles the *.c directly in a board's directory, so these never reach an image. A driver
or platform test stays in tests/host/, named by what it tests, even when one radio needed it first.
The UI simulator and the gallery
The UI runs unchanged on the host. Three tools use it:
| Command | Does |
|---|---|
make -C tests/host run-ui |
gfxTest (drawing) and uiFlowTest: replays every task flow of task-flows.md against the press budgets of usability-tests.md on four display and key map pairs, and crawls every screen for the rules in ui.md 7 |
tests/host/uiSim --geom 160x120i5 --script "ok 4" --out f.png |
Renders one screen after a key script. The options are in the header comment of tests/host/uiSim.c, the script tokens in tests/host/sim/ui_sim.c |
tools/beep_reel.py |
Makes build/beep-reel/beep_reel.mp4: every beep in the scene that sounds it, on the simulated screen, captioned. Needs ffmpeg built with libass (its subtitles filter draws the captions). make -C tests/host run-ui runs its checks without the video (radio-features.md 4) |
make -C tests/host ui-gallery |
Renders every gallery screen at every display class into docs/ux/screens/<class>/, and one contact sheet per class into docs/ux/screens/sheets/<class>.png |
make -C tests/host ui-sheets [GEOM=128x64] |
The contact sheets only, into tests/host/out/sheets/ (not committed): every gallery screen at 2x under its name on one page, to audit a layout change at a glance. GEOM takes any geometry --geom does |
The display classes are 160x120i5 (160x120 with 5 px hidden left and right), 160x128,
240x320i10x5 (16 colors, anti-aliased fonts one size up, 10 px hidden left and right and 5 px top and bottom),
128x64 and 240x320; the wireframes README says which radio uses which.
The user manual uses the 160x120i5 images.
After a UI change, run ui-gallery and commit the PNGs with the change, so the review shows the difference. The
gallery target deletes and rewrites docs/ux/screens/*/; commit only what you meant to change. Commit the contact
sheet of each class whose screens changed. A new screen gets an
entry in the gallery[] table at the top of tests/host/uiSim.c.
Hardware in the loop
The HIL tests drive a radio through the debug link: they press keys, read the radio's state and take screenshots. Each test leaves the radio as it found it. Without a radio they all skip.
Opt-in flags allow what changes the radio or the air around it:
| Flag | Allows |
|---|---|
| none | Radio-only tests. Nothing transmits |
--allow-rf |
Transmitting, from the radio or the SDR bench |
--allow-service-stop |
Accepted and ignored: a bench board stops nothing |
--allow-usb-switch |
Switching the radio to USB network mode and back |
--allow-codeplug-write |
Writing the codeplug over CPS (backed up and restored) |
--allow-reboot |
Restarting the radio |
The full list, the options and the rig assumptions are in tests/hil/README.md; "Writing a test" there has the fixtures.
A skip is not a pass. Every skip names its kind, and a bring-up run (tools/bringup.py run --radio <id>) runs a
radio's feature checklist, fails a required test that skips for a reason its feature doesn't declare, and records the
result in docs/testing/results/ (tests/hil/README.md).
The SDR bench
A second station for on-air tests: one or two boards that serve the bench line protocol (tools/bench/bench_line.py).
They send calls, texts and data to the radio and check what the radio transmits (IDs, color code, slot timing, ramps, carrier
offset, FM tones and deviation). A board's address comes from --bench-api or RFOUNDRY_BENCH_API, never from files
in the repository. Setup and the manual cases: docs/testing/dmr-bench.md.
A bug found on the bench should end as a host test, so it stays fixed without the bench.