HOVEL // chapter 26
Chapter 26 Part 05 / Engineering

Testing Strategy

Hovel tests contracts first. The main risk is not only that a function breaks; it is that CLI, one-shot execution, MCP, daemon RPC, module hosts, storage, and SDKs disagree about the same operation, chain, throw, provider, payload, session, artifact, or event.

Local Gates

All checks run through Task. For sparse checkout work, use the available-slice gate:

task checkout:status
task check

task check detects the repository slices present in the checkout and runs their slice checks. Missing optional slices are reported as skipped. Real failures in present slices still fail the task.

The full local gate is:

task ci

task ci first requires a full checkout. It then runs repository policy, lint, version-updater tests, remote-compatible docs verification, build, tests, the Go race gate, fuzz smoke, and coverage ratchets. Go compilation also runs Bazel-native nogo analysis, including nilness checks, across core, SDK, and module targets.

VHS rendering is an explicit host-service boundary. The task docs:demos, task docs:demos:wine, and task docs:demos:all refresh commands drive tmux and Chrome, optionally Docker for Wine, and replace the reviewed GIFs under docs/site/public/assets/demos/. Normal docs gates consume those checked-in assets, so task docs:check, task docs:build, and task docs:ci remain deterministic and remote-compatible.

CI Gate

The repository split changed the CI shape. The local and hosted gates are modular but Task-backed:

  1. core: task build, task test, race tests, fuzz smoke, lint, and coverage for the core workspace.
  2. sdk: task sdk:ci builds and tests the Go, Python, and Rust SDK slices.
  3. modules: task modules:ci builds and tests in-repo examples, Squatter provider code, module packaging checks, and workspace install behavior, then task release:modules-package verifies the module package/index release path.
  4. docs: task docs:check tests and assembles the complete declared site, including generated SDK API pages and internal link validation; task docs:report materializes the same site with current test evidence.
  5. squatter-wine: installs Wine in CI, then runs task modules:wine-test and task docs:demos:wine.

CI, Pages, and publish jobs append GitHub Actions job summaries with the commands they ran, coverage ratchets, generated artifacts, staged site statistics, or publish targets where those details apply.

Astro always publishes the test report application. Run task docs:report to execute the report-producing test workflow and attach its JSON, logs, and XML to the application in _site/. A normal task docs:build deterministically publishes the same route without test evidence. The report preserves Bazel target-level evidence, suite-level rollups, per-case XML where present, assertion failure messages, captured case output, full console logs, raw XML, and undeclared outputs for drilldown across the monorepo.

Docs verification stays in the docs slice instead of becoming a dependency of every core checkout. The full aggregate task ci requires a complete checkout and runs the wired remote-compatible slice gates together. Hosted CI runs host-service demo rendering as a separately named step so failures cannot be confused with hermetic build or test failures.

Test Inventory

The root Bazel test suite covers the main product surfaces:

  1. Go host tests for domain packages, application services, command registry, daemon runtime, daemon manager, SQLite/filesystem storage, descriptors, CLI, command mode, root CLI, daemon RPC, MCP, and the Hovel binary entrypoint.
  2. Module-runtime tests for Python RPC, Go SDK, Go SDK test helpers, Python SDK, Rust SDK, Go/Python/Rust example modules, and the MS17-010 Python example modules.
  3. Chain, throw, confirmation, launch-key, artifact, event, installed-payload, session, and operator-session behavior through application and adapter tests.
  4. Schema smoke tests and documentation smoke tests.
  5. Squatter provider tests, PE/source-contract tests, client wire/shell/SMB pipe unit tests, mux/protocol integration tests, file transfer tests, and Wine-backed functional tests in the Wine gate.
  6. Module packaging tests for in-repo example modules and workspace installation.
  7. Demo harness tests, including standard throw/session/MCP verification and the MCP agent demo harness.
  8. Shared throw-audit contract assertions reused by command mode, CLI, and MCP tests.
  9. A fuzz smoke target for pure domain parsing behavior.

What Each Layer Protects

Layer What it protects
Domain Validated value objects, state transitions, IDs, and safety-sensitive invariants without adapter imports.
Application services Planning, confirmation, throw control, chain runtime, operator session state, installed payload records, artifacts, events, and launch-key behavior.
Adapters CLI, one-shot command mode, root commands, daemon RPC, MCP tools, terminal logging, descriptor loading, and filesystem storage.
SDKs and modules JSON-RPC framing, handshake/schema paths, logging, step hooks, sessions, provider methods, packaging, and language-specific helper APIs.
Squatter Provider behavior, Windows PE shape, no-import-table constraints, mux framing, process/file modules, transports, and Wine-backed payload operation.
Docs and demos Bazel-declared operator demo GIFs, staged Pages content, SDK API docs, and internal links.

Safety-Sensitive Contracts

Tests around throw execution must preserve these rules:

  1. A throw cannot start without a persisted plan and matching confirmation record.
  2. throw --now skips the prompt but still persists an auditable bypass confirmation.
  3. Dangerous modules require the explicit dangerous-module allowance before execution.
  4. CLI, one-shot execution, MCP, and future adapters must use the same planning, confirmation, guardrail, event, artifact, and persistence paths. The shared audit contract asserts durable plan, now-bypass confirmation, and throw lifecycle events after real front-end execution.
  5. Installed payload records are created only from explicit provider descriptors and remain visible when reconnect or cleanup fails.
  6. Operator-controlled configuration values are not silently redacted or dropped.

Coverage Metrics

Coverage is a ratchet, not a vanity number. task coverage runs Bazel merged LCOV coverage for the Go domain and application layers through task coverage:go, then runs Python SDK coverage through the standard-library trace module. It writes generated reports to coverage/.

Domain ratchet: 75%
Application ratchet: 65%
Python SDK ratchet: 80%

The ratchets start below the existing baselines so the gate fails on regressions without pretending the first enforced number is the long-term target. Run task coverage for the current measured values. The intended long-term goals remain domain 90%+, application services 85%+, Python SDK 85%+, and adapter coverage judged primarily by contract behavior rather than raw line count.

Stress Gates

task test:race runs the application service layer with the Go race detector through rules_go. task fuzz:smoke runs a short fuzz campaign against the domain event type parser. Both are part of task ci and the GitHub Actions build-test job.

Future Coverage

New product surfaces should bring their own contract tests before becoming durable product promises:

  1. The TUI should have model/update tests, rendering tests, and daemon-boundary integration tests.
  2. REST/OpenAPI should verify generated API contracts, authorization/session binding, and parity with application services.
  3. The general service process runtime should have lifecycle, health-check, supervision, logging, and shutdown tests.
  4. Richer MCP resources and prompts should verify that they expose context without granting authority or bypassing confirmation.
  5. Coverage ratchets should tighten as focused tests land, rather than jumping directly to aspirational percentages.