FrameAssistant/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Working rules (agreed with the user, always apply)

- **Proper software engineering:** follow established procedures and processes, not ad-hoc
  hacking.
- **Nothing without explicit permission.** Propose first, then act only on the user's go-ahead.
  Permission covers only what was asked: no extra changes, renames, rewrites or "while I'm at
  it" fixes. Suggest those instead.
- **Ask, don't assume.** When a requirement, decision or detail is unclear, ask for
  clarification before proceeding. Don't fill gaps with your own defaults.
- **Fast, iterative development cycle.** Each iteration runs through these phases in order, and
  each phase ends with the user's approval before the next starts:
  1. **Requirements engineering:** what is needed and why, with acceptance criteria.
  2. **Planning and design:** how it will be built, alternatives, trade-offs.
  3. **Implementation:** only what the approved design covers.
  4. **Testing:** verify against the acceptance criteria and report the results faithfully.
  5. **Assessment and decision:** review the outcome together and decide how to proceed.
- Keep iterations small, so each cycle stays fast.
- An informal "ok" from the user is enough to approve a phase.
- **Document everything:** requirements, designs, decisions, test results. Documentation is
  direct and concise, but complete. It is Markdown and lives in `docs/`. Whenever something
  changes, also update the supporting documents (README, CLAUDE.md, other docs) to match.

## What this is

**FrameAssistant**, "Steam Frame companion for Home Assistant" (in the spirit of HASS.Agent):
it reports headset/environment sensors to Home Assistant, offers services Home Assistant can
call (e.g. notifications in the headset), and shows a Home Assistant dashboard as a SteamVR
dashboard tab. Status: the overlay platform works; the Home Assistant parts are being planned.

**Start every session by reading `docs/PROGRESS.md`** (current state, next steps, log).

User-facing docs: `README.md` (quick start, development, CLI), `docs/REQUIREMENTS.md`
(requirements, open questions, decisions), `docs/PROGRESS.md`, `docs/ARCHITECTURE.md`, `docs/STEAM-FRAME.md`.
Keep them in sync with changes. This file holds what a developer needs on top of that.

Platform: a SteamVR **dashboard overlay**, built from the author's vrdashboard template,
targeting the Steam Frame (aarch64, SteamOS) and x86_64 Linux. Stack: C++20, Qt 6 Quick (QML,
≥ 6.5) and OpenVR `IVROverlay`. Two render backends behind one interface, chosen at runtime
with `--graphics-api vulkan|opengl`.
- **Naming:** the app is FrameAssistant (`project(frameassistant)`, executable
  `frame-assistant`, app key `net.matzman.frameassistant`, tab "Frame Assistant"). The
  overlay platform keeps its template names (`vrdashboard_core`, `VRDASH_*`, `vrdash_*`,
  `DashboardApplication`); don't rename them. Avoid "Steam", "Home Assistant" and "HASS" in
  product names (trademarks; the user dislikes "HASS").
- Keep it simple: no FetchContent dependency, no plugin system.
- **Vulkan is the default** where it is compiled in: on the Frame it is about 1.9× faster than
  OpenGL, which runs there through zink.
- Without an explicit `--graphics-api`, a Vulkan start-up failure falls back to OpenGL
  (`startBackend()` in `src/app/DashboardApplication.cpp`). With an explicit choice there is no fallback.

## Commands

```sh
cmake --preset native-debug && cmake --build --preset native-debug   # system Qt
ctest --preset native-debug                                          # all tests
build/native-debug/bin/tst_inputbridge clickUsesGlCoordinates        # one QtTest function
build/native-debug/bin/frame-assistant --desktop       # UI preview in a window, no SteamVR
build/native-debug/bin/frame-assistant --snapshot x.png  # one frame via the real offscreen path
scripts/container-build.sh cross    # x86_64 → aarch64 against the Frame's system Qt (default)
scripts/container-build.sh native   # same for the host arch (use on the Frame)
scripts/container-build.sh cross-bundle   # fallback: self-contained, Qt bundled (also native-bundle)
scripts/container-build.sh cross shell    # debug inside the cross container
scripts/container-test.sh           # all tests in Debian incl. Vulkan + validation layers
VK_DRIVER=lvp scripts/container-test.sh  # same on lavapipe (software Vulkan)
build/native-debug/bin/frame-assistant --graphics-api vulkan --benchmark 600  # frame ms incl. GPU
```

- Use `--snapshot` and then read the PNG to check UI or rendering changes without a headset.
- `tst_render` runs once per backend (`VRDASH_TEST_API`, separate CTest entries, because the
  Qt graphics API is process-global). It QSKIPs without a GPU context or when Vulkan isn't
  compiled in.
- `tst_inputbridge` runs with `QT_QPA_PLATFORM=offscreen`.
- The Vulkan backend is optional (`VRDASH_VULKAN=AUTO|ON|OFF`). It needs the Vulkan headers
  and a Qt built with Vulkan. Hosts without headers build OpenGL only.
- **Any Vulkan change: run `scripts/container-test.sh`.** It must report "No Vulkan validation
  messages". `VRDASH_VULKAN_VALIDATION=1` enables the layers in any run.
- Cross bundles can be smoke-tested on x86 with `qemu-aarch64` plus Xvfb, inside a Debian
  container that has the arm64 X11/GL runtime libraries and *no* Qt.

## Code style

- **C++ follows the Google C++ style**, enforced by `.clang-format` (`BasedOnStyle: Google`):
  2-space indent, 80 columns, opening braces on the same line everywhere (functions too),
  `int* p`, sorted include blocks.
  - Run `clang-format -i $(find src tests -name '*.cpp' -o -name '*.h')`.
    `clang-format --dry-run --Werror …` must be clean.
  - Lines may exceed 80 columns only for user-visible strings (log/error messages).
  - Never let clang-format split a trailing comment: put the comment on its own line above.
  - `src/AppConfig.h.in` is formatted by hand.
- Naming stays Qt style, not Google naming (CamelCase classes, camelCase methods, `m_`
  members, `k` constants), so overrides, signals and QML-visible names match Qt's APIs.
- QML, CMake and shell scripts keep their usual conventions (QML: 4 spaces).

## Architecture

The data flow is QML → off-screen texture → SteamVR overlay. SteamVR overlay events flow back
into Qt events.

- `src/app/DashboardApplication` is the **base class of every app** (in `vrdashboard_core`):
  CLI, logging, SteamVR connection, backend selection, surfaces, controller navigation, and the
  modes (overlay, `--desktop`, `--snapshot`, `--benchmark`, manifest install).
  - **Inheritance for the app, components for the surfaces** (the user's decision, after
    trying composition): the app derives from `DashboardApplication`. In its constructor it
    configures (`addSurface`, optional `setAppKey`/`setDisplayName`) and it
    overrides the hooks `loading()` (backends + shared engine exist, QML not loaded) and
    `started()` (everything runs). In the hooks, protected accessors give the running state:
    `mode()`, `appKey()`, `appName()`, `engine()`, `surfaces()`, `surface()` (primary),
    `surface(id)`, `vrRuntime()`, `navigation()`. The app is `src/Dashboard.h/.cpp`.
  - The base constructor creates the `QGuiApplication` (`QCoreApplication` for manifest
    commands), so derived members can be QObjects with timers.
  - All QML and surfaces are torn down **at the end of `exec()`**, while the derived object
    and its members (e.g. QML singletons) still exist.
- `src/app/Surface` is **one view**: its QML root (from the app's module), renderer, render
  backend and overlay. All surfaces share one `QQmlEngine`.
  - It's a component: configured via `SurfaceConfig` (id, qmlRootType, displayName,
    textureSize, thumbnail), with signals `visibilityChanged`/`activated`. Never subclassed.
  - Overlay key = `<app key>.<id>`, or the app key itself for an empty id. The app key (manifest,
    `IdentifyApplication`) is one per process; `--app-key`/`--app-name` override it.
  - Backends: the first surface decides the API (incl. the Vulkan→OpenGL fallback); the
    others use the same API, because `QQuickWindow::setGraphicsApi` is process-wide.
  - Controller navigation sends keys to the *active* surface: the one last opened or pointed
    at (`Surface::activated`), if visible; else any visible one.
  - `vr/Overlay` is the overlay base (input, visibility, frames). `DashboardOverlay` is the
    only kind so far; a `WorldOverlay` would be another subclass (see "Planned extension").
  - `vrdash_add_dashboard()` (`cmake/Dashboard.cmake`) defines a dashboard executable:
    QML module, bytecode-only QML, OpenVR lib, install, registration for Qt bundling.
    The QML module's URI (`QML_URI`) is written down only there: a generated source file
    registers it as the app's default module (`DashboardApplication::setDefaultQmlModule`
    at static-init time). `setQmlModule()` exists only to load from another module.

- `render/RenderBackend.h` is the **only** place graphics-API code may live:
  - context/device setup, texture allocation, layout handling, `vr::Texture_t` + UV bounds
    for `SetOverlayTexture`, and CPU readback.
  - The header documents the call order (initialize → configureWindow → attachWindow →
    createRenderTarget → per frame beginFrame/finishFrame → detachWindow).
  - `createRenderBackend()` is the factory.
  - `QQuickWindow::setGraphicsApi()` must run before any `QQuickWindow` exists (see
    `startBackend()` in `DashboardApplication.cpp`).
  - In overlay mode the backend is initialized *after* `VR_Init`, because Vulkan needs
    `VrRuntime::renderInterop()`: the instance/device extensions SteamVR requires and the HMD's
    `VkPhysicalDevice`.
- `VulkanRenderBackend`:
  - Qt Quick creates the `VkDevice`; the backend owns the `VkImage`.
  - OpenVR requires `TRANSFER_SRC_OPTIMAL` plus `TRANSFER_SRC|SAMPLED` usage. Qt leaves the
    target in `COLOR_ATTACHMENT_OPTIMAL`, so `finishFrame()`/`beginFrame()` transition between
    the two with small synchronous submits. QQuickRenderControl offscreen frames already wait
    for the GPU.
  - Everything created on Qt's device must be freed in `detachWindow()`, before
    QQuickRenderControl destroys the device.
  - The instance requests API version ≤ 1.3. QVulkanInstance's 1.0 default made Qt itself
    trigger validation errors.
- `render/OffscreenQuickRenderer` is API-agnostic:
  - Drives `QQuickRenderControl` and renders on demand (on scene change or animation),
    capped at `--max-fps`.
  - Pacing uses an absolute schedule from frame starts. Waiting a fixed interval *after*
    each frame cost about 10% on the Frame and about 50% under load;
    `tst_render pacingHoldsFrameRateWithSlowFrames` guards this.
  - Each frame goes to a `FrameSink` callback **while the context is current**, which OpenGL
    `SetOverlayTexture` requires. `Surface` sets it to its overlay.
- `vr/VrRuntime` handles `VR_Init(VRApplication_Overlay)` and polls events on a Qt timer. It
  routes per-overlay events to handlers registered by overlay handle.
  - **`VR_Shutdown` must run before the graphics device is destroyed** (openvr.h: "Be sure to
    call OpenVR_Shutdown before destroying these resources").
  - Order in `runOverlay()` (`DashboardApplication.cpp`): the surfaces (renderers, backends)
    are members and die after it returns; `VrRuntime` is a local, so it shuts down first; an
    `OverlayGuard` declared after it destroys the overlays before that. Otherwise SteamVR's
    client frees its query pool on a dead VkDevice at exit, which crashed on the Frame.
- `vr/DashboardOverlay` is `CreateDashboardOverlay` plus the tab thumbnail. **Rendering follows
  `VREvent_OverlayShown/Hidden`, not the dashboard's open/closed state.** Users can float or pin
  dashboard overlays so they stay visible in games; tying rendering to the dashboard would
  freeze them there.
- `input/OverlayInputBridge` turns OpenVR overlay events into Qt events, so QML never knows
  about VR:
  - Mouse coordinates arrive in **GL convention (y from the bottom)** and must be flipped.
  - Smooth scroll becomes `QWheelEvent` (with a sub-pixel accumulator).
  - When a text item gets active focus, it opens the SteamVR keyboard
    (`KeyboardFlag_Minimal`: characters stream in; `\b` and `\n` mean Backspace and Return).
  - The offscreen window gets an explicit `FocusIn` event (in `OffscreenQuickRenderer`),
    otherwise `activeFocusItem` stays null.
- `input/ControllerNavigation` turns SteamVR Input actions (`resources/input/actions.json`,
  default bindings for `frame_controller`) into Tab/Backtab/arrow/Space/Escape key events.
  Qt Quick's own focus chain does the rest; no QML glue is needed.
- **QML singletons owned by the app:** keep them as members of the app class. A singleton
  must not be default-constructible, or QML builds its own instance instead of calling
  `create()` (e.g. give the constructor a required `QObject* parent` and init it `{nullptr}`).
- Publish per-frame values (frame stats etc.) to QML at most about once per second. Binding
  per-frame values in QML re-triggers the on-demand renderer forever.
- **The executable is `frame-assistant` (`VRDASH_EXECUTABLE_NAME`), never `vrdashboard`.**
  SteamVR identifies processes by executable name. With `vrdashboard`, the Frame's SteamVR
  treated our process as *its own dashboard* (watchdog, "VRDashboard process failed to start
  10 times"). CMake rejects names of SteamVR processes.
- The version is in `project()` in the root `CMakeLists.txt` (currently 0.1.0); release
  packages are named after it.
- App identity (app key, name, executable name, texture size) lives in CMake cache variables in the root
  `CMakeLists.txt` and is generated into `AppConfig.h`.

## Planned extension: world overlay (not implemented)

Several surfaces per process exist (tested by `tests/tst_surfaces`). Still to come, for a
notification app among others: a **`WorldOverlay`** (subclass of `vr/Overlay`, created via
`CreateOverlay`) with configurable placement and optional interactivity. Placement modes:
room, HUD, hand, and "spawn in front then stay". Interactivity uses
`VROverlayFlags_MakeOverlaysInteractiveIfVisible`, which takes the system laser while
visible. `SurfaceConfig` would get a kind (dashboard/world) plus placement; the app shows and
hides world surfaces itself. Keep the teardown rule: overlays, then SteamVR, then backends.

## Build layout and deployment

- The build tree mirrors the install layout (`bin/`, `lib/`, `share/frameassistant/`). The binary
  finds its data at `<exe>/../share/frameassistant`, and `libopenvr_api.so` through
  RPATH `$ORIGIN/../lib`.
- `manifest.vrmanifest` uses paths relative to itself, so `--install-manifest` works from both
  the build tree and an install.
- `cmake/OpenVR.cmake` downloads only `openvr.h`, `LICENSE` and the matching
  `bin/linux64|linuxarm64/libopenvr_api.so` into `third_party/openvr-<ver>/` (gitignored).
- **Deploying to the Frame uses its system Qt by default.** Bundling Qt is the fallback for
  when a SteamOS update ships a Qt that no longer fits.
- `VRDASH_DEPLOY_QT=ON` (presets `native-bundle` and `cross-arm64-bundle`) runs `cmake/BundleQt.cmake`
  and `bundle-install.cmake.in` at install time:
  - `qmlimportscanner` collects the QML modules.
  - `file(GET_RUNTIME_DEPENDENCIES)` with the target's objdump collects the libraries.
  - `patchelf` sets relative RPATHs, and the script writes `bin/qt.conf`.
  - Qt's own `qt_generate_deploy_*` is deliberately not used: it skips cross-compiled Linux
    targets, and with a `/usr` Qt it would bundle glibc.
- **`VRDASH_BUNDLE_EXCLUDE` must keep excluding the GPU/display stack** (libGL/EGL/GLX,
  libdrm/gbm, Vulkan, Wayland, X11/xcb, libffi, glibc, libstdc++). These must come from SteamOS,
  or GPU access breaks.
- **Release packages** (`scripts/package-release.sh [--bundle-qt]`) contain only the
  app, `install.sh` (install/update: stops a running instance, registers,
  launches), `uninstall.sh` and `README.txt`. `deploy-to-frame.sh` rsyncs the newest one to
  the Frame and runs its `install.sh`.
- **Licenses:** MIT (`LICENSE`); `THIRD_PARTY_NOTICES.md` and OpenVR's license are installed to
  `share/doc/frameassistant/`. Bundled builds also get the Debian copyright files of every bundled
  library plus the referenced common license texts (collected via `dpkg -S` in
  `bundle-install.cmake.in`, after resolving paths: trixie registers `/usr/lib` paths).
- **Qt bundling runs once for all dashboards** (`vrdash_bundle_qt()` at the end of the root
  `CMakeLists.txt`). Bundling per executable made the second run see each library twice.
- **System-Qt builds** (the default; presets `cross-arm64` and `native-systemqt`,
  `container-build.sh cross|native`): no Qt bundle, so the package is about 3 MB instead of
  46 MB.
  - `VRDASH_QML_BYTECODE_ONLY=ON` keeps qmlcachegen's version-locked C++ out of the binary.
  - The binary then needs only stable `Qt_6` symbols, `qt_version_tag@Qt_6.8` (Qt ≥ 6.8),
    and Qt's Vulkan function wrappers. Those are tagged `Qt_6_PRIVATE_API` but generated from
    the Vulkan spec.
  - Verified: the Debian-built binary (Qt 6.8.2) runs on Arch's Qt 6.8.0 (Arch Linux Archive
    snapshot 2024-10-20, image `archlinux:base-20241020.0.271562`), with both backends.
  - Check with `objdump -T bin/frame-assistant | grep Qt_6_PRIVATE_API`: only `QVulkan*` may appear.
- **`-mno-direct-extern-access` is added whenever the compiler supports it** (root
  `CMakeLists.txt`).
  - Arch/SteamOS build Qt with protected symbols, so consumers must not use copy relocations.
    Debian's Qt doesn't pass that flag on.
  - Without it, an x86-64 Debian build fails against Arch's Qt
    (`GNU_PROPERTY_1_NEEDED_INDIRECT_EXTERN_ACCESS`). aarch64 GCC has no such flag, and arm64
    PIE never uses copy relocations.
- The Frame's `/etc/pacman.conf` points to private Valve repositories (URLs marked
  "DO_NOT_SHARE"). Never put those URLs into this repo or into scripts.
- Cross builds use Debian multiarch:
  - host (amd64) Qt tools via `QT_HOST_PATH=/usr`, target Qt from `/usr/lib/aarch64-linux-gnu`.
  - Cross presets turn tests off.

## Verified on the Steam Frame

- Over SSH, `run.sh opengl` finds `DISPLAY=:0`, `VR_Init` succeeds, and the bundled Qt starts
  (2026-10-04, SteamOS holo 0.4.3, system Qt 6.8.0).
- **OpenGL on the Frame runs through zink (GL on Vulkan) on Turnip/Adreno 750.** There is no
  native GL driver, so the Vulkan backend avoids a translation layer there.
- Both backends work in SteamVR: display orientation (no texture flip needed for either
  backend), laser, scroll direction, keyboard and D-pad navigation (SteamVR Input actions
  arrive while the dashboard is open) are all OK.
- Floating works: a floated/pinned dashboard tab keeps rendering in games and stays
  interactive.
- Started by SteamVR itself (`install.sh --launch`, auto-start), the app finds a working
  display connection; no `QT_QPA_PLATFORM` override is needed.
- **The system-Qt package works on the Frame** (2026-10-04): the Debian-built binary (Qt 6.8.2)
  runs on SteamOS's Qt 6.8.0 with both backends. The package contains no Qt, so it can only
  have used the system Qt.
- Vulkan now exits cleanly via "Quit app" (the OpenVR shutdown-order fix is confirmed).
- **Install/launch works end to end** (2026-10-04): `install.sh` registers both test overlays,
  SteamVR knows both keys, `--launch` starts them, they auto-start again after a reboot, and
  `uninstall.sh` unregisters them.
- **SteamVR on the Frame reads the program path from `binary_path_linux_arm`** (its own
  manifests in `/opt/steamvr` use it). Entries with only `binary_path_linux` are skipped
  ("must specify binary_path for launch_type binary"), and the key then counts as unknown.
  `resources/CMakeLists.txt` picks the key from the target architecture.
- **SteamVR on the Frame refuses `VRApplication_Utility`.** `VR_Init()` always queries
  `IVRSystem`, which fails with `VRInitError_Init_NotAvailableToUtilityApps` (123). Manifest
  commands therefore use `VRApplication_Background`, which needs a running SteamVR and never
  starts it. Don't switch back to Utility.
- `benchmark.sh` on the Frame (600 frames, 1600×1000, incl. GPU wait): **Vulkan 1.01 ms avg
  (982 fps), OpenGL/zink 1.89 ms avg (528 fps)**, so Vulkan is about 1.9× faster.
- After the pacing fix both run at 60 fps in VR. The (since removed) diagnostics example's frame time (Vulkan about
  2.4 ms, GL about 1.2 ms) isn't comparable, because GL leaves out its asynchronous GPU work.
- Before the pacing fix, Vulkan showed 55 fps and GL 60. Cause: the render time was added
  to the frame interval, and GL's frame time leaves out its asynchronous GPU work.

## Not yet verified on a Steam Frame

Check these when testing on the device, and update this list:

- `scripts/deploy-to-frame.sh` and the release package (`install.sh` install/update,
  `uninstall.sh`).
- The restructured app code (base class `DashboardApplication`, `Surface`, `Overlay` base,
  active-surface navigation) has only been tested off the device so far.
- Several dashboard tabs from one process (works in `tst_surfaces`, not tried in SteamVR).