FrameAssistant/docs/ARCHITECTURE.md

# Architecture

A dashboard is **one process**. It renders QML scenes off-screen into textures and hands
them to SteamVR as overlays. Input flows the other way: SteamVR reports laser, scrolling,
keyboard and controller actions, and the app turns them into ordinary Qt events. The QML
never knows it is in VR, which is why it also runs in a plain desktop window (`--desktop`,
`--desktop-novr`).

## Structure

```mermaid
flowchart LR
    app["<b>Your app</b><br/>class Dashboard : DashboardApplication<br/>members, loading() / started()"]
    subgraph base["DashboardApplication"]
        direction TB
        engine["QQmlEngine<br/>(shared by all surfaces)"]
        runtime["VrRuntime<br/>SteamVR connection"]
        nav["ControllerNavigation"]
    end
    subgraph surfaces["Surfaces: one per view"]
        direction TB
        s1["<b>Surface 'main'</b><br/>QML root → Renderer → RenderBackend<br/>DashboardOverlay + InputBridge"]
        s2["<b>Surface 'settings'</b> (optional)<br/>same parts"]
    end
    steamvr(["SteamVR"])
    app -- "inherits" --> base
    base -- "creates, drives" --> surfaces
    surfaces -- "textures" --> steamvr
    steamvr -- "laser, scroll, keyboard" --> surfaces
    steamvr -- "controller actions" --> nav
    nav -- "keys to the active surface" --> surfaces
```

**`DashboardApplication`** (`src/app/DashboardApplication.*`) is the base class of every app.
It owns everything that isn't app-specific: command line, logging, the SteamVR connection,
choosing the render backend, the surfaces, controller navigation, the auxiliary modes, and
the order in which all of that is created and destroyed.

Your app derives from it:
- **Constructor:** configuration (`addSurface()`, optionally `setAppKey()`,
  `setDisplayName()`, `setQuickStyle()`, `setQmlModule()`). The app's own command-line
  options go to the base constructor as a list of `QCommandLineOption`s; they are parsed and
  listed in `--help` with the built-in ones, and read with `isOptionSet()`/`optionValue()`.
  Its commands (flags such as `--ha-remove`) go there as a second list: like the manifest
  commands they get a `QCoreApplication` (no display needed, works over SSH), and `exec()`
  runs `prepare()` and then `runCommand(name)` in mode `Command` instead of a mode.
- **Hooks:**
  - `prepare(QString* error)`: first, once mode and app key are known, before the QML
    engine, the surfaces and SteamVR exist (not for manifest commands). Returning `false`
    ends `exec()` with the error and exit code 1, e.g. for invalid settings.
  - `loading()`: the shared QML engine, the surfaces and their render backends exist; QML
    isn't loaded yet. Set up what the QML needs here.
  - `started()`: everything runs. Connect to surfaces and input here.
  - `runCommand(name)`: one of the app's commands, after `prepare()`; no QML engine,
    surfaces or SteamVR. Its result is the exit code.
- **Accessors:** `mode()`, `appKey()`, `appName()`, `isOptionSet()`, `optionValue()`,
  `engine()`, `surfaces()`, `surface()` (the first), `surface(id)`, `vrRuntime()`,
  `navigation()`.
- **Lifetime:** the base constructor creates the `QGuiApplication` (`QCoreApplication` for
  commands), so your members can be
  QObjects with timers. All QML and surfaces are torn down at the end of `exec()`, while your
  object and its members still exist.

**`Surface`** (`src/app/Surface.*`) is one view: a QML root, its renderer, its render backend
and its overlay. It's a component you configure (`SurfaceConfig`: id, QML root type, tab name,
texture size, icon) and connect to (`visibilityChanged`, `activated`), never subclass.
- All surfaces share one QML engine, so models and singletons are visible in every view.
- Overlay key: `<app key>.<id>`, or the app key itself for an empty id.
- A surface renders only while its overlay is visible.

**Render layer** (`src/render/`):
- **`OffscreenQuickRenderer`** drives Qt Quick through `QQuickRenderControl`, without a window.
  - It renders on demand (scene change or animation), capped at `--max-fps`, on an absolute
    schedule so slow frames don't lower the frame rate.
- **`RenderBackend`** is the *only* place with graphics-API code: device setup, texture,
  hand-off description for SteamVR, readback.
  - **`VulkanRenderBackend`** (default): Qt creates the device on the GPU SteamVR names, with
    the extensions SteamVR requires. The backend owns the image and moves it to
    `TRANSFER_SRC_OPTIMAL` after each frame, the layout OpenVR wants, and back before the next.
  - **`OpenGLRenderBackend`** (fallback): on the Frame it runs through zink on Vulkan.

**VR layer** (`src/vr/`):
- **`VrRuntime`**: `VR_Init` as overlay app, polls SteamVR events and routes them by overlay
  handle. It also reports SteamVR's Vulkan requirements, and registers and launches manifests.
- **`Overlay`**: base of all overlays. It handles input routing, visibility and frame
  submission.
  - **`DashboardOverlay`** is the dashboard tab with its icon. A world overlay would be
    another subclass (planned).

**Input layer** (`src/input/`):
- **`OverlayInputBridge`** (one per overlay):
  - **Laser:** turned into mouse events, with the y-axis flipped (OpenVR counts from the
    bottom).
  - **Scrolling:** thumbstick scrolling becomes wheel events.
  - **Keyboard:** opens the SteamVR keyboard when a text field gets focus, and turns typed
    characters into key events.
- **`ControllerNavigation`** (one per app): SteamVR Input actions (`resources/input/`, rebindable
  in SteamVR) become Tab/arrow/Space/Escape for the *active* surface, i.e. the one last opened
  or pointed at. Qt Quick's own focus handling does the rest.

## One frame

```mermaid
sequenceDiagram
    participant QML as QML scene
    participant R as OffscreenQuickRenderer
    participant B as RenderBackend (Vulkan)
    participant O as Overlay
    participant S as SteamVR
    QML->>R: scene changed / animation tick
    Note over R: wait for the next slot (≤ max-fps)
    R->>B: beginFrame (image back to COLOR_ATTACHMENT)
    R->>R: polish, sync, render (Qt Quick)
    R->>B: finishFrame (image → TRANSFER_SRC, GPU done)
    R->>O: frame sink: texture + bounds
    O->>S: SetOverlayTexture (SteamVR copies it)
```

## Input

```mermaid
sequenceDiagram
    participant S as SteamVR
    participant V as VrRuntime
    participant I as OverlayInputBridge
    participant Q as QML (QQuickWindow)
    S->>V: overlay event (mouse, scroll, keyboard)
    V->>I: routed by overlay handle
    I->>Q: QMouseEvent / QWheelEvent / QKeyEvent
    Q-->>I: text field got focus
    I->>S: ShowKeyboardForOverlay
```

## Start and shutdown (overlay mode)

1. **Start:**
   1. Call `prepare()`; stop with exit code 1 if it fails.
   2. Connect to SteamVR (`VR_Init`).
   3. Create one render backend per surface. The first surface decides the API, including a
      fallback to OpenGL; the others use the same API, because Qt's graphics API is
      process-wide.
   4. Call `loading()`.
   5. Load each surface's QML.
   6. Create the overlays.
   7. Set up controller navigation.
   8. Call `started()`, then run the event loop.
2. **Shutdown runs in reverse**, with one rule found on the Frame: **overlays first, then
   SteamVR, then the graphics devices.** OpenVR requires shutting down before the devices it
   got textures from are destroyed. Otherwise its client library frees objects on a dead
   `VkDevice` at exit, which crashed on the Frame. `runOverlay()` encodes this order through
   the declaration order of its objects.

## Building blocks in CMake

- **`vrdash_add_dashboard()`** (`cmake/Dashboard.cmake`) turns sources plus QML files into a
  dashboard executable. It covers:
  - the QML module, whose `QML_URI` becomes the app's default module through a generated
    source file
  - bytecode-only QML for system-Qt builds
  - `libopenvr_api.so` next to the executable
  - install rules and registration for Qt bundling
  - the check against SteamVR process names
- **`cmake/Mosquitto.cmake`** (app, not platform) finds libmosquitto through pkg-config. With
  `FRAMEASSISTANT_SHIP_MOSQUITTO=ON` (all package presets) it installs the library into `lib/`
  under its soname plus Debian's copyright file, and excludes it from the Qt bundler.
- **App identity** (root `CMakeLists.txt`): `project()` name and version, `VRDASH_APP_KEY`,
  `VRDASH_DISPLAY_NAME`, `VRDASH_EXECUTABLE_NAME`, texture size. CMake generates from them
  `AppConfig.h` and the SteamVR manifest, with `binary_path_linux_arm` for ARM builds and
  `binary_path_linux` for x86_64.

## Planned: world overlays

For e.g. a notification app: a `WorldOverlay` (another `Overlay` subclass, created with
`CreateOverlay`) that the app shows and hides itself.
- **Placement modes:** room, HUD, hand, "spawn in front, then stay".
- **Optionally operable:** while visible, it takes the system laser
  (`VROverlayFlags_MakeOverlaysInteractiveIfVisible`).

The surface structure is ready for it: `SurfaceConfig` would gain a kind and a placement.

## FrameAssistant on top

The app itself (`src/Dashboard.*`, `src/config/`, `src/ha/`, `src/qml/`) builds on this
platform; its design (Home Assistant connection, entities, overlays, threads) is in
[DESIGN.md](DESIGN.md).