FrameAssistant/docs/REQUIREMENTS.md

# Requirements

Status: **approved** 2026-10-05 (iteration 1, phase 1). Changes from now on are recorded as
decisions below.

## Goals

| ID | Goal |
|----|------|
| G1 | **Sensors:** report the state of the headset and its surroundings to Home Assistant (HA). |
| G2 | **Services:** offer actions on the headset that HA can call, e.g. notifications. |
| G3 | **Dashboard:** show HA's user interface inside VR. |

## Context and constraints

Known facts, not requirements:

- Target device: Steam Frame (aarch64, SteamOS, system Qt 6.8). Also runs on x86_64 Linux with
  SteamVR.
- Platform: SteamVR dashboard overlay (C++20, Qt 6 Quick, OpenVR), see
  [ARCHITECTURE.md](ARCHITECTURE.md) and [STEAM-FRAME.md](STEAM-FRAME.md).
- SteamVR starts the app as an overlay application; it runs while SteamVR runs.

## Requirements

Priority: **M**ust, **S**hould, **C**ould (confirmed by the user, D13).
**MVP** marks the scope of the first iteration (D8). "Device" means one headset running
FrameAssistant.

### Connection (CON)

| ID | Requirement | Prio | MVP | Acceptance criteria |
|----|-------------|------|-----|---------------------|
| CON-1 | Connect to the configured MQTT broker using an existing MQTT client library. Login and TLS are supported as far as that library supports them. | M | ✓ | With valid settings the device connects; connection state and errors are logged. |
| CON-2 | Publish all entities via HA MQTT discovery, grouped as one HA device per headset. | M | ✓ | Entities appear in HA without manual HA configuration. |
| CON-3 | Report availability: online while running, offline when stopped or crashed (last will). | M | ✓ | Normal exit or killed process → HA shows the device's entities as unavailable. |
| CON-4 | When the broker is unreachable, reconnect with backoff; after reconnecting, republish discovery and current state. Nothing is buffered. | M | ✓ | Broker restart → entities return to correct states without restarting the app. |
| CON-5 | Several headsets work side by side with one HA, each as its own device, identified by its configured device name. | M | ✓ | Two devices with different names → two HA devices, no shared entities. |

### Configuration (CFG)

| ID | Requirement | Prio | MVP | Acceptance criteria |
|----|-------------|------|-----|---------------------|
| CFG-1 | Settings are read from a JSON file in `~/.config/frameassistant/`. No settings UI. | M | ✓ | Changing the file and restarting changes the behaviour. |
| CFG-2 | The device name is required. If it (or another required setting) is missing or the file is invalid, the app exits with an error message in the log file and on stderr and a non-zero exit code. | M | ✓ | Missing name → no MQTT connection, clear error in log and stderr, exit code ≠ 0. |
| CFG-3 | Broker connection settings (host, port, credentials, TLS options as supported by the library) are configurable. | M | ✓ | Device connects to a broker given only in the file. |

### Sensors (SEN)

| ID | Requirement | Prio | MVP | Acceptance criteria |
|----|-------------|------|-----|---------------------|
| SEN-1 | Presence: headset worn / user active. | M | ✓ | Putting the headset on/off changes the state in HA. |
| SEN-2 | Standby: headset in standby. | M | ✓ | Standby/wake changes the state in HA. |
| SEN-3 | Headset battery level and charging state. | M | ✓ | Values match the headset's own display; plugging in toggles charging. |
| SEN-4 | Per controller: connected, battery level, charging state. | M | ✓ | Turning a controller off/on and its battery are reflected in HA. |
| SEN-5 | Current VR app/game (name; none when idle). | M | ✓ | Starting/quitting a game updates the sensor. |
| SEN-6 | SteamVR dashboard open. | M | ✓ | Opening/closing the dashboard toggles the state. |
| SEN-7 | Tracking state of headset and controllers. | S | | Covering the cameras / losing tracking is reflected. |
| SEN-8 | Play area: size and seated/standing. | S | | Values match SteamVR's room setup. |
| SEN-9 | Near boundary (boundary visible). | S | | Walking to the boundary toggles the state. |
| SEN-10 | Passthrough (room view) shown. | S | | Toggling passthrough toggles the state. |
| SEN-11 | Session duration since the headset was put on. | S | | Resets on put-on; increases while worn. |
| SEN-12 | Network: SSID, signal strength, IP address. | S | | Values match the system's network settings. |
| SEN-13 | Audio: output device, volume, mute. | S | | Changing volume/mute in SteamOS updates HA. |
| SEN-14 | Sensors publish on change, throttled per sensor (minimum interval). | M | ✓ | A rapidly changing value produces at most one update per interval. |
| SEN-15 | The set of sensors is fixed: every device always offers the same entities. A sensor whose data the device doesn't support or currently has no value for is shown as **unavailable** in HA. | M | ✓ | On a device without a value (e.g. no battery reported), the entity exists and is unavailable; it becomes available once a value exists. |

### Notifications (NOT)

| ID | Requirement | Prio | MVP | Acceptance criteria |
|----|-------------|------|-----|---------------------|
| NOT-1 | HA can show a notification with title and text that hides after a timeout. | M | ✓ | Sent from HA → visible in VR (also during a game) → disappears after the timeout. |
| NOT-2 | Placement configurable: head-locked, spawn in front and stay, on the hand. | S | | Each placement works as described. |
| NOT-3 | Priority levels with a distinct look (colour/icon). | S | | Each level is visually distinguishable. |
| NOT-4 | Sounds: built-in sound per priority; HA may give a sound URL instead. | S | | Default sound plays per level; a given URL plays instead. |
| NOT-5 | Action buttons; the chosen action is sent back to HA. | S | | Choosing an action triggers an HA automation. |
| NOT-6 | A notification is confirmed by: user dismisses it, user picks an action, timeout (non-critical only), or HA clears it. | S | | Each of the four paths removes it from the list. |
| NOT-7 | Unconfirmed notifications are listed in the main dashboard tab. | S | | A critical notification not yet acknowledged appears in the list. |
| NOT-8 | Critical notifications bypass do-not-disturb, stay until acknowledged, play a sound and vibrate the controllers. | S | | All four behaviours observable. |
| NOT-9 | Behaviour for several notifications at once (Q1). | S | | Defined after design exploration. |
| NOT-10 | Report to HA as an event how each notification was confirmed: dismissed, timed out or cleared (picked actions are covered by NOT-5). Only if easy to implement. | C | | Each confirmation path produces an HA event naming the notification and the reason. |

### Other services (SRV)

| ID | Requirement | Prio | MVP | Acceptance criteria |
|----|-------------|------|-----|---------------------|
| SRV-1 | Attention signals: vibrate controllers, fade the view to a colour, show the boundary. | S | | Each triggerable from HA. |
| SRV-2 | Launch a VR app by app key; quit the current app. | S | | Both work from HA. |
| SRV-3 | System control: volume, mute, screenshot, standby. | S | | Each works from HA (where SteamOS/SteamVR allows it). |
| SRV-4 | Open the SteamVR dashboard on FrameAssistant's tab, optionally on a given HA view. | S | | Dashboard opens on the tab. |
| SRV-5 | Do-not-disturb switch, controllable from HA and in VR; suppresses non-critical notifications. | S | | Toggle in either place is reflected in the other. |
| SRV-6 | Status cards (e.g. progress, countdown, door open), set, updated and cleared by HA, in their own small dashboard tab that exists only while at least one card exists (D9). | S | | First card adds the tab; clearing the last removes it; the tab can be floated. |
| SRV-7 | The status tab returns to its previous floating position when it reappears. | C | | Floated tab reappears at the old position. |
| SRV-8 | Play audio (sound or HA TTS) from a URL on the headset. | S | | HA TTS message is heard in the headset. |

### Dashboard (DSH)

| ID | Requirement | Prio | MVP | Acceptance criteria |
|----|-------------|------|-----|---------------------|
| DSH-1 | A dashboard tab to control HA entities in VR (approach: Q2, Q3). | S | | Defined after design exploration. |

### Non-functional (NFR)

| ID | Requirement | Prio | MVP | Acceptance criteria |
|----|-------------|------|-----|---------------------|
| NFR-1 | Runs on the Steam Frame and on x86_64 Linux with SteamVR. | M | ✓ | Release package works on the Frame. |
| NFR-2 | Runs only while SteamVR runs (started by SteamVR). | M | ✓ | Starts with SteamVR, ends with it. |
| NFR-3 | No measurable impact on VR performance: idle when nothing changes, no rendering while tabs are hidden, low CPU for sensors. | M | ✓ | Game frame rate/timing unchanged with FrameAssistant running (measurement method in design). |
| NFR-4 | UI in English and German, following the system language. | S | | Both languages complete. |
| NFR-5 | Home LAN only; no remote access features. | M | ✓ | — |
| NFR-6 | HA side needs only the MQTT integration; nothing custom installed in HA. | M | ✓ | Works on a stock HA with MQTT. |
| NFR-7 | Targets the current HA release. Only current HA features are used, nothing deprecated or legacy; older HA versions are not supported. | M | ✓ | Works with the current HA release; HA logs no deprecation warnings for FrameAssistant's entities. |
| NFR-8 | Unit tests for logic that runs without headset or broker; a manual on-device checklist per iteration. The test strategy is adjusted as the project progresses. | M | ✓ | Tests pass; checklist done and documented. |
| NFR-9 | No latency requirement for notifications. | — | | — |

### Out of scope

- Images/camera snapshots in notifications.
- A select entity listing installed apps.
- Sensors for mic in use and screenshots.
- Running without SteamVR; remote access; a settings UI.
- Publishing (private for now).

## Open questions

| # | Question | Status |
|---|----------|--------|
| Q1 | Several notifications at once: stack, queue, replace by tag? | open, design exploration |
| Q2 | Dashboard: HA's own web UI vs. native VR UI? | open, design exploration (Qt WebEngine exists on the Frame, Appendix C) |
| Q3 | Dashboard data: HA WebSocket API (token) in addition to MQTT, or MQTT only? | open, design |
| Q4 | Which MQTT client library? Licence, availability on SteamOS, TLS/login support. | answered: libmosquitto, shipped in the package (D14, Appendix B) |
| Q5 | Status card: own tab or part of the main tab? | answered: own tab, on demand (D9) |
| Q6 | MVP sensor list? | answered (MVP column) |
| Q7 | Minimum supported HA version? | answered: current release, no legacy features (D10) |
| Q8 | Sensors the Frame doesn't provide: don't create the entity, or show it as unavailable? | answered: fixed set, unavailable (D11) |
| Q9 | Should HA be told when a notification is dismissed or times out? | answered: yes, as an event, if easy (D12) |

## Decisions

| # | Date | Decision | Reason |
|---|------|----------|--------|
| D1 | 2026-10-05 | Connect to HA via **MQTT with HA MQTT discovery** | Any entity type, commands possible; same approach as HASS.Agent |
| D2 | 2026-10-05 | Use the user's **existing MQTT broker** | Already runs with HA |
| D3 | 2026-10-05 | **Home LAN only**; no remote access | User's use case |
| D4 | 2026-10-05 | **Several headsets, one HA**: each headset is its own HA device | User's use case |
| D5 | 2026-10-05 | Settings in JSON at `~/.config/frameassistant/`, device name required | All Frames share one default hostname |
| D6 | 2026-10-05 | Run only while SteamVR runs | Simplicity |
| D7 | 2026-10-05 | Use an existing MQTT library, support what it supports | Don't reimplement MQTT/TLS |
| D8 | 2026-10-05 | MVP = MQTT + sensors + basic notification | User's priority |
| D9 | 2026-10-05 | Status cards get their own dashboard tab, present only while cards exist | Float only the small panel; no empty tab |
| D10 | 2026-10-05 | Target the current HA release; no deprecated/legacy features | Avoid legacy baggage |
| D11 | 2026-10-05 | Fixed sensor set; unsupported or missing values are "unavailable" | Same entities on every device, predictable in HA |
| D12 | 2026-10-05 | Notification confirmations are reported to HA as events, if easy | Cheap to add, useful for automations |
| D13 | 2026-10-05 | Requirements approved as drafted, priorities included (MVP = Must, other wanted features = Should, SRV-7 and NOT-10 = Could) | End of phase 1 |
| D14 | 2026-10-05 | MQTT client: **libmosquitto** from Debian's package in the build container, shipped in the release package like `libopenvr_api.so` | Not on the Frame; Qt MQTT would need the exact Qt version (Appendix B) |
| D15 | 2026-10-05 | HA MQTT discovery: **own small layer on Qt's JSON classes**, using device-based discovery (one `<prefix>/device/<id>/config` topic with all entities under `cmps`) | No suitable library (Appendix B); HA recommends device-based discovery for multi-entity devices (NFR-7) |

## Appendix A: data sources

What the headset can report, by source. Checked against `openvr.h` (OpenVR 2.15.6). **None of
it is verified on the Frame yet**: SteamVR on the Frame may not fill every property.

### OpenVR vs. OpenXR

- **OpenVR** (what we use) gives system-wide state: all tracked devices, batteries, the running
  app, the dashboard, the play area, standby. It also runs as a background/overlay app next to
  games.
- **OpenXR** is an API for *the one app* that renders the scene. It has no access to other
  apps, device batteries or the dashboard, and SteamVR's overlay/dashboard features exist only
  in OpenVR. **Not useful for this project.**

### OpenVR

| Data | API | Smart-home use |
|------|-----|----------------|
| Worn / user active | `GetTrackedDeviceActivityLevel` (UserInteraction, Idle, Standby), events `TrackedDeviceUserInteractionStarted/Ended` | Presence ("in VR"), lights, do-not-disturb |
| Standby | events `EnterStandbyMode` / `LeaveStandbyMode` | Headset put away |
| Battery, charging (headset, controllers, trackers) | `Prop_DeviceBatteryPercentage_Float`, `Prop_DeviceIsCharging_Bool`, `Prop_DeviceProvidesBatteryStatus_Bool` | Low-battery alerts, charging reminders |
| Devices connected | events `TrackedDeviceActivated/Deactivated`, `Prop_ControllerRoleHint_Int32` | Controllers on/off |
| Tracking state | pose `eTrackingResult`, `bPoseIsValid` | Tracking lost |
| Running app / game | `GetCurrentSceneProcessId` → `GetApplicationKeyByProcessId` → app name; `GetSceneApplicationState`; event `SceneApplicationChanged` | "Playing X", automations per game |
| Dashboard open | `IsDashboardVisible`, events `DashboardActivated/Deactivated` | User is in menus |
| Passthrough (room view) | events `RoomViewShown/Hidden` | User can see the real room |
| Play area | `GetPlayAreaSize`, `GetPlayAreaRect`, `GetCalibrationState`, `GetTrackingSpace` (seated/standing) | Room-scale vs. seated |
| Near boundary | `AreBoundsVisible` (polled) | User close to walls/furniture |
| Screenshot taken | event `ScreenshotTaken` | HA event |
| Audio devices | `k_pch_audio_*` settings, event `AudioSettingsHaveChanged` | Which output is used (no volume) |

Actions OpenVR offers (input for the services round): fade the view to a colour (`FadeToColor`),
show the boundary (`ForceBoundsVisible`), vibrate controllers (`TriggerHapticPulse`), take a
screenshot (`RequestScreenshot`), launch apps (`IVRApplications`), notifications
(`IVRNotifications`; reportedly unreliable in current SteamVR, own overlay is the safer path).

### Linux system (outside OpenVR)

| Data | Source | Smart-home use |
|------|--------|----------------|
| Network: SSID, signal, IP | NetworkManager (D-Bus) or `/proc`, Qt `QNetworkInterface` | Which network, connection quality |
| Audio: volume, mute, mic in use | PipeWire/PulseAudio | "In a call" → do-not-disturb |
| Device battery (Frame is standalone) | `/sys/class/power_supply` | Fallback if OpenVR has none |

### Other candidates ("environment")

- **VR session duration** (time since put on): break reminders, statistics.
- **Mic in use**: user is talking (voice chat), so don't play announcements.
- **Streaming from a PC** vs. standalone (Steam Link), if detectable.

## Appendix B: MQTT client library (Q4)

Evaluated 2026-10-05 for D14.

| | libmosquitto (C) | Qt MQTT | Eclipse Paho C/C++ |
|---|---|---|---|
| Licence | EPL-2.0 / EDL-1.0 | GPL-3 or commercial (no LGPL) | EPL-2.0 / EDL-1.0 |
| On the Frame (SteamOS, checked) | no | no | no |
| Arch repos / Debian | `mosquitto` / `libmosquitto-dev` | `qt6-mqtt` / Qt add-on | AUR only / `libpaho-mqtt-dev` |
| Login, TLS | yes (OpenSSL) | yes (QSslSocket) | yes |

- **Qt MQTT is ruled out**, not because of its licence (the user accepts GPL-3), but because it
  uses Qt's private API: `QMqttClientPrivate` derives from `QObjectPrivate`. QtCore then requires
  the *exact* Qt version (`checkForIncompatibleLibraryVersion()` in `qobject_p.h` calls
  `qFatal("Cannot mix incompatible Qt library …")`). A Qt MQTT built against Debian's Qt 6.8.2
  aborts on the Frame's Qt 6.8.0 and would break with every SteamOS Qt update. It would only
  work with Qt bundled (about 46 MB instead of about 3 MB).
- **libmosquitto** does not depend on Qt. Debian trixie's `libmosquitto1` 2.0.21 (arm64)
  needs only `libssl.so.3`, `libcrypto.so.3` and `libc.so.6`, with symbol versions up to
  `OPENSSL_3.0.0` and `GLIBC_2.34`. The Frame has OpenSSL 3.2.1 and glibc 2.39 (checked
  2026-10-05), so it fits. Licence per Debian: EPL-2.0 or EDL-1.0 (`uthash.h`: BSD-1-Clause);
  the licence text must ship with the package.
- Paho: no advantage over libmosquitto; uses its own threads.
- Integration into the Qt event loop (`QSocketNotifier` on `mosquitto_socket()`) is part of the
  design.

### HA discovery libraries (D15)

No library fits; discovery itself is just JSON on a config topic.

- [hacpp](https://github.com/pblxptr/hacpp) (C++, Linux): rejected. Early stage ("everything
  can be changed soon"); only binary sensor, sensor, button, cover; buffers messages on
  reconnect (contradicts CON-4); own MQTT client (async_mqtt) on Boost.Asio ≥ 1.90 (not on the
  Frame), Conan 2, GCC 14+; its own event loop next to Qt's; TLS/login undocumented.
- [HAMqttDevice](https://github.com/plapointe6/HAMqttDevice),
  [KLib-HomeAssistantMqtt](https://github.com/KazaLab/KLib-HomeAssistantMqtt),
  [hamqtt](https://registry.platformio.org/libraries/dalib666/hamqtt): Arduino/embedded payload
  generators with per-component topics.
- homie-cpp: implements the Homie convention, not HA discovery.

## Appendix C: libraries per requirement

Checked on the Frame 2026-10-05 (SteamOS, system Qt 6.8.0). Rule: **a Qt add-on module must
come from the Frame's system Qt.** Shipping one fails like Qt MQTT (Appendix B): Qt's modules
use Qt's private API, which requires the exact Qt version.

| Need | Requirements | Candidate | On the Frame | New dependency |
|---|---|---|---|---|
| JSON | CFG-1, CON-2 | `QJsonDocument` (QtCore) | ✓ | no |
| HA discovery | CON-2/3/5, SEN-15 | own layer (D15) | — | no |
| MQTT | CON-1 | libmosquitto (D14) | ✗, shipped; its deps `libssl.so.3`, `libcrypto.so.3` ✓ (Appendix B) | yes |
| IP address | SEN-12 | `QNetworkInterface` (QtNetwork) | ✓ | no |
| SSID, signal | SEN-12 | NetworkManager via QtDBus | ✓ `libQt6DBus`, `nmcli` | no |
| Volume, mute, output | SEN-13, SRV-3 | PipeWire via libpulse, or `wpctl`/`pactl` | ✓ `libpulse.so.0`, `libpipewire-0.3.so.0`, `wpctl`, `pactl` | to decide in design |
| Sounds, audio from URL | NOT-4, SRV-8 | Qt Multimedia + QtNetwork | ✓ `libQt6Multimedia`, FFmpeg plugin | no |
| HTTPS (downloads, WebSocket) | SRV-8, Q3 | Qt TLS plugin | ✓ `libqopensslbackend.so` | no |
| Translations | NFR-4 | `QTranslator`, `lupdate`/`lrelease` | ✓ (build-time tools) | no |
| Dashboard, HA web UI | DSH-1, Q2 | Qt WebEngine (Quick) | ✓ `libQt6WebEngineCore`, `libQt6WebEngineQuick`, `libQt6WebView` | no; works with our off-screen renderer and Vulkan? Needs a spike |
| Dashboard, native UI | DSH-1, Q2/Q3 | HA WebSocket API via Qt WebSockets | ✓ `libQt6WebSockets` | no |
| Haptics, fade, apps | SRV-1/2 | OpenVR | ✓ | no |
| Tests | NFR-8 | QtTest | ✓ (host only) | no |

Also present and possibly useful later: `libQt6TextToSpeech`, Qt's `networkinformation` plugin.

Open points:
- The Debian build container needs the matching `-dev` packages (amd64 and arm64) for every
  module used. Not checked yet.
- WebEngine and Multimedia come with SteamOS's desktop (KDE) packages; a SteamOS update could
  remove them.