D31/D32: ids, topics, retain and availability for the HA MQTT interface, MVP entity list. D33: MVP notification via the notify entity. D34: configuration file format. D35: shipping libmosquitto. D36: NFR-3 measurement. D37: MVP implemented in four iterations, DSH-1 first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
447 lines
30 KiB
Markdown
447 lines
30 KiB
Markdown
# Design
|
||
|
||
Status: **in progress**, iteration 1, phase 2 (planning and design). High-level architecture
|
||
and MVP details done; next: platform changes and the implementation plan. Requirements and their decisions (D1–D15):
|
||
[REQUIREMENTS.md](REQUIREMENTS.md). Design decisions start at D16.
|
||
|
||
## Process and threads
|
||
|
||
- **One process** started by SteamVR does everything: HA connection, sensors, services, all
|
||
overlays (D16).
|
||
- Several threads inside it for concurrent work (D16). **A thread is added only where work can
|
||
block** and would stall rendering and SteamVR events (D29):
|
||
|
||
| Thread | Runs |
|
||
|---|---|
|
||
| main | Qt event loop, QML, off-screen rendering of all surfaces, SteamVR event polling and texture submission, **all OpenVR calls** (I/O wrappers), HA device and entities, managers |
|
||
| MQTT | the MQTT client (libmosquitto): DNS lookup, TCP and TLS connect block for seconds when the broker is unreachable |
|
||
|
||
- OpenVR stays in the main thread: it isn't documented as thread-safe, and all events come
|
||
through one queue.
|
||
- The boundary is three signals with value arguments (queued connections):
|
||
|
||
```
|
||
main thread MQTT thread
|
||
HA device ── publish(topic, payload) ──▶ MQTT client
|
||
HA device ◀── messageReceived(topic, payload) ── MQTT client
|
||
HA device ◀── connectionChanged(state, error) ── MQTT client
|
||
```
|
||
|
||
- The MQTT client lives in its thread via `moveToThread()`; it creates its socket notifiers
|
||
and timers there, after the move.
|
||
- Not needed as own threads: WebEngine (own processes), Qt Multimedia and QtNetwork
|
||
(asynchronous), later D-Bus and `QProcess` for network/audio (asynchronous). Rendering stays
|
||
in the main thread as long as NFR-3 holds.
|
||
- Rules: no direct calls on objects of another thread (signals or
|
||
`QMetaObject::invokeMethod` with `Qt::QueuedConnection`); lambdas connected across threads
|
||
always get a context object; no `BlockingQueuedConnection`; only values, no shared
|
||
pointers, cross the boundary.
|
||
|
||
## Layers
|
||
|
||
Three layers; dependencies only go downward (D17).
|
||
|
||
```
|
||
Presentation QML (view and controller)
|
||
▼
|
||
Business logic entities, services, managers
|
||
▼
|
||
I/O HA connection (MQTT), SteamVR, system (network, audio)
|
||
```
|
||
|
||
- Qt Quick already enforces separating presentation from logic.
|
||
- The overlay platform (`vrdashboard_core`) spans two layers: overlays, rendering and input
|
||
are presentation infrastructure; SteamVR access for sensors and actions is I/O.
|
||
|
||
## HA channels
|
||
|
||
- **MQTT is the only channel for everything device-related:** entities, services, availability
|
||
(D24).
|
||
- MQTT can't show HA as a whole (other entities' states, service calls for them), which the
|
||
dashboard (DSH-1) needs. The dashboard gets **its own channel, separate from MQTT**:
|
||
- Q2a, HA web UI in Qt WebEngine: the browser talks to HA itself (HTTP/WebSocket, HA's own
|
||
login); config: HA URL.
|
||
- Q2b, native QML dashboard: HA's WebSocket API with a token (Q3) as another I/O component;
|
||
config: HA URL and token.
|
||
- Both fit the three layers. The MVP contains the web UI test (DSH-1, D28), which also serves
|
||
as the spike on WebEngine in the off-screen renderer; Q2/Q3 are decided for DSH-2.
|
||
- Known DSH-1 risks: WebEngine must be initialised before the `QGuiApplication` (created by the
|
||
platform's base constructor), so the platform needs a small change; WebEngine with our
|
||
off-screen renderer, especially Vulkan, is untested; text fields inside the page probably
|
||
don't open the SteamVR keyboard (fallback: log in once in a desktop mode).
|
||
|
||
## Start-up and modes
|
||
|
||
- **Start-up hook (D21):** the platform (`DashboardApplication` in `vrdashboard_core`) gets a
|
||
third hook, e.g. `virtual bool prepare(QString* error)`. `exec()` calls it once mode and app
|
||
key are known, before the QML engine, the surfaces and `VR_Init`. On `false`, `exec()` ends
|
||
with exit code 1 and reports the error through the existing path (log and stderr). The
|
||
default returns `true`, so other apps built on the template don't change. FrameAssistant
|
||
loads its configuration there and decides from `mode()` whether it needs it (CFG-2).
|
||
- **Modes (D22):**
|
||
|
||
| Mode | Configuration + HA connection | SteamVR |
|
||
|---|---|---|
|
||
| overlay (default) | yes | required (overlay app) |
|
||
| `--desktop` | yes | connects as background app if SteamVR runs; otherwise VR entities are unavailable (SEN-15) |
|
||
| `--desktop-novr` | yes | never |
|
||
| `--snapshot`, `--benchmark` | no | no |
|
||
|
||
- `--desktop` on the Frame over RDP shows the live UI with real data without wearing the
|
||
headset. Stop the SteamVR-started instance by hand first: two instances with the same
|
||
device name would knock each other off the broker.
|
||
- `--config <path>` loads another configuration file instead of
|
||
`~/.config/frameassistant/config.json`, e.g. on a PC with its own device name (CON-5).
|
||
|
||
## Overlays
|
||
|
||
Two overlay kinds in the platform (D25):
|
||
|
||
| Overlay | Kind | Visibility | MVP |
|
||
|---|---|---|---|
|
||
| Main tab (UI-1, later NOT-7, do-not-disturb) | dashboard tab | always there; SteamVR shows/hides it | ✓ |
|
||
| Notifications | world overlay | the app (notification manager) shows/hides it | ✓ |
|
||
| Status tab (SRV-6) | dashboard tab | exists only while cards exist (D9) | ✗ |
|
||
| HA dashboard (DSH-1, web UI test) | dashboard tab | always there | ✓ |
|
||
|
||
- **World overlays (MVP):** a new overlay kind next to the dashboard tab (CLAUDE.md, "Planned
|
||
extension"). The surface gets a kind and a placement; renders only while visible (NFR-3).
|
||
In the desktop modes (D22) a world overlay is a plain window the app shows and hides.
|
||
- **Premultiplied alpha:** Qt Quick renders premultiplied alpha; SteamVR assumes straight
|
||
alpha unless `VROverlayFlags_IsPremultiplied` is set, which the platform doesn't do yet.
|
||
Fully opaque/transparent pixels are right, semi-transparent ones (antialiasing, shadows)
|
||
would be too dark. Set the flag in the platform; verify on the device.
|
||
- **Runtime surfaces (later, SRV-6):** creating a surface at runtime repeats the start-up
|
||
steps for one surface. Destroying one while SteamVR runs is the risk: each surface has its
|
||
own backend (with Vulkan its own `VkDevice`), and SteamVR's client keeps objects on devices
|
||
it got textures from (the reason for the teardown order). Needs a spike on the Frame; ways
|
||
out: one device shared by all surfaces, or reuse instead of destroy.
|
||
- **Interactive world overlays (later, NOT-5):** the laser hits the whole overlay rectangle,
|
||
including transparent parts.
|
||
|
||
### Notifications (D26, D27)
|
||
|
||
- **One world overlay for all notifications** (D26). QML lays out the visible notifications
|
||
as cards inside it; transparent areas around them are invisible. No runtime surfaces needed,
|
||
one renderer and texture (NFR-3). One placement for all; per-notification placement is not
|
||
required (NOT-2 makes placement configurable, not per notification) and could later become
|
||
one overlay per placement.
|
||
- **MVP placement: spawn in front, then stay** (D27): about 1 m away, slightly below the line
|
||
of sight, positioned once when it appears. No continuous tracking. Lazy follow and hand
|
||
placement come with NOT-2.
|
||
- Placement techniques:
|
||
- head-locked: SteamVR keeps the overlay relative to the HMD; runs in the compositor, smooth.
|
||
- spawn in front, then stay: one absolute transform when shown.
|
||
- lazy follow: the app updates the transform continuously from the head pose; CPU cost and
|
||
jitter if updated too rarely.
|
||
- UI guidelines from research: Appendix A.
|
||
|
||
## Object classes
|
||
|
||
The app is HA-centric, so its building blocks follow HA's concepts (D18). Objects talk to each
|
||
other through Qt signals and slots (Qt's observer pattern). Senders don't know their receivers,
|
||
and signals across threads are delivered through the receiver's event loop.
|
||
|
||
| Class | Role | MVP |
|
||
|---|---|---|
|
||
| **HA entity** | One HA entity, both directions. Entity → HA: state and JSON attributes. HA → entity: commands; HA's MQTT integration turns its own services into them (e.g. `button.press`, `switch.turn_on`, `number.set_value`, `notify.send_message`). Observes value signals of other objects and publishes them; turns MQTT commands into signals other objects act on. **Native entity types are preferred** wherever their meaning fits (D23). | ✓ |
|
||
| **HA service** | **Fallback** where no entity type fits (D23): a JSON command on an own MQTT command topic (HA side: the standard `mqtt.publish` action), e.g. rich notifications or vibration with parameters. Results go back asynchronously, e.g. through an `event` entity. How the user wraps it in HA (scripts, automations, pyscript) is their choice; FrameAssistant only requires stock HA with MQTT (NFR-6). | depends on the NOT-1 choice |
|
||
| **HA device** | Owns the MQTT connection, discovery and availability, and owns all entity and service objects. One per headset (CON-5). | ✓ |
|
||
| **I/O wrappers** | Data sources and actuators; know nothing about HA. One wrapper object per OpenVR interface used (e.g. `IVRSystem`, `IVRApplications`, `IVRChaperone`, `IVRSettings`). System wrappers (network, audio) later. | OpenVR ✓, system ✗ |
|
||
| **Overlays** | View and controller of MVC (QML), as dashboard tabs or world overlays. Get their data (the model) from other objects. | ✓ |
|
||
| **Managers** | One object per stateful feature that manages and coordinates it, using entities, services and overlays. Notification manager (MVP); status card manager (SRV-6/7) later. | notifications ✓ |
|
||
| **HA web UI** | Encapsulates Qt WebEngine for the dashboard tab (DSH-1; Q2 for DSH-2). | ✓ (DSH-1) |
|
||
| **Configuration** | Settings from the JSON file, loaded once by the app as an immutable value. Each component gets the part it needs through its constructor (D19). | ✓ |
|
||
| **App** | The managing object: owns all of the above as components and wires them together. | ✓ |
|
||
|
||
### Entities and services (D23)
|
||
|
||
- **One class per HA entity type** on a common base. The base does the discovery part, topics,
|
||
availability and throttling, so SEN-14, SEN-15 and NFR-7 are handled in one place.
|
||
- A concrete entity is an **instance with a description** (id, name, device class, unit,
|
||
throttle interval), not a class of its own.
|
||
- Our actions map to native types where the meaning fits:
|
||
|
||
| Action | Type | HA service |
|
||
|---|---|---|
|
||
| Sensors (SEN-*) | `sensor`, `binary_sensor` | — |
|
||
| Notification (NOT-1) | `notify` | `notify.send_message` |
|
||
| Screenshot, standby, quit app, show boundary (SRV-1/2/3) | `button` | `button.press` |
|
||
| Do-not-disturb (SRV-5) | `switch` | `switch.turn_on/off` |
|
||
| Volume (SRV-3) | `number` | `number.set_value` |
|
||
| Launch app by key (SRV-2) | `text` | `text.set_value` |
|
||
| Attention signal with duration/volume (SRV-1), to check | `siren` | `siren.turn_on` |
|
||
| Feedback (NOT-5, NOT-10) | `event` | — |
|
||
|
||
- **Wiring:** the app wires simple entities (I/O wrapper signal ↔ entity). Managers get their
|
||
entities and services from the HA device and wire them internally. The HA device owns them
|
||
all (D18).
|
||
- **MQTT `notify` drops the title:** HA's `async_send_message()` renders only `message` into
|
||
the payload (`homeassistant/components/mqtt/notify.py`). NOT-1 needs title and text; options
|
||
(MVP detail): first line is the title, JSON in the message, or an own service.
|
||
|
||
Notes for later:
|
||
- OpenVR delivers all events through one queue; one dispatcher (today `VrRuntime`) hands them
|
||
to the wrappers.
|
||
- OpenVR is not documented as thread-safe (relevant for the thread decision).
|
||
|
||
## Open
|
||
|
||
1. Detailed plan for implementation iteration 1 (see [PROGRESS.md](PROGRESS.md), "Next steps").
|
||
|
||
## HA MQTT interface
|
||
|
||
Checked against HA's [MQTT documentation](https://www.home-assistant.io/integrations/mqtt/)
|
||
(NFR-7). The documentation prescribes no topic layout and shows both per-entity topics and a
|
||
shared JSON state topic without preference.
|
||
|
||
### IDs (D31)
|
||
|
||
- **Device id:** `ha_unique_id` from the configuration if given, else derived from the device
|
||
name. HA allows `[a-zA-Z0-9_-]`, e.g. "Wohnzimmer Frame" → `wohnzimmer_frame`. With a
|
||
derived id, renaming the device makes HA see a new device; `ha_unique_id` avoids that.
|
||
- **Entity `unique_id`:** `frameassistant_<device id>_<entity>`, e.g.
|
||
`frameassistant_wohnzimmer_frame_headset_battery`.
|
||
|
||
### Topics, retain, availability (D32)
|
||
|
||
Layout follows common practice (e.g. Zigbee2MQTT's `…/<device>/set`); one topic per entity.
|
||
|
||
| Purpose | Topic | Retained |
|
||
|---|---|---|
|
||
| Discovery (device-based, all entities under `cmps`) | `homeassistant/device/<device id>/config` | yes |
|
||
| Device availability (last will: `offline`) | `frameassistant/<device id>/availability` | yes |
|
||
| Entity availability (SEN-15) | `frameassistant/<device id>/<entity>/availability` | yes |
|
||
| State | `frameassistant/<device id>/<entity>/state` | yes |
|
||
| Attributes (e.g. app key of the running app) | `frameassistant/<device id>/<entity>/attributes` | yes |
|
||
| Command (`notify`, later `switch`, `button`, …) | `frameassistant/<device id>/<entity>/set` | no |
|
||
| HA birth message (subscribed) | `homeassistant/status` | — |
|
||
|
||
- Discovery uses the base topic `~` = `frameassistant/<device id>`.
|
||
- One topic per entity: per-entity throttling (SEN-14) and availability (SEN-15) stay simple.
|
||
- Each entity has two availability topics (device, entity) with `availability_mode: all`: it
|
||
is available only while the device is online *and* the entity has a value.
|
||
- Discovery, availability and states are retained. The headset is often off (FrameAssistant
|
||
runs only with SteamVR); retained discovery keeps its entities defined in HA across HA
|
||
restarts, showing "unavailable" through the retained `offline`.
|
||
- On HA's birth message (`homeassistant/status` = `online`) discovery and all states are sent
|
||
again after a random delay (HA's documentation recommends the delay against broker load).
|
||
- Against "ghost entities": on uninstall, FrameAssistant clears its retained topics (empty
|
||
payloads), e.g. through a command `uninstall.sh` calls (`--ha-remove`; detail later).
|
||
- Removing an entity from a newer version: publish its component with only `p` once, then
|
||
omit it (HA's documented procedure).
|
||
|
||
### MVP entities
|
||
|
||
| Entity | Type | Device class | Requirement |
|
||
|---|---|---|---|
|
||
| Headset worn | `binary_sensor` | `occupancy` | SEN-1 |
|
||
| Standby | `binary_sensor` | — | SEN-2 |
|
||
| Headset battery | `sensor` | `battery` (%) | SEN-3 |
|
||
| Headset charging | `binary_sensor` | `battery_charging` | SEN-3 |
|
||
| Controller left/right connected | 2 × `binary_sensor` | `connectivity` | SEN-4 |
|
||
| Controller left/right battery | 2 × `sensor` | `battery` (%) | SEN-4 |
|
||
| Controller left/right charging | 2 × `binary_sensor` | `battery_charging` | SEN-4 |
|
||
| Running app | `sensor` (text), app key as attribute | — | SEN-5 |
|
||
| Dashboard open | `binary_sensor` | — | SEN-6 |
|
||
| Notification | `notify` | — | NOT-1 |
|
||
|
||
- 14 entities. Controllers are fixed as left and right (SEN-15: fixed set).
|
||
- Device info (`dev`): name = device name; manufacturer and model from OpenVR where available.
|
||
Origin (`o`): FrameAssistant and its version.
|
||
|
||
### MVP notification (D33)
|
||
|
||
- HA's MQTT `notify` entity passes only `message`; it drops `title`
|
||
(`async_send_message()` in `homeassistant/components/mqtt/notify.py`).
|
||
- **MVP: the `notify` entity with a convention: the first line of the message is the title**,
|
||
e.g. `notify.send_message` with `message: "Washing machine\nis done"`. A message without a
|
||
line break has no title. Native and selectable in HA's automation editor.
|
||
- **Later (with NOT-3 onward): an own JSON command** (`mqtt.publish`, e.g.
|
||
`{"title": …, "message": …, "priority": …}`) for everything the entity can't carry. Both
|
||
paths feed the same notification manager.
|
||
- Display time: one value from the configuration, default 10 s (Appendix A). Per-notification
|
||
durations come with the JSON command.
|
||
|
||
## Configuration file (D34)
|
||
|
||
- Location: `~/.config/frameassistant/config.json` (`$XDG_CONFIG_HOME` respected); `--config
|
||
<path>` overrides it (D22).
|
||
|
||
```json
|
||
{
|
||
"device": {
|
||
"name": "Wohnzimmer Frame",
|
||
"ha_unique_id": "wohnzimmer_frame"
|
||
},
|
||
"mqtt": {
|
||
"host": "homeassistant.local",
|
||
"port": 1883,
|
||
"username": "frameassistant",
|
||
"password": "…",
|
||
"tls": {
|
||
"enabled": false,
|
||
"ca_file": "/path/to/ca.crt",
|
||
"cert_file": "…",
|
||
"key_file": "…"
|
||
}
|
||
},
|
||
"homeassistant": {
|
||
"discovery_prefix": "homeassistant",
|
||
"url": "http://homeassistant.local:8123"
|
||
},
|
||
"notifications": {
|
||
"timeout_s": 10
|
||
}
|
||
}
|
||
```
|
||
|
||
| Key | Required | Default | Purpose |
|
||
|---|---|---|---|
|
||
| `device.name` | **yes** | — | device name in HA (CFG-2) |
|
||
| `device.ha_unique_id` | no | derived from the name | D31 |
|
||
| `mqtt.host` | **yes** | — | CFG-3 |
|
||
| `mqtt.port` | no | 1883, 8883 with TLS | |
|
||
| `mqtt.username` / `password` | no | no login | |
|
||
| `mqtt.tls.enabled` | no | `false` | |
|
||
| `mqtt.tls.ca_file` | no | system certificates | own CA, e.g. a self-signed broker certificate |
|
||
| `mqtt.tls.cert_file` / `key_file` | no | none | client certificate, if the broker requires one |
|
||
| `homeassistant.discovery_prefix` | no | `homeassistant` | |
|
||
| `homeassistant.url` | for DSH-1 only | — | missing: the dashboard tab shows a hint instead of the web UI |
|
||
| `notifications.timeout_s` | no | 10 | D33 |
|
||
|
||
- Errors (missing required values, invalid JSON, wrong types): exit with a message and exit
|
||
code ≠ 0 (CFG-2, via the start-up hook D21).
|
||
- Unknown keys: a warning in the log, no abort (catches typos without breaking across
|
||
versions).
|
||
- Password in plain text: warn if the file is readable by others; otherwise file permissions
|
||
(`chmod 600`).
|
||
- Throttle intervals (SEN-14) are fixed per entity in the code for the MVP.
|
||
- A `config.example.json` ships in the package.
|
||
- To check on the Frame when TLS is implemented: where the system certificates are
|
||
(`/etc/ssl/certs`?), for the `ca_file` default.
|
||
|
||
## Shipping libmosquitto (D35)
|
||
|
||
Modelled on OpenVR (`libopenvr_api.so` copied to `lib/`, found via RPATH `$ORIGIN/../lib`,
|
||
excluded from the Qt bundler, licence in `share/doc/frameassistant/`).
|
||
|
||
1. Found at build time through `pkg-config` (`libmosquitto.pc`; Debian and Arch).
|
||
2. Build containers: `libmosquitto-dev` (native) or `libmosquitto-dev:arm64` (cross); the
|
||
broker `mosquitto` in the test container for the integration tests (D30).
|
||
3. A CMake option (e.g. `FRAMEASSISTANT_SHIP_MOSQUITTO`) installs the real file behind
|
||
`libmosquitto.so.1` into `lib/`: on for the package presets (system Qt and bundled), off for
|
||
host builds, which use the system library. `libmosquitto` joins the Qt bundler's exclusion
|
||
list, like OpenVR, so it isn't shipped twice.
|
||
4. Licence: Debian's `libmosquitto1/copyright` into `share/doc/frameassistant/third-party/`,
|
||
plus the EPL-2.0/EDL-1.0 texts if that file lacks them (check when implementing); an entry
|
||
in `THIRD_PARTY_NOTICES.md`.
|
||
5. App-specific (e.g. `cmake/Mosquitto.cmake`, `src/CMakeLists.txt`); the platform stays free
|
||
of MQTT (D17).
|
||
6. Guard: `package-release.sh` checks with `readelf -V` that shipped libraries need no newer
|
||
symbol versions than the Frame has (today at most `OPENSSL_3.2`, `GLIBC_2.39`).
|
||
|
||
## Test strategy
|
||
|
||
Four levels (D30), implementing NFR-8:
|
||
|
||
| Level | What | How | Where |
|
||
|---|---|---|---|
|
||
| 1. Unit tests | configuration (CFG-2), discovery payloads, throttling (SEN-14), availability logic (SEN-15), entities, notification manager | QtTest with **fakes at the boundaries**: a fake MQTT client for the three signals (D29), fake I/O wrappers instead of OpenVR | host and test container, `ctest` |
|
||
| 2. Integration tests with a real broker | CON-1–4: connect, last will when killed, reconnect after broker restart, login/TLS | the test starts a local `mosquitto` on a free port (host: installed; test container: add the package) | host and test container, `ctest` |
|
||
| 3. UI and rendering | main tab and notification cards, transparency | existing tools: `--snapshot`, `tst_render`, desktop modes | host |
|
||
| 4. Manual | against the user's HA: entities appear, no deprecation warnings (NFR-7). On the Frame: sensors, notification in VR and in a game, DSH-1, premultiplied alpha, NFR-3 | checklist per iteration, results documented in `docs/` | user's HA, the Frame |
|
||
|
||
- OpenVR wrappers stay thin; logic goes into the testable parts behind them. The wrappers
|
||
themselves are only checked on the device (level 4).
|
||
- HA itself is tested manually for now (an HA container would be possible but costly);
|
||
revisit later (NFR-8 allows adjusting the strategy).
|
||
|
||
### NFR-3 measurement (D36)
|
||
|
||
1. **FrameAssistant's own cost:**
|
||
|
||
| Measure | Method | Target |
|
||
|---|---|---|
|
||
| CPU when idle (tabs closed, no notification) | process CPU time from `/proc/<pid>/stat` over 60 s | practically 0 (e.g. < 1 % of one core) |
|
||
| Rendering only while visible | renderer frame counter in the log | 0 frames while nothing is visible |
|
||
| GPU time per frame when visible | existing `--benchmark` | reference value |
|
||
|
||
2. **Impact on the game (A/B):** the same quiet scene (e.g. a game, standing still), 2 minutes
|
||
each: A without FrameAssistant (twice, to know the run-to-run noise), B with FrameAssistant
|
||
idle, C with a notification every 10 s. Compare dropped and reprojected frames and the
|
||
game's GPU time. Acceptance: B and C within A's noise.
|
||
3. **Tool:** in the MVP manually, with SteamVR's "Advanced Frame Timing". Later a measuring
|
||
mode (e.g. `frame-assistant --vr-stats 120`, background app, reads the compositor's
|
||
`GetCumulativeStats`/`GetFrameTimings` and prints a summary).
|
||
|
||
## Appendix A: notification UI in VR (research)
|
||
|
||
Collected 2026-10-05 for the notification design.
|
||
|
||
**Studies:**
|
||
- [Rzayev et al., CHI PLAY 2019](https://dl.acm.org/doi/abs/10.1145/3311350.3347190), 24
|
||
participants, tasks incl. gaming; placements head-up display (fixed in view), on-body,
|
||
floating, in-situ (embedded in the scene). On-body and floating: best understandability,
|
||
usability and presence. HUD: most intrusive and disturbing; only for urgent notifications.
|
||
- [Hsieh et al., CHI 2020](https://dl.acm.org/doi/10.1145/3313831.3376228), 40 participants;
|
||
displays on the HMD, on the controller, movable panel:
|
||
- Users dislike notifications blocking their focus region (HMD: inflexible location;
|
||
controller: blocks the work area). Recommendation: let users choose the location from a
|
||
set of peripheral positions.
|
||
- The more engaged the user, the more disruptive notifications feel. Then rather a small
|
||
peripheral indicator (e.g. unread count), content later.
|
||
- Don't reuse a modality the VR content already uses (e.g. game vibration hides
|
||
notification vibration). Urgent: more obtrusive means such as audio.
|
||
- Display time in the study was 12 s; longer is likely more disruptive.
|
||
- [NotifiVR, Ghosh et al. 2018](https://pubmed.ncbi.nlm.nih.gov/29543163): haptic alone was the
|
||
least effective modality.
|
||
|
||
**Platforms and tools:**
|
||
- [Meta Quest design guidelines](https://developers.meta.com/horizon/design/mr-design-guideline/):
|
||
avoid head-locked UI; anchor in space or "lazy follow" (follows the head delayed and
|
||
smoothed); about 1 m away, slightly below the line of sight. Users can set where
|
||
notifications appear.
|
||
- [visionOS](https://developer.apple.com/videos/play/wwdc2023/10078/): content to read goes
|
||
centred and slightly below the line of sight; periphery for context, centre for focus.
|
||
- [XSOverlay](https://xsoverlay.vercel.app/settings) (common SteamVR overlay tool): popups at a
|
||
configurable offset from the head, smoothly following;
|
||
[xsoverlay-tweak](https://github.com/chaixshot/xsoverlay-tweak) adds a "leash" mode (stays
|
||
until the head turns too far away).
|
||
|
||
**Consequences for FrameAssistant:**
|
||
1. Placement: not rigidly head-locked. Lazy follow/leash, spawn in front then stay, and hand
|
||
are well supported (matches NOT-2). Head-locked at most for critical notifications (NOT-8).
|
||
2. Position: about 1 m, below the line of sight or towards the periphery, not centred; offset
|
||
configurable.
|
||
3. Content: compact (title, short text), fade in/out, timeout around 10 s; longer content in
|
||
the main tab's list (NOT-7).
|
||
4. Several at once: few visible, stacked, newest in front (fits D26).
|
||
5. Modalities: sound as the main signal for important notifications (NOT-4/8), vibration only
|
||
as a supplement.
|
||
|
||
## Decisions
|
||
|
||
| # | Date | Decision | Reason |
|
||
|---|------|----------|--------|
|
||
| D16 | 2026-10-05 | **One process** started by SteamVR does everything; several threads inside it for concurrent work | Meets NFR-2 without extra work, no IPC; the platform already supports several surfaces per process |
|
||
| D17 | 2026-10-05 | **Three layers:** presentation (QML), business logic, I/O. The overlay platform spans presentation (overlays, rendering, input) and I/O (SteamVR access) | Simple and enough for the start; Qt Quick already enforces separating presentation from logic |
|
||
| D18 | 2026-10-05 | **HA-centric object classes:** HA entity, HA service, HA device, I/O wrappers, overlays, managers, HA web UI, configuration, app; communication through signals and slots | The app is HA-centric; signals/slots decouple objects and work across threads |
|
||
| D19 | 2026-10-05 | **Configuration is injected:** the app loads it once as an immutable value and passes each component the part it needs; no singleton | Explicit dependencies, trivial tests with own configs, thread-safe; the app creates all components anyway |
|
||
| D21 | 2026-10-05 | Start-up hook in the platform that may fail (`prepare()`), before the engine, surfaces and `VR_Init`; failure → exit code 1 | CFG-2 needs a clean abort; `loading()`/`started()` can't fail; backward compatible |
|
||
| D22 | 2026-10-05 | Overlay and both desktop modes load the configuration and connect to HA; `--desktop` also connects to SteamVR as a background app if it runs; `--desktop-novr` never does; `--snapshot`/`--benchmark` stay without configuration; new option `--config <path>` | Live development over RDP on the Frame with real data; PC development without SteamVR under its own device name |
|
||
| D23 | 2026-10-05 | **One class per HA entity type** on a common base; concrete entities are described instances. Native entity types (and the HA services the MQTT integration provides for them) are preferred; an own JSON command service only as fallback | Mirrors HA's model; discovery, availability and throttling in one place; actions are visible and selectable in HA without `mqtt.publish` |
|
||
| D24 | 2026-10-05 | MQTT for everything device-related; the dashboard gets its own channel to HA, chosen with Q2/Q3 | MQTT can't provide HA as a whole; both dashboard options fit the layers without affecting the MVP |
|
||
| D25 | 2026-10-05 | Two overlay kinds in the platform: dashboard tabs and world overlays (MVP: one placement, not interactive). Set `VROverlayFlags_IsPremultiplied`. Runtime surfaces and interactive world overlays later | Notifications need world overlays; Qt Quick renders premultiplied alpha |
|
||
| D26 | 2026-10-05 | One world overlay for all notifications; QML lays out the cards | No runtime surfaces, fewer resources; per-notification placement isn't required |
|
||
| D27 | 2026-10-05 | MVP notification placement: spawn in front (about 1 m, slightly below the line of sight), then stay | Research favours non-head-locked placement; no continuous tracking needed |
|
||
| D29 | 2026-10-05 | Threads: only the MQTT client runs in its own thread; everything else, incl. all OpenVR calls and the HA layer, in the main thread. New threads only for work that can block | Blocking connect must not stall rendering; OpenVR isn't documented as thread-safe; the narrowest possible thread boundary |
|
||
| D30 | 2026-10-05 | Test strategy in four levels: unit tests with fakes at the boundaries, integration tests against a local mosquitto, UI/render tools, manual checklist (user's HA, the Frame) | Covers NFR-8; the narrow boundaries (D29, I/O wrappers) make fakes cheap; HA automation deferred |
|
||
| D31 | 2026-10-05 | Device id from optional `ha_unique_id` in the configuration, else derived from the device name; entity ids `frameassistant_<device id>_<entity>` | Renaming shouldn't have to create a new HA device |
|
||
| D32 | 2026-10-05 | One topic per entity under `frameassistant/<device id>/`, common-practice layout; discovery, availability and states retained; republish on HA's birth message with random delay; device + entity availability with `availability_mode: all`; clear retained topics on uninstall | HA prescribes no layout; the headset is often offline, so retained discovery keeps its entities defined; per-entity topics keep throttling and availability simple |
|
||
| D33 | 2026-10-05 | MVP notification through the `notify` entity, first line = title; own JSON command later for richer notifications; display time from the configuration (default 10 s) | MQTT `notify` drops the title; native action for the common case, extensible path later |
|
||
| D34 | 2026-10-05 | Configuration file format (sections `device`, `mqtt`, `homeassistant`, `notifications`); required: device name and broker host; unknown keys only warn | Covers CFG-1–3, D31, D33, DSH-1 with few required values |
|
||
| D35 | 2026-10-05 | Ship libmosquitto like OpenVR: pkg-config at build time, copied to `lib/` by a CMake option for package presets, excluded from the Qt bundler, licence shipped; `package-release.sh` checks symbol versions against the Frame | Not on the Frame; keeps the platform MQTT-free; a Debian update that breaks compatibility shows up when packaging |
|
||
| D36 | 2026-10-05 | NFR-3 measured as own cost (CPU idle, frames while hidden, GPU per frame) plus an A/B comparison of the game's compositor statistics; manual in the MVP, a `--vr-stats` mode later | Measurable acceptance for NFR-3 without extra code in the MVP |
|
||
| D37 | 2026-10-05 | The MVP is implemented in four iterations: 1 DSH-1 + configuration, 2 MQTT/HA device/UI-1, 3 sensors, 4 notifications | Small iterations (working rules); DSH-1 first because the user wants to see it early and it doesn't depend on MQTT |
|