FrameAssistant/docs/REQUIREMENTS.md

Requirements

Status: approved 2026-10-05 (iteration 1, phase 1). Changes from now on are recorded as decisions below.

Goals

ID	Goal
G1	Sensors: report the state of the headset and its surroundings to Home Assistant (HA).
G2	Services: offer actions on the headset that HA can call, e.g. notifications.
G3	Dashboard: show HA's user interface inside VR.

Context and constraints

Known facts, not requirements:

• Target device: Steam Frame (aarch64, SteamOS, system Qt 6.8). Also runs on x86_64 Linux with SteamVR.
• Platform: SteamVR dashboard overlay (C++20, Qt 6 Quick, OpenVR), see ARCHITECTURE.md and STEAM-FRAME.md.
• SteamVR starts the app as an overlay application; it runs while SteamVR runs.

Requirements

Priority: Must, Should, Could (confirmed by the user, D13). MVP marks the scope of the first iteration (D8). "Device" means one headset running FrameAssistant.

Connection (CON)

ID	Requirement	Prio	MVP	Acceptance criteria
CON-1	Connect to the configured MQTT broker using an existing MQTT client library. Login and TLS are supported as far as that library supports them.	M	✓	With valid settings the device connects; connection state and errors are logged.
CON-2	Publish all entities via HA MQTT discovery, grouped as one HA device per headset.	M	✓	Entities appear in HA without manual HA configuration.
CON-3	Report availability: online while running, offline when stopped or crashed (last will).	M	✓	Normal exit or killed process → HA shows the device's entities as unavailable.
CON-4	When the broker is unreachable, reconnect with backoff; after reconnecting, republish discovery and current state. Nothing is buffered.	M	✓	Broker restart → entities return to correct states without restarting the app.
CON-5	Several headsets work side by side with one HA, each as its own device, identified by its configured device name.	M	✓	Two devices with different names → two HA devices, no shared entities.

Main tab (UI)

ID	Requirement	Prio	MVP	Acceptance criteria
UI-1	The main dashboard tab shows the connection status: device name, broker, connected or not, last error.	M	✓	Broker stopped → tab shows "not connected" and the error; broker back → "connected".

Configuration (CFG)

ID	Requirement	Prio	MVP	Acceptance criteria
CFG-1	Settings are read from a JSON file in ~/.config/frameassistant/. No settings UI.	M	✓	Changing the file and restarting changes the behaviour.
CFG-2	The device name is required. If it (or another required setting) is missing or the file is invalid, the app exits with an error message in the log file and on stderr and a non-zero exit code.	M	✓	Missing name → no MQTT connection, clear error in log and stderr, exit code ≠ 0.
CFG-3	Broker connection settings (host, port, credentials, TLS options as supported by the library) are configurable.	M	✓	Device connects to a broker given only in the file.

Sensors (SEN)

ID	Requirement	Prio	MVP	Acceptance criteria
SEN-1	Presence: headset worn / user active.	M	✓	Putting the headset on/off changes the state in HA.
SEN-2	Standby: headset in standby.	M	✓	Standby/wake changes the state in HA.
SEN-3	Headset battery level and charging state.	M	✓	Values match the headset's own display; plugging in toggles charging.
SEN-4	Per controller: connected, battery level, charging state.	M	✓	Turning a controller off/on and its battery are reflected in HA.
SEN-5	Current VR app/game (name; none when idle).	M	✓	Starting/quitting a game updates the sensor.
SEN-6	SteamVR dashboard open.	M	✓	Opening/closing the dashboard toggles the state.
SEN-7	Tracking state of headset and controllers.	S		Covering the cameras / losing tracking is reflected.
SEN-8	Play area: size and seated/standing.	S		Values match SteamVR's room setup.
SEN-9	Near boundary (boundary visible).	S		Walking to the boundary toggles the state.
SEN-10	Passthrough (room view) shown.	S		Toggling passthrough toggles the state.
SEN-11	Session duration since the headset was put on.	S		Resets on put-on; increases while worn.
SEN-12	Network: SSID, signal strength, IP address.	S		Values match the system's network settings.
SEN-13	Audio: output device, volume, mute.	S		Changing volume/mute in SteamOS updates HA.
SEN-14	Sensors publish on change, throttled per sensor (minimum interval).	M	✓	A rapidly changing value produces at most one update per interval.
SEN-15	The set of sensors is fixed: every device always offers the same entities. A sensor whose data the device doesn't support or currently has no value for is shown as unavailable in HA.	M	✓	On a device without a value (e.g. no battery reported), the entity exists and is unavailable; it becomes available once a value exists.

Notifications (NOT)

ID	Requirement	Prio	MVP	Acceptance criteria
NOT-1	HA can show a notification with title and text that hides after a timeout.	M	✓	Sent from HA → visible in VR (also during a game) → disappears after the timeout.
NOT-2	Placement configurable: head-locked, spawn in front and stay, on the hand.	S		Each placement works as described.
NOT-3	Priority levels with a distinct look (colour/icon).	S		Each level is visually distinguishable.
NOT-4	Sounds: built-in sound per priority; HA may give a sound URL instead.	S		Default sound plays per level; a given URL plays instead.
NOT-5	Action buttons; the chosen action is sent back to HA.	S		Choosing an action triggers an HA automation.
NOT-6	A notification is confirmed by: user dismisses it, user picks an action, timeout (non-critical only), or HA clears it.	S		Each of the four paths removes it from the list.
NOT-7	Unconfirmed notifications are listed in the main dashboard tab.	S		A critical notification not yet acknowledged appears in the list.
NOT-8	Critical notifications bypass do-not-disturb, stay until acknowledged, play a sound and vibrate the controllers.	S		All four behaviours observable.
NOT-9	Behaviour for several notifications at once (Q1).	S		Defined after design exploration.
NOT-10	Report to HA as an event how each notification was confirmed: dismissed, timed out or cleared (picked actions are covered by NOT-5). Only if easy to implement.	C		Each confirmation path produces an HA event naming the notification and the reason.

Other services (SRV)

ID	Requirement	Prio	MVP	Acceptance criteria
SRV-1	Attention signals: vibrate controllers, fade the view to a colour, show the boundary.	S		Each triggerable from HA.
SRV-2	Launch a VR app by app key; quit the current app.	S		Both work from HA.
SRV-3	System control: volume, mute, screenshot, standby.	S		Each works from HA (where SteamOS/SteamVR allows it).
SRV-4	Open the SteamVR dashboard on FrameAssistant's tab, optionally on a given HA view.	S		Dashboard opens on the tab.
SRV-5	Do-not-disturb switch, controllable from HA and in VR; suppresses non-critical notifications.	S		Toggle in either place is reflected in the other.
SRV-6	Status cards (e.g. progress, countdown, door open), set, updated and cleared by HA, in their own small dashboard tab that exists only while at least one card exists (D9).	S		First card adds the tab; clearing the last removes it; the tab can be floated.
SRV-7	The status tab returns to its previous floating position when it reappears.	C		Floated tab reappears at the old position.
SRV-8	Play audio (sound or HA TTS) from a URL on the headset.	S		HA TTS message is heard in the headset.

Dashboard (DSH)

ID	Requirement	Prio	MVP	Acceptance criteria
DSH-1	A dashboard tab shows HA's web UI (Qt WebEngine) at a configured URL, as a "does it work out of the box" test.	M	✓	The page loads in VR and can be operated with the laser; whatever doesn't work out of the box is documented and moved to a later iteration.
DSH-2	The full dashboard to control HA entities in VR: everything beyond DSH-1, e.g. login comfort, opening a given HA view (SRV-4), possibly a native approach (Q2, Q3).	S		Defined after design exploration.

Non-functional (NFR)

ID	Requirement	Prio	MVP	Acceptance criteria
NFR-1	Runs on the Steam Frame and on x86_64 Linux with SteamVR.	M	✓	Release package works on the Frame.
NFR-2	Runs only while SteamVR runs (started by SteamVR).	M	✓	Starts with SteamVR, ends with it.
NFR-3	No measurable impact on VR performance: idle when nothing changes, no rendering while tabs are hidden, low CPU for sensors.	M	✓	Game frame rate/timing unchanged with FrameAssistant running (measurement method in design).
NFR-4	UI in English and German, following the system language.	S		Both languages complete.
NFR-5	Home LAN only; no remote access features.	M	✓	—
NFR-6	HA side needs only the MQTT integration; nothing custom installed in HA.	M	✓	Works on a stock HA with MQTT.
NFR-7	Targets the current HA release. Only current HA features are used, nothing deprecated or legacy; older HA versions are not supported.	M	✓	Works with the current HA release; HA logs no deprecation warnings for FrameAssistant's entities.
NFR-8	Unit tests for logic that runs without headset or broker; a manual on-device checklist per iteration. The test strategy is adjusted as the project progresses.	M	✓	Tests pass; checklist done and documented.
NFR-9	No latency requirement for notifications.	—		—

Out of scope

• Images/camera snapshots in notifications.
• A select entity listing installed apps.
• Sensors for mic in use and screenshots.
• Running without SteamVR; remote access; a settings UI.
• Publishing (private for now).

Open questions

#	Question	Status
Q1	Several notifications at once: stack, queue, replace by tag?	open, design exploration
Q2	Dashboard: HA's own web UI vs. native VR UI?	open, design exploration (Qt WebEngine exists on the Frame, Appendix C)
Q3	Dashboard data: HA WebSocket API (token) in addition to MQTT, or MQTT only?	open, design
Q4	Which MQTT client library? Licence, availability on SteamOS, TLS/login support.	answered: libmosquitto, shipped in the package (D14, Appendix B)
Q5	Status card: own tab or part of the main tab?	answered: own tab, on demand (D9)
Q6	MVP sensor list?	answered (MVP column)
Q7	Minimum supported HA version?	answered: current release, no legacy features (D10)
Q8	Sensors the Frame doesn't provide: don't create the entity, or show it as unavailable?	answered: fixed set, unavailable (D11)
Q9	Should HA be told when a notification is dismissed or times out?	answered: yes, as an event, if easy (D12)

Decisions

#	Date	Decision	Reason
D1	2026-10-05	Connect to HA via MQTT with HA MQTT discovery	Any entity type, commands possible; same approach as HASS.Agent
D2	2026-10-05	Use the user's existing MQTT broker	Already runs with HA
D3	2026-10-05	Home LAN only; no remote access	User's use case
D4	2026-10-05	Several headsets, one HA: each headset is its own HA device	User's use case
D5	2026-10-05	Settings in JSON at ~/.config/frameassistant/, device name required	All Frames share one default hostname
D6	2026-10-05	Run only while SteamVR runs	Simplicity
D7	2026-10-05	Use an existing MQTT library, support what it supports	Don't reimplement MQTT/TLS
D8	2026-10-05	MVP = MQTT + sensors + basic notification	User's priority
D9	2026-10-05	Status cards get their own dashboard tab, present only while cards exist	Float only the small panel; no empty tab
D10	2026-10-05	Target the current HA release; no deprecated/legacy features	Avoid legacy baggage
D11	2026-10-05	Fixed sensor set; unsupported or missing values are "unavailable"	Same entities on every device, predictable in HA
D12	2026-10-05	Notification confirmations are reported to HA as events, if easy	Cheap to add, useful for automations
D13	2026-10-05	Requirements approved as drafted, priorities included (MVP = Must, other wanted features = Should, SRV-7 and NOT-10 = Could)	End of phase 1
D14	2026-10-05	MQTT client: libmosquitto from Debian's package in the build container, shipped in the release package like libopenvr_api.so	Not on the Frame; Qt MQTT would need the exact Qt version (Appendix B)
D15	2026-10-05	HA MQTT discovery: own small layer on Qt's JSON classes, using device-based discovery (one <prefix>/device/<id>/config topic with all entities under cmps)	No suitable library (Appendix B); HA recommends device-based discovery for multi-entity devices (NFR-7)
D20	2026-10-05	New requirement UI-1 (status display in the main tab), MVP	Helps testing on the device; little effort
D28	2026-10-05	DSH-1 split: DSH-1 (MVP) = HA web UI in a tab as an out-of-the-box test; DSH-2 (Should) = the full dashboard. The MVP (D8) now also contains DSH-1	The user wants to see early whether the simple variant works

Design decisions (D16 onward) are in DESIGN.md.

Appendix A: data sources

What the headset can report, by source. Checked against openvr.h (OpenVR 2.15.6). None of it is verified on the Frame yet: SteamVR on the Frame may not fill every property.

OpenVR vs. OpenXR

• OpenVR (what we use) gives system-wide state: all tracked devices, batteries, the running app, the dashboard, the play area, standby. It also runs as a background/overlay app next to games.
• OpenXR is an API for the one app that renders the scene. It has no access to other apps, device batteries or the dashboard, and SteamVR's overlay/dashboard features exist only in OpenVR. Not useful for this project.

OpenVR

Data	API	Smart-home use
Worn / user active	GetTrackedDeviceActivityLevel (UserInteraction, Idle, Standby), events TrackedDeviceUserInteractionStarted/Ended	Presence ("in VR"), lights, do-not-disturb
Standby	events EnterStandbyMode / LeaveStandbyMode	Headset put away
Battery, charging (headset, controllers, trackers)	Prop_DeviceBatteryPercentage_Float, Prop_DeviceIsCharging_Bool, Prop_DeviceProvidesBatteryStatus_Bool	Low-battery alerts, charging reminders
Devices connected	events TrackedDeviceActivated/Deactivated, Prop_ControllerRoleHint_Int32	Controllers on/off
Tracking state	pose eTrackingResult, bPoseIsValid	Tracking lost
Running app / game	GetCurrentSceneProcessId → GetApplicationKeyByProcessId → app name; GetSceneApplicationState; event SceneApplicationChanged	"Playing X", automations per game
Dashboard open	IsDashboardVisible, events DashboardActivated/Deactivated	User is in menus
Passthrough (room view)	events RoomViewShown/Hidden	User can see the real room
Play area	GetPlayAreaSize, GetPlayAreaRect, GetCalibrationState, GetTrackingSpace (seated/standing)	Room-scale vs. seated
Near boundary	AreBoundsVisible (polled)	User close to walls/furniture
Screenshot taken	event ScreenshotTaken	HA event
Audio devices	k_pch_audio_* settings, event AudioSettingsHaveChanged	Which output is used (no volume)

Actions OpenVR offers (input for the services round): fade the view to a colour (FadeToColor), show the boundary (ForceBoundsVisible), vibrate controllers (TriggerHapticPulse), take a screenshot (RequestScreenshot), launch apps (IVRApplications), notifications (IVRNotifications; reportedly unreliable in current SteamVR, own overlay is the safer path).

Linux system (outside OpenVR)

Data	Source	Smart-home use
Network: SSID, signal, IP	NetworkManager (D-Bus) or /proc, Qt QNetworkInterface	Which network, connection quality
Audio: volume, mute, mic in use	PipeWire/PulseAudio	"In a call" → do-not-disturb
Device battery (Frame is standalone)	/sys/class/power_supply	Fallback if OpenVR has none

Other candidates ("environment")

• VR session duration (time since put on): break reminders, statistics.
• Mic in use: user is talking (voice chat), so don't play announcements.
• Streaming from a PC vs. standalone (Steam Link), if detectable.

Appendix B: MQTT client library (Q4)

Evaluated 2026-10-05 for D14.

	libmosquitto (C)	Qt MQTT	Eclipse Paho C/C++
Licence	EPL-2.0 / EDL-1.0	GPL-3 or commercial (no LGPL)	EPL-2.0 / EDL-1.0
On the Frame (SteamOS, checked)	no	no	no
Arch repos / Debian	mosquitto / libmosquitto-dev	qt6-mqtt / Qt add-on	AUR only / libpaho-mqtt-dev
Login, TLS	yes (OpenSSL)	yes (QSslSocket)	yes

• Qt MQTT is ruled out, not because of its licence (the user accepts GPL-3), but because it uses Qt's private API: QMqttClientPrivate derives from QObjectPrivate. QtCore then requires the exact Qt version (checkForIncompatibleLibraryVersion() in qobject_p.h calls qFatal("Cannot mix incompatible Qt library …")). A Qt MQTT built against Debian's Qt 6.8.2 aborts on the Frame's Qt 6.8.0 and would break with every SteamOS Qt update. It would only work with Qt bundled (about 46 MB instead of about 3 MB).
• libmosquitto does not depend on Qt. Debian trixie's libmosquitto1 2.0.21 (arm64) needs only libssl.so.3, libcrypto.so.3 and libc.so.6, with symbol versions up to OPENSSL_3.0.0 and GLIBC_2.34. The Frame has OpenSSL 3.2.1 and glibc 2.39 (checked 2026-10-05), so it fits. Licence per Debian: EPL-2.0 or EDL-1.0 (uthash.h: BSD-1-Clause); the licence text must ship with the package.
• Paho: no advantage over libmosquitto; uses its own threads.
• Integration into the Qt event loop (QSocketNotifier on mosquitto_socket()) is part of the design.

HA discovery libraries (D15)

No library fits; discovery itself is just JSON on a config topic.

• hacpp (C++, Linux): rejected. Early stage ("everything can be changed soon"); only binary sensor, sensor, button, cover; buffers messages on reconnect (contradicts CON-4); own MQTT client (async_mqtt) on Boost.Asio ≥ 1.90 (not on the Frame), Conan 2, GCC 14+; its own event loop next to Qt's; TLS/login undocumented.
• HAMqttDevice, KLib-HomeAssistantMqtt, hamqtt: Arduino/embedded payload generators with per-component topics.
• homie-cpp: implements the Homie convention, not HA discovery.

Appendix C: libraries per requirement

Checked on the Frame 2026-10-05 (SteamOS, system Qt 6.8.0). Rule: a Qt add-on module must come from the Frame's system Qt. Shipping one fails like Qt MQTT (Appendix B): Qt's modules use Qt's private API, which requires the exact Qt version.

Need	Requirements	Candidate	On the Frame	New dependency
JSON	CFG-1, CON-2	QJsonDocument (QtCore)	✓	no
HA discovery	CON-2/3/5, SEN-15	own layer (D15)	—	no
MQTT	CON-1	libmosquitto (D14)	✗, shipped; its deps libssl.so.3, libcrypto.so.3 ✓ (Appendix B)	yes
IP address	SEN-12	QNetworkInterface (QtNetwork)	✓	no
SSID, signal	SEN-12	NetworkManager via QtDBus	✓ libQt6DBus, nmcli	no
Volume, mute, output	SEN-13, SRV-3	PipeWire via libpulse, or wpctl/pactl	✓ libpulse.so.0, libpipewire-0.3.so.0, wpctl, pactl	to decide in design
Sounds, audio from URL	NOT-4, SRV-8	Qt Multimedia + QtNetwork	✓ libQt6Multimedia, FFmpeg plugin	no
HTTPS (downloads, WebSocket)	SRV-8, Q3	Qt TLS plugin	✓ libqopensslbackend.so	no
Translations	NFR-4	QTranslator, lupdate/lrelease	✓ (build-time tools)	no
Dashboard, HA web UI	DSH-1, DSH-2, Q2	Qt WebEngine (Quick)	✗ present but not loadable: 6.7.0 next to Qt 6.8.0, built against FFmpeg 6, which the Frame lacks (STEAM-FRAME.md)	yes, only bundled with all of Qt (experiment, see PROGRESS.md)
Dashboard, native UI	DSH-2, Q2/Q3	HA WebSocket API via Qt WebSockets	✓ libQt6WebSockets	no
Haptics, fade, apps	SRV-1/2	OpenVR	✓	no
Tests	NFR-8	QtTest	✓ (host only)	no

Lesson: "present" was checked by file only. Check that a library loads (ldd … | grep "not found") and that its package version matches the rest of Qt (pacman -Q | grep qt6-).

Also present and possibly useful later: libQt6TextToSpeech, Qt's networkinformation plugin.

Open points:

• The Debian build container needs the matching -dev packages (amd64 and arm64) for every module used. Not checked yet.
• WebEngine and Multimedia come with SteamOS's desktop (KDE) packages; a SteamOS update could remove them.