FrameAssistant/docs/PROGRESS.md

Progress

Where the project stands, and what happened in each session. Newest first. The process is in CLAUDE.md ("Working rules").

Current state

• Iteration 1 closed (2026-10-06); implementation iteration 2, phase 3 (implementation). Requirements (see "Next steps", D41) and design (DESIGN.md, "Iteration 2: MQTT and HA device", D42–D44) approved. DSH-1 had a negative result: HA's web UI via Qt WebEngine doesn't work on the Frame, so the "HA Dashboard" tab and the WebEngine dependency were removed (D40). The rest of iteration 1 (platform hooks, settings file, --config, log file for SteamVR-started instances) works on the Frame with the system-Qt package (3.7 MB).

• Frame test results of iteration 1 (2026-10-05):

  #	Check	Result
  1	Started by SteamVR: process runs, log file from the manifest's --log-file, settings loaded, rendering with Vulkan (Turnip, Adreno 750)	✓
  2	In VR: tab "Frame Assistant" visible and usable	✓
  3	Settings without device.name, ./install.sh → app exits, log says "Setting 'device.name' is required"	skipped (no device access), postponed
  4	Settings restored, ./install.sh → runs again	skipped (no device access), postponed

  The error itself is covered off the device by tst_config. Checks 3 and 4 run in the next session on the Frame (iteration 2 testing needs it too).

• Implementation follows the order in DESIGN.md (steps 1–7). Steps 1 (libmosquitto in CMake and the containers), 2 (MqttClient, MosquittoClient, tst_mqtt) 3 (HaEntity, HaSensor, HaDevice, unit tests) and 4 (app wiring, --desktop-novr, UI-1) and 5 (TLS) done; next: step 6 (app commands, --ha-remove, uninstall.sh).

Next steps

Requirements and design cover the whole MVP; it is implemented in four small iterations (D37), each with implementation, testing and assessment:

Iteration	Content	Platform change	Checked by
1	DSH-1: HA web UI in a dashboard tab (failed, removed: D40); configuration (D34) with start-up hook and --config; log file for SteamVR-started instances	start-up hook (D21), app command-line options (D38)	DSH-1 result documented; app starts on the Frame with the settings file, settings errors in the log
2	MQTT client (own thread), HA device with discovery and availability, libmosquitto in the package, status display UI-1	modes (D22)	device appears in HA, online/offline right, reconnect works
3	OpenVR wrappers and the 13 sensor entities	--desktop with optional SteamVR (D22)	sensor values in HA on the Frame
4	notification manager, notify entity, world overlay	world overlay, premultiplied flag (D25)	notification from HA in VR, also in a game

Requirements of implementation iteration 2: MQTT, HA device, UI-1

Approved 2026-10-06. Goal: FrameAssistant connects to the user's broker, appears in HA as a device that goes online and offline correctly, and the main tab shows the connection status. No real sensors yet: a dummy sensor stands in for them (D41).

• Dummy sensor "Uptime": a sensor with the seconds since start, entity_category: diagnostic, updated every 10 s. In the code a generic test entity whose value is set from outside: the app feeds it a timer, unit tests set values directly. Ships in iteration 2 builds; iteration 3 decides whether it is removed from HA (component with only p once) or kept as a diagnostic entity.

Item	Acceptance criterion
CON-1, CFG-3	Connects without login, with login, and with TLS (self-signed CA); connection state and errors are logged
CON-2	The device with the dummy sensor appears in HA via device-based discovery, without configuration in HA
CON-3	Online while running; normal exit and kill -9 make it unavailable
CON-4	After a broker restart it reconnects with backoff and republishes discovery and states; also republishes after HA's birth message
CON-5	Two settings files with different device names give two HA devices
SEN-14	The dummy sensor's updates are throttled
UI-1	The main tab shows device name, broker, connected or not, and the last error, and follows a broker stop and start
Modes (D22)	Overlay, --desktop and the new --desktop-novr load the settings and connect; in iteration 2 the two desktop modes behave alike (neither uses SteamVR yet)
--ha-remove (D32)	Clears all retained topics, after which the device is gone from HA; uninstall.sh calls it
D35	libmosquitto is in both packages; package-release.sh's symbol-version check passes
Tests (D30)	Unit tests with a fake MQTT client; integration tests against a local mosquitto (connect, last will, reconnect, login, TLS)
Manual	CON, UI and --ha-remove checks from the PC against the user's HA (possible because nothing needs SteamVR). On the Frame: the package with libmosquitto runs; the postponed checks 3 and 4 of iteration 1

To find out during the iteration: where the Frame keeps its system certificates (default for mqtt.tls.ca_file).

Plan for implementation iteration 1: DSH-1 and configuration

Goal: a second dashboard tab "HA Dashboard" shows HA's web UI at the URL from the configuration; what works out of the box is tested and documented. The main tab stays "Frame Assistant".

Platform (vrdashboard_core):

1. Start-up hook prepare() (D21).
2. App-specific command-line options (D38): the base constructor takes an optional list of extra QCommandLineOptions, adds them to parsing and --help, and exposes their values.

App: 3. QtWebEngineQuick::initialize() in main.cpp before the app object (D39); no platform change. 4. Configuration (src/config/): immutable value plus load(path, &error), complete per D34; path from --config, else $XDG_CONFIG_HOME or ~/.config/frameassistant/config.json; loaded in Dashboard::prepare() (overlay and desktop modes). 5. HA web UI (src/ha/HaWebUi, D18): holds the URL (injected), exposed to QML as a singleton owned by Dashboard; a named, persistent WebEngine profile under ~/.local/share/frameassistant/ so the HA login survives restarts (D39). 6. New surface dashboard ("HA Dashboard"), QML root HaDashboard.qml with a WebEngineView; without a URL it shows a hint. 7. Build: find_package(Qt6 … WebEngineQuick); qt6-webengine-dev (native and :arm64) in the containers; a new static library for app logic (configuration, HaWebUi) that tests link.

Tests (D30): 8. tst_config: valid file, missing required fields, invalid JSON, wrong types, defaults, derived and explicit ha_unique_id, unknown keys, --config. 9. Platform: tst_surfaces extended for prepare() (success/abort) and extra options. 10. Manual, results documented: host --desktop; Frame desktop mode over RDP; Frame in VR with Vulkan and OpenGL: page loads, laser click, scrolling, SteamVR keyboard in the page's input fields, login kept after restart; CPU/GPU while the tab is open. 11. clang-format clean, ctest green; container-test.sh only if Vulkan code is touched.

Docs: 12. README (configuration file, --config, new tab), ARCHITECTURE.md and CLAUDE.md (new hook, app options), THIRD_PARTY_NOTICES (Qt WebEngine from the system), STEAM-FRAME.md (results), PROGRESS.md.

Known limitation: the bundled-Qt package does not support WebEngine in this iteration.

Log

2026-10-06: iteration 1 closed

• Frame checks 3 and 4 (settings error when started by SteamVR) skipped: the user had no access to the device. Postponed to the next session on the Frame; listed in CLAUDE.md under "Not yet verified on a Steam Frame".
• Phase 5 (assessment) of iteration 1: the user agreed to close it with checks 3 and 4 postponed and to start iteration 2.
• Requirements of iteration 2 approved (see "Next steps"). User's choices: a dummy sensor instead of a real one (D41); TLS, --desktop-novr and --ha-remove in this iteration. Since nothing in iteration 2 needs SteamVR (--desktop doesn't call VR_Init), the MQTT/HA checks run from the PC; the Frame only for the package.
• Design of iteration 2 written (DESIGN.md). User's decisions: libmosquitto's own thread (D42), app commands in the platform for --ha-remove (D43), committed TLS test certificates (D44). Design approved; phase 3 started.
• Step 1: cmake/Mosquitto.cmake (pkg-config; FRAMEASSISTANT_SHIP_MOSQUITTO=ON in all package presets installs lib/libmosquitto.so.1 and Debian's copyright file, which already contains the EPL-2.0/EDL-1.0 texts); libmosquitto-dev(:arm64) in the build containers, mosquitto in the test container; THIRD_PARTY_NOTICES entry. Checked: host configure finds 2.1.2; container-test.sh 8/8, no Vulkan validation messages; the cross build stages 2.0.21 (aarch64) and the copyright file. The binary doesn't need the library yet (nothing calls it; --as-needed), which changes in step 2. On the host, the GUI tests can't run in this session (no display).
• User: no need to ask for permission to continue between the steps of this phase.
• Step 2: MqttClient interface, MosquittoClient, tst_mqtt (12 cases against a local mosquitto). Reading libmosquitto 2.0.21's source showed that mosquitto_loop_start() doesn't fit (blocking DNS in the caller, silent thread end on TLS/protocol errors); the user chose an own std::thread (D45, amends D42). Findings: a failed first connect is retried; libmosquitto loses the DNS error code; strerror() is localised (German on the dev host), so errors use strerrordesc_np(). Test pitfall: QTRY_* evaluates its expression once more at the end, so it must not consume data (readAll…()). Checked: host 3 × 12/12; container-test.sh 9/9 (Debian's libmosquitto 2.0.21), no Vulkan validation messages; cross build: private Qt symbols only QVulkan*, at most GLIBC_2.34; the app binary links libmosquitto only from step 4 on (first use).
• Step 3: HaEntity (description, discovery component, throttling, availability), HaSensor, HaDevice (discovery payload, publish all after connect and after HA's birth message with random delay, last will, removeFromHa()), FakeMqttClient for tests. tst_haentity 11/11, tst_hadevice 14/14 on the host.
• Step 4: platform mode DesktopNoVr (--desktop-novr, behaves like --desktop for now; tst_surfaces surfaces_desktop_novr); Dashboard creates client and device in loading(), connects in started(), feeds the dummy sensor every 10 s and disconnects gracefully on aboutToQuit; ConnectionStatus QML singleton and the status card in Main.qml (UI-1). Checked:
  • end to end against a local mosquitto (--desktop-novr, watched with mosquitto_sub): discovery, online, state and availability in the designed order; broker restart → "connection lost", "connection refused", reconnect and full republish; kill -9 → will offline. (mosquitto also sends the will when it shuts down itself.)
  • screenshots under Xvfb: unconfigured (snapshot), connected, not connected with last error.
  • container-test.sh 12/12, no Vulkan validation messages; cross build links libmosquitto.so.1, at most GLIBC_2.34, private Qt symbols only QVulkan*.
  • Not checked yet: the graceful exit through aboutToQuit in the app (only in tst_mqtt); it needs a real window close or "Quit app" in SteamVR (manual test).
• Step 5: TLS in MosquittoClient (ca_file or the system's CAs, client certificate, checks before connecting); test certificates in tests/data/ (D44); 9 TLS cases in tst_mqtt. Findings:
  • SIGPIPE crash: with TLS, a write to a connection the broker closed killed the process (OpenSSL uses write()). Fixed by blocking SIGPIPE in the client thread.
  • libmosquitto's tls_set() needs a CA file or directory, so with system CAs only MOSQ_OPT_TLS_USE_OS_CERTS is set, and a client certificate needs ca_file (documented, reported as a settings error).
  • A failed handshake returns only "TLS error"/"Protocol error"; the reason is in libmosquitto's log, which is now appended.
  • Checked: host 3 × 21/21; container-test.sh 12/12 (libmosquitto 2.0.21), no Vulkan validation messages.
  • Still open: which CA store the Frame's OpenSSL uses by default (for ca_file empty); check on the device.

2026-10-05: requirements approved

• User approved REQUIREMENTS.md including the priorities (D13). Phase 2 started.
• Q4 decided: libmosquitto, shipped in the package (D14). Qt MQTT ruled out: private Qt API requires the exact Qt version. Checked on the Frame: no MQTT library installed; libssl/ libcrypto 3 and libcjson 1 present.
• HA discovery: own small layer on QJson, device-based discovery (D15); hacpp and the embedded libraries rejected.
• Checked on the Frame which Qt modules, plugins and audio/network tools exist; library list per requirement in REQUIREMENTS.md Appendix C. Notable: Qt WebEngine and Qt WebSockets are present (relevant for Q2/Q3).
• libmosquitto ABI check: Debian's 2.0.21 needs at most OPENSSL_3.0.0/GLIBC_2.34; the Frame has OpenSSL 3.2.1 and glibc 2.39. No libcjson dependency. D14 holds.
• Frame test of the remaining iteration 1: started by SteamVR with log file and settings, Vulkan rendering, tab usable in VR (checks 1, 2 ✓); checks 3, 4 left for the next session.
• DSH-1 closed (D40): the bundling experiment failed (sandbox, ICU resources, Vulkan abort, empty page with OpenGL and in a desktop window; details in STEAM-FRAME.md). At the user's decision the "HA Dashboard" tab, HaWebUi, the WebEngine build dependency and the bundler's WebEngine support were removed (reverts of the corresponding commits). Qt WebView is no alternative (WebEngine backend on Linux). homeassistant.url stays reserved for DSH-2, where a native dashboard is the candidate.
• Phase 4 (testing on the Frame), first findings:
  • deploy-to-frame.sh exited silently without a bundled package in dist/ (fixed).
  • The system-Qt package doesn't start: the Frame's Qt WebEngine is 6.7.0 (rest of Qt 6.8.0) and built against FFmpeg 6, which the Frame lacks (STEAM-FRAME.md). DSH-1's out-of-the-box variant therefore fails on the Frame; and since the app links WebEngine, the whole app fails.
  • At the user's request, experiment B: the bundled package now ships Qt WebEngine (Debian 6.8.2, own FFmpeg inside) with its helper process and resources: 141 MB archive, 352 MB unpacked. Needs GLIBC_2.38 and GLIBCXX_3.4.32 from the Frame. Test pending.
  • Other options noted: load WebEngine only at runtime so the app survives without it; a native dashboard (DSH-2, Q2b). Qt WebView is no alternative (on Linux its backend is WebEngine; native views are separate windows, not rendered into the texture).
• Phase 3 of implementation iteration 1:
  • Platform: prepare() hook and app-specific command-line options, tested in tst_surfaces (incl. surfaces_prepare_fail).
  • Settings file (src/config/, new library frameassistant_lib), --config; tst_config (37 cases). config.example.json in the package; README.txt explains the set-up.
  • "HA Dashboard" tab: WebEngineView, HaWebUi with a persistent profile, WebEngine initialisation in main(). Host: desktop mode loads a local test page, profile data lands in ~/.local/share/frameassistant/webengine; snapshot renders both tabs.
  • Build containers with Qt WebEngine. Cross build: the aarch64 binary needs no private Qt symbols, at most Qt_6.8 and GLIBC_2.34. Container tests 8/8 (incl. Vulkan with validation layers, no messages).
  • Snags: moc misses classes in files with raw strings containing // (CLAUDE.md); generated QML type registration includes headers by file name; Qt ≥ 6.9 hints at WebEngineProfilePrototype (not in the Frame's 6.8). Deviations: HaWebUi lives in the app, not the library; the password warning needs a password in the file.
  • Log file for SteamVR-started instances: the manifest passes --log-file logs/frame-assistant.log (user's choice among: default log path in the app, manifest argument, document only).
  • Not verified yet: WebEngine with a URL in the off-screen pipeline (Frame, phase 4); the bundled package (not supported for WebEngine in this iteration).
• Plan for implementation iteration 1 approved; phase 3 started.
• Plan for implementation iteration 1 written; new tab "HA Dashboard" (D38, D39).
• MVP implemented in four iterations; DSH-1 first, at the user's wish (D37).
• NFR-3 measurement method, manual in the MVP (D36). MVP details done.
• Shipping libmosquitto (D35).
• Configuration file format (D34).
• MVP notification: notify entity with first line as title; JSON command later (D33).
• HA MQTT interface: ids (D31), topics, retain, availability (D32), MVP entity list.
• Test strategy in four levels (D30). High-level architecture done.
• Threads: MQTT client in its own thread, everything else incl. OpenVR in the main thread (D29).
• DSH-1 split: web UI out-of-the-box test in the MVP, full dashboard as DSH-2 (D28).
• Overlays: two kinds (D25), one overlay for all notifications (D26), MVP placement "spawn in front, then stay" (D27). Researched notification UI in VR (DESIGN.md, Appendix A). Found: premultiplied-alpha flag missing in the platform.
• HA channels: MQTT for everything device-related; dashboard channel decided with Q2/Q3 (D24).
• Entities: one class per HA entity type, native types preferred, own service only as fallback (D23). Found: MQTT notify drops the title.
• Start-up hook prepare() in the platform (D21); modes, --desktop-novr and --config (D22).
• New requirement UI-1: status display in the main tab (D20).
• The user installed mosquitto on the dev host, so native builds can use libmosquitto.
• Started docs/DESIGN.md: one process (D16), three layers (D17), HA-centric object classes (D18). HA side: FrameAssistant needs only stock HA; how the user wraps commands (scripts, pyscript) is up to them (NFR-6 unchanged).
• Agreed to settle the high-level architecture first (list under "Next steps"), before component details.
• Stay with the Debian build container: an Arch container wouldn't match SteamOS's own package versions (private repos, rolling Arch) and has no arm64 multiarch for cross builds.

2026-10-04/05: project set-up and requirements

• Copied the vrdashboard template. Removed what the app doesn't need:

  • examples/ (diagnostics app) and the VRDASH_BUILD_EXAMPLES option
  • the Frame test package (scripts/package-frame-test.sh, docs/FRAME-TESTING.md, screenshots)
  • the demo UI (src/qml/DemoPage.qml)

  Main.qml shows a placeholder card.

• Named the app FrameAssistant, "Steam Frame companion for Home Assistant":

  • CMake project frameassistant, executable frame-assistant
  • app key net.matzman.frameassistant, tab "Frame Assistant", QML module FrameAssistant

  Names rejected:

  • "Steam…" and "…Home Assistant": trademarks
  • "HASS…": the user dislikes it, because "Hass" is German for hate.

• Agreed working rules (in CLAUDE.md):

  • explicit permission for everything; ask instead of assuming
  • iterative five-phase cycle; an informal "ok" approves a phase
  • document everything in Markdown in docs/

• git init. Baseline commit 4a5b090.

• Requirements engineering (several question rounds with the user). The results are in REQUIREMENTS.md:

  • decisions D1–D12
  • 51 requirements
  • research on OpenVR data sources (Appendix A)

• Checks: native-debug builds, ctest passes 6/6 (Vulkan surface test skipped, no Vulkan on this host), --snapshot renders.