Initial FrameAssistant baseline from the vrdashboard template
Template stripped to the overlay platform (examples, demo UI and Frame test package removed), renamed to FrameAssistant, and working rules for the project recorded in CLAUDE.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -0,0 +1,162 @@
|
||||
# Architecture
|
||||
|
||||
A dashboard is **one process**. It renders QML scenes off-screen into textures and hands
|
||||
them to SteamVR as overlays. Input flows the other way: SteamVR reports laser, scrolling,
|
||||
keyboard and controller actions, and the app turns them into ordinary Qt events. The QML
|
||||
never knows it is in VR, which is why it also runs in a plain desktop window (`--desktop`).
|
||||
|
||||
## Structure
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
app["<b>Your app</b><br/>class Dashboard : DashboardApplication<br/>members, loading() / started()"]
|
||||
subgraph base["DashboardApplication"]
|
||||
direction TB
|
||||
engine["QQmlEngine<br/>(shared by all surfaces)"]
|
||||
runtime["VrRuntime<br/>SteamVR connection"]
|
||||
nav["ControllerNavigation"]
|
||||
end
|
||||
subgraph surfaces["Surfaces: one per view"]
|
||||
direction TB
|
||||
s1["<b>Surface 'main'</b><br/>QML root → Renderer → RenderBackend<br/>DashboardOverlay + InputBridge"]
|
||||
s2["<b>Surface 'settings'</b> (optional)<br/>same parts"]
|
||||
end
|
||||
steamvr(["SteamVR"])
|
||||
app -- "inherits" --> base
|
||||
base -- "creates, drives" --> surfaces
|
||||
surfaces -- "textures" --> steamvr
|
||||
steamvr -- "laser, scroll, keyboard" --> surfaces
|
||||
steamvr -- "controller actions" --> nav
|
||||
nav -- "keys to the active surface" --> surfaces
|
||||
```
|
||||
|
||||
**`DashboardApplication`** (`src/app/DashboardApplication.*`) is the base class of every app.
|
||||
It owns everything that isn't app-specific: command line, logging, the SteamVR connection,
|
||||
choosing the render backend, the surfaces, controller navigation, the auxiliary modes, and
|
||||
the order in which all of that is created and destroyed.
|
||||
|
||||
Your app derives from it:
|
||||
- **Constructor:** configuration (`addSurface()`, optionally `setAppKey()`,
|
||||
`setDisplayName()`, `setQuickStyle()`, `setQmlModule()`).
|
||||
- **Hooks:**
|
||||
- `loading()`: the shared QML engine, the surfaces and their render backends exist; QML
|
||||
isn't loaded yet. Set up what the QML needs here.
|
||||
- `started()`: everything runs. Connect to surfaces and input here.
|
||||
- **Accessors:** `mode()`, `appKey()`, `appName()`, `engine()`, `surfaces()`, `surface()`
|
||||
(the first), `surface(id)`, `vrRuntime()`, `navigation()`.
|
||||
- **Lifetime:** the base constructor creates the `QGuiApplication`, so your members can be
|
||||
QObjects with timers. All QML and surfaces are torn down at the end of `exec()`, while your
|
||||
object and its members still exist.
|
||||
|
||||
**`Surface`** (`src/app/Surface.*`) is one view: a QML root, its renderer, its render backend
|
||||
and its overlay. It's a component you configure (`SurfaceConfig`: id, QML root type, tab name,
|
||||
texture size, icon) and connect to (`visibilityChanged`, `activated`), never subclass.
|
||||
- All surfaces share one QML engine, so models and singletons are visible in every view.
|
||||
- Overlay key: `<app key>.<id>`, or the app key itself for an empty id.
|
||||
- A surface renders only while its overlay is visible.
|
||||
|
||||
**Render layer** (`src/render/`):
|
||||
- **`OffscreenQuickRenderer`** drives Qt Quick through `QQuickRenderControl`, without a window.
|
||||
- It renders on demand (scene change or animation), capped at `--max-fps`, on an absolute
|
||||
schedule so slow frames don't lower the frame rate.
|
||||
- **`RenderBackend`** is the *only* place with graphics-API code: device setup, texture,
|
||||
hand-off description for SteamVR, readback.
|
||||
- **`VulkanRenderBackend`** (default): Qt creates the device on the GPU SteamVR names, with
|
||||
the extensions SteamVR requires. The backend owns the image and moves it to
|
||||
`TRANSFER_SRC_OPTIMAL` after each frame, the layout OpenVR wants, and back before the next.
|
||||
- **`OpenGLRenderBackend`** (fallback): on the Frame it runs through zink on Vulkan.
|
||||
|
||||
**VR layer** (`src/vr/`):
|
||||
- **`VrRuntime`**: `VR_Init` as overlay app, polls SteamVR events and routes them by overlay
|
||||
handle. It also reports SteamVR's Vulkan requirements, and registers and launches manifests.
|
||||
- **`Overlay`**: base of all overlays. It handles input routing, visibility and frame
|
||||
submission.
|
||||
- **`DashboardOverlay`** is the dashboard tab with its icon. A world overlay would be
|
||||
another subclass (planned).
|
||||
|
||||
**Input layer** (`src/input/`):
|
||||
- **`OverlayInputBridge`** (one per overlay):
|
||||
- **Laser:** turned into mouse events, with the y-axis flipped (OpenVR counts from the
|
||||
bottom).
|
||||
- **Scrolling:** thumbstick scrolling becomes wheel events.
|
||||
- **Keyboard:** opens the SteamVR keyboard when a text field gets focus, and turns typed
|
||||
characters into key events.
|
||||
- **`ControllerNavigation`** (one per app): SteamVR Input actions (`resources/input/`, rebindable
|
||||
in SteamVR) become Tab/arrow/Space/Escape for the *active* surface, i.e. the one last opened
|
||||
or pointed at. Qt Quick's own focus handling does the rest.
|
||||
|
||||
## One frame
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant QML as QML scene
|
||||
participant R as OffscreenQuickRenderer
|
||||
participant B as RenderBackend (Vulkan)
|
||||
participant O as Overlay
|
||||
participant S as SteamVR
|
||||
QML->>R: scene changed / animation tick
|
||||
Note over R: wait for the next slot (≤ max-fps)
|
||||
R->>B: beginFrame (image back to COLOR_ATTACHMENT)
|
||||
R->>R: polish, sync, render (Qt Quick)
|
||||
R->>B: finishFrame (image → TRANSFER_SRC, GPU done)
|
||||
R->>O: frame sink: texture + bounds
|
||||
O->>S: SetOverlayTexture (SteamVR copies it)
|
||||
```
|
||||
|
||||
## Input
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant S as SteamVR
|
||||
participant V as VrRuntime
|
||||
participant I as OverlayInputBridge
|
||||
participant Q as QML (QQuickWindow)
|
||||
S->>V: overlay event (mouse, scroll, keyboard)
|
||||
V->>I: routed by overlay handle
|
||||
I->>Q: QMouseEvent / QWheelEvent / QKeyEvent
|
||||
Q-->>I: text field got focus
|
||||
I->>S: ShowKeyboardForOverlay
|
||||
```
|
||||
|
||||
## Start and shutdown (overlay mode)
|
||||
|
||||
1. **Start:**
|
||||
1. Connect to SteamVR (`VR_Init`).
|
||||
2. Create one render backend per surface. The first surface decides the API, including a
|
||||
fallback to OpenGL; the others use the same API, because Qt's graphics API is
|
||||
process-wide.
|
||||
3. Call `loading()`.
|
||||
4. Load each surface's QML.
|
||||
5. Create the overlays.
|
||||
6. Set up controller navigation.
|
||||
7. Call `started()`, then run the event loop.
|
||||
2. **Shutdown runs in reverse**, with one rule found on the Frame: **overlays first, then
|
||||
SteamVR, then the graphics devices.** OpenVR requires shutting down before the devices it
|
||||
got textures from are destroyed. Otherwise its client library frees objects on a dead
|
||||
`VkDevice` at exit, which crashed on the Frame. `runOverlay()` encodes this order through
|
||||
the declaration order of its objects.
|
||||
|
||||
## Building blocks in CMake
|
||||
|
||||
- **`vrdash_add_dashboard()`** (`cmake/Dashboard.cmake`) turns sources plus QML files into a
|
||||
dashboard executable. It covers:
|
||||
- the QML module, whose `QML_URI` becomes the app's default module through a generated
|
||||
source file
|
||||
- bytecode-only QML for system-Qt builds
|
||||
- `libopenvr_api.so` next to the executable
|
||||
- install rules and registration for Qt bundling
|
||||
- the check against SteamVR process names
|
||||
- **App identity** (root `CMakeLists.txt`): `project()` name and version, `VRDASH_APP_KEY`,
|
||||
`VRDASH_DISPLAY_NAME`, `VRDASH_EXECUTABLE_NAME`, texture size. CMake generates from them
|
||||
`AppConfig.h` and the SteamVR manifest, with `binary_path_linux_arm` for ARM builds and
|
||||
`binary_path_linux` for x86_64.
|
||||
|
||||
## Planned: world overlays
|
||||
|
||||
For e.g. a notification app: a `WorldOverlay` (another `Overlay` subclass, created with
|
||||
`CreateOverlay`) that the app shows and hides itself.
|
||||
- **Placement modes:** room, HUD, hand, "spawn in front, then stay".
|
||||
- **Optionally operable:** while visible, it takes the system laser
|
||||
(`VROverlayFlags_MakeOverlaysInteractiveIfVisible`).
|
||||
|
||||
The surface structure is ready for it: `SurfaceConfig` would gain a kind and a placement.
|
||||
@@ -0,0 +1,71 @@
|
||||
# Steam Frame
|
||||
|
||||
How to build, deploy and run dashboards on the Steam Frame, and what was learned on the
|
||||
device. Facts below were verified on a Frame with SteamOS `holo` 0.4.3 (system Qt 6.8.0,
|
||||
Adreno 750 GPU with Mesa Turnip).
|
||||
|
||||
## Building
|
||||
|
||||
Frame builds run in a Debian container (Docker or Podman), either cross-compiled on an x86_64
|
||||
PC (no emulation) or natively on the Frame:
|
||||
|
||||
```sh
|
||||
scripts/container-build.sh cross # on the PC → build/cross-arm64/stage
|
||||
scripts/container-build.sh native # on the Frame itself (podman/distrobox)
|
||||
scripts/container-build.sh cross-bundle # fallback variant, Qt bundled
|
||||
scripts/container-build.sh cross shell # a shell in the build container
|
||||
```
|
||||
|
||||
**System Qt (default).** Builds use SteamOS's own Qt; packages are about 3 MB.
|
||||
- The container builds against Debian's Qt 6.8.2; at runtime the Frame's Qt 6.8.0 is used.
|
||||
Qt keeps binary compatibility within 6.8, as long as the program uses no Qt-internal API.
|
||||
- **QML is compiled to bytecode only** (`VRDASH_QML_BYTECODE_ONLY`), because Qt's QML-to-C++
|
||||
compiler output is tied to the exact Qt version.
|
||||
- The binary then needs only the stable Qt API, `qt_version_tag` (Qt ≥ 6.8), and Qt's Vulkan
|
||||
wrapper functions. Verified with an Arch container frozen at Qt 6.8.0 and on the Frame.
|
||||
- After a SteamOS update with a different Qt, rebuild, or use the bundled variant.
|
||||
|
||||
**Bundled Qt (fallback, `-bundle` modes, about 46 MB).**
|
||||
- Qt, its plugins and QML modules, and their non-system dependencies are copied next to the
|
||||
program (`cmake/BundleQt.cmake`).
|
||||
- The GPU and display stack (GL/EGL, Vulkan loader, Wayland, X11, libffi, glibc) always comes
|
||||
from SteamOS; bundling it would break GPU access.
|
||||
- The Debian copyright files of all bundled libraries are included, as their licenses require.
|
||||
|
||||
## Packaging and deploying
|
||||
|
||||
```sh
|
||||
scripts/package-release.sh # → dist/<exe>-<version>-linux-arm64/ + .tar.gz
|
||||
scripts/package-release.sh --bundle-qt # → …-linux-arm64-bundled
|
||||
scripts/deploy-to-frame.sh steamos@<frame> # rsync the newest package to ~/<exe>, run install.sh
|
||||
```
|
||||
|
||||
A release folder contains the program, `install.sh` (install or update: stops a running
|
||||
instance, registers with SteamVR, starts it), `uninstall.sh`, `README.txt` and the licenses.
|
||||
SteamVR stores the folder's absolute path, so after moving the folder, run `install.sh`
|
||||
again.
|
||||
|
||||
Without the deploy script: copy the `.tar.gz`, unpack it on the Frame, run `./install.sh`
|
||||
(SteamVR must be running). Over SSH this works too: SteamVR starts the dashboard itself.
|
||||
|
||||
## What the Frame requires (all handled by the overlay platform)
|
||||
|
||||
| Finding | Consequence |
|
||||
|---------|-------------|
|
||||
| SteamVR is installed system-wide under `/opt/steamvr`; its log is `~/.local/share/Steam/logs/vrserver.txt` | Look there when a manifest or start fails |
|
||||
| The manifest needs **`binary_path_linux_arm`**; entries with only `binary_path_linux` are silently skipped ("must specify binary_path") | The manifest template picks the key from the target architecture |
|
||||
| SteamVR refuses **`VRApplication_Utility`** (`VRInitError_Init_NotAvailableToUtilityApps`) | Manifest commands connect as `VRApplication_Background` (needs a running SteamVR, never starts one) |
|
||||
| An executable named **`vrdashboard`** is taken for SteamVR's own dashboard (its watchdog then "restarts" it) | Executable `frame-assistant`; CMake rejects SteamVR process names |
|
||||
| **OpenGL runs through zink** on Vulkan (no native GL driver for the Adreno) | Vulkan is the default: 1.01 ms vs 1.89 ms per frame (`--benchmark`, 1600×1000) |
|
||||
| OpenVR must shut down **before** the graphics device is destroyed | Teardown order in `DashboardApplication::runOverlay()` (else a crash at exit with Vulkan) |
|
||||
| Qt needs a display connection even for off-screen rendering; over SSH none is set | SteamVR-launched instances inherit it; over SSH, set `DISPLAY=:0` |
|
||||
| Process names are cut to 15 characters | Scripts find running instances by command line, not by name |
|
||||
| Floated/pinned dashboard tabs stay visible and operable in games | Rendering follows the overlay's visibility, not the dashboard's |
|
||||
|
||||
## Notes
|
||||
|
||||
- **The Frame's package repositories** in `/etc/pacman.conf` are private (URLs marked
|
||||
"DO_NOT_SHARE"). Don't put them into scripts or this repository.
|
||||
- **Valve's public Holo Core preview container** (`registry.gitlab.steamos.cloud/holo/
|
||||
holo-core-aarch64-preview/base-devel`) ships Qt 6.10, newer than the Frame's 6.8. It is not
|
||||
usable for system-Qt builds.
|
||||
Reference in New Issue
Block a user